A collection of statistical tests

The tests module gathers a selection of statistical tests that are not routinely available in other packages.

# imports
import pandas as pd
import numpy as np
from stats_misc import tests

Wald based interaction test

The following test can be used to determine whether there is a statistical difference between two subgroup specific estimates.

res = tests.wald_interaction_test([0.1, 0.05], [0.01, 0.03])
print(f"The difference: {res.point_estimate:.2f} and its p-value {res.p_value:.3f}.")

The difference: 0.05 and its p-value 0.114.

Testing against a non-zero null

The null_value parameter shifts the null hypothesis to H₀: β₁ − β₂ = δ. This is useful for equivalence testing or when the scientific question is whether the difference exceeds a clinically meaningful threshold δ rather than zero.

# Test whether the difference exceeds a threshold of 0.02
res_nz = tests.wald_interaction_test([0.1, 0.05], [0.01, 0.03], null_value=0.02)
print(f"Difference vs null=0.02: estimate={res_nz.point_estimate:.2f}, p-value={res_nz.p_value:.3f}.")

Difference vs null=0.02: estimate=0.05, p-value=0.343.

Comparing observed data against an arbitrary theoretical distribution

TThe ks_test function compares observed data against a null distribution across known groups.

data = pd.DataFrame({
    'group': ['A', 'A', 'B', 'B', 'C'],
    'values': [0.1, 0.2, 0.3, 0.4, 0.5]
})
res = tests.ks_test(data, 'group', 'values', nulldistribution='uniform')
res["B"].pvalue

np.float64(0.32000000000000006)

Using alternative null distributions

The nulldistribution argument accepts any continuous distribution name recognised by scipy.stats. A common alternative is 'norm' to test whether group data is consistent with a standard normal distribution.

res_norm = tests.ks_test(data, 'group', 'values', nulldistribution='norm')
print({group: round(result.pvalue, 3) for group, result in res_norm.items()})

{'A': np.float64(0.424), 'B': np.float64(0.292), 'C': np.float64(0.617)}

Aggregate data one-way ANOVA

An one-way ANOVA based on aggregate data - the results will be identical to a one-way ANOVA using the individual participants data which gave rise to the aggregate data.

means = [2.0, 3.0, 8.0]
variances = [0.2, 0.2, 0.8]
sizes = [10, 11, 9]
res = tests.anova_one_way(means, variances, sizes)
print(f'The F-statistic is {res.test_statistic:.2f} with a p-value of {res.p_value:.4f}.')

The F-statistic is 256.99 with a p-value of 0.0000.

Fisher z-test for a correlation coefficient

correlation_test applies the Fisher z-transformation to test whether a Pearson correlation coefficient differs from a null value:

z = (arctanh(r) - arctanh(ρ₀)) × sqrt(n - 3)

Caveats:

• The p-value may be unreliable for smaller sample sizes (e.g. 20). Use a permutation test in these cases.

• The default null_value=0.0 tests H₀: ρ = 0; supply a non-zero value to test H₀: ρ = ρ₀.

# Test H₀: ρ = 0
res_zero = tests.correlation_test(0.45, 50)
print(f"H₀: ρ=0   → z={res_zero.test_statistic:.3f}, p={res_zero.p_value:.4f}")

# Test H₀: ρ = 0.3  (comparison against a non-zero reference)
res_nonzero = tests.correlation_test(0.45, 50, null_value=0.3)
print(f"H₀: ρ=0.3 → z={res_nonzero.test_statistic:.3f}, p={res_nonzero.p_value:.4f}")

H₀: ρ=0   → z=3.323, p=0.0009
H₀: ρ=0.3 → z=1.201, p=0.2298