Meta-analysis and heterogeneity methods

The following code can be used to conduct fixed and random effects meta-analysis and explore between estimate/study heterogeneity.

from stats_misc import meta_analysis as ma
from stats_misc.constants import (
    NamesMetaAnalysis as NamesMA,
)

# Example data
estimates = [0.1, -0.2, 0.23, 0.59]
standard_errors = [0.01, 0.03, 0.02, 0.08]
weights = [0.2, 1.2, 1.0, 0.9]

Fixed effect meta-analysis

A fixed effect meta-analysis assumes all the estimates are sourced from the same (hypothetical) population and hence that there is no between study/estimate variability. Often the between study variability is indicate by the Greek letter tau with an superscript 2, i.e., a fixed effect meta-analysis assumes that the tau-square is zero.

point, se = ma.fixed_effect(estimates, standard_errors)
print('The point estimate `{:.2f}` and its standard error `{:.2f}`.'.format(point, se))

The point estimate `0.10` and its standard error `0.01`.

Random effects meta-analysis

Compared to an fixed effect meta-analysis and random effects meta-analysis allows the tau-squared to be zero or larger. This represents a more believable model were the studies/estimates may be sourced from distinct (but unknown) populations. For this we do first need a good estimate of the tau-squared.

# first get the tau-squared estimate
tausquared = ma.estimate_tau(estimates, standard_errors)
# get the random effects estimate
point, se = ma.random_effects(estimates, standard_errors,
                              between_estimate_variance=tausquared)
print('The point estimate is {:.2f} and its standard error is {:.2f}, '
      'based on the tau-squared estimate of {:.3f}.'.format(point, se, tausquared))

The point estimate is 0.17 and its standard error is 0.16, based on the tau-squared estimate of 0.103.

Comparing tau-squared estimators

estimate_tau supports six estimation methods, differing in their weighting scheme and whether they iterate to convergence:

Method	Description
dl	DerSimonian–Laird method of moments (non-iterative)
ca	Cochrane ANOVA method of moments (non-iterative)
dl2	Two-stage DL: refines weights using the DL tau² estimate
ca2	Two-stage CA: refines weights using the CA tau² estimate
mm-it	General method of moments iterated to convergence (default)
pm-it	Paule–Mandel iterative estimator

All methods are described in DerSimonian & Kacker (2007). The iterative methods (mm-it, pm-it) are generally preferred as they reduce sensitivity to the initial weight choice.

methods = ['dl', 'ca', 'dl2', 'ca2', 'mm-it', 'pm-it']
for method in methods:
    tau2 = ma.estimate_tau(estimates, standard_errors, method=method)
    print(f'{method:>6}: tau² = {tau2:.4f}')

    dl: tau² = 0.0296
    ca: tau² = 0.1052
   dl2: tau² = 0.0994
   ca2: tau² = 0.1034
 mm-it: tau² = 0.1034
 pm-it: tau² = 0.1034

Evaluate between study heterogeneity

The tua-squared provides a direct way to evaluate the presence of between study/estimate heterogeneity. This can be formally tested using the Q-test and the I-squared. For this we first need to supply an estimate of the overall estimate.

# re-use the fixed effect estimate (can also use a random effects estimate)
point, _ = ma.fixed_effect(estimates, standard_errors)
# The actual heterogeneity cal
res = ma.heterogeneity(estimates, standard_errors, overall_estimate=point)

isquared_ci_form = '({:.2f}, {:.2f})'.format(getattr(res, NamesMA.ISQR_CI)[0], getattr(res, NamesMA.ISQR_CI)[1])
print('The Q-test p-value: {:.2f} and the I-squared estimate {:.2f}% with its confidence interval {}'.format(
    getattr(res, NamesMA.QPVAL),
    getattr(res, NamesMA.ISQR),
    isquared_ci_form,
))

The Q-test p-value: 0.00 and the I-squared estimate 98.33% with its confidence interval (97.88, 98.68)

Note that this class also return the DerSimonian and Laird method of moments tau-squared estimates which is equal to the dl results from estimate_tau.

print('The `heterogeneity based tau-squared estimates: {},\n'
     'Compared to the `DL` estimate from `estimate_tau`: {}.'.format(
    res.tau_squared, ma.estimate_tau(estimates, standard_errors, method='dl')))

The `heterogeneity based tau-squared estimates: 0.029609947089947092,
Compared to the `DL` estimate from `estimate_tau`: 0.029609947089947092.

Optional parameters for heterogeneity

Two parameters are often overlooked:

• ``alpha`` controls the width of the I² confidence interval (default 0.05 → 95% CI). Adjust this when reporting at a different significance level.

• ``tau2`` lets you supply a pre-computed tau² instead of having the function estimate it internally via DerSimonian–Laird. This is useful when you want the heterogeneity statistics to be consistent with a specific estimator (e.g. the mm-it estimate computed above).

# 90% CI for I-squared (alpha=0.10)
res_90 = ma.heterogeneity(estimates, standard_errors, overall_estimate=point, alpha=0.10)
ci_90 = getattr(res_90, NamesMA.ISQR_CI)
print(f'90% CI for I²: ({ci_90[0]:.2f}, {ci_90[1]:.2f})')

# Supply an external tau² (here the mm-it estimate)
tau2_mmit = ma.estimate_tau(estimates, standard_errors, method='mm-it')
res_ext = ma.heterogeneity(estimates, standard_errors, overall_estimate=point, tau2=tau2_mmit)
ci_ext = getattr(res_ext, NamesMA.ISQR_CI)
print(f'\nUsing mm-it tau²={tau2_mmit:.4f}: I²={res_ext.i_squared:.2f}%  95% CI ({ci_ext[0]:.2f}, {ci_ext[1]:.2f})')

90% CI for I²: (97.96, 98.63)

Using mm-it tau²=0.1034: I²=98.33%  95% CI (97.88, 98.68)