One of the most persistent problems in published research is publication bias: the systematic tendency to publish studies with statistically significant results while leaving those with null or contrary findings in the file drawer. In a meta-analysis, this bias can artificially inflate the combined effect size, because the available sample of studies is not representative of all the research that has been conducted on the topic. The funnel plot is the classic visual tool for evaluating this problem, but its interpretation is subjective. Egger's test offers a formal and quantitative alternative.
The statistical logic of Egger's test
The test was proposed by Matthias Egger and colleagues in an article published in the BMJ in 1997 that has become one of the mandatory references in any meta-analysis. The fundamental idea is simple and elegant: if publication bias does not exist, small studies (with large standard errors) and large studies (with small standard errors) should be symmetrically distributed around the true effect. Small studies will have more dispersed estimates simply because of their greater imprecision, but on average they should yield the same result as large studies.
The test formalizes this idea through a simple linear regression. The dependent variable is the standardized effect, calculated by dividing each study's effect size by its standard error (that is, d/SE, or equivalently, the study's z-statistic). The independent variable is the study's precision, defined as the inverse of the standard error (1/SE). If there is no bias, the regression line should pass through the origin, meaning the intercept should equal zero. An intercept significantly different from zero suggests funnel plot asymmetry.
To understand why this works, consider what happens when we divide both sides of the effect equation by SE. If the true effect is theta, then E[d/SE] = theta/SE = theta * (1/SE). This is a line through the origin with slope theta. When there is publication bias of the small-study effects type (small studies show larger effects than large ones), studies with low precision (low 1/SE) show disproportionately high standardized effects, which shifts the intercept upward.
The Egger regression plot
The graph associated with Egger's test plots precision (1/SE) on the X-axis and the standardized effect (d/SE) on the Y-axis. Each study appears as a point in this space. The weighted regression line is superimposed on the data, and its Y-intercept is the key statistic of the test. A positive intercept indicates that less precise studies (further to the left) tend to show disproportionately large standardized effects, which is consistent with the presence of publication bias or with the small-study effects phenomenon more generally.
Interpreting the intercept
The result of Egger's test is summarized by three values: the estimated intercept, its standard error, and the associated p-value. The null hypothesis is that the intercept equals zero, which would indicate symmetry in the funnel plot. Egger et al. originally recommended a significance level of p < .10 rather than the conventional p < .05, because the test has low statistical power and a stricter threshold increases the risk of failing to detect real bias. However, this convention is not universal, and many researchers use p < .05.
A positive intercept suggests that small studies tend to overestimate the effect (assuming that larger effects are coded as positive). A negative intercept would suggest the opposite: that small studies tend to underestimate the effect, which is less common in the context of publication bias but can occur in certain situations. The magnitude of the intercept informs about the severity of the asymmetry: intercepts close to zero, even if significant, suggest mild asymmetry; large and significant intercepts suggest a more serious problem.
Important limitations
Egger's test has several limitations that every researcher should know. The most cited is its low statistical power when the meta-analysis includes fewer than ten studies. With few studies, the test frequently fails to reach statistical significance even when real bias exists, producing false negatives that can give a false sense of security. Cochrane explicitly recommends against using the test when the meta-analysis includes fewer than ten studies.
Another fundamental limitation is that funnel plot asymmetry is not synonymous with publication bias. A significant Egger's test can result from genuine heterogeneity between studies, from differences in methodological quality between small and large studies, or from an inappropriate choice of effect measure. For example, odds ratios have a natural correlation between effect magnitude and standard error, which can produce artifactual asymmetry. Peters et al. (2006) proposed an alternative test that uses 1/n instead of 1/SE as the precision measure for binary effect measures, precisely to avoid this artifact.
It should also not be forgotten that Egger's test only detects a specific type of bias: small-study effects, that is, the tendency of small studies to show more extreme effects. There are forms of publication bias that do not produce this pattern, such as selective reporting of outcomes within a study (outcome reporting bias), which Egger's test cannot detect.
Comparison with other tests
The Begg and Mazumdar (1994) test, based on rank correlation between effect size and its variance, is a less powerful alternative to Egger's test and is used less frequently in contemporary practice. Peters' test (2006) is preferable when working with odds ratios or relative risks. Duval and Tweedie's trim and fill is not strictly a test but rather an adjustment method that estimates the number of potentially missing studies and calculates a corrected effect, although its assumptions are strong and its usefulness has been questioned.
Implementation in R and reporting
In R, the test can be run directly. With the metafor package, the regtest() function applied to an rma object produces Egger's test. With the meta package, the metabias() function provides the same result along with a graphical representation. Both functions allow selection of test variants (Egger, Thompson, Peters) through their arguments.
When reporting the result, it is recommended to indicate the type of test, the intercept value with its standard error, and the p-value. An example report following APA style could be: "Egger's regression test suggested no significant funnel plot asymmetry (intercept = 0.42, SE = 0.38, p = .287), although the statistical power of this test was limited given the small number of included studies (k = 8)." Including the mention of statistical power is important when the number of studies is low, to prevent readers from interpreting the absence of significance as evidence of absence of bias.
In summary, Egger's test is a valuable but imperfect tool for evaluating the presence of publication bias in a meta-analysis. It should be used as a complement to visual inspection of the funnel plot and to other bias assessment methods, never as definitive proof on its own. Its interpretation must always consider the context of the meta-analysis, including the number of studies, the presence of heterogeneity, and the type of effect measure used.