Publication bias is one of the most serious and persistent problems in empirical science. It occurs when studies with statistically significant results or those that confirm a hypothesis are more likely to be published than those with null or contradictory results. The cumulative effect is devastating: the published literature systematically overestimates the magnitude of real effects, and meta-analyses that rely exclusively on published studies inherit this bias. The funnel plot is the most widely used visual tool for assessing whether a meta-analysis may be affected by this problem.
Structure of the funnel plot
A funnel plot is a scatterplot in which each point represents a study included in the meta-analysis. The horizontal axis (X) displays the effect size estimated by each study, while the vertical axis (Y) shows a measure of precision, typically the standard error. There is a crucial detail in the orientation of the Y-axis: it is inverted, so that smaller values of standard error (that is, the more precise studies with larger samples) appear at the top of the plot, and the less precise ones at the bottom. This convention is not arbitrary: it reflects the idea that the most precise studies sit at the apex of the evidence.
The plot includes a vertical line indicating the combined effect estimate from the meta-analysis, and two diagonal lines representing the 95% confidence interval boundaries around that estimate for each level of precision. These diagonal lines form the funnel shape that gives the plot its name. The logic is as follows: more precise studies (upper portion) should cluster near the combined estimate, while less precise studies (lower portion) will scatter more widely. In the absence of bias, studies should be distributed symmetrically around the central line, filling the funnel evenly.
Interpreting symmetry
The key to interpretation lies in evaluating the symmetry of the plot. A symmetric funnel plot, where points are distributed evenly on both sides of the central line, suggests no visual evidence of publication bias. This does not mean bias is absent, only that the visual pattern does not detect it. An asymmetric funnel plot, on the other hand, raises a red flag. The most common form of asymmetry associated with publication bias is the absence of points in the lower-left region of the plot (or the lower-right, depending on the direction of the effect). This region would correspond to small studies that found null effects or effects in the opposite direction: exactly the type of studies least likely to be published.
When you observe this asymmetry, the interpretation is that unpublished studies may exist that, had they been included, would have reduced the combined effect estimate. In other words: the effect reported by your meta-analysis may be inflated because the evidence base is incomplete. This is a profoundly uncomfortable finding for any researcher, but ignoring it constitutes poor scientific practice.
Fixed effects vs. random effects models
The type of model used in the meta-analysis affects the appearance of the funnel plot. Under a fixed effects model, it is assumed that all studies estimate the same true population effect, and the variability between studies is attributed exclusively to sampling error. In the funnel plot, this translates to a single vertical line representing that common effect, and the funnel boundaries are relatively narrow because the expected variability is limited to sampling error alone.
Under a random effects model, each study is allowed to estimate a slightly different effect, due to variations in populations, contexts, interventions, or measurement instruments. In the funnel plot, the distribution of points tends to be more dispersed even in the upper portion of the plot, and the combined estimate incorporates an additional variance component (tau-squared) that reflects between-study heterogeneity. The choice between fixed and random effects models affects not only the point estimate and confidence interval of the combined effect, but also the visual interpretation of the funnel plot: a funnel plot that appears asymmetric under a fixed effects model may look more reasonable under a random effects model, and vice versa.
The trim-and-fill method
When a funnel plot suggests asymmetry, one of the most widely used sensitivity analysis tools is the trim-and-fill method (Duval and Tweedie, 2000). This method operates in three steps. First, it identifies and removes (trims) the smallest studies from the asymmetric side of the funnel plot. Second, it recalculates the combined effect estimate using the remaining studies. Third, it reintroduces the removed studies along with their symmetric counterparts (fill), that is, imputed studies that the method estimates are missing. The result is a new adjusted combined effect estimate that incorporates the hypothetically absent studies.
It is essential to understand what trim-and-fill is and what it is not. It is not a definitive correction for publication bias. It is a sensitivity analysis: it shows how much the effect estimate would change if the funnel plot asymmetry were entirely due to publication bias and if the missing studies had exactly the imputed characteristics. If the adjusted estimate differs substantially from the original, the meta-analytic conclusion is fragile and depends on the completeness of the study base. If it barely changes, the conclusion is more robust.
Limitations of the funnel plot
The funnel plot has important limitations that should not be overlooked. The most relevant is that asymmetry is not synonymous with publication bias. There are other legitimate causes of funnel plot asymmetry, such as genuine heterogeneity between studies (small studies may have used different populations or interventions), the correlation between effect size and standard error (which occurs, for example, when using odds ratios), or differences in methodological quality. An asymmetric funnel plot is a warning sign, not a definitive diagnosis.
Another practical limitation is that the funnel plot loses utility when there are few studies. With fewer than ten studies, the visual assessment of symmetry is essentially subjective and unreliable. For meta-analyses with a small number of studies, it is preferable to use formal tests such as Egger's test (regression of the effect size on its standard error) or Begg's test (rank correlation), although these tests also have low statistical power with small study samples.
Implementation in R
In R, the funnel() function from the metafor package generates high-quality funnel plots with a single line of code. Starting from an rma object (the result of the meta-analytic model), simply running funnel(model) produces the basic plot. The function allows customization of the vertical axis type (standard error, inverse variance, sample size), colors, labels, and the inclusion of the confidence interval around the combined effect. For trim-and-fill analysis, the trimfill() function from the same package applies the method and can be visualized directly on the funnel plot, displaying imputed studies as hollow points differentiated from the real studies.
A complete workflow would include estimating the meta-analytic model with rma(), generating the funnel plot with funnel(), applying Egger's test with regtest(), and running trim-and-fill with trimfill(). This set of analyses provides a robust assessment of publication bias that goes beyond visual inspection and that any reviewer would expect to find in a well-prepared meta-analysis manuscript.
Before you submit: the publication bias section is one of the most commented on in the first review round, especially when the funnel plot shows up on its own, with no formal test and no sensitivity analysis alongside it. If your manuscript is already written, run it through the AI Paper Reviewer: a free Reviewer 2 style pre-review that tells you what they will push back on while you can still fix it, instead of three months later in the editor's letter.