Reviewer 2: Where the Meme Comes From, What the Data Say and How to Answer One

Everyone who has submitted a paper has met them. Reviewer 1 suggests a few clarifications and wishes you luck. Reviewer 2 says the contribution is not novel, the sample is too small, the analysis should have been done in a completely different way, and, while you are at it, you should cite four papers that happen to share an author. The joke spread because almost every researcher has a story that fits it.

This article is about what to do with that story. Where the meme comes from, what the evidence actually says about Reviewer 2, the six comments you will get from one and how to answer each, when to involve the editor, and how to meet Reviewer 2 before the journal does.

What "Reviewer 2" means

Reviewer 2 is not a person, it is a type. The type is easy to recognise:

• wants you to have done a different study from the one you did;
• declares the work "not novel" without citing what it repeats;
• asks for analyses that have nothing to do with your question;
• asks you to cite papers whose main merit is their authorship;
• criticises something you did not write;
• writes in a tone that would get them removed from any meeting room;
• and contradicts Reviewer 1 on the one point where you had finally agreed with yourself.

Why the number 2? There is nothing special about it. The number on your decision letter usually reflects nothing more than the order in which reviewers were invited or sent their reports. With two reviewers per paper as the common minimum, "the other one" became Reviewer 2, and the name stuck.

Is Reviewer 2 really the worst? What the data say

Someone checked. David Peterson (2020) used the reviewer database of the journal Political Behavior to test whether Reviewer 2 is more negative about manuscripts than the other reviewers, or more out of line with them. He found no evidence of either. What he did find is that Reviewer 3 was more likely to rate a paper more than one category below the other reviewers. His conclusion, tongue firmly in cheek: Reviewer 3 behaves so badly that Reviewer 2 takes the blame. It is one journal in one field, but it is the only direct test of the meme, and the meme lost.

Two other findings explain why the meme survives anyway.

Reviewers disagree a lot, as a rule. A meta-analysis of 48 studies covering about 19,000 manuscripts found that agreement between reviewers of the same paper is low: a mean intraclass correlation of .34 and a mean Cohen's kappa of .17 (Bornmann et al., 2010). One reviewer liking your paper and the other disliking it is not bad luck; it is close to the normal state of peer review. Whoever happens to be listed second takes the blame.

Unprofessional reviews are not imaginary. In an online survey of 1,106 researchers in science, technology, engineering and mathematics from 46 countries, 58% said they had received an unprofessional review, and 70% of those had received more than one (Silbiger & Stubler, 2019). People from groups underrepresented in STEM were the most likely to report that the experience damaged their sense of their own ability, their productivity and their career. The survey was voluntary, so the exact percentage is probably inflated, but the problem is real.

So: the number is noise, the behaviour exists. Answer the behaviour, whichever number it comes with.

The six Reviewer 2 comments, and what to do with each

1. "The contribution is not novel"

Usually it means one of two things: your introduction never stated the gap in a sentence the reviewer could quote back, or the reviewer knows a paper you missed. Name the gap explicitly, cite the closest studies and say exactly what you add (a population, a design, a measure, a longer follow-up). If the reviewer named a paper, cite it and spell out the difference. Do not answer "to our knowledge, this is the first study", which is the sentence that provoked the comment.

2. "The sample is too small"

This one is often a request for a justification you did not give. Report the a priori power analysis if you had one; if you did not, a sensitivity power analysis tells the reviewer the smallest effect your sample could detect. Show confidence intervals so the reader sees the precision of each estimate. Do not promise to collect more data unless you can, and do not call an underpowered null result "evidence of no effect". The guide on what to do when the reviewer asks for a larger sample has the full set of options.

3. "Why didn't you use method X?"

Sometimes the reviewer is right, sometimes it is a preference. If running method X is feasible, run it as a sensitivity analysis and report both: if the conclusions hold, the objection becomes a strength. If X is not appropriate for your data, explain why, with a methodological reference. If it is a method you do not know well, read what to do when a reviewer asks for an analysis you have not mastered before you say yes, and how to reanalyse your data for a reviewer once you do.

4. "The authors should cite the following papers"

If the papers are relevant, cite them and thank the reviewer: it costs you nothing. If they are obviously the reviewer's own work and have nothing to do with your paper, you are not obliged to add them. The COPE guidelines for peer reviewers ask reviewers to refrain from suggesting citations to their own or an associate's work merely to increase citation counts, and say that suggestions must be based on valid academic or technological reasons (COPE Council, 2017). Explain calmly in your reply why the papers fall outside your scope and, if the pattern is blatant, mention it to the editor in the confidential comments.

5. The misreading

The reviewer criticises an analysis you did not run or a claim you did not make. Before you get angry, assume the text was unclear, because it often was. Quote the passage, rewrite it so it cannot be misread again, and point to the page and lines. Never write "the reviewer has not read the paper carefully": the editor will read it as an attack, and the reviewer will read it too.

6. The tone

Sarcasm, condescension, remarks about your English or your competence. Separate the tone from the content and answer only the content, as if the comment had been written politely. If a comment crosses into personal or derogatory remarks, tell the editor: COPE asks reviewers to be professional and to refrain from being hostile or inflammatory and from making derogatory personal comments (COPE Council, 2017). An editor can set aside a review that breaks those rules, but only if someone tells them.

How to answer Reviewer 2 without losing the editor

Your real audience is the editor, who will decide on the paper and who also has to deal with Reviewer 2. A reply that is calm, complete and easy to check makes you the reasonable party in the room. And a harsh Reviewer 2 often arrives inside a major revision, which is better news than it feels: here is what a major revision means and your odds of acceptance. The rules for the reply are few:

• Answer every point. Quote the comment, reply, and show the change with page and line numbers.
• Concede what is fair. Agreeing quickly with the good points buys credibility for the points you contest.
• Disagree with evidence. A reference, an analysis or a methodological argument; never an adjective.
• Thank briefly, do not grovel. One line of thanks per reviewer is enough.
• No irony, ever. Write every sentence as if the reviewer were sitting next to the editor while they read it.

A pattern for polite disagreement that works with almost any reviewer:

We thank the reviewer for this suggestion. We considered a mixed-effects model but retained the ANCOVA because each participant contributes a single post-treatment measurement, so there is no within-person clustering to model. To show that the conclusions do not depend on this choice, we now report the mixed-effects model as a sensitivity analysis (Supplementary Table S3); the treatment effect is virtually unchanged (page 12, lines 240–246).

When Reviewer 1 and Reviewer 2 contradict each other, follow the editor's letter if it takes a side. If it does not, choose, explain your choice in both replies, and refer to the other reviewer's comment so the editor sees you handled the conflict rather than ignoring one of them. If the contradiction is about something fundamental, ask the editor before you rewrite the paper. Noble (2017) collects ten practical rules for this letter, and the guide on how to write a response to reviewers walks through its structure with examples.

Meet Reviewer 2 before the journal does

In health and behavioural research, most Reviewer 2 comments are about method and statistics, and most of them are predictable: no justification for the sample size, effect sizes missing, assumptions never checked, causal language in a cross-sectional design, an item of the reporting guideline that is nowhere in the text. They are hard to see in your own paper because you know what you meant. The cheapest insurance is a colleague willing to read the manuscript as a hostile reviewer would, and to write down everything they would object to.

If you do not have that colleague this week, Previum (AI peer review) reads your manuscript the way two journal reviewers would and returns their objections before you submit, while you can still fix them. You can see what a full report looks like in the annotated peer review report example. And if the objections are already on your desk and they are statistical, I can help you answer them: that is what the statistical support for your reply to reviewers is for.

And if you are the one reviewing

Sooner or later you will be Reviewer 2 for someone else. Review the paper they wrote, not the one you would have written. Separate what is essential for the conclusions to hold from what is merely a suggestion, and label them that way. Suggest a citation only when the paper genuinely needs it. Write every comment so the authors can act on it. And read your report once more before you send it, imagining it arriving in your own inbox on a Friday evening.

Frequently asked questions

What does "Reviewer 2" mean?

It is academic slang for the harsh, dismissive or off-target peer reviewer: the one who wants a different study, calls the work not novel, asks for unrelated analyses or for citations to their own papers. The number itself means nothing; it usually reflects the order in which reviewers were invited or replied.

Is Reviewer 2 always the harshest?

No. The only published test of the meme, using the reviewer database of the journal Political Behavior, found no evidence that Reviewer 2 is more negative or more out of line than the other reviewers. Reviewer 3 was the one more likely to rate a paper well below the others (Peterson, 2020).

Should I tell the editor a review is unfair?

Only when it crosses a line: personal or derogatory remarks, or requests to cite irrelevant papers by the reviewer. Do it calmly in the confidential comments to the editor. For ordinary disagreements, answer the content in your reply and let the editor judge.

Do I have to cite the papers a reviewer suggests?

Cite them if they are relevant. If they are not, you can decline and explain why. COPE's guidelines for peer reviewers say citation suggestions must be based on valid academic reasons, not on increasing the reviewer's citation count.

What if Reviewer 1 and Reviewer 2 contradict each other?

Follow the editor's letter if it takes a side. If it does not, choose one option, justify it in both replies and mention the other reviewer's comment, so the editor can see you handled the conflict. If the disagreement is fundamental, ask the editor before rewriting.

References

Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). A reliability-generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS ONE, 5(12), Article e14331. https://doi.org/10.1371/journal.pone.0014331

COPE Council. (2017). Ethical guidelines for peer reviewers (Version 2). Committee on Publication Ethics. https://doi.org/10.24318/cope.2019.1.9

Noble, W. S. (2017). Ten simple rules for writing a response to reviewers. PLOS Computational Biology, 13(10), Article e1005730. https://doi.org/10.1371/journal.pcbi.1005730

Peterson, D. A. M. (2020). Dear Reviewer 2: Go f' yourself. Social Science Quarterly, 101(4), 1648–1652. https://doi.org/10.1111/ssqu.12824

Silbiger, N. J., & Stubler, A. D. (2019). Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. PeerJ, 7, Article e8247. https://doi.org/10.7717/peerj.8247