How to Check That a Reference Actually Exists

Someone in your cohort cites a study that does not exist. There was no bad faith: a model handed them the reference, it carried two surnames that sound right for the field, a journal that publishes exactly that, and a DOI that looked like a DOI, and nobody opened it. The thread appears on r/GradSchool most weeks with the same shape — one person finds it by accident, everyone else starts wondering how many are sitting in their own list — and it ends the same way every time, because the reviewer does open it. Checking whether a reference exists takes about two minutes, and a whole bibliography can be run in one pass; what does not work is looking at an entry and deciding whether it feels right. The other ways a reference list goes wrong — the shifted year, the preprint cited as published, the retracted paper still being cited — are in how to check your reference list before you submit. This piece is only about whether the work exists at all.

A model is not lying. It is reconstructing from memory

A language model does not hold a bibliographic database it can look things up in. It holds the SHAPE of a citation, learned from having seen millions of them, and when you ask for three studies on a topic it produces the most probable continuation: a surname that really does publish in that literature, a year consistent with when the topic took off, a title assembled from the exact vocabulary of the field, a journal that really does publish that work, and a DOI carrying the right publisher prefix (10.1016 is Elsevier, 10.1037 is the APA). Every component is real. The combination is not.

That produces a consequence which runs against instinct: the fabricated reference looks better than many of the genuine ones. The awkward real entry — a 1994 chapter in an edited volume, an unpublished thesis with no DOI, a government technical report — looks suspicious and is impeccable. The fabricated one comes out clean, recent, with an identifier and flawless APA formatting. Screening by what sounds serious screens in precisely the wrong direction.

The dangerous case is not the wholly invented entry, which sometimes gives itself away. It is the half-true one. In a manuscript I reviewed in August 2026, two entries carried the correct title and the correct DOI while three of every four co-authors had never signed that paper: they had been assembled from the article's record rather than copied from the source. Reading cannot catch it, because on the page everything fits. The reviewer who clicks the DOI catches it in two seconds, and from that point reads the rest of your manuscript with different eyes. The rule I took from it is literal: never compose a reference from an abstract, a database record or anybody's memory, including your own.

No prompt fixes this, and neither does telling the model to cite only real work. Fabrication is not a failure of obedience, it is what the machine does when you ask for a fact it does not have in front of it; what you can do is stop letting it have the final word on what exists. Where a model genuinely helps and where it does not, in ChatGPT and AI for your thesis statistics.

Reading the answer: four verdicts, and only one means fabricated

Confirmed. At least two independent databases recognize it and agree on title and authorship. This is the only verdict you can state flatly. One database is not enough, and not out of paranoia: catalogs copy records from each other, so an entry with a typo propagates exactly as fast as a clean one.

Single source. It exists, but without consensus. This is common with very recent articles, with small journals deposited by a single aggregator, and with conference proceedings. It is not an alarm. It is a prompt to open the link yourself.

No trace. No DOI declared and nothing found anywhere. This is the NORMAL outcome for books, book chapters, theses, technical reports, proceedings and local journals, which make up a large slice of most reference lists outside the anglophone core. Treating "I cannot find it" as "you made it up" is worse than saying nothing: it sends you to spend an hour defending an impeccable entry, and when a tool does it, the false accusation destroys the credibility of everything else the tool tells you.

A DOI nobody recognizes, or a DOI pointing somewhere else. Here there is a real signal. If the entry declares a DOI and no database has it registered, that is the one case where silence is informative, because a DOI exists in order to be in a registry. One caveat before you accuse anyone: not every DOI is issued by Crossref. Theses, datasets and institutional repositories usually carry DataCite DOIs, so the test that settles it is whether doi.org resolves the identifier; if it resolves and shows your work, the work exists even though Crossref never heard of it. If the DOI resolves but returns a different paper, you have a hard, verifiable finding: it happens when a reference manager drags identifiers out of alignment after the list is reordered, and it is the failure a reviewer spots before you do. One operational caveat: when a database times out or rate-limits you — Semantic Scholar returns a 429 readily — that counts as unavailable, never as disagreement.

Checking one reference by hand, in two minutes

If it has a DOI, start there. Paste https://doi.org/10.xxxx/yyyy into the browser and read the page that opens: the title and the author list have to be the ones in your entry. That the link opens something proves nothing about your reference, only that the identifier is registered. This is the step most people do halfway, and it is the one that catches the most expensive error.

If there is no DOI, search the full title, in quotes, in two places. Crossref (search.crossref.org) covers nearly everything a publisher has deposited with a DOI. OpenAlex (openalex.org) is the broadest free catalog and picks up a good deal of what Crossref misses. If both return the same work with the same authors, the reference exists and you can move on.

If it is health research, add PubMed. Search article title[Title] at pubmed.ncbi.nlm.nih.gov. It covers biomedicine and clinical psychology very well and covers nothing else, so a social psychology paper missing from PubMed tells you precisely nothing.

If the journal is Spanish-language or Latin American, add DOAJ. The Directory of Open Access Journals (doaj.org) indexes a large part of the Ibero-American catalog that anglophone databases handle badly. For a book or a chapter, do not expect any of the five to have it: the right places are a library catalog or OpenLibrary, and for physics, math and computing, arXiv.

Which database covers what, and why so much of a list comes back not indexed

Crossref is not an index of the literature. It is the registry of DOIs that publishers deposit, so anything a publisher never deposited is absent no matter how good the journal is. OpenAlex does try to be a general index and covers the widest ground. PubMed sees biomedicine only. Semantic Scholar holds an enormous corpus and throttles you the moment you push. DOAJ sees open access only. None of the five is a catalog of knowledge; they are five different crops with five different holes, which is exactly why the rule is two agreeing sources rather than trust in one.

The holes are measured, not assumed. When I built the verification step for Previum I tested twelve genuine references from Spanish and Latin American journals: without DOAJ in the set, only three reached two agreeing sources, meaning a Spanish-speaking author would have seen three of every four clean entries flagged. In a separate real report on a Peruvian macroeconomics paper, eight of forty references came back as not indexed, and all eight were books, chapters, theses and Spanish-language journals. Nothing was wrong with any of them.

SciELO, the one that would help most for Latin America, currently has no usable public endpoint: its search interface answers with an anti-bot challenge and its other interface only serves records by internal identifier, which is no use for locating a reference. Much of SciELO arrives anyway through Crossref, because it deposits DOIs, and through DOAJ. The point generalizes: a gap in a database is a fact about the database, not about the reference.

There is one case where absence does say something, though not about whether the article exists. If nobody indexes the journal, it is not in DOAJ, it deposits no DOIs and its website promises peer review in 72 hours, the question is no longer whether the work exists but what you are citing.

Checking the whole list in one pass

Sixty references by hand is an afternoon nobody has. All eight databases (Crossref, OpenAlex, PubMed, Semantic Scholar, DOAJ, OpenLibrary, Google Books and arXiv) have public APIs with no key required, so fifteen lines of script will walk the list. With a DOI the query is direct: https://api.crossref.org/works/10.xxxx/yyyy returns JSON with title, authors and year. Without one, search by text: https://api.crossref.org/works?query.bibliographic=TITLE+AUTHORS&rows=3, and if that finds nothing, the same query using query.title, which behaves better with long titles.

The other four are just as easy to ask. OpenAlex: https://api.openalex.org/works?search=TITLE, or https://api.openalex.org/works/doi:10.xxxx/yyyy when you have the identifier. PubMed takes two steps, esearch.fcgi with term[Title] and then esummary.fcgi with the identifiers it returns. Semantic Scholar has an endpoint built for exactly this job, /graph/v1/paper/search/match, which matches one title against the corpus instead of handing you a list of near misses. DOAJ takes https://doaj.org/api/search/articles/bibjson.title:"TITLE". Any of those addresses can be pasted straight into a browser without writing a line of code.

Four details that save you false positives, all four learned the hard way. Put your email in Crossref's mailto= parameter, which puts you in their polite pool and answers far more reliably. Wait about a second between calls or you will be throttled halfway down the list. Compare title and authorship rather than the year, because Crossref reports the online-first year while APA wants the issue year, so a one-year gap is almost never an error. And match titles on runs of consecutive words rather than exact equality, because the record usually carries a subtitle your citation omits, and bilingual journal names (Canadian Psychology / Psychologie canadienne) are not a discrepancy.

If you would rather not write the script, rcrossref in R and habanero in Python make the calls for you, and there is a low-tech shortcut that works surprisingly well: add each reference to Zotero by DOI. Zotero asks Crossref and fills in the metadata, so an entry that retrieves nothing is your signal, and the ones that do retrieve leave you with the correct details instead of whatever the draft contained.

The day my own Professional report invented 58 references

In August 2026 I was generating the public example reports for Previum on a real open-access paper when I found that the Professional report was auditing bibliographies nobody had sent it. The flaw was mine and it was structural: the component that locates the reference list returned the whole document whenever it fitted inside its character limit — that is, for any normal paper — and a model was then asked to transcribe its reference list. When the manuscript carried no bibliography, the model sometimes answered that there was none and sometimes rebuilt one from memory.

Measured on the same stripped paper across four languages, Spanish and Portuguese behaved; the English run invented 58 references and the French run 55. Those fabrications were then delivered to the customer as a serious warning about their bibliography, with 57 of them additionally flagged as not indexed. A paid report telling a researcher their reference list has problems, when that list never existed, is not the kind of error a footnote can cover.

The fix was one conceptual line: a reference only enters the audit if its title appears as a contiguous phrase in the document. Short titles must match in full; for long ones, a window of six consecutive words has to appear verbatim in the text. The principle underneath is that transcribing means copying, not recalling, and an invented phrase almost never matches the paper word for word. The first version of the guard measured what share of the title's words appeared anywhere in the text, and it was useless: in a paper about medication satisfaction, words like patient, questionnaire and factor are everywhere, so 9 of 23 fabricated entries walked through. Requiring six consecutive words, none did. The following day I closed the second leak: a separate screening field bypassed the guard entirely and, in the English report, was still declaring those 58 references where there were none.

The lesson applies to whatever pipeline you use, not just to mine. When a tool hands you an analysis of your bibliography, the useful question is not how many references it checked but what it does when there is nothing to transcribe: if it still returns a list, it is not reading your document. What a tool like that should actually be checking, in what an AI paper reviewer flags.

What to do with the one that does not exist

Do not try to repair the reference. The problem is not the bibliography entry, it is the sentence in the manuscript that leaned on it: a claim was made and the evidence is missing. Go back to the claim and decide whether a real study supports it — search by the content, not by the phantom title — or whether the sentence has to go. Hunting for a substitute that says roughly the same thing, purely to fill the slot, repeats the original mistake with extra steps.

If the fabricated entry came from a model, check every reference from that same session rather than just this one. Fabrications rarely arrive alone: they show up in the answer where you asked for literature the model knew thinly, and they tend to come in clusters. Check the in-text citations too, because a fabricated reference drags a fabricated claim behind it — the sample size, the effect or the finding you attributed to that paper does not exist either.

If the manuscript is already under review, write to the editor yourself, before a reviewer gets there. A voluntary correction three days later is an administrative note; the same entry found in review becomes a research integrity question, and the editor stops reading the paper and starts reading you. Where the line sits between a manuscript that can be fixed and one that gets desk-rejected, in why Q1 journals reject your paper.

Frequently asked questions

How do I check whether a paper actually exists?

Search the full title in two independent databases and confirm they agree on title and authorship, not just on the year. Crossref and OpenAlex are the default pair; add PubMed for health research and DOAJ for Spanish-language or Latin American journals. If the entry has a DOI, open it at https://doi.org/ and read the landing page, which must show your title and your authors rather than a similar paper. Two agreeing sources and the work exists.

Why do AI-generated references look so convincing?

Because they are not invented from nothing, they are reconstructed. The model has seen millions of citations and learned their shape, so it assembles one from authors who really do publish in the field, a journal that really does publish the topic, a plausible year and a DOI with the publisher's correct prefix. Every component is real and only the combination is false, which is what makes the "I check the ones that look odd" strategy useless. The entries you need to check are not the odd-looking ones, they are the ones you have not looked up.

What does it mean when a reference appears in no database at all?

Almost always that it is not indexed, not that it is fake. Books, book chapters, doctoral theses, technical reports, conference proceedings and local journals without DOIs sit outside Crossref, OpenAlex, PubMed and DOAJ by design, and they make up a substantial share of most reference lists. The different case is a reference that declares a DOI no database recognizes: there the silence is informative, because a DOI exists in order to be registered.

Can I check an entire bibliography at once?

Yes, and you do not need to be a programmer. All eight databases expose public APIs with no key, so a short script walks the list and queries them one entry at a time: Crossref by DOI or title, OpenAlex by search, PubMed in two steps, Semantic Scholar through its title-matching endpoint, DOAJ by title, OpenLibrary by title and ISBN for books, and arXiv by title for preprints. R has rcrossref and Python has habanero. Leave about a second between calls and put your email in Crossref's mailto parameter to join their polite pool.

Can I just ask ChatGPT to verify its own references?

It does not work, because the check would be run by the same memory that produced the reference. A model with no real access to the databases will tell you the entry exists with exactly the confidence it used to write it, and if you press, it will happily confirm something it fabricated a moment earlier. It only counts when the tool genuinely queries the catalogs and shows you what each one returned; a verdict that arrives without the source behind it is not a verification.

What should I do if I have already submitted with a fabricated reference?

Write to the editor before a reviewer finds it, listing what you have checked and what the correction is. A note from you within days is a piece of admin; the same entry surfaced in review opens an integrity question that is no longer about the paper but about you. Use the occasion to check the rest of the list, because fabricated references usually travel in groups, and so do the claims that rested on them.

A fabricated reference is never caught by reading, only by looking it up, and the criterion stays the same throughout: two independent databases agreeing on title and authorship, and no accusation where all you have is an indexing gap. If what you are facing is no longer a doubtful entry but a reviewer who has questioned your bibliography and asked you to redo the analyses as well, that is what my statistical consulting is for.