A literature review written by a language model reads like the work of someone who knows the field. The paragraphs flow, the authors sound plausible, the journals exist. The problem is in the small print: some of the references point to nothing.
That has been measured. William Walters and Esther Wilder asked ChatGPT-3.5 and ChatGPT-4 for short literature reviews on 42 varied topics. They looked up all 636 references in the 84 texts across several databases and websites. Of the GPT-3.5 references, 55 percent were fabricated, and of the GPT-4 references, 18 percent. And of the references that did lead to a real work, 43 percent (GPT-3.5) and 24 percent (GPT-4) contained substantive errors. The authors call GPT-4 a major improvement, and conclude that problems remain.
That study is from 2023 and tested two models. It doesn't tell you what today's models do. But the conclusion for your work is simple: a list of references from a model is a list of candidates, not a list of sources.
Why this goes wrong so easily
A model doesn't look literature up. It writes what a reference statistically ought to look like: an author who publishes in that field, a year in a sensible range, a title that touches the topic, a journal that fits. Sometimes that's an existing article. Sometimes it's a new combination of parts that are all real, except the whole.
That's why it doesn't stand out on a first read. A fabricated reference isn't sloppy, it's convincing. And in a review with forty references, nobody checks forty times, certainly not close to a deadline.
Which phase this is
SEAL describes the four phases knowledge work goes through: Source, Edit, Approve, Launch. In every phase the traceability of a source can be lost, and the weakest phase determines what the work is worth.
This is a Source problem. The question for that phase is: where does the information come from, and does that source actually exist? A claim enters the work with a source AI has invented, and without a recorded source the rest of the chain can't be saved. You can write a fine paragraph, have it proofread and have it approved, but if the source underneath doesn't exist, all the effort afterwards was wasted.
Three steps per reference
Do it per reference, and do it before the review goes to anyone. Not afterwards, because after delivery an invented source is a correction, not a check.
1. Does the work exist? Look the reference up somewhere outside the model: the publisher's website, a library catalogue, Crossref or Google Scholar. Search by title, not by the link the model gives you. A DOI that doesn't resolve is a warning sign, but one that does resolve proves little: it can point to a different article than the title suggests. So compare what you find with what's written.
2. Are the details right? Compare authors, year, journal, volume and page numbers with the source itself. This isn't pedantry. In Walters and Wilder's study, nearly half of the real GPT-3.5 references had a substantive error. Put in the wrong page number and you send the reader, and the next researcher, hunting for a passage that isn't there.
3. Does the source say what the text claims? Open the work and find the passage. An existing article can be about the topic without supporting the claim. This is the step that takes the most time, and the only one that needs someone with subject knowledge. If a reference fails here, it isn't an invented source but a shifted claim. That belongs to the Edit phase, and that error needs a different check.
A log you can hand over with the work
Keep three outcomes per reference, in a table with one row per source:
- No.: 7
- Reference: Jansen & De Wit (2021)
- Found (where, date): Crossref, 12 Oct
- Details correct: No: volume 14 should be 41, corrected
- Passage checked: Yes, p. 112
- No.: 12
- Reference: Made-up example: Bakker (2019)
- Found (where, date): Not found
- Details correct: n/a
- Passage checked: n/a
(The rows are made up, for illustration.)
Three outcomes per row are enough: found, deviating, not found. Only what's found stays in the review. What isn't found comes out, or you find the real source yourself and rebuild the paragraph on it, rather than asking the model for "another source". That just produces the next reference you have to check.
The log is more than housekeeping. Clients who ask how the review came about get a document instead of a promise. And whoever updates the review later can see which references have already been tested.
Better than checking afterwards
The strongest answer is that the model doesn't have to invent references at all. A system that answers only from a collection you've supplied can't name a reference that isn't in it. In TRACES, the six layers on which you assess an AI system, that's called Retrieval. The system should also say when the answer is missing, rather than invent something. You still need step 3, but steps 1 and 2 largely disappear.
If you use a standalone model with no such collection, the list it gives you is a starting point for searching. You fetch the sources yourself, and you copy the reference from the work, not from the answer.
What you can do this week
- Take the latest review where AI was used, finished or still in draft. Pick ten references.
- Look each one up outside the model, by title. Note: found, deviating or not found.
- For the works you find, check that author, year, volume and pages are right.
- Count how many of the ten didn't exist or contained errors. That number is the leak in your own working method at Source, and you can set it next to the study's.
Source
Walters, W.H. & Wilder, E.I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, 14045. doi.org/10.1038/s41598-023-41032-5