Last updated: October 2026
If you use ChatGPT or Gemini for anything serious, sooner or later you ask for sources. And sooner or later you wonder the same thing I did: are those references real, or does the AI just make them up?
I did not want to guess, so I tested it. I asked Claude, the AI assistant that helps me with this website, to run the test for me, and then check every reference one by one. It found something I did not expect, and I think it is worth knowing before you trust an AI with your sources.
The Short Version
- On a well-studied topic, both ChatGPT and Gemini gave me real references. All 10 checked out.
- On a narrow topic, ChatGPT still did well: 5 out of 5 were real.
- Gemini was a different story. Its answer had two lists. The first one was mostly real. The second one had no reference that I could verify, and one even said “Example record” next to the DOI.
- The lesson: check every source, especially when the topic is narrow.
How the Test Was Done
I wanted a fair and simple test that anyone could repeat. This is what we did:
- We used the free web versions of ChatGPT (without logging in) and Gemini (in a temporary chat).
- We asked both the exact same question, and we kept the first answer, with no retries.
- We asked for 5 academic references, with authors, year, title, journal and DOI.
- Then Claude checked every DOI in Crossref, a public database of scientific publications. It compared the authors, title, journal and pages, and it searched the title in case the DOI was wrong.
I did two tests: one on a well-known topic and one on a very narrow one. It is a small test, so read it as a snapshot and not as a ranking. Also, both answers showed source labels, which suggests the tools may have searched the web while answering.
Test 1: A Well-Known Topic
The question: “Give me 5 academic references about the effect of remote work on employee productivity. For each one include the authors, year, title, journal and DOI.”
| Tool | References | Real and correct |
|---|---|---|
| ChatGPT | 5 | 5 of 5 |
| Gemini | 5 | 5 of 5 |
Both tools gave real papers, and several were the same, such as the well-known experiment on working from home in a Chinese company and a classic review of telecommuting. Every DOI existed, and the authors, titles, journals and pages matched. The only differences were small, like the year of the first online publication.
For a popular topic, with plenty of research, the AI did what I hoped it would do.
Test 2: A Very Narrow Topic
The question: “Give me 5 academic references about how remote work affects the productivity of public-sector employees in Spain. For each one include the authors, year, title, journal and DOI.”
This is the kind of question where there may not be five perfect papers, so I wanted to see what would happen.
ChatGPT
All 5 references existed and matched. ChatGPT also warned me, on its own, that not all of them measure productivity directly and that the list should be treated as a starting point. One detail: for one Spanish paper, it gave the title in English translation instead of the original Spanish title. The paper is real, but a title like that would be hard to find if you copied it.
Gemini
Gemini’s answer was strange. It gave me one list of five references, and the fifth was cut off in the middle of a DOI. Right after that, it started a second list of five different references. So I checked both lists.
| List | What I found |
|---|---|
| First list (5) | 4 were real and matched. The fifth was a real paper, but the DOI was incomplete, and the real title says the study is about Ecuador, not Spain. |
| Second list (5) | None could be verified. Three DOIs did not exist. Two DOIs existed but belonged to completely different papers, one about age and austerity policies and one about cryptocurrencies. |
One entry in the second list even had a note next to the DOI saying “Example record”. In other words, the tool itself was telling me that the reference was a placeholder. I have not named the authors it attributed those references to, because I could not find those papers, and I do not want to put names to something that may not exist.
What I Noticed
- With a famous topic, the sources were reliable. That is the case most people try, and it can give a false sense of security.
- With a narrow topic, the risk went up a lot, and it went up for one of the two tools in our test.
- The fake references looked real. They had believable authors, titles that sounded right, and DOIs with the correct format. You cannot tell by looking.
- A real DOI does not mean a correct reference. Two DOIs in the second list worked, but they opened a different paper.
- Some answers hide small mistakes. One was a real study about another country, and one was a translated title.
Why Does This Happen?
As I understand it, these tools write text that sounds right based on patterns they have learned. When they do not find a perfect match, they can produce something that fits the pattern, even if it does not exist. I am not an expert on how they work, so take this as a simple explanation and not as a technical one. What matters is the result: the tool does not warn you when it is inventing something.
How to Check a Reference in Two Minutes
- Copy the DOI and open it at doi.org. If it says the DOI cannot be found, be suspicious.
- Check that the page is the same paper. Compare the title, authors and journal with what the AI told you.
- Search the title in Crossref or Google Scholar. A real paper appears.
- Look at the year, volume and pages. Small differences can mean the reference was mixed up.
- If you cannot find it, do not use it. It does not matter how real it looks.
My Advice if You Use AI for Sources
- Never copy a reference without checking it. Every one, every time.
- Ask for fewer, not more. The more references you ask for, the more chances for a mistake.
- Be extra careful with narrow topics. That is where I saw the problems.
- Ask for links, and open them. A tool that can search the web is usually safer than one that answers from memory, but it is not perfect.
- Use the AI as a starting point. It can help you find ideas and keywords. The checking is up to you.
If you want ready-made examples of how to ask for what you need, see our guide to the best ChatGPT prompts. For another comparison of tools for research and sources, read Perplexity vs ChatGPT for research.
Frequently Asked Questions
Does ChatGPT make up sources?
In my test it did not: all 10 of its references were real. But I only tested two questions, and other people have reported different results, so check every source.
Does Gemini make up sources?
In my narrow-topic test, one of its two lists had no reference I could verify. In the well-known topic, all its references were real. It depends on the question, so check them.
How can I tell if a reference is fake?
Open the DOI and compare the paper with what the AI said, then search the title in Crossref or Google Scholar. If you cannot find it, do not use it.
Why is a fake reference hard to spot?
Because it looks real. It has believable authors, a plausible title and a DOI in the right format.
Is it safe to use AI for school or research?
It can help you find ideas, but you are responsible for what you hand in. Check every source and follow the rules of your school or journal.
Which tool is better for finding sources?
I only tested two free tools on two questions, so I cannot say. Tools that search the web and show links are usually easier to check.
My Take
I am glad I ran this test, because it changed how I use AI. Most of the time, the sources were real, and that is good news. But the one time they were not, they looked perfectly real. So my rule now is simple: the AI can give me a first list, but I do not trust a single reference until I have opened it myself.
Keep Reading
- Perplexity vs ChatGPT for Research (2026)
- ChatGPT vs Claude vs Gemini: Which Is Best in 2026?
- How I Choose Which AI to Use for Each Task
About This Test
This article is independent. I have no sponsorship from OpenAI, Google or any other company mentioned, and I do not earn commissions from the tools covered. The test was run by Claude, the AI assistant that helps me with this website, following my instructions, on October 21, 2026, with the free versions of ChatGPT and Gemini and two questions. Results may be different with other questions, versions and dates.
Disclaimer: AI products and features change quickly. This article is informational and not academic or legal advice. Always verify any reference before you use it.

