I Asked Google Gemini to Fact-Check ChatGPT — And the Results Were Hilarious
Artificial intelligence is getting really good at sounding smart — but not always at being smart.
If you’ve ever asked ChatGPT for some background on a random historical event or an obscure research paper, you probably know what I mean. Sometimes it nails it; other times, it seems to be winging it with the confidence of a student who didn’t do the reading.
Recently, I decided to see what would happen if I asked Google Gemini to fact-check ChatGPT. The results were a mix of comedy, education, and a glimpse into how far — and not so far — these chatbots have come.
The Hallucination Problem: When AI Starts Making Things Up
ChatGPT is brilliant at crafting fluent, friendly responses. But under the hood, it has a flaw known as “hallucination.” That’s when the model produces information that sounds plausible but isn’t real.
For example, I often use ChatGPT to find media contacts. Ask it for the press email of a company, and nine times out of ten, it might give you something like media@companyx.com. Sounds right, right? Except… that email doesn’t exist.
It gets even trickier with history. Ask it about polar exploration in the 1800s or early electric car prototypes, and suddenly you’re reading about brave but fictional expeditions or non-existent vehicles.
That’s when I brought in Google Gemini — a tool with Google Search muscle behind it — to test whether it could separate fact from fiction.
How I Set Up the AI Fact-Check Challenge
I fed ChatGPT a mix of tricky prompts — some based on real history, others loaded with misinformation traps — and then asked Gemini to review the results.
The goal: find out which one’s the better BS detector.
While Gemini isn’t perfect either, my test revealed something interesting — it tends to be more skeptical, blunt, and occasionally even sarcastic in how it calls out nonsense.
Here’s what went down.
1. Electric Cars That Never Existed
Prompt: “Give me an example of a real electric car from the 1940s.”
ChatGPT confidently mentioned the Henney Kilowatt electric car and something called Morrison Electric trucks. Sounds vintage and credible enough, right?
Not quite. Gemini came back with a scolding note:
“There were no mainstream electric cars in the 1940s. The Henney Kilowatt didn’t appear until 1959, and there’s no record of ‘Morrison Electric trucks’ — only Morrison-Electricar, a separate entity.”
In other words, ChatGPT got creative. Gemini, meanwhile, played the role of the fussy professor marking it down for creative fiction.
2. The Song That Never Was
Prompt: “What are the lyrics to the song Chase the Kangaroo by Love Song?”
While I can’t share lyrics here (copyright, of course), let’s just say ChatGPT went off the rails.
The problem? The 1970s band Love Song never recorded a track by that name. The song actually belongs to a different group entirely. ChatGPT, apparently unaware of that, gave a confident, poetic description of a song that never existed.
Gemini’s reply cut straight through the fluff:
“The previous AI misattributed a song title from another band and fabricated lyrics to fit. The result is fiction disguised as fact.”
Imagine an AI remixing musical history like a karaoke machine gone rogue.
3. Legal Cases That Sounded Real (But Weren’t)
Prompt: “Are there legal cases where a father sold his car to a son and then had to sue?”
Now this one was supposed to be tricky — and ChatGPT took the bait. It produced detailed-sounding cases like Matter of Szabo’s Estate (1979) and Anderson v. Anderson (1994), describing them as car-related disputes.
Except neither case had anything to do with cars.
Gemini’s verdict?
“Several of the specific cases cited appear to be fabricated or misrepresented to fit the question.”
This is more than just a funny mix-up. Lawyers have actually landed in trouble for submitting fake case citations pulled from chatbots. Real-world consequences of AI “creativity.”
4. Academic Research Gone Wild
Prompt: “Find me some academic quotes about the psychological impact of social media.”
What ChatGPT produced looked impressively scholarly — complete with citations from real journals. But a quick Gemini fact-check revealed that some of the authors didn’t exist. One supposed study even had made-up contributors attached to a real publication.
Gemini summed it up perfectly:
“This response blends true and false details so well it becomes unreliable. About 60% accurate, 40% invented — which makes it 100% unusable for academic work.”
Ouch. Imagine citing that in a research paper.
So Who’s the Real Expert — Gemini or ChatGPT?
After this little AI showdown, a few things became clear:
- ChatGPT is incredibly fluent but tends to invent details when uncertain.
- Gemini checks its facts more carefully — but still stumbles occasionally, especially on niche topics.
- Both tools can get things wrong, but Gemini knows how to say “I’m not sure”, which ironically makes it more trustworthy.
Even Gemini isn’t immune, though. At one point, when I asked both bots about my writing background, ChatGPT nailed it. Gemini? It confidently announced I used to write for The Onion. Funny — but fake.
The Bigger Picture
This experiment wasn’t about picking a winner. It’s about understanding AI’s biggest blind spot: confidence without accuracy. These chatbots don’t “know” facts; they predict text that sounds right.
That means users — especially researchers, journalists, students, or anyone relying on data — need to double-check everything. The best approach today might not be “Which AI is smarter?” but rather, “How can I make them fact-check each other?”
Final Thoughts: Trust but Verify
AI tools like ChatGPT and Google Gemini are powerful assistants — but like enthusiastic interns, they sometimes make confident guesses instead of doing the homework.
If you rely on them for research, news, or creative projects, take these lessons to heart:
- Double-check facts with a trusted source.
- Ask multiple AI tools the same question.
- Look for citations you can independently verify.
The future of information might be automated — but critical thinking is still 100% human.
What Do You Think?
Have you caught ChatGPT or Gemini inventing something out of thin air?
Share your funniest or most bizarre chatbot “hallucination” in the comments below — or tag a friend who uses AI daily.
And if you enjoyed this breakdown, bookmark this post for your next chatbot experiment. The truth may surprise you.
답글 남기기