😳 Holy shit… this paper reveals why AI invents fake citations, sections, and page numbers out of thin air. LLMs aren’t “getting creative.” They’re structurally incentivized to manufacture details whenever they hit a knowledge gap and the paper breaks down exactly how the failure happens. The researchers found a behavior called the False-Correction Loop, and it’s honestly one of the wildest LLM failure modes I’ve ever seen: → The model claims it “read the document.” → It cites page 12, page 24, Section 4, Theorem 2 none of which exist. → You point it out. → It apologizes. → Then confidently fabricates new fake pages, fake DOIs, fake figures… → You point it out again. → It apologizes again. → Rinse. Repeat. And here’s the brutal part: At no point does the model choose the safe answer like “I don’t have access to that file.” The paper explains why: The reward structure values: ✔ sounding coherent ✔ staying engaged over ✘ being factually correct ✘ admitting uncertainty So the model does the only thing its incentives push it toward:...