Because the model is predicting plausible next words, not looking anything up. Canadian courts now have a name for it: a hallucination.
Short answer
Because the model is predicting plausible-sounding next words, not looking anything up — when it doesn't actually know a real source, it can generate one that reads exactly like a genuine citation, complete with a title, an author and a page number, none of which exist. Canadian courts have already given this a name: a “hallucination.”
The Federal Court defines the term in its own notice to litigants: “‘Hallucination’ is a term used to refer to facts, citations, and other content generated by AI that are not true, and have been fabricated by AI in response to a prompt or request.” (Federal Court, Notice to the Parties and the Profession, May 7, 2024) Ontario's Superior Court of Justice describes the same failure the same way: “Hallucinations can consist of non-existent cases, mischaracterizations of case law, and fabricated quotations.” (Ontario Superior Court of Justice, Consolidated Civil Provincial Practice Direction)
A language model generates text one probable word at a time, based on patterns in what it was trained on — it has no built-in step that looks up a real database and checks whether a source exists. Asked for a citation, it produces the shape of one, because that shape was extremely common in training data, whether or not the specific source is real. That's the same underlying gap behind why AI doesn't reliably know today's date and why it tends to agree with a confident-sounding premise — nothing forces the model to verify anything before it answers.
The Federal Court requires a formal Declaration when AI-generated content that resembles co-authorship appears in a filing, and under its own “human in the loop” heading states plainly: “The Court urges verification of any AI-created content in these documents.” (Federal Court, Notice to the Parties and the Profession, May 7, 2024) Ontario's practice direction is blunter still about who bears the risk: the court will not tolerate inadvertence, “regardless of whether they directly interacted with the technology.”
See how a reviewer verifies AI-assisted work before anyone signs off on it.
