RAG vs. ChatGPT: What Source-Based AI Does Differently
ChatGPT and others now search the web and cite sources. Invented case law has become rarer, but errors remain. What changes with an AI built on official texts in their current version and precise citations – and how to recognise a good answer.
Orlando Kahanek · · Updated:
In short: General-purpose AI assistants such as ChatGPT now search the web and link to their sources. Completely invented answers have become rarer for simple legal questions. The assistants are still not error-free, and specialised legal AI makes mistakes too. The difference lies in where the sources come from and how precisely a statement can be checked: a legal corpus of official texts in their current version allows citations down to the article and paragraph.
How it started: invented case law in court
On 22 June 2023, a federal court in New York sanctioned two lawyers and their firm with a USD 5,000 penalty. In a brief, they had cited six court decisions that ChatGPT had invented (Mata v. Avianca, S.D.N.Y.). The case became a cautionary tale for the entire profession. It shows the underlying problem: a language model without access to sources produces text by probability, not by the record.
A team at Stanford University measured how large the problem was at the time. On verifiable questions about decisions of US federal courts, the 2023 models were wrong in 69% (GPT-3.5), 72% (PaLM 2) and 88% (Llama 2) of cases (Dahl et al., Journal of Legal Analysis 2024). These models worked without web search and without sources.
What has changed since
ChatGPT, Claude, Gemini and Perplexity can now search the web and cite their sources with a link. Technically, this is itself a form of retrieval-augmented generation (RAG): the model searches first and then answers on the basis of the pages it found. For simple questions, the basic statements are therefore often correct.
The errors do not disappear, however; they shift:
- The source exists but does not say what is claimed. In a second Stanford study, even specialised legal tools with their own databases were wrong in 17% (Lexis+ AI) to 33% (Westlaw AI-Assisted Research) of cases, often with real but unsuitable references (Magesh et al., Journal of Empirical Legal Studies 2025).
- Outdated versions: web search also finds older guides and superseded statutory texts.
- Fine distinctions: time limits in special cases, void or merely voidable, federal or cantonal law. This is exactly where details decide the outcome.
Both studies concern US law. We are not aware of comparable published measurements for Swiss law.
What RAG is, and what matters
RAG (retrieval-augmented generation) combines two steps:
- Retrieval (source search): before answering, the system searches a corpus for the relevant sources. The search is semantic, meaning it is based on meaning and not just on keywords.
- Generation (answer): the language model receives the sources found and formulates its answer on that basis, with a reference to the citation.
The corpus is what makes the difference. A general-purpose assistant searches the open web. A legal RAG application searches official statutory texts, case law and legal literature.
Example: the question "Can my landlord terminate my lease if I have not paid two months' rent?" leads directly, within the legal corpus, to Art. 257d CO (tenant in arrears), to Art. 266l CO (form of notice) and to the relevant case law.
Web search or legal corpus: the comparison
| Criterion | General-purpose assistant with web search | RAG on a legal corpus |
|---|---|---|
| Sources | Web pages of all kinds: authorities, guides, forums | Official statutory texts, case law, commentaries |
| Version | Depends on the page found, sometimes outdated | Current version of the statutory text |
| Citation | Usually a link to a page | Article, paragraph and letter, with a link to the official source |
| Cantonal law | As far as it is easy to find on the web | The cantons' legal collections in the corpus |
| Errors | Rarer than without web search, but present | Rarer than without sources, but present |
The last row is the most important: no AI is error-free. Checking sources remains essential. A precise citation simply makes checking faster.
How to recognise a good answer
An example from employment law: "Can an employer dismiss an employee with immediate effect during the probation period?"
A good answer separates two things. During the probation period, either party may terminate the employment relationship at any time by giving seven days' notice (Art. 335b para. 1 CO). Termination with immediate effect, however, is permitted during the probation period too only for good cause (Art. 337 CO).
For every AI answer, check:
- Citation: are the article and paragraph given, and does the text really say what is claimed?
- Version: is the cited text still in force?
- Level: federal or cantonal law? For procedures, taxes, building projects and authorities, the canton often decides.
- Legal consequence: void or voidable? With or without a time limit?
- Case law: are decisions cited with a number that can be looked up?
What a legal corpus must deliver in Switzerland
Multilingualism
Switzerland has three official languages, and each language version of a federal act is equally authoritative. A legal corpus can cite the same article in the language of the question:
- DE: Obligationenrecht (OR)
- FR: Code des obligations (CO)
- IT: Codice delle obbligazioni (CO)
Current version
Laws change all the time. A legal corpus works with the wording that applies today rather than with whatever happens to be on a web page.
Federalism
The 26 cantons have their own laws and their own courts. A question on building law in Zurich needs Zurich law, a tax question in Ticino needs Ticino law. A corpus that contains the cantonal legal collections can include both in a targeted way.
Limits and responsibility
RAG-based systems do not replace a lawyer. They speed up research and make sources easier to check. Important:
- Always check sources: the citation is the starting point of the check, not a substitute for it.
- No legal assessment at the push of a button: assessing the individual case remains the task of a qualified professional.
- Gaps are possible: highly specialised areas of law and the most recent decisions may be missing from the corpus.
Conclusion
The question is no longer "AI with or without sources" but: which sources, in which version and how precisely cited. General-purpose assistants with web search have become useful for a first overview. For legal work, what counts are official texts in their current version, citations down to the article and paragraph, and your own check.
Sources
- Mata v. Avianca, Inc., No. 1:22-cv-01461 (S.D.N.Y.), sanctions decision of 22 June 2023: overview (Wikipedia)
- Dahl, Magesh, Suzgun, Ho: Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models, Journal of Legal Analysis 16 (2024): academic.oup.com
- Magesh et al.: Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, Journal of Empirical Legal Studies (2025): onlinelibrary.wiley.com
- Swiss Code of Obligations (SR 220), current version: fedlex.admin.ch
This article was written by the RechtsKI team and revised in October 2026. RechtsKI uses RAG with federal law, the cantonal law of all 26 cantons, case law and commentaries as its source corpus.