RAG vs. ChatGPT: Why Source-Based AI is Superior for Legal Research
ChatGPT hallucinates court decisions – RAG doesn't. Learn why source-based AI technology is superior for legal research in Switzerland. With practical comparison and real BGE references.
Orlando Kahanek ·
RAG vs. ChatGPT: Why Source-Based AI is Superior for Legal Research
In May 2023, New York attorney Steven Schwartz was sanctioned by the court after citing six completely fabricated court decisions in a legal brief – generated by ChatGPT. The case Mata v. Avianca (1:22-cv-01461, S.D.N.Y.) became a cautionary tale for the entire profession: generative AI without source grounding is unsuitable and dangerous for legal work.
But does this mean AI is fundamentally unfit for legal research? No – it means the right architecture is decisive.
The Hallucination Problem: Why ChatGPT is Unreliable for Lawyers
Large Language Models (LLMs) like GPT-4 generate text based on statistical probabilities. They calculate word by word which token is most likely to follow. This leads to a fundamental problem:
- Fabricated case numbers: ChatGPT generates plausible-sounding but non-existent Federal Court decisions
- Incorrect legal articles: Art. 42 CO is correctly cited, but the content doesn't match the actual legal text
- Mixed legal systems: German BGB is confused with Swiss CO, EU law presented as Swiss law
- Outdated information: Legislative changes after the training cutoff are ignored
A 2023 Stanford study showed that ChatGPT-3.5 hallucinated in up to 69% of cases for legal questions. GPT-4 improved this to approximately 36% – still unacceptable for professional legal work.
What is RAG? The Architecture Behind Reliable Legal AI
RAG (Retrieval-Augmented Generation) is a two-step process:
Step 1: Retrieval (Source Search)
Before the AI responds, it searches a curated legal corpus – in our case, the complete Swiss Classified Compilation of Federal Legislation (SR), Federal Court decisions (BGE), and cantonal court decisions. The search is semantic: not just exact keywords, but content relevance.
Example: The question «Can my landlord evict me if I don't pay rent for two months?» automatically finds:
- Art. 257d CO (Tenant's payment default)
- Art. 266l CO (Form of termination)
- BGE 140 III 591 (Protection against termination for payment default)
Step 2: Generation (Answer Generation)
The LLM receives the found sources as context and generates the answer exclusively based on these sources. Every statement is linked to a specific source citation.
Practical Comparison: ChatGPT vs. RAG-Based Legal AI
| Criterion | ChatGPT (without RAG) | RAG-Based Legal AI | |---|---|---| | Source citations | None or fabricated | Verifiable SR/BGE numbers | | Currency | Training cutoff (months old) | Up-to-date legal corpus | | Swiss law | Mixed with DE/AT/EU law | Exclusively Swiss legal system | | Hallucination rate | 36-69% for legal questions | < 5% through source grounding | | Traceability | Black box | Transparent source block | | Multilingual | Good, but without legal context | DE/FR/IT with correct terminology |
Case Example: Employment Law Question Compared
Question: «An employer in Zurich wants to summarily dismiss an employee during the probation period. What rules apply?»
Typical ChatGPT Answer:
> «During the probation period, the employment relationship can be terminated at any time with 7 days' notice...» (correct but incomplete – no distinction between ordinary and summary dismissal)
RAG-Based Answer:
> «During the probation period (Art. 335b CO), a notice period of 7 days applies. However, summary dismissal is only permissible during the probation period for just cause pursuant to Art. 337 CO. The Federal Court has consistently held that the just cause must be so serious that the terminating party cannot reasonably be expected to continue the employment relationship. Mere dissatisfaction with performance is insufficient (established Federal Court case law on Art. 337 CO).»
Limitations and Responsibility
Even RAG-based systems do not replace a lawyer. They are tools that accelerate legal research and improve the quality of source work.
- Always verify sources: RAG provides citations – use them for verification
- No substitute for legal analysis: The legal evaluation remains the lawyer's task
- Limits in niche areas: Very specialized legal fields may be underrepresented in the corpus
Conclusion: Architecture Makes the Difference
ChatGPT and RAG both use LLM technology – but the difference is fundamental. Only RAG-based systems offer the reliability and traceability that professional legal work demands.
---
This article was written by the RechtsKI team. RechtsKI uses RAG technology with complete Swiss federal law, Federal Court decisions, and cantonal jurisprudence as its source corpus.