Why our tutoring AI answers from documents, not memory
A tutor that invents curriculum is worse than no tutor. Saarthians answers from classroom material first and generates second — here is the architecture that makes that promise hold.
The failure mode: confident, wrong, untraceable
A general-purpose language model asked about a chapter will produce a fluent answer whether or not it knows the chapter. In a classroom that failure mode is not an edge case — it is the product failing at its only job.
So the first architectural decision in Saarthians was a negative one: the tutor is never allowed to answer from memory alone. Every response must trace to retrieved material — a PDF section, a note, a grounded chunk.
Retrieval first, generation second
The pipeline order is deliberate: ingest classroom PDFs, chunk them with structure preserved, embed, retrieve the top passages for a question, and only then generate — with the retrieved passages in context and citations pointing back.
This ordering changes what can go wrong. When an answer is bad, the failure is inspectable: either retrieval fetched the wrong chunk (a data problem) or generation strayed (a prompt problem). Each has a different fix, and neither requires retraining anything.
Classroom scoping is a retrieval problem too
A tutor serving many classrooms must never leak one classroom's material into another's answers. In Saarthians this is enforced where it cannot be bypassed: row-level security in Postgres scopes every retrieval query before generation ever sees it.
Frontend checks would be a promise. Database policy is a guarantee — and it is verified with live testing, not assumed.
Evaluation before launch, not after complaints
Before classrooms depended on it, answers were checked against source material in loops: does this response trace to a real chunk? Does it stay inside the classroom's scope? Anything unverifiable stayed out of the product.
The rule we ship with: what cannot be measured is not shipped.
- Trace every answer to a retrieved chunk
- Scope every query by classroom policy
- Test flows live, not just in notebooks
- Keep the honest unknowns visible
What this means for your documents
The same pattern ports directly: curricula, manuals, policies, knowledge bases — any corpus where wrong answers cost trust. Ingestion and evaluation do the heavy lifting; the model is the last mile, not the foundation.
If your material matters enough that hallucinations are unacceptable, the architecture above is where we would start with you too.