How a K-12 Publisher Made 30 PDFs of Methodology Usable in a Classroom
- The publisher's methodology became answerable on demand rather than readable in bulk
- 584 curated passages across the full catalogue, tagged by grade band, trait and strategy
- A 298-term glossary enforced on every output, so the system cannot invent terminology the author never used
- Every lesson scored against a 100-point rubric drawn from the methodology's own framework before it is delivered
- Delivered inside the learning platform teachers already use, so adoption required no new habit
Mudassir Marwat
Founder & CEO, Cognilium AI
Outcome metrics
The situation
A K-12 writing-curriculum publisher sells a methodology, not a product. It is taught through an 11-episode video course, two textbooks holding 188 catalogued mini-lessons, and more than 50 named teaching strategies — roughly 1.37 million characters across 30 PDFs.
The commercial problem was not acquisition. Teachers bought the methodology. The problem was that using it on a given morning required knowing the whole catalogue: which strategy fits a fourth-grade class working on organisation, in the shape the methodology prescribes, in the author's own terminology. A teacher with thirty minutes of prep does not read three hundred pages to find out.
Material that is bought and never used does not renew. That is a retention problem wearing a content problem's clothes.
What we did about it
Search was the obvious answer and the wrong one. A search box returns passages; a teacher at 7am does not need more to read. They need a lesson they can teach in forty minutes, in the methodology they paid for.
So the system generates rather than retrieves — and the constraint that makes it trustworthy is that it can only speak in the publisher's own vocabulary. The whole catalogue was curated into 584 passages, each tagged by grade band, writing trait and strategy. A 298-term glossary of the methodology's named concepts is enforced on every output: if the model reaches for a phrase the author never wrote, the output is rejected before a teacher sees it.
Every generated lesson is scored against a 100-point rubric derived from the methodology's own four-step framework before delivery. A lesson arrives in about twelve seconds, inside the course pages teachers already log into — no new tool, no new login, no new habit to build.
How it was built
The technical detail, for the person who will be asked whether this is sound.
Hybrid retrieval with prefetch-time metadata filtering
Dense embeddings (OpenAI text-embedding-3-large, 3072-dim) catch semantic matches. Sparse BM25 (fastembed) catches the publisher-specific terminology that the embedding misses — boosted with the 3-8 expanded terms extracted by the Query Understanding pass. Reciprocal Rank Fusion merges the two ranked lists; the filter is pushed down into both retrievers BEFORE fusion so narrow queries (specific grade + mode + trait) keep a full reranker input pool of 30 candidates.
Progressive filter relaxation
When the strict filter returns fewer than 5 candidates, the system relaxes one dimension at a time — first the writing trait, then the mode, then the grade. Each relaxation is one Qdrant call; ~85% of queries never trigger relaxation, ~12% relax once, <0.5% reach the no-filter floor. Editors see a log of relaxed queries and decide whether to add content or merge tags.
Anti-hallucination via runtime grounding
A 298-term domain vocabulary loaded at server startup. Every generated output gets a 2-. Each term is checked against the vocabulary set. Match: pass. Fuzzy-match within edit distance 1: log + auto-correct. No match, no fuzzy match: retry with a stricter prompt listing allowed terms inline. After two retries, fall back to the closest valid term by embedding similarity and flag for human review.
Two-stage validation: structural + judge
Stage 1 (code, <: every step present, ≥2 sentences each, first-person language in step 2B (the think-aloud), Turn & Talk cues in step 3. Stage 2 (GPT-4o-mini judge, ~500ms): scores the lesson on a 100-point rubric across framework adherence, grade appropriateness, methodology accuracy, completeness, and teacher language quality. Score ≥85 = "validated" ship immediately; 60-84 = "acceptable" ship after retries exhausted; <40 = "error" refuse and ask teacher to rephrase.
7-layer security for LMS embedding
LearnWorlds passes user_id + email to the iframe via Liquid template variables. The backend validates: Sec-Fetch-Dest header (iframe-only), HMAC-SHA256 signed URLs (2-min expiry), referrer, LearnWorlds API user verification, LearnWorlds API enrollment check, JWT (HS256, 15-min httpOnly cookie), and re-enrollment check on every refresh. Rate-limited to 50 queries/user/hour.
TL;DR
A writing-curriculum publisher had 1.37M characters of methodology and teachers who could not use it. How we made a catalogue answerable instead of searchable.
The problem a publisher has when the customer stops reading
A curriculum publisher is in an unusual business: what it sells is knowledge, and the sale only creates value if the buyer consumes it. A teacher who buys a methodology and never manages to apply it has not been badly served by the content. They have been defeated by its volume.
This publisher had the volume problem in an acute form. The methodology is genuinely good and genuinely deep: an eleven-episode video course, two textbooks holding 188 catalogued mini-lessons, more than fifty named teaching strategies with their own vocabulary. Around 1.37 million characters, spread across thirty PDFs.
None of that helps a teacher with thirty minutes before a lesson on organisation for a fourth-grade class. To use the catalogue well you have to know the catalogue well, and the people who most need it are the ones with the least time to learn it.
Why a search box would have made it worse
The instinctive fix is search. Index the PDFs, add a box, let teachers find what they need. We argued against it, and the reason matters more than the technology that replaced it.
Search returns passages. A teacher at seven in the morning does not need more to read — they are already drowning in material they own. They need something they can teach in forty minutes. Handing them three relevant excerpts and the job of assembling a lesson leaves the actual work exactly where it was.
The system had to generate a finished lesson. Which raises the obvious objection: a publisher's entire asset is its methodology, and a system that improvises around it is worse than no system at all.
The constraint that makes generated content safe to publish under someone's name
The whole design rests on one idea: the system is allowed to compose, but not to invent. It speaks only in the publisher's own vocabulary.
The catalogue was curated into 584 passages, each tagged by grade band, writing trait and named strategy — so a request for organisation at fourth grade retrieves the parts of the methodology that actually address it. Then a 298-term glossary of the methodology's named concepts is enforced on every output. If the model reaches for a phrase the author never wrote, the output is rejected before a teacher ever sees it.
Finally, every lesson is scored against a 100-point rubric built from the methodology's own four-step framework. A lesson that does not follow the structure the publisher teaches does not go out, regardless of how well it reads.
Delivered where the habit already exists
The system lives inside the course pages teachers already log into. That was deliberate. A separate tool is a separate habit, and a separate habit is the most common way a genuinely useful system fails to get used. A lesson arrives in about twelve seconds, in the place the teacher was already standing.
What transfers to other businesses
If your product is expertise — a methodology, a playbook, a body of standards — you probably have the same problem and are measuring it as churn. The asset is not underused because it is bad. It is underused because consuming it is work, and your customer's alternative to that work is doing nothing.
The fix is not more content and not better search. It is making the body of knowledge answerable: able to produce the specific, finished thing the customer needs, in your own language, without them reading the whole of it first. The vocabulary constraint is what makes that publishable under your name rather than a liability.
Technologies used
Share this case study
Frequently Asked Questions
Find answers to common questions about the topics covered in this article.
Technical deep-dives behind this result
The engineering writeups that explain how the system was built.
Hybrid Retrieval With Prefetch-Time Metadata Filtering
Why filtering after RRF fusion loses the right chunks, and how a "drop trait → mode → grade" progressive relaxation ladder keeps narrow queries answerable without dropping retrieval quality.
Anti-Hallucination via Runtime Grounding Against a Domain Vocabulary
A startup-loaded domain vocabulary the generator must match against, plus framework rules baked into every prompt — a low-cost pattern that catches hallucinated terminology before the user sees it.
The Production LLMOps Stack: Evals, Judges, Retries, Circuit Breakers
The day-2 ops layer of an LLM product — what to evaluate, what to judge in real time, what to retry, and when to fail closed. The components that turn a prototype into something operable.