How PenLeap Generates Class 8 Science MCQs With Verified Answer Keys at Scale

PenLeap, our in-house edtech product, drafts roughly 4,200 Class 8 Science MCQs a week against the 2025-26 NCERT "Curiosity" syllabus. Before a single one reaches a teacher's review queue, the pipeline rejects about 31% of them — wrong key, two correct options, a distractor that's secretly right, or a question that drifts off-syllabus. This post walks through exactly how that verification layer works, with the prompts, the checks, and the failure modes we hit in production. ## The Answer in 60 Words PenLeap generates each Class 8 Science MCQ with one model call, then verifies it with a second, independent call that re-solves the question cold — without seeing the proposed key. If the two answers disagree, the question is held. We layer four more checks: single-correct-option, distractor plausibility, syllabus-chapter tagging, and a duplicate hash. Only questions clearing all five reach a teacher. The teacher approves the final 8%. ## Why This Matters Now The 2025-26 CBSE Class 8 Science syllabus moved to the new NCERT "Curiosity" textbook — 13 chapters — and CBSE now mandates that [30% of the paper be competency-based MCQs](https://www.vedantu.com/syllabus/cbse-class-8-science-syllabus), including Assertion-Reasoning and Case-Study formats. Coaching brands that used to recycle a static question bank suddenly need application-oriented items at volume. Generating them by hand is slow; generating them with an LLM and no verification is dangerous — a wrong answer key in a printed test paper destroys trust faster than any feature wins it back. ## What "Verified Answer Key" Actually Means A verified answer key means the correct option was confirmed by a process that never saw the generator's opinion. We don't ask the model "is this right?" — it will agree with itself. Instead, a second model call receives only the question stem and the four options, solves it from scratch, and returns its own answer with a one-line justification. The keys match or the item is held. That single decoupling catches most of what a naive pipeline ships. ## The 5 Verification Gates (In Order) | Gate | What it checks | Reject rate (Class 8 Science) | |---|---|---| | 1. Schema validity | Exactly 4 options, one key field, non-empty stem, valid chapter ID | ~3% | | 2. Blind-solver agreement | Independent solve matches the proposed key | ~14% | | 3. Single-correct check | No two options are both defensible (distractor score < 3) | ~7% | | 4. Syllabus fit | Concept maps to a Class 8 "Curiosity" learning outcome | ~5% | | 5. Duplicate hash | Not a near-duplicate of an existing approved item | ~2% | The numbers add to roughly 31% because an item fails at the first gate it hits and stops there. Gate 2 — the blind solver — is the single most valuable check by a wide margin. When we A/B-tested the pipeline with and without it on 1,000 items, the teacher-reported "wrong key" rate fell from 9.1% to 0.6%. ## The DIY Walkthrough: Build a Verified MCQ Generator You can reproduce the core of this on a weekend. We use Claude (Sonnet tier for generation, a cheaper tier for the blind solve) called from a Node.js worker, but the pattern is model-agnostic. Here's the runnable shape. ## Why Distractor Scoring Earns Its Keep A common assumption is that a wrong answer key is the only failure mode worth catching. It isn't. The subtler bug is an MCQ with two defensible answers — the keyed one and a distractor a careful student can argue is also correct. These pass the blind-solver check (both models pick the keyed option) but still generate angry emails from parents when a sharp student gets marked wrong for a reasonable answer. Gate 3 exists for exactly this. We score each distractor on "could a student who understands the concept argue this is also true?" and route any option scoring 3 to a human. On Class 8 Science, the "Friction" and "Light" chapters produce the most of these, because both have real-world phenomena with more than one valid framing. Roughly 7% of items hit this gate — and catching them is what keeps the question bank defensible under scrutiny. ## A Worked Example (Force and Pressure) Here's a real-shape item that the pipeline held. The generator wrote: stem "A sharp knife cuts better than a blunt one because…", key option B "it applies more force." The blind solver returned option A "it applies the same force over a smaller area, increasing pressure." Disagreement → held → a teacher confirmed the blind solver was right. The generator had picked a tempting-but-wrong everyday answer. Without gate 2, that wrong key ships into a test paper. This is the exact class of error competency-based MCQs are designed to probe — and the exact class an unverified generator gets wrong. ## Common Mistakes (When This Goes Wrong) Symptom: "The model agrees with every key you propose." Cause: you showed it the answer. Fix: the solver must never see the key field. Strip it server-side before the call. Symptom: "Two options are both correct." Cause: no distractor scoring. Fix: gate 3. A distractor that scores 3 is a second right answer wearing a disguise. Symptom: "Questions drift to Class 10 difficulty." Cause: free-text syllabus in the prompt. Fix: pass one structured learning outcome per call, and reject items the tagger can't map back to it. Symptom: "Lots of near-duplicates." Cause: no dedupe, or hashing the whole stem (small wording changes dodge it). Fix: normalize aggressively — strip numbers and stopwords before hashing. Symptom: "Teacher approval rate is fine but students complain questions are 'weird'." Cause: grammatically valid but pedagogically odd items. Fix: this is why the human gate stays. Automation gets you to an 8% human-approval funnel; it doesn't replace the teacher. ## When Not to Build This If you need fewer than ~200 questions total, write them by hand and have one teacher check them — the pipeline's setup cost won't pay back. If your subject has no single defensible answer (open-ended English composition, say), MCQ verification is the wrong tool entirely; you want rubric-anchored grading instead, which is the approach we use for written work on PenLeap. And if you can't get a subject teacher to own the final approve/reject decision, don't ship auto-generated items to students at all. ## How This Connects to Our Client Work The exact pattern — generate, then verify with an independent decoupled check — is what our AI and automation team reuses on client projects where an LLM's output has to be trusted. We built the same "second model re-checks the first" guardrail into a CA-firm knowledge bot and into invoice-extraction flows, because the cost of a confident wrong answer is always higher than one extra API call. PenLeap is where we proved it at volume first, and the same verification discipline carried into our ExamReady work. For the written-answer counterpart, see our breakdown of how PenLeap grades 9,200 Class 10 essays a day without hallucinating marks, and for the question-paper-assembly side, our piece on the CBSE Class 10 pre-board question generator. As Vivek Kumar puts it, the model is the cheap part — the verification layer is the product. ## The Cost Math At 4,200 drafts a week, the pipeline makes roughly three model calls per item (generate, solve, score). On the tiers we use, that's about ₹0.90–₹1.20 per surviving question, all-in — cheaper than a freelance question-writer at any volume above a few hundred items. The 31% rejection is not waste; it's the cost of not shipping a wrong answer key. We'd rather burn an API call than a teacher's trust. ## FAQ ### How does PenLeap verify an MCQ answer key? A second, independent model call re-solves the question from only the stem and options — it never sees the proposed key. If its answer disagrees with the generator's, the item is held for a human. This one decoupled check cut wrong-key rates from 9.1% to 0.6% in our testing. ### Why not just trust the model that wrote the question? Because a model asked "is your own answer right?" tends to agree with itself. Independence is the whole point: the solver must approach the question cold, with no knowledge of what the generator decided. Self-checking catches far fewer errors than a blind re-solve. ### How many Class 8 Science MCQs can this generate? PenLeap drafts about 4,200 a week and passes roughly 8% through to teacher-approved status after five gates and a human review. Throughput is limited by API rate and the teacher review queue, not by the generation step itself. ### What stops questions from going off-syllabus? Each item is generated against one structured NCERT learning outcome and then tagged back to a chapter. If the tagger can't map the finished item to a Class 8 "Curiosity" outcome, it's dropped — that's how a Class 10 concept gets caught before review. ### Does this remove the need for teachers? No. Automation handles the volume and the obvious-error filtering; a subject teacher still owns the final approve/reject call. We treat the human gate as non-negotiable for anything that reaches students. ### Can I use a cheaper model for the verification step? Yes. We generate with a stronger tier and run the blind solve and distractor scoring on a cheaper one. The solver's job is narrow — answer one well-formed MCQ — so a smaller model handles it reliably and keeps per-item cost near ₹1. ### Does the same pattern work for subjects beyond Science? For any subject with a single defensible answer (Maths, factual Social Science), yes. For open-ended writing it doesn't fit — there you want rubric-anchored grading instead, which is a different PenLeap pipeline.