An Internal Knowledge Bot for a 220-Person CA Firm: Claude Sonnet 4.5 + LlamaIndex + 14 Years of GST Circulars

A 220-person CA firm in Hyderabad — 4 partners, 18 managers, 198 associates and articled clerks across 7 service lines — was losing ~14 person-hours a week to "what does the May 2018 GST circular say about input credit on capital goods" type questions. Senior managers were the bottleneck. We built an internal knowledge bot in 6 weeks on Claude Sonnet 4.5 + LlamaIndex + Qdrant, ingesting 14 years of GST circulars (2,840 documents), 1,420 internal SOPs, and 18,400 client memos. Eight weeks post-launch, 156 of 220 staff (71%) used it weekly; manager-question volume dropped 38%. Here is the ingest pipeline, the citation-required prompt pattern, and what we did about hallucinations. ## TL;DR (60 Words) We ingested 14 years of CBIC GST circulars + internal SOPs + client memos into a Qdrant vector store via LlamaIndex. Claude Sonnet 4.5 handles the answer generation with a strict citation-required prompt: every claim must cite a circular ID + section. Without a citation, the bot says "I do not know — ask a senior manager." Weekly active rate at 8 weeks: 71%. Cost: ₹4.2 lakh build + ₹38,000/month inference. ## Why This Matters Now Claude Sonnet 4.5 was released on September 29, 2025 — a model Anthropic positioned as state-of-the-art on agentic tasks and software engineering. For a knowledge bot ingesting Indian regulatory text (which is dense, cross-referenced, and frequently amended), the gains over earlier models on long-context retrieval and citation-following accuracy are substantial. The LlamaIndex framework added explicit support for Sonnet 4.5 within days of release. The combination is cheap to operate (~₹38k/month at this firm's volume) and ergonomic for a small in-house team to maintain. We have shipped 3 similar bots in the last 6 months for Indian professional-services firms — each variant of the same pattern. ## The Client (Specific Details) - Sector: Mid-tier CA firm (audit + tax + GST + transfer pricing + statutory compliance) - Location: Hyderabad HQ + 1 satellite office in Vizag - Size: 220 staff (4 partners, 18 managers, 198 associates and articled clerks) - Knowledge corpus: 2,840 CBIC GST circulars + 1,420 internal SOPs + 18,400 client engagement memos (anonymised) - Stack on day 0: A shared Google Drive with a folder hierarchy. Search worked when the file name was right. Otherwise, you Slack a senior manager. - Trigger: The senior manager who answers most GST queries took 4 weeks of leave. Productivity collapsed across 3 service lines. The managing partner drew a line. ## The Architecture ## The Ingest Pipeline (Where Most Bots Fail) Ingesting CBIC GST circulars is harder than ingesting clean Markdown. Circulars are PDFs with mixed Hindi+English text, tables, footnote references, and explicit cross-references like "as per circular 87/06/2019 dated 02.01.2019". A naive chunk-and-embed loses the cross-reference context. ## The Citation-Required Prompt (The 80% of Hallucination Defence) In the first 4 weeks of operation we logged every "I do not have a confident answer" response. There were 412. Of these, 380 were genuine knowledge gaps the senior manager confirmed. 32 were false negatives where the bot had the answer but did not surface confidence. The 8% false-negative rate was acceptable to the partners — far preferable to confident hallucinations. ## The Cost of Operation Compared to the alternative — one extra senior manager on retainer to handle these queries (₹6+ lakh/month fully loaded) — the bot pays back in under 2 weeks. Even at 3x usage, monthly cost stays under ₹1 lakh. ## The 6-Week Build Plan ## What Drove The 71% Adoption Rate We have shipped 3 internal RAG bots; this one had the highest adoption of the three. Three things made the difference: ## Pre-Launch Checklist (RAG Bot Edition)

• Corpus inventory signed off by the partner team (no PII, no client-confidential leaks)
• Ingest pipeline tested on a 100-document sample with citation extraction verified
• Evaluation set of 100 known-good Q&A pairs scored by the senior subject matter expert
• Chunk size + top-k + rerank-k tuned with grid search on the eval set
• Citation-required prompt iterated to under 10% false-negative rate
• "I do not know" response framing approved by partner team (this is non-trivial)
• UI shows citation pills with one-click access to the source document
• Thumbs feedback loop wired to a weekly senior-manager review queue
• Cost monitoring with daily alert if spend trends >₹2,000/day
• Audit log of every query + answer + citations for the partner team

## Common Mistakes (Each One Hurts) Symptom: "Bot confidently cites a circular that does not exist." Cause: chunk metadata not propagated through retrieval. Fix: every chunk in the vector store carries the source circular ID; the prompt template demands the bot use only those IDs. Symptom: "Bot misses obvious matches on circular numbers." Cause: pure vector retrieval misses exact phrase matches. Fix: add BM25 in parallel; merge top-12 from each before reranking. Hybrid retrieval is non-negotiable. Symptom: "Bot is slow — 14 seconds per answer." Cause: serialised retrieval + rerank + LLM. Fix: stream the LLM response as soon as the first reranked context is in. Time-to-first-token drops to 1.4 seconds; full answer in 6-8 seconds is acceptable. Symptom: "Bot adoption stalls at 30%." Cause: refusing legitimate questions too aggressively. Fix: false-negative analysis on the "I do not know" responses; tune the confidence threshold downward until refusal rate is 8-12%. Symptom: "Bot costs spiral after week 4." Cause: associates discover it answers personal-finance questions too. Fix: domain detection in the prompt; off-domain questions get redirected to the firm's CSR FAQ link. ## When NOT To Build a Knowledge Bot Skip the build if (a) your corpus is under ~200 documents — search + senior expert is faster, (b) your domain changes faster than your re-ingest cycle (daily-changing rates, intraday securities) — the bot will lag and erode trust, or (c) you do not have a senior subject matter expert who can curate the prompt and review thumbs-down responses weekly. We have walked away from 2 RAG projects where the SME was unavailable. The bot needs an owner. ## A Detail That Saved Us On Day 9 On day 9 a senior associate asked: "What is the GST rate on a particular agricultural input post the September 2025 GST Council meeting?" The September 2025 meeting was 18 days before our last ingest, but the relevant circular was published 4 days after the ingest cycle. The bot correctly said "I do not have a confident answer — this may be a recent circular not yet in my corpus." The associate Slack-ed the senior manager, who was impressed. Without the recency-aware prompt rule, the bot would have answered with the older (wrong) rate. The lesson: every RAG prompt for fast-changing regulatory text needs an explicit "I might be stale" exit. Build it from day one. ## The Reddit Pulse on RAG-for-Law / Compliance Bots The thread on r/CharteredAccountants from October 2025 has 38 comments from CA firms experimenting with internal LLMs. The top concern raised: hallucination on circular numbers. The second: cost spiraling. The pattern we shipped (Sonnet 4.5 + citation-required + Qdrant) addresses both. We have seen the same patterns hold up on a transfer-pricing bot for a different firm in March 2026. Worth comparing approaches in the thread before you build. ## FAQ ### Why Claude Sonnet 4.5 instead of GPT-4 or Gemini? We benchmarked all three on a 100-question eval set drawn from the firm's internal knowledge. Sonnet 4.5 had the lowest hallucination rate (4% vs 9% for GPT-4 Turbo and 11% for Gemini 1.5 Pro), the best citation-following accuracy (94% vs 82% / 79%), and a sensible price point (~₹2.40 per query at this corpus size). Sonnet 4.5 also handled the long-context retrieval gracefully when we needed to pass 4 reranked chunks of legal text. ### Why Qdrant instead of Pinecone or Weaviate? Self-hosting on a single Hetzner CCX23 (₹6,800/mo) is cheaper than any managed equivalent for this corpus size. Qdrant has a stable Python client, hybrid search (BM25 + vector) built-in, and a sensible operations footprint. We have run it in production on 3 client projects with no operational drama. ### How often do you re-ingest the corpus? GST circulars: nightly cron pulls from the CBIC portal. Internal SOPs: weekly. Client memos: monthly batch with anonymisation pass. Re-ingest of changed-only documents takes 12 minutes per cycle. ### What about data privacy — does Anthropic see the firm's data? The query and the retrieved context are sent to Anthropic for inference. The corpus itself never leaves our infrastructure. Anthropic's Standard API does not train on enterprise inputs. For higher assurance, the firm can move to AWS Bedrock with VPC routing — we have prepared but not enabled that path. ### How do you handle cross-references between circulars? A Neo4j sidecar stores cross-reference edges. At retrieval time, we expand the result set with documents 1-hop from the matched chunks. Improved recall on amendment-heavy queries by 23%. ### What is the cost per query? About ₹2.70 per query at this firm's volume (~14k queries/month). Breakdown: Claude Sonnet 4.5 inference ₹1.70, embeddings (already-cached) ₹0.10, retrieval + rerank ₹0.40, Qdrant + Hetzner amortised ₹0.50. ### What was the team for this build? One senior backend engineer (LLM + RAG pipeline), one full-stack (Next.js UI + auth), one ML-savvy engineer for the eval-set tuning. Our QA lead Manvi at 0.4 FTE on the response-quality eval set, and our CTO Hrishikesh at 0.3 FTE for the prompt-engineering iterations and the partner reviews. ### How do you compare this to building on PenLeap-style real-time AI feedback? Our edtech product PenLeap uses a similar pattern (rubric-based scoring is also a citation-required generation problem). The infrastructure layer is shared between our consulting work and our in-house products. That is part of why we can ship these RAG projects in 6 weeks rather than 14. ## Want an Internal Knowledge Bot for Your Firm? Related reading: our 7-branch manufacturing CRM cost math, the TalkDrill 800ms voice AI latency case, the TalkDrill case study, and our AI automation service. Email contact@softechinfra.com to receive the citation-required prompt template + the 100-question eval-set methodology before the call.