Build a Hinglish Voice IVR for a Clinic: Weekend Appointment Booking
Build a multilingual voice IVR that books clinic appointments in Hinglish over a weekend. Twilio + Whisper + Claude, slot-filling, and a Google Calendar write-back. Real flow, gotchas, and costs.
Hrishikesh Baidya
May 25, 202514 min read
0%
A 4-doctor clinic in Indore was missing roughly 18 calls a day. The single receptionist could not answer the phone, check the doctor's calendar, and handle a walk-in at the same time. Each missed call was often a missed appointment. We built a voice IVR that picks up every call, talks to the patient in Hinglish, finds an open slot, and writes the booking straight into Google Calendar — and we built it over a weekend. This walkthrough gives you the call flow, the Hinglish prompt design, the slot-filling logic, the calendar write-back, and the three gotchas that will trip you up.
~18/day
Calls Previously Missed
2 days
Weekend Build Time
Hinglish
Hindi + English, One Flow
~₹6–9
Cost Per Completed Call
## The Answer in 60 Words
Use Twilio to answer the call and stream audio, Whisper (or an Indic ASR) to transcribe Hinglish speech, Claude to run the conversation and decide the next question, and a TTS voice to speak back. The bot collects name, doctor, and preferred time through slot-filling, checks Google Calendar for openings, books the slot, and reads back a confirmation. Weekend build, modest per-call cost.
## Why This Matters Now
Small Indian clinics live and die on the phone. Patients call to book, reschedule, and ask "kya doctor available hai aaj?" — and they switch between Hindi and English mid-sentence without thinking. A receptionist who steps away for five minutes loses bookings to voicemail nobody checks. By mid-2025, speech-to-text on Indian-accented Hinglish had become good enough, and Claude's conversation handling reliable enough, that a small clinic could run a booking IVR that actually books — not the rigid "press 1 for appointments" trees patients hang up on. This is the same voice-AI stack we use on TalkDrill, our in-house English-speaking app, where Indian-accent speech recognition is the core problem we solve every day.
## What You'll Need
A Twilio account with an Indian or virtual phone number and Programmable Voice
A speech-to-text engine — Whisper (large-v3) or an Indic ASR like Sarvam for Hinglish
An Anthropic API key for Claude (we use a fast model for low latency)
A text-to-speech voice that handles Hinglish acceptably (test a few — accents vary)
A Google Cloud project with the Calendar API enabled and a service account
A small server (Node.js) to hold the call session and orchestrate the pieces
The clinic's real doctor schedule and slot length (e.g., 15-minute slots)
## The Five Pieces of the Stack
📞
Twilio (Telephony)
Answers the call, streams audio to your server over a Media Stream, and plays your TTS reply back. Handles the boring parts of phone calls — DTMF, call forwarding, hang-up detection — so you don't.
🎧
ASR (Speech-to-Text)
Turns Hinglish speech into text. Whisper large-v3 is a strong default; an India-tuned engine often wins on names and numbers. This is the piece that makes or breaks accuracy.
🧠
Claude (Conversation)
Runs the dialogue. Reads the transcript, tracks which slots are filled, decides the next question, and returns structured data plus the line to speak. A fast model keeps latency low.
🔊
TTS (Text-to-Speech)
Speaks Claude's reply in a natural Hinglish voice. Pre-warm it and stream the audio so the patient hears a reply fast, not after an awkward silence.
📆
Google Calendar API
The source of truth for availability. The freebusy endpoint finds open slots; an event create writes the booking. The doctor sees it in the same calendar they already use.
🗄️
Session Store (Node.js)
Holds the per-call state: who's calling, slots filled so far, the proposed booking. A simple in-memory map keyed by Twilio's call SID is enough for a single clinic.
## The Call Flow
1
Patient calls → Twilio answers
2
Greet in Hinglish, ask the need
3
Transcribe → Claude decides next slot
4
Check calendar for openings
5
Confirm + write booking back
## The DIY Walkthrough (A Weekend, Step by Step)
1
Answer the call and capture audio with Twilio
Point your Twilio number's voice webhook at your server. Return TwiML that greets the caller and either uses <Gather input="speech"> or opens a Media Stream over WebSocket for tighter control. Verification: call the number and confirm your server receives the audio or the speech result.
2
Transcribe the Hinglish reply
Send the captured audio to Whisper large-v3 or an Indic ASR. For Hinglish, test both — Whisper handles code-switching surprisingly well, but an India-tuned engine often wins on numbers and names. Verification: say "mujhe Dr Sharma ke saath kal subah ka appointment chahiye" and confirm the transcript is usable.
3
Run the conversation with Claude (slot-filling)
Feed the transcript to Claude with a system prompt that defines the slots to fill: patient name, doctor, date, and time. Claude returns the next thing to say and the structured slots it has captured so far. It asks only for what's missing. Verification: a transcript with the doctor but no date makes Claude ask for the date — not re-ask the name.
4
Check Google Calendar for open slots
Once you have doctor + date, query the doctor's calendar with the Calendar API freebusy endpoint for that day. Return the open 15-minute slots. If the patient's requested time is taken, offer the two nearest free slots. Verification: a fully-booked morning makes the bot offer afternoon options instead.
5
Speak back with TTS and confirm
Convert Claude's reply to speech and play it through Twilio. Read the proposed slot back: "Theek hai, Dr Sharma ke saath kal 10:30 baje. Confirm karun?" Wait for a yes. Verification: the spoken confirmation matches the slot you're about to book — no mismatch between what's said and what's written.
6
Write the booking back to the calendar
On confirmation, create the event via the Calendar API with the patient name and phone number in the description. Send the patient an SMS confirmation too. Verification: the event appears in the doctor's calendar within seconds and the patient gets the SMS.
7
Add a human handoff for anything off-script
If the caller asks something the bot can't handle (an emergency, a complaint, a complex query), forward the call to the receptionist or take a callback number. Never trap a patient in a loop. Verification: saying "mujhe doctor se baat karni hai" forwards the call instead of re-prompting.
## Designing Hinglish Prompts (The Part People Get Wrong)
The mistake is writing English prompts and hoping the TTS sounds natural. Write the bot's lines the way an Indore receptionist actually speaks: "Namaste, [clinic] mein aapka swaagat hai. Aap kis doctor ke saath appointment lena chahte hain?" Mixed Hindi-English, short sentences, polite. Patients respond in kind, which makes the ASR's job easier.
A few rules that worked for us. Keep each bot turn to one question. Confirm numbers and names by reading them back, because that is where ASR errors hide. Offer two concrete slots instead of an open "kab aana chahenge?" — a closed choice is faster to fill and easier to transcribe. And keep a small lexicon of the doctors' names and common medicine terms so the ASR doesn't mangle "Dr Bhatnagar" into "doctor but nagar."
We learned most of this the hard way building Indic voice features. Our deep-dive on a TalkDrill Hindi voice bot and a pronunciation lexicon that cut WER 31% covers the lexicon trick in detail, and our latency work on a 740ms voice round-trip on Indian 4G explains why every 200ms of delay makes a phone bot feel broken.
## The 3 Gotchas
Latency makes or breaks the feel. A 4-second pause after the patient speaks reads as "the line dropped," and they hang up. Stream audio instead of waiting for the full utterance, use a fast model for the conversation step, and pre-warm your TTS. Aim for under 1.5 seconds from end-of-speech to start-of-reply.
Numbers and dates are where ASR fails. "Teen baje" vs "teen tareekh," "10:30" heard as "10:13" — these errors book the wrong slot. Always read the date and time back and require a confirmation before writing anything. We covered the same trap in our Tally CA-helpdesk IVR built in 2 days.
Calendar race conditions double-book. Two patients call at the same minute and both get offered 10:30. The fix: do a final freebusy check inside the write transaction, and if the slot just filled, apologize and offer the next one. Don't trust the availability you read 20 seconds ago.
Never let a voice bot handle medical emergencies. The very first branch of the flow should detect urgency words ("emergency," "chest pain," "saans nahi aa rahi") and immediately route to a human or read out the emergency number. A booking bot that tries to schedule an emergency is a liability, not a feature.
## A Cost Breakdown Per Call
At roughly ₹6–9 a completed call, a clinic that recovers even five missed bookings a day pays for the entire system many times over. The exact figure depends on your ASR and TTS vendors and average call length; benchmark with your own traffic.
## Common Mistakes (When Not to Build This)
Your call volume is tiny. If the clinic gets 10 calls a day and the receptionist handles them fine, a voice IVR is solving a problem you don't have. Build it when calls are genuinely being missed.
You need rich clinical conversation. A booking bot books. If you need symptom triage or insurance pre-auth over voice, that is a much bigger, regulated build — not a weekend project. Keep the first version to scheduling.
You skip the human handoff. A bot with no escape hatch frustrates exactly the patients you most want to keep. Wire the handoff in step one, not as an afterthought.
## Real Example: The Indore Clinic
The 4-doctor Indore clinic we built this for was losing roughly 18 calls a day to a single overworked receptionist. After the weekend build, the IVR answered every call, booked the straightforward appointments in Hinglish, and forwarded anything complex. The receptionist's phone load dropped enough that she could focus on patients at the desk.
The same voice pipeline runs in production on TalkDrill, our in-house English-speaking app, where the hard problem is exactly this: understanding Indian-accented, code-switched speech in real time and replying fast enough that a conversation feels natural. Everything we learned there about ASR lexicons, streaming, and latency budgets went straight into this clinic build. That is the advantage of building voice features for our own product first — we hit the rough edges on our own users before a client ever does.
For a text-channel version of the same idea, see our patient booking portal for a 9-clinic diagnostics group that halved no-shows, and for the WhatsApp route, our multilingual Hindi/English/Tamil chatbot. This whole pattern sits inside our AI and automation practice, and the engineer who'd lead your build is Hrishikesh, our CTO.
## Frequently Asked Questions
### Can a voice IVR really understand Hinglish?
Yes, with the right stack. Whisper large-v3 and India-tuned ASR engines handle code-switched Hindi-English well in 2026, especially when you add a lexicon of the clinic's doctor names and common terms. Always read numbers and dates back to the caller to catch the errors that remain.
### How long does it take to build a clinic booking IVR?
A focused booking IVR — answer, converse in Hinglish, check a calendar, book, confirm — is a weekend build for an experienced team. Adding triage, payments, or EHR integration extends it significantly. Start with scheduling only and expand once it proves itself.
### What does each IVR call cost to run?
Roughly ₹6–9 for a three-minute completed booking call, split across Twilio voice minutes, speech-to-text, the Claude conversation step, and text-to-speech. Your exact cost depends on vendors and call length. Benchmark with your own traffic before committing.
### How do you stop the bot from double-booking a slot?
Do a final freebusy check inside the write transaction, not just when you first offer the slot. If two callers race for the same 10:30, the second one's booking fails the recheck and the bot offers the next open slot instead. Never trust availability read seconds earlier.
### What happens if the patient has an emergency?
The very first step of the flow detects urgency words in Hindi and English and routes the call to a human or reads the emergency number aloud. A booking bot must never attempt to schedule an emergency. The human handoff is non-negotiable.
### Which is better for Hinglish — Whisper or an Indic ASR?
Both work; test on your own audio. Whisper handles code-switching well and is easy to start with. India-tuned engines like Sarvam often edge ahead on Indian names and numbers. Run a small benchmark of real clinic phrases and pick on accuracy and latency together.
Want a Voice IVR That Books Appointments?
We build Hinglish and regional-language voice bots for clinics, dealerships, and service businesses — answer every call, fill slots, write to your calendar, hand off to a human when needed. Typical first version ships in 1–2 weeks. First call is technical, with the engineer who'd lead your build.