Voxa: an AI phone receptionist that books appointments

A salon chain in Japan kept losing bookings for one simple reason: the phone rang while everyone was busy with a client, or after closing. So we put an AI on the line. It picks up, understands what the caller wants, checks the booking rules, reads the booking back, and logs every call with its recording and transcript for the staff to review.

Admin demo login: password / password. Every call, name and phone number in there is made up.

First, a quick honest note

The real system belongs to the client, so I can't show its code, its data or its name. What I can show you is how I work. I studied the original, then rebuilt the whole staff side from scratch under a new brand called Voxa, with fictional data, and put it live so you can click around. The architecture, the rules and the security choices below come from the original. The screens you see are the rebuild.

The problem

A salon lives on its booking calendar, and most bookings still come in by phone. When a stylist is halfway through a color job, nobody picks up. When the shop is closed on Monday, nobody picks up. Every one of those missed calls is a customer who probably books somewhere else.

The ask was simple to say and hard to build: answer every call in natural spoken Japanese, take new bookings, changes and cancellations, never promise a slot that doesn't exist, and give the 1 to 3 staff per store a clean place to check what happened on each call.

The architecture

Here's the whole thing on one page. The caller dials the salon number, Twilio streams the live audio to our Fastify backend, and the backend bridges it to the OpenAI Realtime API. Speech goes in, the model reasons and calls our tools, speech comes back out. No robotic menu, no "press 1". Every call lands in PostgreSQL, and the staff review them in the admin panel.

The hard parts, and how I solved them

Putting an AI on a phone line sounds easy until a real person calls. These are the problems that took actual engineering.

• Silence kills a phone call. On a website, a two second wait is fine. On the phone it sounds like the line dropped, and people hang up. So I kept the audio in Twilio's native μ-law format end to end with no conversion step, used the Realtime API so speech goes straight in and straight out, and trimmed the prompt down to just the current task once the AI knows what the caller wants. Fewer tokens per turn, faster replies.
• Real callers interrupt, and change their mind. Nobody waits politely for a bot to finish. Semantic voice activity detection works out when someone actually finished a sentence, and the moment the caller starts talking, the AI stops. If the caller corrects the booking, the AI takes the correction, up to three times. After that it ends the call politely and logs it for staff to follow up, instead of looping forever.
• Language models love to say yes. The scariest bug in a booking bot is "you're all set!" when nothing was booked. So the system won't let the AI report success unless the confirm tool actually ran. If it didn't, the call is logged as failed with a reason the staff can read and act on.
• Names and numbers over a phone line. Japanese names, phone numbers, dates, all spoken over a cheap phone line. The AI finds a free slot first and only then asks for a name and number, so nobody spells their name for a time that isn't available. Then comes a two turn read back: the AI repeats everything, asks "is that correct?", and waits for a clear yes.
• Rules that can't bend. Closed days, opening hours, a 30 minute slot grid, at least 60 minutes notice for same day bookings, up to 7 days ahead. These live in code, not in the prompt, so the AI can't talk its way around them. When a request breaks a rule, the caller always hears alternatives instead of a flat no, and availability is checked one last time right before confirming.

How I write the code

I try to write code the next engineer can read without calling me. In practice that means a few habits I don't skip.

• TypeScript in strict mode everywhere. Unused code fails the type check, so dead code doesn't pile up.
• Validate everything that comes from outside. Forms, URL parameters and environment variables all go through Zod. A broken link like ?page=abc falls back to page 1 instead of crashing the screen.
• Every screen has all its states. Loading, empty, error, and the happy path. The call log list alone has a skeleton loader, an empty row, an error row with retry, and pagination for 10, 20, 50 or 100 rows.
• Errors a human can understand. A failed call doesn't just say "error". It says the caller hung up before the booking was confirmed, so the staff knows to call them back.
• Small commits with clear names. One feature per branch, Conventional Commits like feat(call-logs) or fix(ui), merged without squashing so the history tells the story of the build.

The process, step by step

I use a method I call ATM, which in Indonesian stands for Amati, Tiru, Modifikasi: observe, imitate, modify. You study what already works, rebuild it faithfully, and only then change things on purpose.

• 1
  Observe. I read the original end to end and wrote it all down: screens, call flow, business rules, API shapes, design tokens. Before writing a single line, I knew exactly what "done" meant.
• 2
  Design first. Every screen and every state got designed before code, including the date picker open, the failed call and the 404 page.
• 3
  Plan in stages with a dependency map. Setup, login, app shell, call logs, call detail, system pages, landing page, final check. Each stage has its own branch and can't start until the one it depends on is verified.
• 4
  Build with AI agents, review like a lead. I run AI coding agents in parallel, each in its own git worktree. My job is the one a tech lead has: split the work, check each stage against the design, and only then merge. The agents type fast. I'm the one who says what's good enough.
• 5
  Prove it with screenshots. Every stage left evidence behind: 74 screenshots across desktop, tablet and phone, plus a final report of 18 screen and state checks. All 18 passed, with type check and build both clean.

Security, and how I checked it

A phone line that can create bookings is a door into the business, so I treat it like one. This is what the original system does:

• Only Twilio can talk to the voice endpoint. Every request, both the HTTP webhook and the live audio socket, has its Twilio signature verified. A fake request gets dropped.
• Staff login done properly. Passwords hashed with argon2. The session lives in a signed, httpOnly, SameSite strict cookie backed by a session row in the database, so it can be revoked. Requests from other origins are rejected.
• Rate limits and security headers on every route, including a Content Security Policy.
• Customer data stays private, even from staff. Phone numbers come out of the API masked to the last 4 digits, and sensitive fields are redacted from the server logs.

And in the rebuild itself: the login redirect blocks open redirect tricks (javascript:, data:, protocol relative and cross origin URLs), every route behind login has a guard, and a 401 anywhere clears the session and sends you back to login. I also took the client and vendor names out of every public file, and kept my own analysis of the original out of the repo entirely.

Fast where your customers feel it

A caller won't sit through silence, and someone checking you out on their phone won't wait for a slow page. Both cost you bookings.

• No awkward silence on the line. Replies come back quickly enough to sound like a conversation, and the AI stops the moment the caller cuts in. Nobody wonders if the line dropped.
• Opens cleanly, in two languages. The website is plain static pages, so nothing freezes and nothing jumps around while it loads. Japanese is the default for the salon's customers, and English is one tap away.
• Staff never wait either. Call logs open instantly, and a recording plays right away once it has been opened before. Checking yesterday's calls takes minutes, not a coffee break.

Delivery

The original ran as a working PoC on AWS in Tokyo: the backend in a container on ECS, PostgreSQL next to it, secrets in SSM Parameter Store, and GitHub Actions running lint, type check and tests on every pull request, then deploying automatically once the code is merged. The rebuild is split into two apps, each one its own Vercel project deployed straight from main. Every change goes out the same way and can be rolled back in one click.

What you'd get at the end is not just a link. You get the repo, a plan that shows what was built in what order, a README that gets a new developer running in minutes, and the evidence folder that proves each screen works.

What you get when you work with me

• A working piece before you commit. I build a small part of your idea first, running for real. You decide after you've used it.
• AI that's safe to put in front of customers. Guardrails like the fake success check are part of how I build, not an extra you have to ask for.
• Fast where it matters. Every screen is checked on desktop and on a phone before you ever see it.
• You own everything. Code, repo, docs and deploys are yours, written so your next hire won't have to rebuild it.