How to test multilingual accuracy in an AI receptionist demo

Every AI receptionist vendor now claims multilingual support. The demos all sound impressive: the AI switches to Spanish mid-call like it was born doing it. But multilingual is where the gap between a demo and a Tuesday afternoon is widest. Language models handle common languages well and uncommon ones badly, and your business needs to know which end of that spectrum your callers sit on.

Here is how to test multilingual accuracy before you buy, using your own callers’ reality instead of the vendor’s script.

Bring your own test calls

Do not let the vendor pick the test language samples. Write down the five most common situations your non-English callers call about, and get a real person, ideally a staff member or a customer who speaks the language, to run the demo with you. Vendor demos use clean, slow, textbook speech. Your callers will speak fast, mix in English words, and use slang. The test has to match the callers, not the brochure.

Test code-switching

Most bilingual callers do not stay in one language. A Spanish-speaking caller might say “necesito una cita para el viernes, and also, how much is the cleaning?” Code-switching, moving between languages inside one sentence, is where multilingual systems stumble. Run a demo call that does it on purpose and check whether the AI follows both halves. If the system locks into one language and ignores the other half of the sentence, your callers will repeat themselves and get frustrated.

Check the hard parts: names, numbers, addresses

Transcription accuracy drops for proper nouns and numbers in every language, and it drops faster in languages with less training data. Have your tester give a spelled-out name, a street address, and a phone number, then read the transcript. If “Calle Ocho” becomes “Cali Ocho,” that is cosmetic. If the callback number is wrong, that is a lost lead. Test the details that cost you money when they are wrong.

Ask how language detection works

There are two ways a system can go multilingual. It can detect the caller’s language and switch, or you can assign a language to a phone number and route accordingly. Detection sounds smarter, but it adds a failure point: the system has to decide correctly in the first few seconds. Ask how long detection takes and what happens when it guesses wrong. Ask whether a caller can switch languages mid-call if they want to. The answers tell you how much control you have when the clever part fails.

Test the languages in order of your call volume

Spanish will be fine on every major platform. The question is your second and third languages. If a meaningful share of your callers speak Vietnamese, Haitian Creole, or Tagalog, test those specifically, because accuracy varies a lot below the top five languages. Ask the vendor which languages they support natively and which run through translation, because those are different products with different accuracy. A vendor who is honest about the weaker languages is more trustworthy than one who claims all fifty are equal.

Check the summary language

Here is the detail buyers miss: the call happens in Spanish, but the summary lands in your inbox in what language? If your staff reads English, the summary should be in English even when the call was not. Ask to see a summary of a non-English call. If the summary comes back in the caller’s language, your team will be pasting it into a translator, which is exactly the friction you paid to remove.

Ask about accent coverage

“Spanish” is not one thing. Mexican, Puerto Rican, Dominican, and Colombian Spanish sound different, and so do the English accents of your bilingual staff. Ask whether the system was tested on the accents your callers actually have. This is hard for vendors to prove, which is why your own test callers matter more than any claim on a spec sheet.

Our multilingual AI receptionist guide covers which languages matter most for US small businesses and how the leading systems handle them. Pair it with this testing checklist and you will know more about the vendor’s multilingual chops than their own sales rep does.