Define the job
Write one page before you watch a demo: the hours that matter, who calls, what each caller needs, who takes the urgent calls, and which systems hold your schedule.
Details
Start with a one-page job description for your phone. List the hours that matter: evenings for an HVAC contractor, lunch hours for a salon, overnight for a property manager. List who calls, from new customers and regulars to suppliers, job seekers and wrong numbers. Next to each, write what should happen. A taken message and a booked appointment are different jobs. Factual calls about hours, prices and schedules suit software best, since people trust algorithms more for tasks that feel factual20. Name the person who takes urgent transfers, plus a backup. In a 2023 Gartner survey, customers' top fear about AI service was that it would make reaching a person harder14. Last, list the systems that hold your schedule.
Coverage window
caller types
required actions
escalations
systems
Open Requirements worksheet
Requirements worksheet
- 1. Coverage windowWrite down the hours your phone matters most, such as evenings for an HVAC contractor, lunch for a salon or overnight for a property manager.
- 2. caller typesList everyone who calls, from new customers and regulars to suppliers, job seekers and wrong numbers, and what should happen for each.
- 3. required actionsMark which calls need a taken message and which need a booked appointment, because those are two different jobs.
- 4. escalationsName the person who takes urgent transfers, plus a backup for when that person is busy, asleep or on vacation.
- 5. systemsNote every system that holds your schedule, like a salon booking app or the shared Outlook calendar at a law firm.
The table above is the accessible HTML version. Downloads contain the same blank prompts and no hidden scoring.
Set must-pass gates
Set your deal breakers before you score anything. Location, sensitive data, real bookings, a working transfer and your budget ceiling can each knock a vendor out alone.
Details
Any one of these knocks a vendor out. Location comes first. Some voice products serve only one country, and First AI Employee, for one, is for businesses operating in the United States3. Sensitive data comes next. If calls carry patient health details, as at a dental or medical office, get a written answer on whether the product is built for that. First AI Employee is not a fit for those calls3. Then check real booking: a taken request or a texted link is not an appointment on your calendar. Then check the transfer, with a named person who answers when the rules run out. Set your budget ceiling before you open a single pricing page.
Geography
regulated data
booking truth
transfer fallback
budget ceiling
Open Must-pass gate builder
Must-pass gate builder
Pick only the rules your business can really test and enforce. Leave the rest unchecked.
Run ten realistic calls
Make the same ten calls to every vendor, including a wrong number, a question it cannot answer and a broken calendar. Judge what landed in your systems, not how the call sounded.
Details
Make ten test calls to every vendor from your own phone, with the same script each time. Ask a price, change a booking detail and request something the assistant does not know. Repeat the same booking in quiet and background noise, and verify the calendar afterward. The 2026 tau-Voice benchmark evaluates completed tasks separately from voice interaction across 278 simulated customer-service tasks29. It is not a small-business accuracy estimate. It shows why a convincing conversation and a correctly completed action deserve separate checks. Then disconnect the calendar and try to book. The assistant should acknowledge the failure and preserve a useful message for your team.
New lead
repeat customer
wrong number
urgent request
unknown answer
calendar failure
Open Ten-call test script
Ten-call test script
- 1. New leadCall as a new customer and ask a price, like the cost of a haircut or a drain cleaning. Note whether you get an answer or a promise to call back.
- 2. repeat customerCall as a regular who expects to be known, and see whether it treats you any differently from a stranger.
- 3. wrong numberDial as someone who reached the wrong business, and check that it ends the call politely without logging a fake lead.
- 4. urgent requestReport a flooded basement at 9pm and time how long it takes before a real person is on the line.
- 5. unknown answerAsk something you never gave it an answer to. A confident wrong answer is still yours to stand behind.
- 6. calendar failureDisconnect the calendar, then try to book. A good system says it cannot confirm instead of pretending the slot is yours.
- 7. New lead: caller with a strong accentHave someone with a strong accent make the same new-customer call, and check that the name, number and request come through correctly.
- 8. Repeat customer: moving a bookingCall back as an existing client to move an appointment, and check that the old slot is freed and the new one shows on the calendar.
- 9. Wrong number: a sales pitchCall pretending to sell advertising, and check that the pitch is screened out instead of passed on to you as an urgent message.
- 10. Urgent request: nobody picks upRepeat the urgent call while your transfer person ignores the phone, and listen to what the caller hears next.
The table above is the accessible HTML version. Downloads contain the same blank prompts and no hidden scoring.
Score the providers that pass
Score only the vendors that passed. Set your weights before any demo, and treat anything you could not test as unproven, not as a probable yes.
Details
Set your weights before the first demo, or the best salesperson sets them for you. Score call clarity, completed actions, setup work, cost, recording controls and support. Include security when the assistant connects to business tools. NIST's March 2026 research summary describes successful hijacking attacks against all 13 tested models in a public competition6. Those were tool-use, coding and computer-use scenarios, not phone tests. Our buying recommendation is to ask which actions the receptionist may take and how those limits are enforced. Score the actual result as well as the spoken confirmation. A booking that never landed outweighs a pleasant voice.
Call experience
action reliability
setup burden
commercial terms
privacy and control
support
Open Weighted scorecard
Weighted scorecard
- 1. Call experienceScore how the call sounds from recordings: the pauses, the clarity, and whether a caller who has trouble hearing could follow it.
- 2. action reliabilityGive this the most weight. Score whether the booking, message or transfer actually happened, not what the summary claimed.
- 3. setup burdenScore how much of the setup lands on you, from writing the call rules to updating prices when they change.
- 4. commercial termsScore what the plan costs at your real monthly volume, including any overage and what it takes to cancel.
- 5. privacy and controlScore who controls recordings and transcripts, how long they are kept, and whether you can delete them.
- 6. supportScore how support acts when something breaks on a Saturday, based on what you saw rather than what sales promised.
The table above is the accessible HTML version. Downloads contain the same blank prompts and no hidden scoring.
Normalize pricing
Vendors bill per call, per unique caller or per minute, so headline prices do not compare. Run your own month of calls through each pricing model before you decide.
Details
On a typical shortlist, each vendor bills in its own way. Smith.ai charges per call against included call tiers, with overage7. Goodcall bills by unique customers a month8. Rosie bills monthly included minutes9. Dialzara bills included minutes plus a per-minute overage that varies by plan10. First AI Employee sells three voice plans, Essential, Growth and Scale2. So take last month's call count and average call length, and run them through each model. Per-call billing hurts a business with many short calls. Per-minute billing hurts one with long calls, and a month of new faces runs up a per-customer bill. Add only the extras you need, and ask what leaving early costs.
Usage profile
overage
add-ons
commitment
Open Pricing-normalization worksheet
Pricing-normalization worksheet
- 1. Usage profileWrite down last month's call count and average call length. Price every vendor's model against those same two numbers.
- 2. overageNote what each plan does once its included calls, minutes or customers run out, and the rate it charges past that point.
- 3. add-onsList only the extras you would really turn on, such as a live-agent option, and price each one on top of the plan.
- 4. commitmentAsk what leaving early costs and whether the plan runs month to month. Write the answer down, or mark it not stated.
The table above is the accessible HTML version. Downloads contain the same blank prompts and no hidden scoring.
Check the contract and data path
Read the contract for recording, retention, data use, cancellation and who owns your number. Regulators have acted on voice recordings kept too long28, so get answers in writing.
Details
Read the contract before you sign. Ask to hear the recording notice. Federal law allows recording with one party's consent25, but about 11 states require all parties26, California among them24. Ask how long recordings and transcripts are kept, and whether you can delete one. Keep recordings long enough to check transcripts: a 2023 test of one engine found invented phrases in about 1% of them19. In a children's privacy case, Amazon agreed to a $25 million civil penalty after regulators said it kept voice recordings indefinitely28. Ask whether your calls train the vendor's AI; the FTC says AI vendors must keep their privacy promises27. Ask what leaving costs and who keeps the number. This is not legal advice.
Recording
retention
subprocessors
cancellation
number portability
Open Export/print controls
Export/print controls
Shortlist fit examples
Different needs pick different winners. Use these examples to sort your shortlist, then run the test calls yourself before you sign anything.
Details
Each suits a different buyer. Want setup handled for you? First AI Employee is a managed AI receptionist built around a business's call rules1, and Rosie has guided setup9. Want a live agent in reach? Smith.ai describes an AI receptionist with optional live-agent involvement7, so ask what triggers it and what it costs. Goodcall lets you describe the job while its builder sets up the agent8. Want help with the script? Dialzara includes initial prompt setup on Elite and transfers calls to your own team10. Need outbound callbacks? First AI Employee answers inbound calls only3. Under FCC rules, AI-voice calls you place need prior consent unless an exemption applies15. This is not legal advice.
First AI Employee
- First AI Employee offers a managed AI receptionist built around a business's call rules. Checked 2026-10-20; sources S01.
- The voice product answers inbound calls and does not place outbound or cold calls. Checked 2026-10-20; sources S04.
- The AI receptionist is for businesses operating in the United States. Checked 2026-10-20; sources S02, S04, S09.
- The AI receptionist answers calls 24/7, including nights, weekends and holidays. Checked 2026-10-20; sources S01.
- It books appointments straight onto a connected Google, Outlook or Calendly calendar, or onto its own built-in calendar. Checked 2026-10-20; sources S01.
- It transfers urgent calls to a person by rules the business sets. Checked 2026-10-20; sources S01, S04.
Smith.ai
Smith.ai. Its own pages describe it as an AI receptionist with optional live-agent involvement, billed by per call with included call tiers and overage (stated on its own pages), last checked 2026-10-20. Whether it fits still depends on the jobs it must do for you and the limits you can live with.
Rosie
Rosie. Its own pages describe it as an AI answering service, billed by monthly included minutes (stated on its own pages), last checked 2026-10-20. Whether it fits still depends on the jobs it must do for you and the limits you can live with.
Goodcall
Goodcall. Its own pages describe it as a voice AI phone agent, billed by monthly plan with a unique-customer allowance; extra customers billed at the plan rate (stated on its own pages), last checked 2026-10-20. Whether it fits still depends on the jobs it must do for you and the limits you can live with.
Dialzara
Dialzara. Its own pages describe it as an AI receptionist, billed by monthly included minutes plus plan-specific per-minute overage (stated on its own pages), last checked 2026-10-20. Whether it fits still depends on the jobs it must do for you and the limits you can live with.
Frequently asked questions
Quick answers to common questions. More than one in four US adults has a disability22, and DOJ guidance says businesses must make what they offer online accessible23. Check any booking page your assistant sends. This is not legal advice.
Details
Frequently asked questions
What should I test in an AI receptionist demo?
Make unscripted calls. Include a question it has no answer to, a time slot that is taken, a transfer nobody picks up, background noise, a repeat caller and a request outside what you offer.
Should I choose the provider with the most features?
No. Start with the things it must do. Extra features do not make up for a core job that fails, careless handling of customer data or a billing model that does not fit your calls.
How should I weight the scorecard?
Put the most weight on the call results you need and on what happens when something fails. Setup work, price and convenience come after the service has shown it can handle your real calls.
Should I require a contract?
Many small-business products run month to month, while tools built for large companies may ask for a commitment. Look at cancellation, refunds, getting your data out and taking your number with you, not only the contract length.
What if a vendor will not answer a question?
Mark that line unknown, not yes. If it is something the service must do, an unknown counts as a fail until the vendor proves it.