Dynamic Syms
Researching what it takes to make an AI customer feel real, without letting it go off the rails.
First, what is Symtrain
Symtrain is an AI powered simulation training platform for contact centre and customer facing teams. Instead of learning only through slides and then practising live on real customers, employees rehearse realistic customer interactions in a safe, measurable environment.
These simulated interactions are called Syms. A manager or trainer creates a Sym, assigns it, and a trainee practises it. The platform scores the performance, surfaces where the trainee struggled, and feeds that back into coaching. It moves training from passive learning into repeatable, measurable practice.
The Symtrain loop · practice, measure, coach, improve, repeat
From a fixed script to a real conversation
Most Syms follow established content and logic. The customer says the same things in the same order, whatever the trainee does. Dynamic Syms change that. The AI customer responds to what the trainee actually says, so no two runs are the same.
Reliable and easy to score, but predictable. A trainee can pass by memorising the path.
Realistic and adaptive. It tests whether a trainee can actually think, not just recall a script.
This matters because real customer conversations are messy. Customers interrupt, raise objections, change their mind, and get emotional. A trainee needs to practise handling that variation. Dynamic Syms open the door to it, but they also introduce new questions around control, reliability, and realism. That is where the research began.
Predictable practice does not build real skill
Predictable
A fixed Sym trains recall. Trainees learn the one path and repeat it, which rarely survives contact with a real, unpredictable customer.
Adaptive
An AI customer that reacts to the trainee builds real adaptability. But dynamic behaviour is only useful if it stays believable, controllable, and safe.
How might we make simulated customer conversations feel more realistic, while keeping them useful, controllable, and reliable for training?
What I set out to understand
Realism
How realistic Dynamic Syms actually feel to trainees.
Immersion breaks
Where AI behaviour breaks the illusion of a real customer.
Conversation quality
How consistent and coherent the AI generated conversations are.
Voice and TTS
The quality of the synthesised voice and where pronunciation fails.
Creation friction
How hard it is to author a good Dynamic Sym.
Control
How much control creators need over what the AI can and cannot do.
Intent vs behaviour
The gap between how the product should behave and how it did.
Training fit
Whether Dynamic Syms can support genuinely realistic training scenarios.
How I researched it
This was not a single study. It was a continuous loop of testing an emerging AI product, watching where it broke, and turning technical behaviour into UX implications.
Competitive analysis
Benchmarked the wider landscape of AI coaching and simulation training products.
Structured scenario testing
Ran Dynamic Syms across defined scenario types to probe behaviour.
AI and TTS evaluation
Compared voice technologies and models on realism and reliability.
Usability and UAT
Tested the built product against intended behaviour and logged what broke.
Conversation quality review
Assessed coherence, context, and adherence across generated conversations.
Qualitative synthesis
Clustered recurring issues into higher level UX themes and recommendations.
Grounded in testing and feedback
Findings came from hands on testing and feedback gathered across training and product contexts. Exact counts below are placeholders for verified numbers.
Numbers marked XX are placeholders, to be replaced with verified figures. Nothing here is invented.
Four lenses for judging a Dynamic Sym
To keep testing consistent, I evaluated every Dynamic Sym across four dimensions rather than reacting to issues one at a time.
Testing Dynamic Syms across real situations
I tested defined scenario types, then pushed them with custom real world situations to see whether the AI customer stayed believable and in scenario.
| Test type | Scenario | What I tested | Representative observation |
|---|---|---|---|
| 1A · Objection handling | Fleet software upgrade | Whether the AI raised and held objections believably | Objections could soften too easily, rewarding weak responses |
| 1B · Inbound discovery | Airline flight change | Whether the customer answered discovery questions in character | Context held well, but factual details needed tighter constraints |
| 1C · Sales pitch | Alaska cruise booking | Whether the trainee could pitch against a reactive customer | Realistic variation, occasional drift from the scenario goal |
| Custom | Country Delight, StarMovies | Everyday service situations across contexts | Believable range, with recurring pacing and ending issues |
The voice is half the realism
A Dynamic Sym only feels real if the customer sounds real. I evaluated six voice technologies, and tested ElevenLabs models closely, on the dimensions that affect a live conversation.
Ratings are directional, from hands on testing, not published benchmarks. Latency matters most, a slow reply breaks a conversation faster than an imperfect voice.
Where a wrong syllable teaches the wrong thing
In a real service call, agents read out account numbers, prices, and email addresses. If the AI mispronounces them, the Sym quietly trains the wrong behaviour. This was a recurring, high impact issue.
Flight and account numbers read as words or grouped wrongly, which changes meaning on a live call.
Symbols and domains spoken inconsistently, a problem when verifying a customer.
Prices misread, which undermines billing and sales scenarios where the number is the point.
Unusual terms and language codes handled unevenly, affecting accent and pronunciation consistency.
Where the realism broke down
Testing the built product surfaced nine recurring failure moments. Each one is small on its own, together they decide whether a trainee believes the customer.
The Sym stopped
The simulation could halt or fail to continue when it should have carried on.
Hold requests ignored
When a trainee asked the customer to hold, the AI sometimes kept talking.
Weak factual constraints
The AI could invent or drift on policy and factual details it should have held.
Uneven pacing
Typing and transcription speed made the conversation rhythm feel inconsistent.
Abrupt endings
Some conversations terminated in ways that felt unnatural and unresolved.
Prompt engineering friction
Authoring a good Dynamic Sym demanded too much prompt engineering skill.
Playback and generation delays
Playback or response generation could lag or fail to produce.
Audio and video lag
Synchronisation and latency pulled the trainee out of the moment.
Unclear chat experience
The chat lacked guidance, so trainees were unsure what to do next.
The power, and the friction, in one screen
Nine issues, five themes
Listing bugs is not research. Clustered, the failure moments pointed to five UX themes that a fix has to address at the root.
Dynamic does not automatically mean realistic.
The goal was never to make the AI more unpredictable. It was to make it dynamic enough to feel human, and constrained enough to stay useful, safe, and on scenario.
Turning findings into product direction
Each recommendation answers a theme from the synthesis, so the direction stays traceable to the research.
Scenario constraints
Let creators define allowed and prohibited topics, customer personality, objectives, escalation conditions, and factual boundaries, so dynamic stays inside the training goal.
Guidance over prompt engineering
Replace open prompt fields with structured, guided inputs so a trainer can author a strong Dynamic Sym without being an AI expert.
Conversation controls
Give creators direct control over tone, objection strength, difficulty, customer behaviour, and conversation length.
Explainable feedback and recovery
When an interaction fails, say why, and let creators restart, regenerate, or adjust the behaviour rather than start over.
Voice and pronunciation handling
Improve handling of numbers, currency, email, and language codes, and favour lower latency models for live conversation.
Persona and difficulty presets
Offer ready made customer personalities and difficulty levels so the same scenario can train adaptability without extra authoring effort.
Labelled as recommendations and proposed direction. No redesigned screens are shown as shipped.
UX quality lived at the intersection
This was not testing one feature. The experience a trainee felt depended on a chain of systems, and a weak link anywhere broke the realism.
From research to product decisions
Surfaced what to fix first
Named the critical UX and reliability gaps, the creation friction, and where better controls were needed.
A way to judge a Dynamic Sym
Established a four lens framework and a structured AI and voice evaluation the team could reuse.
Actionable direction
Gave product, design, and engineering clear, prioritised findings, and contributed to UAT and iteration.
Verified business metrics can be added here. No revenue, adoption, or satisfaction figures are claimed.
As UX Researcher, I
- Planned and ran structured product testing
- Evaluated Dynamic Sym behaviour across scenarios
- Investigated AI and voice quality
- Tested and compared six voice technologies
- Identified UX and interaction problems
- Ran UAT and logged what broke
- Analysed the competitor landscape
- Synthesised recurring issues into themes
- Documented findings for the team
- Translated technical AI issues into UX implications
- Proposed research led product improvements
- Collaborated with product, design, and engineering
Building believable AI simulations is not only an AI problem. It is a UX problem.
The more dynamic an AI experience becomes, the more intentionally its boundaries need to be designed.
← Back to all work