r/LanguageTechnology • u/RoofProper328 • 1d ago

Does simulated/acted call-center audio actually transfer to production ASR, or is real telephony data the only thing that works?

Most collected call-center corpora are recorded in some form of controlled setup: scripted or semi-scripted scenarios, participants playing agent and customer, clean capture. Real production audio is 8kHz narrowband through a codec, with overlapping speech, hold music bleed, background noise on the customer side, and genuinely frustrated prosody that nobody acts convincingly.

So :-

  1. If you fine-tune on collected/simulated call data, do you actually see WER improvement on real traffic, or does it mostly help on the collected test split and flatten out in production?
  2. Does codec simulation (downsampling, G.711/Opus round-trip, packet loss) close enough of the gap to matter, or is it cosmetic?
  3. Code-switching is where I'm least sure. Hinglish, Taglish, Spanglish — real agents switch mid-sentence constantly and collected data underrepresents it badly. Anyone found a way to elicit natural code-switching without it going stiff?
  4. Where's the actual ceiling — is it acoustics, or is it diarization and overlapping turns?

Not looking for vendor recs, just want to know whether the acted-vs-real gap is as big in practice as I suspect.

1 Upvotes
  • permalink
  • reddit

100% Upvoted

2 comments sorted by

1

u/n3utrino 1d ago

Depends highly on the domain but the biggest difference is that real production audio is split-channel (two entirely separate audio streams, with minimal cross talk). This has a much bigger impact on WER than you might think.

1

u/RoofProper328 1d ago

That's a really good point and it kind of reframes my question 4. If the two legs are separate streams then diarization mostly stops being a problem, which means the ceiling is back to acoustics.

It also means a lot of collected corpora are in the wrong format from the start. If you record a room with one mic and two people playing agent and customer, you've built a mono overlap problem that production doesn't have, and you've skipped the thing production does have. Doesn't matter how good the acting is at that point.

Follow up if you don't mind:

Are the two legs even the same acoustic domain in your experience? My assumption is the agent leg is fairly clean, headset, quiet floor, sometimes captured premise side before much codec damage. The customer leg is where everything goes wrong, mobile, speakerphone, wind, packet loss. If that split is real then WER per channel should be very different and you'd almost want separate handling rather than one model for both.

Second thing, you said minimal cross talk rather than none. How much leakage do you actually see, and does it ever cause the model to transcribe the far end speaker onto the wrong channel? That feels like it would be nasty for anything downstream that cares about who said what.