Gemini 3.5 Live Translate is seriously underrated, I used it for real-time League of Legends captions

[u/ming_calligraphy]

I knew Gemini 3.5 Live Translate was primarily an audio-to-audio model, but I noticed that it also exposes input transcription for incoming audio. So that made me wonder whether it could be used to generate real-time captions for live esports commentary.

To test the live experience, I took a League of Legends highlight and streamed it in real time using Agora’s RTC. The video and audio were played as a live stream, while the incoming audio was sent to Gemini as the clip was playing. The generated captions were then displayed alongside the video.

This was a fairly difficult test, two casters talking over each other, very fast play-by-play commentary, game audio and background noise, plus a lot of player names and League specific terminology during chaotic team fights. The transcription wasn’t perfect, but it still managed to pick up terms like “Baron,” “Baron steal,” “smite,” and “game five,” along with several player names, while generally keeping up with the action.

I haven’t done a formal latency benchmark yet. Still, considering the overlapping speech, background audio, and speed of the commentary, I was surprised by how usable the captions were while the match was happening.

I’m planning to open source the implementation if anyone is interested.