A new architecture for transcription (and more)
Back in 2017 we shipped a speech-to-text prototype for Jitsi Meet. It was built on Jigasi, and for the better part of a decade that is how transcription in Jitsi worked. It served us well — but it also accumulated a fair amount of technical debt. We now have a new architecture that replaces Jigasi for transcription: it is simpler to run, cheaper to operate, works with a range of speech-to-text providers, and it is cloud-native.
Why replace Jigasi?
Jigasi started life as a SIP gateway — its job was to bridge regular telephony into a conference. Transcription was bolted on later: to transcribe a meeting, a Jigasi instance joins the conference as a hidden participant, receives everyone’s audio, and relays it to a speech-to-text service.
That works, but it carries a lot of baggage. Jigasi is a full XMPP client with a SIP stack attached. Every transcribed conference ties up a stateful Jigasi that has to join the MUC, and audio has to be routed out of the bridge, into Jigasi, and back again. Scaling it means running and load-balancing a fleet of Jigasi instances. That is a lot of moving parts for what is, fundamentally, “send some audio to an API and get text back.”
The new architecture
The key observation is that the media already flows through the Jitsi Videobridge (JVB). So instead of pulling audio back out into a separate component, we let the bridge send it straight to a transcription service.
Jicofo decides when a conference should be transcribed and tells the bridge which service URL to connect to. The bridge opens a single WebSocket to the transcription service and forwards each participant’s Opus audio, tagged per source. The service decodes the audio, streams it to a speech-to-text backend, and streams the results back over the same connection, where they are turned into the captions users see. Jicofo also passes along any HTTP headers, so the service can sit behind an authenticating proxy such as Cloudflare Access.
That is the whole design. There is no extra conference participant, and no second media path — and with it goes an entire class of operational complexity.
A simple protocol on the wire
The bridge and the service talk over a plain JSON-over-WebSocket protocol. We did not invent it from scratch — it is based on VoxImplant’s WebSocket audio-streaming format, in which audio travels as base64 inside a `media` event. Reusing an existing format kept the service compatible with other audio producers (it happily accepts ogg-opus from VoxImplant, for example) and gave the bridge, through a small jicoco-mediajson library, a well-understood thing to implement.
We then extended it with the few events we actually needed: “ping”/”pong” for keepalive, an “info” event so each side can announce itself (application, version, active provider), a per-source “start”, and a “transcription-result” envelope that carries the text back.
A frame of audio going from the bridge to the service looks like this:
{ “event”: “media”,
“media”: { “tag”: “participant-7”, “chunk”: 42, “timestamp”: 1751979219, “payload”: “[base64 Opus]” } }
and a result coming back:
{ “event”: “transcription-result”, “type”: “transcription-result”,
“participant”: { “id”: “participant-7” }, “language”: “en”, “is_interim”: false,
“transcript”: [ { “text”: “hello world”, “confidence”: 0.98 } ] }
A handful of event types is all it takes to move audio in and text — or translated audio — back out.
Pluggable backends
The service itself — opus-transcriber-proxy — is a thin proxy in front of the actual speech-to-text engine. Out of the box it supports xAI, OpenAI’s realtime transcription, Deepgram, and Google Gemini, with a configurable priority order and automatic failover between them. It also speaks to any OpenAI-compatible endpoint, so you can point it at a model you host yourself — no audio has to ever leave your own infrastructure.
…and more
Because the bridge now speaks a simple “audio in, media out” protocol to the service, transcription is only the first thing we can build on it. The same pipeline drives real-time speech-to-speech translation, a feature we’re currently working on where a speaker’s audio is translated, re-encoded as Opus, and played back into the conference in the target language, alongside translated captions. Same architecture, different endpoint.
And there is plenty more the same foundation makes possible — multitrack audio recording straight off the bridge, or connecting a conference directly to an AI agent that can listen and talk back in real time. Anything that wants access to a meeting’s audio no longer needs a Jigasi and a hidden participant; it just needs to speak this protocol.
Docker or Cloudflare, natively
The service runs anywhere. You can run it as a plain Docker container alongside the rest of your deployment, or you can deploy it natively to Cloudflare — it ships as a Cloudflare Container fronted by a Worker, with request routing and container pooling built in, so you can run transcription at the edge without managing any servers of your own.
Try it
Getting started is essentially a `docker run` away:
docker run -d -p 9090:8080 \
-e PROVIDERS_PRIORITY=openai \
-e OPENAI_API_KEY= \
jitsi/opus-transcriber-proxy:latest
Point Jicofo at it, enable transcription in Prosody and `config.js`, and you are transcribing. The full setup — Docker and Cloudflare, the Prosody and Jicofo configuration, and Cloudflare Zero Trust — is documented in the handbook.
This is as much a foundation as it is a feature. Retiring Jigasi for transcription clears a big chunk of technical debt and gives us a clean, pluggable, cloud-native path for real-time AI on top of Jitsi audio — starting with transcription and translation, with more to come. Jigasi-based transcription is now deprecated and will be removed in a future release, so if you are self-hosting, this is a good time to migrate.