If you have already come across Zoom, then you’ve probably heard them make bold claims about their technology like this one for example:
-web
Jitsi founder Emil Ivov recently mentioned in an interview that, in spite of their repeated claims, he hadn’t actually seen Zoom do anything better than WebRTC with regard to quality and video transport. Without proof some felt skeptical of the comparison to the larger commercial player, so we thought, we’d put together a very simple congestion test bed. We ran it on both Zoom and our WebRTC implementation and recorded the results. A picture is worth a thousand words (so at 33 pictures per second, that should make a 3 minute video equivalent to 5,940,000 words 😋):
(for optimal readability, make sure you watch that on YouTube in full screen)
Here’s what happens. We have two 1:1 independent video calls. One with Zoom and one with WebRTC (using Jitsi Meet). The first 10 seconds of the test run on regular Wi-Fi, just like all of us every day. Around second 10, we turn on network impairment for both and limit upstream and downstream bandwidth to 500kbps for both tests. We then measure the following:
- How quickly does each solution notify the user that they are encountering network problems
- How quickly do they resume some partial video flow on a lower definition (i.e., partial adaptation).
- How quickly do they then resume a fully fluid video flow on that low definition (i.e., full adaptation).
Next, we turn off network impairment and observe how quickly the two solutions recover back to normal operation. This time we measure:
- How quickly they reach SD-like quality (at about 1mbps)
- How quickly they reach HD-like quality (at about 2mbps)
For those who don’t want to go through the video, the results are:
[table id=1 /]
Hopefully this would help inform future conversations on the topic …
Important Disclaimer
This is NOT meant to be an exhaustive evaluation of neither Zoom’s media stack nor WebRTC. It is a very simplistic test and it barely scratches the surface. It is however telling and easy to replicate. Here are our test details:
- We played the source content on an independent box and we then used Logitech Screen Share to connect that box and effectively turn it into a webcam for one of the two participants in the conference (the Sender)
- The other participant (the Receiver) had local video muted in both cases
- We used a program called Link Conditioner to impair the bandwidth that comes with Apple’s developer tools
- We ran this on OS X 10.13.6
- The Link Conditioner profile has “Downlink Bandwidth” and “Uplink Bandwidth” both set to 500 Kbps and all other fields left at 0
We look forward to doing more tests in the near future! Ping us on our open Community Forum with questions and comments.
Love,
Your personal Jitsi team!
