RTCP REMB: tweaking video call settings in the browser
Resourceful guys from Tribe not only found their niche at the junction of conference and gamedev'a (which in itself is already interesting), but also know well what is happening under the hood of WebRTC. Product Manager Timothy Le Borne periodically describes the technical side of his offspring; we decided to translate his article about the non-standard, but very interesting functionality of WebRTC. Sure, you will be interested. Well, while you are reading, we will continue to prepare the insider for our updated videoconferences But this is another story.
why we decided to do it and as we have it .
But closer to the point: let's see what happens under the hood.
The WebRTC connection starts with an exchange mechanism called "SDP offer /answer" - the exchange of information on supported codecs and corresponding parameters between the participants.
After that, participants try to communicate directly with each other: they start listening to UDP ports and use ICE, STUN and (hereinafter) TURN to establish a UDP connection.
When a UDP connection is established, participants use it to exchange encryption parameters over the DTLS-SRTP protocol. After that, the participants use:
Protocol SRTP (Secure RTP over UDP) to send and receive media
SRTCP (Secure RTCP over UDP) protocol for flow control and out-of-band data exchange.
Stack of WebRTC-protocols (the changed scheme, the original is here: .webrtc-security.github.io )
We deal with RTCP
It has been mentioned above that RTP and RTCP packets are safely stored in SRTP and SRTCP. What are they used for?
RTP provides end-to-end transport for real-time dаta: audio and video. In some ways, RTCP is the "RTP brother": it is used together with RTP and improves the data transfer protocol by monitoring delivery, as well as management and identification.
In the WebRTC session, the MCU acts as an intermediary between the participants: they are connected to the MCU, but not directly to each other.
WebRTC - basic topology of MCU
For example, take a session with two participants. The MCU receives the media stream from the FOO participant, decodes it, encodes it, and sends it to the BAR participant. Simultaneously, the same thing happens with the flow from BAR to FOO.
MTP and RTCP protocols
The symmetry is obvious, so we will concentrate on one direction and see what is going on:
Focus on one of the sides
When the BAR receives the media stream from the MCU, it also receives statistics from RTCP Sender Reports (SR). The BAR browser will use RTP's sequential packet numbers, RTP timestamps, proper time and Sender Reports to calculate what we call RTCP Receiver Reports (RR).
The BAR browser uses all this data to calculate the maximum expected bitrate, and then sends this value to the MCU using RTCP REMB (Remote Estimate Maximum Bitrate).
We use REMB (effectively)
The RTCP package format is REMB tools.ietf.org/html/draft-alvestrand-rmcat-remb-00#section-???r3r3164.
The calculated maximum bit rate is encoded in the BR Exp and BR Mantissa fields.
An example of implementation can be found in the code Chrome:
In the Go language, we converted this message into the PacketALFBRemb structure:
When you need to know the bitrate from such a structure, you can do so:
If on the contrary, you have a bitrate and you want to populate the Exp and Mantissa fields in the PacketALFBRemb structure, then you'll do something like this:
Bandwidth and RTCP REMB
This is one of the advantages of MCU - the ability to make a REMB RTCP packet to control the outgoing bitrate in the browser:
The MCU sends RTCP REMB to control the outgoing bitrate of the FOO
Also, the MCU can take the bitrate estimate from the receiver and, according to it, change its coding settings so that they coincide with the estimated maximum bit rate; It also optimizes the outgoing bandwidth of the MCU:
The MCU changes the encoding settings using the RTCP REMB values from the BAR member
Manage the bitrate
How can I test the impact of RTCP REMB on the bitrate of the FOO browser? It's quite easy: just send REMB packets with variable bit rate, (for example, 64KB and 128KB every 10 seconds), and then look at the graphics. chrome: //webrtc-internals .
In Go, this would be:
Having opened chrome: //webrtc-internals , you will see something like this:
This is a clear confirmation that you are able to change and monitor the bitrate from your side in real time. Which parameters are better suited to your needs and whether you need to optimize the bandwidth and the quality of the stream - it's up to you :)
You probably noticed that RTCP REMB is now in the status of deprecated . The idea was that the bandwidth should control the data source (and not the remote receiver /participant) - well, in the article we showed that it is still possible and how to do it.
In our case, it was useful - to get point control over the band and quality :)