Architecting your WebRTC application for scalability, Arin Sime

Editor's Notes

• #9: Janus built in C, Jitsi built using Java, MediaSoup built with C++ and Pion uses Go
• #12: Arin’s last slide - “and now Alberto will talk more about media servers and architectural use cases”
• #13: As Arin mentioned, there are different alternatives when building your own WebRTC application. In most cases direct peer to peer communication is really not an option.. In this diagram, at the right, you can see a representation of how a 8 participants peer to peer network would look like. Without a intermediate media server helping us is a bit messy, and specially resource intensive! To reduce the amount of resources used at the edge, there are 2 popular architectures used to scaling this…
• #14: MCU arch and SFU arch
• #17: Now, how do we scale beyond a single MCU or SFU server? IT DEPENDS ON THE USE CASE Media processing operations are very CPU intensive! A XLarge AWS instance, for example, can’t handle more than a couple hundred of sd quality video streams. What happens if we have 1000 viewers? This is how the architecture in a very high level would look like And what about other use cases?
• #18: So, in the case of larger video chat rooms, to have more than 50+ participants in the same room you might be reaching the limits of your infrastructure. It will be a good idea to use multiple MCU/SFU servers and each user can connect to them simultaneously! There are smarter ways to do this but this is an approach.
• #19: And what about many concurrent connections or integrations with telephony systems like IP-PBX and so on. Well… that’s complicated but a simplified flow could look like this where we have web users connecting to an SFU/MCU from web or mobile and other users dialing in from the pstn at the right But there are potential challenges caused by differences in: the SIP implementation, supported resolutions, RTCP Muxing supported or not, different codecs or in case of H264 different profile level ID that define it. ICE (trickle) support…
• #20: So what are the main considerations when deciding which of those architectures to use or implement? One is really evaluating the number of participants (small groups will make things easier). Also, do we need advance functionalities like recording? dial in pstn?
• #21: Now how can we orchestrate this for horizontal scalability and handle 1000s of users calling concurrently? One proposed architecture could look like this diagram here where we have… The challenges we are trying to solve are:... stateful architectures: harder to handle crashes and scaling is difficult. We need to stick to the same server we were using before and reconnect. WebRTC doesn’t do caching like other stateless protocols like hls
• #22: When scaling, if we want to be efficient we want to use auto scaling rules. This rules will… For example, in this charte we assume a connection threshold (this is the start of a new server) after 100 connections. And a server capacity of 400 connections. If our server start time is more than 60sec we wouldn’t have time to handle more than 5 new connections per second (or 300 connections in a minute spikes)
• #23: Another 2 important topics when building scalable and high available webrtc solutions is sticky sessions and data persistence. By sticky sessions I mean connections that will be maintained so user always reach the same server. This is important because… Dayta persistence is what will allow us to distribute traffic between multiple concurrent services and to recover the session in case of a server failure
• #24: After that, once we have our scalable infrastructure ready we will need to load test what we build. To do that it won’t be enough to simulate http requests but we will need to simulate real video and audio traffic. For that typically instances simulating participants are used to connect to the servers and send/receive media. In addition to verifying we are effectively connected and receiving the right number of connections we will need to evaluate the performance by capturing jitter/rtt/packet loss so we can obtain an acceptable quality score or emos. emos mean opinion score usually used to measure a/v quality
• #25: Finally, there are also application and media optimizations you can use to have better quality and have a better experience when scaling. One way is using simulcast or SVC.. SVC is a technique that allows encoding a video stream once in multiple layers. The layers in SVC can be subtracted while maintaining the video, reducing its quality with the reduction of each layer (fps, resolution or snr layers) Another is just limiting the amount of video. Do you really need video? Some great tools just use audio (twitter spaces, clubhouse, slack huddle). In that case we can optimize a lot performance and reach a much larger audience! Or at least just get the video of the active participant using audio detection. Adapt bitrate based on video resolution, this can be done on the fly without renegotiation And for audio we have Opus RED and DTX, DTX shown in the diagram at the bottom right corner.. DTX simply eliminates audio when there is no audio at the other end…
• #26: With that would almost conclude our presentation, but there are many more configuration options and optimizations that will be available in the future. Great teams of engineers are working on different fronts to surpass our current limitations. For example, AV1 and Lyra V2. AV1 has been around for a while already but most devices doesn’t support its hardware acceleration yet so it causes too high cpu usage as of today. Regarding Lyra v2: iOS and other embedded platforms are not supported at this time, but this may change in the future. You can see how lyra achieves same quality with much lower bitrates! And, since before we talked about Opus DTX and how we stop sending audio when is not necessary…Some improvements can be made for better quality. The process of dealing with the missing packets is called packet loss concealment (PLC). The receiver’s PLC module is responsible for creating audio (or video) to fill in the gaps created by packet losses, excessive jitter or temporary network glitches, all three of which result in an absence of data. To realistically continue short speech segments enabling it to fully synthesize the raw waveform of missing speech
• #27: And with that, here concludes our presentation about architecting your WebRTC application for scalability. I really hope you enjoyed and learned from this! And if you are interested about this topics visit us or follow us on Twitter! Thank you so much for watching!