Architecture WebRTC

The Architecture of Pion WebRTC

This document describes in the details the architecture of Pion WebRTC. It shows how data flows through the system and points of importance in the code base.

We also will explain why certain design decisions were made. Some of our designs were made with incorrect assumptions. These could serve as good projects for modernization of the code base.

Exploring the PeerConnection

At a highlevel the components that make up a PeerConnection look like this. At the far right you have public APIs and how data flows to/from them.

                                         <---> pion/srtp ---> OnTrack
                                         |         ^
                                         |         | --------- AddTrack
pion/ice <----> pion/webrtc(internal/mux)
                                         |
                                         <---> pion/dtls <--> pion/sctp -> OnDataChannel
                                                                   ^
                                                                   |---- CreateDataChannel

Receiving Data

pion/ice implements RFC 8445. This is where all network traffic arrives.

pion/webrtc gets all dtls and srtp traffic in one stream, and is in charge of demuxing it. muxfunc inspects each packet that flows through the system and routes it as needed. We later can create handlers and route into specific components.

pion/srtp encrypts/decrypts all media traffic. a muxfunc is created in the DTLSTransport. This muxfunc then passes all media into a srtp.SessionSRTP or srtp.SessionSRTCP. pion/webrtc later will later interact with these sessions.

pion/dtls encrypts/decrypts all encrypted data traffic. a muxfunc is created in the DTLSTransport. This muxfunc then passes all data into pion/dtls.

pion/sctp then handles all decrypted data traffic. dtls.Conn that was created is passed into the sctp.Client. We then call acceptDataChannels and a callback is fired everytime a new SCTP stream is seen. A STCP stream is created for every new DataChannel.

The handling of media streams is broken into two parts. We have explicit media streams for SSRCes that are declared in the Offer/Answer. Those are handled in startRTPReceivers. For each SSRC we explicitly request the stream srtpSession.OpenReadStream. Later we added support for Simulcast where the SSRCes aren't known ahead of time. This is done in handleUndeclaredSSRC where each new SSRC emits a callback. Having these two code paths is a good candidate for refactoring

Sending Data

pion/datachannel wraps a sctp.Client and encodes the messages properly to send. You can see data enter this package in Send. From here WriteDataChannel encodes and sends into the SCTP subsystem. The SCTP subsystem has a direct route out pion/dtls -> pion/ice as explained above.

pion/srtp allows you to open a WriteStreamSRTP. These are opened on the SRTP/SRTCP session objects in the DTLSTransport.

Starting the Connection

Pion is made up of a collection of transports. You have one ICETransport, DTLSTransport and SCTPTransport. You will then have multiple RTPSender and RTPReceiver.

These transports are started by startTransports We need to know the following minimum details from the remote to start the transports.

• DTLS Fingerprint
• ICE User-fragment and Password
• Does the remote support SCTP
• What SSRCes does the remote wish to send
• What codecs does the remote support

All SDP parsing is done in sdp.go and these functions are called through out the PeerConnection.

Generating Offer/Answer

When calling CreateOffer or CreateAnswer Pion will generate a SessionDescription that generates the current representation of the PeerConnection. If SetRemoteDescription has been called it will use generateMatchedSDP Pion will take the remote PeerConnections state into account. If Pion is generating the first SessionDescription it will use generateUnmatchedSDP

FAQ

RTCP/RTCP pipeline and pion/interceptor

A Interceptor is a pluggable RTP/RTCP processor. Via a public API users can easily add and customize operations that are run on inbound/outbound RTP. Interceptors are an interface this means A user could provide their own implementation. Or you can use one of the interceptors Pion will have in-tree. You define an interceptor per SSRC, and you have the following methods you can define.

What is packetio.Buffer, why do we need it?

Inside pion/ice and pion/srtp we have these buffers. These buffers give us the following behaviors.

They allow packets to be processed out of order and ignore streams

Today a user can ignore a OnTrack callback and the packets if they want. This is because we provide a buffer for each RTP and RTCP stream. If we only had one buffer a user would have to accept buffer 0 before they could buffer 1 and 2. Even if they didn't want buffer 0.

It allows fast reading from the network

If the user had code that was slow it could block the network. We want a really fast Read from the UDP socket into a buffer. The user then can pull from that buffer later.

It allows users to control buffering rules easily

The buffers can be size or count based. It is important that users who care about performance have access to all of these things.