Big Ideas
Big Ideas is a list generated from conversations in the Pion community. The only limit to these ideas is that they must help people building RTC projects. They don't have to involve any existing Pion projects, and don't have to be software related at all.
Some of these are rough ideas and still require a lot of thought, others just require an owner. If you are looking to get involved this could be a great place to start! If you put some initial effort into an idea, you might find lots of other people that want to get involved also.
We want to add everyone's ideas, please join Slack and share with us. We would love to have you!
We should explore the value of implementing a pure Go Opus encoder/decoder. If it could open up new interesting projects it could be worth exploring, or it could be a lot of effort for little value.
There is a large demand for embedded/IoT devices. People want something that is easy to build, and has a very small footprint. We need to make Pion WebRTC work with TinyGo and document/explain how to easily use it. This could open up an entire market of devices that communicate inside the same LAN.
WebRTC doesn't work for all Pion users yet. We have added some proprietary extensions, we should work on upstreaming these changes.
- Restrict candidate gathering to only some interfaces/port ranges
- allow addIceCandidate before setRemoteDescription
- More control over latency/loss tradeoffs. User should be able to choose how much loss they are willing to tolerate.
- setCodecPreference doesn't work everywhere yet
- Remove provisional offer/answers
- Push back on APIs that aren't portable (dispatching off of input data types)
When people build a website they don't worry about HTTP at all, they just deploy a server. We should get to that same point with WebRTC. Users should be able to build P2P and Broadcast video applications without having to understand STUN, TURN, Signaling, SDP etc...
They should be able to dive into these things later, but we need to find the balance that can empower users.
We should explore the bleeding edge standards. These technologies could open up new possibilities for what we can build. It also is a great chance to show people how easy it is to prototype things with Pion.
- QUIC (DataChannels and Media)
- WebTransport
We would really love to work with other Open Source projects/small companies. It would be great if we could create a community of people working in the same space. There are lots of things still to learn, and it would be great if there was a community resource of people doing things in the WebRTC space.
Tools like webrtc-cli make a big difference. We should explore what more we can do like vnet to help people get better at debugging/learning about RTC.
Since we already collect webrtc stats based on, https://www.w3.org/TR/webrtc-stats/, we should be able to dump the data and visualize it, similar to how go tool pprof works. And, for the UI, we can follow what chrome://webrtc-internals/ does (see below for the screenshots).
Event Logs (showing events in chronological order):
Stats Graphs:
We need to standardize on commit messages, and auto-cut releases. Currently it is too hard for users to follow what we are doing.
We need to start a ASCII only email list where we don't have access to the emails themselves (but in the hands of a party we trust). It would be great if we could notify users about breaking changes in Pion and just general changes in the WebRTC landscape.
Turn examples into integration tests. See https://github.com/pion/webrtc/issues/139
Design goals:
Use the existing examples
- Have minimal impact on the example code E.g.: add event listeners on the existing textareas.
- Use go test to run the tests
- Place integration tests behind a build flag to avoid bloating unit tests (E.g.: go test github.com/pions/webrtc -tags=integration)
- Communicate test data/results over http in order to minimize use of specialized APIs for test subjects (like a browser, node, ...).
See LiveKit. An OpenSource SFU that comes with full Kubernetes support out of the box.
WebRTC needs to be easier at scale. We need to make it possible for Open Source/small companies to deploy/run WebRTC. This is being solved by ion. We need to do a better job of publishing/supporting the developers who are working on it.
You can use just a single UDP and TCP port.
When using Pion in a server inside a corporate environment ports are usually restricted. If you have a cluster of servers behind a load balancer when expecting a remote connection you cannot know which one will receive it and you need all of them to be expecting them. Having a single port mode would make it easier to coordinate a group of agents where any of them can serve an incoming agent.
Available at pion/mediadevices
Currently we expect users to capture and encode media themselves using GStreamer, ffmpeg or another media library. We should provide portable APIs so users can call getUserMedia in their Go code. We should continue to provide flexibility so users can share pre-recorded content. Many users are re-streaming content (RTMP -> WebRTC) or serving files from the disk.
Available at https://webrtcforthecurious.com
If people would find this useful, I would love to write a book (CreativeCommons) and make it available to everyone. It would also be great to reach out to experts in certain areas and get them to write chapters/sections on things they have deep knowledge about. Here are some of the chapters I think people would find interesting.
- ICE and NAT Traversal (General P2P topics, deep dive on networking topics of programmers)
- DTLS and how it all actually works (TLS handshake and verifying the fingerprint)
- SCTP (sending data over lossy networks and handling back pressure)
- RTP over lossy networks (congestion control, NACK, FEC)
- RTCP (the power of having PLI/BWE vs TCP)
- QUIC (Talk about what it is trying to solve, congestion control across one protocol)
....
Available at pion/awesome-pion
- We need to start an
awesome-pionlist so projects that use Pion are easily discoverable. - We should explore collecting donations so we can redistribute them to support things like WebTorrent
Available at pion/interceptor
We should build a package that allows users to send/receive RTP over lossy networks. It should accept either raw media or RTP packets directly. It can have multiple inbound/outbound tracks, a RTPEngine can be shared across multiple PeerConnectionss so SSRC must not be used as a unique ID. This should live completely outside pion/webrtc so people can use it directly/it can be tested.
It will try to provide the best possible experience possible, but then will provide an API so users can tweak the following.
- Emit signals about suggested bitrate for congestion control.
- Emit signals when PLI/FIR is requested.
- Handle/Emit NACKs. Emit signals so the user can know how much loss is taking place
- Allow the user to choose what amount of loss they are willing to tolerate.
RTPEngineshould have zero loss or zero latency modes (and everything in between) - Easily generates graphs, it should have a
webrtc-internalslike experience so users can easily debug issues.