Participating in an Interoperability Test for WebRTC Protocol Stacks

Introduction

In WebRTC, besides libwebrtc (Chrome/FF/Safari/etc...), there are various implementations such as Pion, aiortc, and sipsorcery.
Basically, these implementations were developed with the goal of communicating with libwebrtc (essentially the browser), and interoperability between implementations other than libwebrtc was not highly prioritized.

In response to this, the author of sipsorcery called for interoperability testing between WebRTC implementations in the Discussion section of the Pion repository. WebRTC protocol stack implementers participated, and the interoperability tests were conducted.

The WebRTC implementations participating at this time are as follows:

	Language	Repository
aiortc	python	https://github.com/aiortc/aiortc
Pion	Go	https://github.com/pion/webrtc
sipsorcery	C#	https://github.com/sipsorcery-org/sipsorcery
werift	TypeScript (Node.js)	https://github.com/shinyoshiaki/werift-webrtc

I am the author of werift.

The interoperability tests are being conducted in the following repository:
https://github.com/sipsorcery/webrtc-echoes
Tests are automatically executed by GitHub Actions triggered by pushes to the master branch.
The current test results are as follows:

About the Interoperability Test

There is documentation regarding the specifications of the interoperability test, which I will introduce here.
https://github.com/sipsorcery/webrtc-echoes/blob/master/doc/EchoTestSpecification.md

Overview

The purpose of the WebRTC echo test is to verify whether a peer connection is established between two peers (Server Peer and Client Peer).

The "echo" in this test refers to the behavior where the Server Peer sends back the audio and video packets it receives to the Client Peer. This method is excellent because testing with a browser as the Client Peer can quickly provide a visual indication of test success.

Signaling

The only signaling operation performed in this test is an HTTP POST request from the Client Peer to the Server Peer. The Client sends an Offer SDP to the server, and the server returns an Answer SDP in the response.

Note: This single-shot signaling method is adopted to reduce signaling complexity. To use single-shot signaling, at least one of the Client Peer or Server Peer must operate in Vanilla ICE mode and include ICE candidates in the SDP. In practice, it is desirable for both peers to be configured to operate in Vanilla ICE mode.

Server Peer

The required flow for the Server Peer is as follows:

• Listen for HTTP POST requests on TCP port 8080. The URL where the HTTP server must listen for POST requests is:
  • http://*:8080/offer
• The body of the POST request from the Client contains a JSON-encoded RTCSessionDescriptionInit object.
• Upon receiving the Offer SDP, create a new "Peer Connection" and perform setRemoteDescription on the Offer.
• Generate an Answer SDP and return it as a JSON-encoded RTCSessionDescriptionInit object in the HTTP POST response.
• Perform the PeerConnection connection.
• Send the RTP media received by the PeerConnection back to the opposing Peer.

Client Peer

The required flow for the Client Peer is as follows:

• Create a new PeerConnection and generate an Offer SDP.
• Send the Offer SDP as a JSON-encoded RTCSessionDescriptionInit object to the Server Peer via an HTTP POST request.
  • http://localhost:8080/offer (adjust the address if the server is not on localhost)
• The HTTP POST response will be the Answer SDP as a JSON-encoded RTCSessionDescriptionInit object.
• Perform setRemoteDescription of the Answer SDP on the PeerConnection.
• Execute the initialization of the PeerConnection.
• Optionally, send audio or video to the server.
• If the Client determines that the PeerConnection connection was successful, terminate the connection and return exit code 0. If the connection fails or times out, the Client returns exit code 1.
  • (The criteria for connection success is not explicitly defined; depending on the implementation, it may be based on the success of the DTLS connection or the echoing of RTP).

Interoperability Test Programs

Since I am the author of werift, I will use the werift test program as an example.
Some parts of the sample code in the text have been omitted for readability.

Server Peer

https://github.com/sipsorcery/webrtc-echoes/blob/master/werift/server.ts

import express from "express";
import { RTCPeerConnection } from "werift";
import https from "https";

const app = express();
app.use(express.static("../html"));
app.use(express.json());

app.post("/offer", async (req, res) => {
  const offer = req.body;

  const pc = new RTCPeerConnection({
    iceConfig: { stunServer: ["stun.l.google.com", 19302] },
  });
  pc.onTransceiver.subscribe(async (transceiver) => {
    const [track] = await transceiver.onTrack.asPromise();
    track.onRtp.subscribe((rtp) => {
      transceiver.sendRtp(rtp);
    });
  });

  await pc.setRemoteDescription(offer);
  const answer = await pc.setLocalDescription(await pc.createAnswer());
  res.send(answer);
});

This code assumes that the Offer SDP contains a sendrecv m-line.
It simply sends back the incoming RTP as is.

Client Peer

https://github.com/sipsorcery/webrtc-echoes/blob/master/werift/client.ts

import { RTCPeerConnection, RtpPacket } from "werift";
import axios from "axios";
import { createSocket } from "dgram";
import { exec } from "child_process";

const url = process.argv[2] || "http://localhost:8080/offer";

// input RTP packet
const udp = createSocket("udp4");
udp.bind(5000);
exec(
  "gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,format=I420 ! vp8enc error-resilient=partitions keyframe-max-dist=10 auto-alt-ref=true cpu-used=5 deadline=1 ! rtpvp8pay ! udpsink host=127.0.0.1 port=5000"
);

new Promise<void>(async (r, f) => {
  setTimeout(() => {
    f(); // timeout, failed
  }, 30_000);

  const pc = new RTCPeerConnection({
    iceConfig: { stunServer: ["stun.l.google.com", 19302] },
  });
  const transceiver = pc.addTransceiver("video", "sendrecv");
  transceiver.onTrack.once((track) => {
    track.onRtp.subscribe((rtp) => {
      console.log(rtp.header);
      r(); // done, succeed
    });
  });

  await pc.setLocalDescription(await pc.createOffer());
  const { data } = await axios.post(url, pc.localDescription);
  pc.setRemoteDescription(data);

  await pc.connectionStateChange.watch((state) => state === "connected");
  udp.on("message", (data) => {
    const rtp = RtpPacket.deSerialize(data);
    rtp.header.payloadType = transceiver.codecs[0].payloadType;
    transceiver.sendRtp(rtp);
  });
})
  .then(() => {
    console.log("done");
    process.exit(0);
  })
  .catch((e) => {
    console.log("failed", e);
    process.exit(1);
  });

A sendrecv Transceiver is prepared to send and receive RTP.
If RTP is received from the ServerPeer, the connection is considered successful.
Since werift does not have features for media generation or encoding/decoding, it uses gStreamer to create the media, receives it via RTP over UDP, and then sends it to the Server Peer through werift.

Issues encountered during the interoperability test

Since werift had only undergone connection tests with Chrome and aiortc, I was not able to connect it to other WebRTC implementations in this interoperability test on the first try. As a result, werift itself required some fixes, and I will write about those details here.

sipsorcery

The flow of the issues that occurred with sipsorcery until they were resolved is documented in this issue. I will summarize the details here.

DTLS Elliptic Curve (RFC4492) negotiation issue

Initially, werift only supported x25519 among Elliptic Curves, but since sipsorcery did not support x25519, an issue occurred where DTLS communication was not possible.

To make matters a bit complicated, although werift only supported x25519, the OpenSSL specification requires supporting at least x25519 and P-256 among Elliptic Curves. Therefore, werift had included P-256 in its supported group of Elliptic Curves just to pass OpenSSL's validation, despite not actually supporting it. This caused some confusion for the author of sipsorcery while investigating the cause of this issue...

In any case, once the cause was identified, I implemented P-256 support in werift, and the issue was resolved.

DTLS extended master secret (RFC7627)

Initially, werift did not support the extended master secret, but since sipsorcery enforces its support, an issue occurred.
I implemented extended master secret support in werift, and this issue was resolved.

At this point, the combination of werift server x sipsorcery client successfully achieved interoperability!

DTLS renegotiation indication extension (RFC5746)

However, the combination of werift client x sipsorcery server was still failing at the DTLS connection stage. When I communicated the situation to the author of sipsorcery, they replied that a similar issue had occurred in the interoperability tests between Pion and sipsorcery. The cause was that Bouncy Castle DTLS, which sipsorcery uses as its DTLS implementation, enforces support for the renegotiation indication extension. Since Bouncy Castle DTLS is also used in Jitsi, implementing the renegotiation indication extension would have a significant impact on improving interoperability, so I decided to support it in werift as well.
The sipsorcery author's prediction was correct, and after implementing the renegotiation indication extension, the combination of werift client x sipsorcery server also succeeded in interoperating!!

Pion

werift's host ice candidate

Initially, werift had an issue with the implementation of obtaining the host ice candidate (local IP address), where it set the host ice candidate to 0.0.0.0 for IPv4 and :: for IPv6. This caused ICE communication with Pion to fail.
After fixing this issue in werift, interoperability with Pion was successful.

Conclusion

Overall, the most challenging part was ensuring DTLS compatibility with sipsorcery. DTLS (TLS) has various extensions, and WebRTC implementations differ in which ones they require. These types of issues are difficult to identify without interoperability testing, so I am glad I participated.

I am very grateful to the author of sipsorcery for automating the interoperability testing with GitHub Actions. Now, when I update the version of werift, I can automatically perform regression and interoperability tests simply by updating the werift version in the testing repository!

In the future, I expect tests for items not currently verified, such as Simulcast and IceRestart, to be added, and I intend to keep up with them as they are introduced!