WEBRTC


METRICS REPORT
2017/03

Usage metrics from real world WebRTC deployments.

A collection of WebRTC pre-call test metrics from September 2017 to

November 2017 brought to you by callstats.io, the realtime
communications monitoring platform.
 WEBRTC METRICS REPORT 2017/03

Greetings from Our CEO

Thank you for downloading callstats.io’s industry report on WebRTC
metrics. This is a special edition analyzing insights from the pre-call
test feature that we rolled out earlier this year.

In 2017, WebRTC moved to the candidate release state and several

browsers announced support for WebRTC v1.0, including Apple’s Safari
and Microsoft’s Edge. With WebRTC gaining maturity, we observe the
application ecosystem growing bigger and more interesting with excit-
ing use cases in e.g. IoT, VR/AR, automation, and conversational de-
vices.

This report is the fourth issue we have published in the last 12 months.
We have received an overwhelming positive response to our report se-
ries and in 2018 we are going to publish repost every four months. Un-
like our earlier reports, which primarily reported on WebRTC’s multi-
media pipeline, i.e., the performance of audio and video, this special
issue is mostly targeted on data channels, i.e., RTCDataChannels. In the
summer of 2017, we started running pre-call tests, i.e., each endpoint
would run a series of network tests before the call was initiated. These
tests are run over our TURN infrastructure, and depending on the type
of tests, they may hit the nearest geographical TURN server or explicit-
ly other TURN server.

In the last quarter, our install-base has continued to grow 20% month
on month, we have over 600 integrated products, and our world-wide
monitored traffic grew by 3-times. Since 2018 is around the corner, we
are making advancements on multiple fronts: improvements for the
dashboard are underway and AI/ML has made its way into more prod-
ucts, for example, in finding correlations in our data across customers,
end-users, and networks.

This 23-page report includes 21 figures that provide easy-to-digest in-

sights into the growing number of WebRTC deployments. We hope it
helps product managers and engineers build better real-time commu-
nication systems. This report answers to questions related to data
channels:

- Do data channels work the same way on every browser?

- Do the pre-call test network metrics correlate well with the confer-
ence that follows?

I hope you enjoy reading this report as much as we enjoyed writing it.

You can find the original page for the report at:

www.callstats.io/industry-reports/webrtc-metrics-report-2017-03/.

Regards,

Varun Singh


© 2017 CALLSTATS I/O Oy 1

 WEBRTC METRICS REPORT 2017/03

Table of contents
Executive Summary ...................................................3
TL;DR 3
1. The WebRTC Media Pipeline ................................4
1.1. WebRTC Pipeline Setup 4
1.2. Applying the WebRTC Statistics 5
2. WebRTC Data Channels ........................................6
2.1. Basics 6
2.2. Our Experiment 7
3. Endpoint Statistics for Pre-Call Test ...................9
3.1. Operating System Distribution 9
3.2. Browser Distribution 9
3.3. Google Chrome (Chromium) 10
3.4. Mozilla Firefox 11
4. Pre-Call Test Results ........................................12
4.1. Bulk Data Transfer Test 12
4.2. Ping Test 14
4.3. Set Timeout Woes 16
5. Conference Comparison ..................................17
5.1. Comparing Throughput 17
5.2. Comparing Average RTT 19

© 2017 CALLSTATS I/O Oy 2

 WEBRTC METRICS REPORT 2017/03

Executive Summary
Web Realtime Communications (WebRTC) is a technology that enables
audio and video calls in the web browser without plugins. WebRTC is
part of the HTML5 standard, which is defined at the World Wide Web
Consortium (W3C).

WebRTC has a statistics API, which can be used to collect data from the
peer connection between two participants. This data used in this re-
port are anonymized WebRTC metrics collected by the callstats.io ser-
vice.

This report discusses metrics sampled across all apps and services us-
ing callstats.io. The metrics include statistics from the data channels
pre-call test and a comparison to the session that followed the tests.

TL;DR
- Chrome still dominates the WebRTC browser market. Likewise, the
operating system distributions remain the same, with Windows lead-
ing.

- 20% of the pre-call tests have an average throughput of 5000 kbps or

more.

- 80% of the pre-call tests have an average throughput above 500

kbps.

- 50% of the pre-call tests do not report any losses.

- In the last 3 months, Firefox and Chrome data channel performance

has improved.

- 90% of connections have an average RTT of below 100ms which is

good for real time communication.

- 10% of the monitored conference calls have an average throughput

of higher than 1000 kbps compared to 60% for the pre-call test.

- 20% of the cases the media session has a higher throughput than the
pre-call test.

- Pre-call test estimates a lower RTT in the case of TURN than the
peer-to-peer.

- 65% of the cases pre-call test for TURN reported lower RTT to the
third-party TURN server used by the conference.


© 2017 CALLSTATS I/O Oy 3

 WEBRTC METRICS REPORT 2017/03

1. The WebRTC Media Pipeline

The metrics in this report are gathered via the WebRTC Statistics API,
which corresponds to individual components across the media pipe-
line. The following section discusses the structure of the media pipe-
line and the metrics associated with the individual component.

1.1. WebRTC Pipeline Setup

The WebRTC getStats() API exposes metrics from various components
within the media pipeline (at the sending and receiving endpoint). Fig-
ure 1.1. shows an example media pipeline. Media is captured at regular
intervals by a device (microphone, camera, or via screen capture). The
raw media frame is then compressed by an encoder and further packe-
tized into MTUs (maximum transmission unit ~1450 bytes) before the
packets are sent over the Internet.

When receiving these packets, the endpoint concatenates them into

frames, complete frames are then sent to the decoder, and finally ren-
dered. Some implementations may discard incomplete frames, or use
concealment mechanisms to hide the missing parts of the frame.
These may have varying impact on the quality of experience.

FIGURE 1.1. THE WEBRTC PIPELINE.

The getStats() API is structured as follows:

1. Sender media capture statistics: corresponds to the media genera-

tion, typically frame rate, frame size, clock rate of the media
source, the name of the codec, etc.

2. Sender RTP statistics: corresponds to the media sender, typically

packets sent, bytes sent, Round Trip Time (RTT), etc.

© 2017 CALLSTATS I/O Oy 4

 WEBRTC METRICS REPORT 2017/03

3. Receiver RTP statistics: corresponds to the media receiver, typically

packets received, bytes received, packets discarded, packets lost,
jitter, etc

4. Receiver media render statistics: corresponds to the media render-

ing, typically frames lost, frames discarded, frames rendered, play-
out delay, etc.

5. Network-related metrics: corresponds to the bytes, packets, sent or

received on that particular interface. Additionally, it may report the
connectivity requests and responses.

1.2. Applying the WebRTC Statistics

When running a WebRTC service, the metrics exposed by the getStats()
API are important for diagnosing issues within a particular session, i.e.
the link between two participants. However, to assess the quality of
the overall service, the data from each session in a conference call
needs to be summarized. Furthermore, the summarized data needs to
be aggregated over period of hours and days to observe and predict
the trends in the service usage.

We get a more comprehensive diagnosis when metrics are collected at

not only at the endpoints, but at various points in the path, such as
media servers, TURN relay servers, and media routers.


© 2017 CALLSTATS I/O Oy 5

 WEBRTC METRICS REPORT 2017/03

2. WebRTC Data Channels

WebRTC is most commonly used to establish audio or video calls be-
tween participants, however, it has the capability to send data as well.
WebRTC data channels are designed for sending and receiving text or
binary messages, which can be used, for example, in chat messages,
object positions in gaming/VR, and distributed file transfer.

2.1. Basics
Transmitting data in WebRTC is pretty simple: It is possible to use the
same transport candidate pair (IceTransport) as audio or as video.
Since the 5-tuple, the source and destination IP address and port is
the same for all WebRTC packets, the connection path does not neces-
sarily change.

While the transmission protocol for audio and video is the Real-Time
Transport Protocol (RTP), data channels currently use the Stream Con-
trol Transmission Protocol (SCTP) over Datagram Transport Layer Secu-
rity (DTLS). Both media and data is multiplexed over UDP. The choice to
use SCTP was made due to the fact that the sending data does not
have strong latency or round-trip time (RTT) requirements as media,
and SCTP is designed for data transfer. This implies that SCTP has simi-
lar acknowledgement, flow control, and congestion control as the
Transmission Control Protocol (TCP). Furthermore, one big advantage
of SCTP is that it supports multiple simultaneous-streams, which re-
duces the impact of Head-of-Line (HoL) blocking.

In addition, SCTP supports a variety of options which makes it a valid

choice for many applications. For example, a WebRTC developer can
choose if the data should be transmitted reliably or not, and if in-or-
der delivery is necessary or not. When transmitting a file, developers
aim at reliable file transmission, thus they likely have both options en-
abled (reliable and in-order delivery). On the other hand, for gaming
applications, it is more important to deliver a message with minimum
latency or RTT, tolerating some minor message losses, thus it might be
a better to have both these options disabled (unreliable and out-of-
order delivery).

From an implementation point of view, RTCDataChannels are pretty

straightforward to use, as they just have to be added to the RTCPeer-
Connection. After that, any message that is provided to the data chan-
nel arrives on the remote side. All complexity of fragmenting and re-
assembly is hidden by the underlying browser implementation.

Data channels are currently used for several non-media applications.

The typical use-case tends to build a distributed network of nodes and
then use these nodes to deliver data or content. This is similar in idea
to BitTorrent. Below are some examples:

- Fetch video from multiple peers. For example, Streamroot, Peer5,

© 2017 CALLSTATS I/O Oy 6

 WEBRTC METRICS REPORT 2017/03

and Hivestreaming.

- This could be extended to fetching content downloaded from multi-

ple peers, for example, Web Torrent, Greta (webpages or content),
Instant (file), and Sharedrop (file).

- ToR-like routing network, i.e., building an Application Layer Distribu-

tion tree with WebRTC nodes to route media or data traffic between
users in the network. An example: Zyptonite.

- Lastly, opportunistic communication between Internet-of-Things de-

vices, e.g. Pipe.

2.2. Our Experiment

The main aim for our pre-call test experiment is to analyze the users’
Internet connection before they get on to a call, and compare the per-
formance of the pre-call test with the performance of the network dur-
ing the call.

The basic set up consists of creating a loopback data channel via a

TURN server. We enforce the usage of a TURN server to ensure that we
measure the connectivity to the Internet and not just the performance
of the local interface. The loopback call sends data from one peer
connection to another and measures various network performance
metrics. For example, ping to measure the RTT, bulk data transfer to
measure throughput and packet losses.

In this report, we present results for the following tests:

- Ping tests that measure the round-trip time (RTT).

- Bulk data transfer tests that measure throughput and packet loss.
RTT and throughput tests are performed in sequence to not interfere
with each other. Apart from the tests, callstats.io also records the type
of ICE candidate pair, i.e., the type of connection to the TURN server.

The bulk data transfer test measures the bits sent and received at
every 100ms interval. The endpoint sends as much data as possible
through the data channel. The aim is to measure the highest through-
put the user’s Internet connection is capable of. From an implementa-
tion point of view, it is important that the buffer of the data channel is
never empty, otherwise the connection is application limited. General-
ly, with the use of data channels, we have only limited control over the
sending behavior and the transport layers underneath, for example,
SCTP, UDP/TCP, and browser scheduling. Hence, it is hard to determine
exactly where the limiting factor is. But, as we will see later in the re-
port, generally data channels are performing good in terms of deter-
mining the quality of a call with WebRTC.

The ping test sends small messages one by one over the data channel.

© 2017 CALLSTATS I/O Oy 7

 WEBRTC METRICS REPORT 2017/03

The aim is to test the lowest possible RTT, which is only achieved if the
packets are not queued in the network. For each of these, the RTT is
measured as the time between sending and receiving. A new packet is
sent when the previous has been received, or every 100ms if round-
trip time is higher.

In total, we ran millions of pre-call tests during September to No-

vember 2017 and this report is a summary of observations. In many
cases the tests were aborted because there were not sufficient time to
run the tests before the connectivity checks began, for example, when
the call starts immediately on page-load.

We will be introducing a new API shortly which will allow the app de-
veloper to delay the call set up until the pre-call is finished.


© 2017 CALLSTATS I/O Oy 8

 WEBRTC METRICS REPORT 2017/03

3. Endpoint Statistics for Pre-Call Test

As WebRTC is a part of the HTML5 standard, it is supported by various
platforms and browsers. Currently, WebRTC is supported by Chrome
(and variants, like Opera), Firefox, Edge, and Safari. Safari’s WebRTC
support is available in Safari 11.0 and later. In the following sections,
we discuss the operating system and browser distribution metrics for
the pre-call test, and deep dive into Chrome and Firefox usage in-
sights. In later sections, this information is insightful when we com-
pare the pre-call test results across browser versions.

3.1. Operating System Distribution

Figure 3.1 shows the distribution of the major operating systems, where
the dominant platform is Windows with 44% of the successfully per-
formed pre-call test. Then, in descending order of the successfully per-
formed pre-call tests across platforms are:

- Mac OS X with 28%

- Android with 18%
- Linux OS with 10%

This distribution is roughly comparable to the results observed in our

previous metrics reports. This means from an OS perspective, we get a
similar user distribution for pre-call test as we did for multimedia ses-
sions.

Windows Mac OS X Android Linux

10 %
18 %
44 %

28 %

FIGURE 3.1. OPERATING SYSTEM DISTRIBUTION.

3.2. Browser Distribution

Each browser has independently implemented the data channel proto-
col atop the fairly ubiquitous Linux native SCTP library (libsctp), al-
though each browser uses different versions of libsctp.

In this section, we show the distribution for browsers among the pre-
call test sessions. Browsers update regularly, with a new stable version

© 2017 CALLSTATS I/O Oy 9

 WEBRTC METRICS REPORT 2017/03

released every six weeks. Safari added support for WebRTC in devel-
oper preview in June 2017 and stable release with iOS 11 and macOS
High Sierra (10.13.x). Apps are still switching to Safari in production,
especially since Safari does not support VP8 and requires universal
support for the H.264 video codec to interoperate with other browsers.
This is visible from our distribution, as the share is still really low (less
than 1%).

Figure 3.2 shows that the leading the WebRTC browser use by a large
margin is Chrome and its variants with a 62% share, followed by Native
apps (33%) and Mozilla Firefox with about 5%.

Chrome Native Firefox

5 %

33 %
62 %

FIGURE 3.2. BROWSER DISTRIBUTION.

3.3. Google Chrome (Chromium)

In the time frame that the tests were run, two stable versions were re-
leased: M61 in early September, M62 in mid October. In the figure, we
see the distribution among these and other browser versions. The top
five are listed separately, the rest are bundled together in the “Other”

62 61 60 Other

13 % 19 %

20 %

48 %

FIGURE 3.3. CHROME VERSION DISTRIBUTION.

© 2017 CALLSTATS I/O Oy 10

 WEBRTC METRICS REPORT 2017/03

category. Figure 3.3 shows, the newest Chrome version already has a
share of 19%, but the previous version is still dominant with 48%. Oth-
er versions are also still present and make up a total of a third. Thus
we can conclude that amongst callstats' data channel users, Chrome
browsers are generally well up to date.

3.4. Mozilla Firefox

Since September, Mozilla released two stable Firefox browser versions,
FF56 at the end of September and FF57 in mid-November. While FF57
was only released during the last 10 days of our tests, it already shows
a share of 13% in Figure 3.4. This is likely due to Mozilla’s promise that
this version would be a heavy rewrite of the browser which increases
the performance of the overall browser significantly (not necessarily of
the WebRTC parts). Still, the biggest part (48%) is taken by the previous
version FF56, and 24% by FF55. Other versions have a share of 15%.


57 56 55 Other

15 % 13 %

24 %
48 %

FIGURE 3.4. FIREFOX VERSION DISTRIBUTION.

© 2017 CALLSTATS I/O Oy 11

 WEBRTC METRICS REPORT 2017/03

4. Pre-Call Test Results

In this section, we share summary statistics of the performed tests.
First, we start with looking at the throughput tests. Bulk data transfer
tests measure the available throughput over the data channel. In addi-
tion to the throughput, we are also able to measure the total packet
loss, fractional packet loss per interval, and burst packet losses. In a
separate ping test, we are able to measure the round-trip time (RTT).

The results are shown in cumulative distributions as an aggregate of

the whole dataset, or categorized by platform, or by browsers. Further,
we also distinguish the situations when we can see the TURN’s public
IP address or some local IP address. The ratio is 76% to 24%, i.e., 1 in 4
calls, we observe no IP address or a local IP address.

4.1. Bulk Data Transfer Test

The bulk data transfer test measures the goodput, i.e., the actual bits
received by the endpoint at 100ms intervals. At the end of session, it
records the average throughput for each pre-call test session. Since
the test sends a large stream of data, we can observe packet loss and
characterize the loss pattern. Figure 4.1 shows the average throughput
distribution for the bulk data transfer test. We observe that 20% of the
tests have a throughput of 5000 kbps or more, however, at least 80% of
tests show an average throughput above 500 kbps.

FIGURE 4.1. CDF OF THROUGHPUT FOR BULK DATA TRANSFER.

In addition to measuring the throughput, the bulk data transfer test

also measures packet loss during the throughput test. The data chan-
nel for the pre-call test is set to unreliable mode, so that the test is
not hampered by SCTP’s retransmissions. Figure 4.2 (in the next page)
shows the distribution for losses. In 50% of all cases, tests do not re-
port any losses. Alternatively, we also observe connections where the

© 2017 CALLSTATS I/O Oy 12

 WEBRTC METRICS REPORT 2017/03

amount of data loss is significantly high to have a audio/video call.

FIGURE 4.2. CDF OF FRACTIONAL LOSSES.

From our data, we can also compare the data channels performance
across browsers and versions. Figure 4.3 shows the comparison of per-
formance across browsers. It is noticeable that the throughput of
Chrome is generally a bit lower than that for Firefox or even Native ap-
plications. Above 5000 kbps is reached for 20% of Chrome measure-
ments, but for almost 30% of other shown browsers. All browsers use
the same library for libsctp, however, different version of it, which may
explain the performance difference.

FIGURE 4.3. CDF OF AVEREAGE THROUGHPUT ACROSS

BROWSERS.

© 2017 CALLSTATS I/O Oy 13

 WEBRTC METRICS REPORT 2017/03

When comparing performance between different versions in Figure 4.4,

we notice that for Firefox, the performance has improved since version
54. The performance of versions 55, 56 and 57 are in general 10% bet-
ter, likely due to the use of newer SCTP implementations.

FIGURE 4.4. CDF OF AVERAGE THROUGHPUT FOR FIREFOX.

In Figure 4.5 for Chrome, we also see a better throughput over data
channels with all of the newer versions.

FIGURE 4.5. CDF OF AVERAGE THROUGHPUT FOR CHROME.

4.2. Ping Test

The ping test sends small messages one by one over the data channel.
The difference between the receive and send timestamp for a particu-
lar packet is the RTT. The aim is to measure the lowest possible RTT,
which can only achieved if the no packets are queued in along the

© 2017 CALLSTATS I/O Oy 14

 WEBRTC METRICS REPORT 2017/03

end-to-end path. Since, we send 10 packets, the RTT is measured for

every packet and the lowest, highest, the average, and variance
recorded.

Figure 4.6 shows the distribution between the two connection types:
the performance is poorer for situations the recorded remote IP is a
local address or null. General observation is that 90% of connections
have an RTT of below 100ms which enables good real time communica-
tion. For users with a private remote IP, the situation looks different:
they have often really high RTTs. For a part of these connections real
time communication is not possible.

FIGURE 4.6. CDF FOR RTT (PING TEST).

When sending the separate ping packets, they might get lost as well.
The case is different from the losses in the bulk download test: in this
case, the endpoint sends a single packet and any packet loss is very
likely due to congestion and the congestion is not due to self-conges-
tion of the path. So the losses on this path are due to cross-traffic or
unreliability in the operator handling UDP traffic. The unreliability
might be due to various reasons, e.g., TCP cross traffic sharing the
same queue as the TCP.

Figure 4.7 (in the next page) shows that in 38% of cases at least one
packet out of the 10 is dropped, but very rarely more than four. The
percentage for the connections with private remote IP is again much
higher.


© 2017 CALLSTATS I/O Oy 15

 WEBRTC METRICS REPORT 2017/03

FIGURE 4.7. CDF OF UNACKNOWLEDGED PINGS.

4.3. Set Timeout Woes

One thing that implementers have to be aware of, is that the usage of
setTimeout function to achieve scheduling has to be used with care.
The browser will schedule this event, but it will only be executed if re-
sources allow it. This can cause severe delays as can be seen from our
Figure 4.8 below. The interval in which the timeout should be called by
the browser during the throughput test is 100ms. It normally is much
higher.

FIGURE 4.8. CDF OF SET TIMEOUT INTERVALS.

© 2017 CALLSTATS I/O Oy 16

 WEBRTC METRICS REPORT 2017/03

5. Conference Comparison
So far in this report, we analyzed the results of the data channel tests
in isolation. However, these tests are run before the conference hap-
pens, typically when the page is loaded or the app is opened. However,
we can compare the pre-call test result to what the user experienced
during a conference call that follows the pre-call test. In 25% of the
sessions, the conference call immediately followed the call. In some
chat applications, we have the opportunity to run the pre-call test pe-
riodically in the background, which gives us a historical view of the
connection as well.

The following results show how similar a data channel pre-call test is
to the conference data. Split by connection type, this shows that 69%
of these conferences where performed over peer-to-peer traffic, 9%
used TURN, and 22% used a Bridge. We will split our graphs by this
separation to get more insight into differences. To remind: in the bulk
of the scenarios, we abort the pre-call test, as soon as callstats.io ob-
serves the ICE connection state moving to “checking”.

5.1. Comparing Throughput

Figure 5.1 shows the distributions of average throughput aggregated
across sessions with both pre-call test and conference data. In general,
only 10% of the monitored conference calls have a throughput of high-
er than 1000 kbps. This is either limited by the bandwidth estimation
of the congestion control, or that the session does not need to send at
a higher bit rate. In contrast, we observe that generally the pre-call
tests has a higher throughput, with 60% of the sessions achieving a
data rate of more than 1000 kbps.

FIGURE 5.1. CDF OF AVERAGE THROUGHPUT.

© 2017 CALLSTATS I/O Oy 17

 WEBRTC METRICS REPORT 2017/03

Figure 5.2 shows the difference between the average throughput mea-
sured by the pre-call test and the conference session. In 10% of the
cases, the pre-call test measures a lower throughput (this may be due
to the set timeout woes discussed in section 3.3).

FIGURE 5.2. CDF OF AVERAGE THROUGHPUT DIFFERENCE.

In the last WebRTC report, we compared the network performance of

the sessions based on connection type, i.e., sessions that were P2P,
Bridge, or TURN, and there were significant difference. Thus, we are re-
evaluating connection types with pre-call test as well. Figure 5.3 shows
the distribution of average throughput for each connection type, i.e.,
compares the performance of the pre-call test for a session with P2P,
bridge, and a TURN. For the endpoint in the conference, we find that
there definitely is a difference: P2P connections are mostly around 800
kbps, only 20% of traffic is either lower or higher. TURN and Bridge

FIGURE 5.3. CDF OF AVG. THROUGHPUT PER CONNECTION TYPE.

© 2017 CALLSTATS I/O Oy 18

 WEBRTC METRICS REPORT 2017/03

connections seem to have a higher percentage of low-throughput con-

nections, but also higher total throughput. So far, we have only seen a
total comparison, but it does not show how well the two values for a
particular session coincide.

The Figure 5.4 shows the difference. As can be seen, in about 80% of
the cases, the pre-call test session achieves a higher throughput than
the corresponding conference. This is expected, as the test is designed
to use all the available Internet connection, while the conference call
potentially only uses what is necessary to transmit the media. The re-
maining 20%, where the test has a lower throughput, can have several
reasons: the browser might be too busy to send data through the data
channel fast enough; the Internet connection is currently used by oth-
er activities of the user, or the network situation of the user has
changed for the worse between test and call.

FIGURE 5.4. CDF OF AVERAGE THROUGHPUT DIFFERENCE PER

CONNECTION TYPE.

The result for a split by connection type shows that the pre-call test is
achieving a higher result than the conference more often if the confer-
ence is using P2P.

5.2. Comparing Average RTT

Analyzing the average RTT between pre-call test and conference is
more difficult than it is to compare the throughput, as RTT heavily de-
pends on the length of the path through the Internet. If the conference
users are located near each other, the RTT is lower than when they are
placed across continents. And if the users use infrastructure like TURN
or Bridge conference servers, the traffic is routed through these
servers. Hence, the location of the infrastructure is of importance
when comparing round-trip time. Due to these reasons, in this section,
we are analyzing only those data points that performed pre-call tests

© 2017 CALLSTATS I/O Oy 19

 WEBRTC METRICS REPORT 2017/03

to the same callstats.io TURN server.

We may revisit this in a future report by comparing the aggregate RTT

to the respective TURN servers and the RTT between the TURN servers
and compare the aggregate RTT with the session RTT. This ensures that
the endpoints are geographically located near each other, and the
data is not polluted significantly by the distance between users. This
narrows the data set for this section to only 10% of the complete
dataset, of which 70% are P2P and 30% are TURN connections. There is
no RTT data for Bridge conference connection with these filters.

FIGURE 5.5. CDF OF AVERAGE RTT.

Figure 5.5 shows the distribution of RTTs for the pre-call test and con-
ference data. We observe that the TURN-connection based pre-call test
has generally a higher RTT than the conferences.

FIGURE 5.6. CDF OF AVERAGE RTT DIFFERENCE.

© 2017 CALLSTATS I/O Oy 20

 WEBRTC METRICS REPORT 2017/03

In Figure 5.6 (in the previous page), we observe that in 70% of the cas-
es, the pre-call test reports a higher RTT value, and in 30% cases a val-
ue lower than the RTT between the endpoints.

Figure 5.7 shows the split by connection type (P2P or TURN). We ob-
serve that the RTT’s reported in the pre-call tests for the two groups
are fairly similar. Similarly, comparing the RTT for the P2P conferences
and TURN conferences (using the same TURN server, not callstats.io's)
we see a clear distinction. Moreover, we observe that the P2P confer-
ence RTT is typically lower to the RTR measured by the pre-call test.
This usually means that the users were in fact closer than our TURN
servers.

FIGURE 5.7. CDF OF AVERAGE RTT PER CONNECTION TYPE.

Surprisingly, we observed that in 70% of the cases the third-party

TURN servers used by the endpoints during the call were further than
our pre-call test TURN servers.

To summarize the additional observations

- The 95%-le RTT reported by the P2P conferences is lower than 100ms
in about 90% of cases.
- For TURN, this threshold holds true only for 40% of users.
- In 20% of cases, TURN connections even have an RTT of higher than
300ms.

Figure 5.8 (in the next page) clearly shows the separation by connec-
tion type for RTT measured during the pre-call test, and the RTT re-
ported by the endpoint in the conference. For P2P connections, we ob-
serve that the pre-call test RTT is higher in as many as 25% of the cas-
es. It is comparable (within 20ms) in about 70% of the cases, and lower
in about 5% of the cases. This was expected, as we filtered for users
that connected to the same callstats.io TURN server and should be ge-

© 2017 CALLSTATS I/O Oy 21

 WEBRTC METRICS REPORT 2017/03

ographically relatively nearby, and did not use a TURN server to estab-
lish the connection. To summarize, when comparing TURN conference
connections: in 65% of the cases the pre-call test is lower than or
equal to the average conference RTT.

FIGURE 5.8. CDF OF AVERAGE RTT DIFFERENCE PER CONNECTION

TYPE.

The analysis in this report shows that pre-call tests ran over the data
channel can be used to measure and predict the performance of the
multimedia session that follows the test.


© 2017 CALLSTATS I/O Oy 22

 WEBRTC METRICS REPORT 2017/03

Editors
Varun Singh is the CEO, Founder of callstats.io, contributor to several
standards at the IETF and W3C. He has over 15 years of industry and
research experience, and a Ph.D and M.Sc in Multimedia, Networking,
and Network Economics from Aalto University.

Lennart Schulte is a Software engineer at callstats.io, and a Ph.D can-

didate at the School of Electrical Engineering at Aalto University. He
received his diploma degree in computer science from RWTH Aachen,
Germany.

Marcin Nagy is the VP of Engineering and Co-Founder of callstats.io,

and a Ph.D candidate at the School of Electrical Engineering at Aalto
University. He has 9 years of industry and research experience in mul-
timedia, network security and privacy and an M.Sc from Aalto Universi-
ty.

Lasse Lumiaho is the Marketing lead at callstats.io, with 8 years of in-

dustry experience in product management and marketing. He received
his M.Sc in User-centered Product Development from Aalto University.

Copyright
© 2017 CALLSTATS I/O Oy, Made by hand in Finland.

All trademarks, service marks, trade names, trade dress, product

names and logos appearing on the site are the property of their re-
spective owners, including in some instances CALLSTATS I/O Oy.

© 2017 CALLSTATS I/O Oy 23