|
TECHNICAL REPORT
Speech and multimedia Transmission Quality (STQ);
QoS aspects of TCP-based video services like YouTube™
---------------------- Page: 1 ----------------------
2 ETSI TR 101 578 V1.3.1 (2018-10)
Reference
RTR/STQ-00220m
Keywords
measurement, QoS, service, TCP-based video
services
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
The present document can be downloaded from:
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
If you find errors in the present document, please send your comment to one of the following services:
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.
© ETSI 2018.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.
TM TM
3GPP and LTE are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M logo is protected for the benefit of its Members.
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI
---------------------- Page: 2 ----------------------
3 ETSI TR 101 578 V1.3.1 (2018-10)
Contents
Intellectual Property Rights . 7
Foreword . 7
Modal verbs terminology . 7
Introduction . 7
1 Scope . 9
2 References . 9
2.1 Normative references . 9
2.2 Informative references . 9
3 Abbreviations . 9
4 Quality of Service measurements for IP-based video services like YouTube™ . 10
4.0 General . 10
4.1 Phases of IP-based video services . 10
4.2 QoS aspects of IP-based video services . 12
4.2.0 Scope of aspects . 12
4.2.1 Video start and time to first picture . 12
4.2.2 Video freezes . 12
4.2.3 Adaptive videostreaming . 12
4.2.4 Perceived video quality . 13
4.3 QoS parameters for IP-based video services . 13
4.3.0 Parameter and trigger points . 13
4.3.1 Video Access Failure Ratio [%] . 14
4.3.2 Video Access Time [s] . 15
4.3.3 Void . 15
4.3.4 Void . 15
4.3.5 Void . 15
4.3.6 Void . 15
4.3.7 Void . 15
4.3.8 Void . 15
4.3.9 Void . 15
4.3.10 Void . 15
4.3.11 Void . 15
4.3.12 Void . 15
4.3.13 Void . 15
4.3.14 Void . 15
4.3.15 Void . 15
4.3.16 Void . 15
4.3.17 Impairment Free Video Session Ratio [%] . 15
4.3.18 Void . 16
4.3.19 Void . 16
4.3.20 Void . 16
4.3.21 Void . 16
4.3.22 Void . 16
4.3.23 Impairment Free Video Session Ratio [%] . 16
4.3.24 Video Playout Cut-off Ratio [%] . 16
4.3.25 Void . 16
4.3.26 Void . 16
4.3.27 Video Playout Duration [s] . 16
4.3.28 Void . 16
4.3.29 Accumulated Video Freezing Duration [s] . 16
4.3.30 Void . 17
4.3.31 Void . 17
4.3.32 Void . 17
4.3.33 Void . 17
4.3.34 Video Freezing Time Proportion . 17
ETSI
---------------------- Page: 3 ----------------------
4 ETSI TR 101 578 V1.3.1 (2018-10)
4.3.35 Video Quality . 17
4.4 Recommended supplementary information for IP-based video service measurements . 17
4.4.0 Introduction. 17
4.4.1 Video Preparation Failure Ratio [%] . 18
4.4.2 Video Preparation Time [s] . 18
4.4.3 Pre-Playout Buffering Failure Ratio [%] . 18
4.4.4 Pre-Playout Buffering Time [s] . 18
4.5 Configuration aspects including timeout recommendations for IP-based video service measurements . 19
4.5.0 Purpose . 19
4.5.1 URL . 19
4.5.2 Timeouts . 19
4.5.2.0 Application of timeouts . 19
4.5.2.1 Video Access timeout . 19
4.5.3 Video Playout Duration . 19
4.5.4 Handling of video freezes . 20
4.5.4.0 Use of freezes . 20
4.5.4.1 Minimum freeze duration . 20
4.5.4.2 Maximum duration of single freeze . 20
4.5.4.3 Maximum duration of all freezes . 20
4.5.4.4 Maximum number of freezes . 20
4.5.5 Timeout and Threshold Frameworks . 20
4.5.6 Hide video during playout . 21
4.5.7 Play until the end . 21
4.5.8 Cache and cookies . 21
4.5.9 Video Resolution characteristics of the clip test . 21
4.6 Impacts of measurement hardware for IP-based video service measurements . 21
Annex A: Void . 23
History . 24
ETSI
---------------------- Page: 4 ----------------------
5 ETSI TR 101 578 V1.3.1 (2018-10)
List of figures
Figure 1: Typical phases of IP-based video services .11
ETSI
---------------------- Page: 5 ----------------------
6 ETSI TR 101 578 V1.3.1 (2018-10)
List of tables
Table 1: Overview of QoS parameters and mapping to typical phases of the video services as experienced by the user.13
Table 2: Overview of the trigger points used for the QoS parameter definition .14
Table 3: Example settings that do model a standard user .21
ETSI
---------------------- Page: 6 ----------------------
7 ETSI TR 101 578 V1.3.1 (2018-10)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This Technical Report (TR) has been produced by ETSI Technical Committee Speech and multimedia Transmission
Quality (STQ).
Modal verbs terminology
In the present document "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be
interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
There are a variety of popular IP-based video services available on the internet, on which users can view, upload and
share videos. These services have become very popular and have a major share of the internet traffic worldwide. Due to
their high popularity in general and use over mobile internet their availability and quality is of key interest for the
provider of mobile internet access, which makes the services a matter for benchmarking. The down-stream scenario, the
probability to access and see a desired video and the quality of the video is the subject of measurement method laid out
in the present document.
Any video content is accessed via a link that is provided by the service on request. This request can be triggered by
selecting a video on a web-page, by selecting a video in a smartphone application or – if the URL is known – by direct
access of a video player with streaming capabilities. A popular example for a video streaming service is YouTube™.
NOTE: YouTube™ is the trade name of a product supplied by Google. This information is given for the
convenience of users of the present document and does not constitute an endorsement by ETSI of the
product named. Equivalent products may be used if they can be shown to lead to the same results.
Today's video streaming services are mainly based on reliable transmission. It is often TCP, but e.g. YouTube™ applies
a proprietary protocol named QUIC. This protocol is based on UDP but secures transmission at a higher layer.
ETSI
---------------------- Page: 7 ----------------------
8 ETSI TR 101 578 V1.3.1 (2018-10)
The source video, either uploaded by a user or provided e.g. by a broadcasting station or live stream is usually in high
quality in high resolution. Typically, the receiving video server re-processes the video, add streaming information and is
usually transcoding it to meet its coding schemes and data rate classes. In practice these videos are transcoded in
different resolutions and stored for down-streaming by the video server.
State of the art video streaming services do not downstream the entire video in one pre-defined resolution (or bitrate),
they adjust the amount of data to transport at the available channel capacity or restrictions given by the operators
(called: adaptive bitrate). To adjust the amount of data the most efficient strategy is to change the image resolution.
Other strategies are decreasing encoding depth or reducing the picture rate (frame-rate) of the video. It is obvious that
the applied compression affects the perceived video quality, the degree of degradation is depending on compression and
the strategy of compression.
Typically, the resolution is not changed continuously. It is usually switching between fixed resolutions as e.g. 240, 360,
480, 720 and 1 080 lines. The most common schemes for adaptive bitrate are DASH and HLS, where the video is
requested in sub-sequent portions of a few seconds in a defined resolution.
On the other hand the clips not need to come physically from the same server since mobile operators employ proxies in
order to move the content closer to their subscriber and the downlink bandwidth could be controlled by both the mobile
operator network and the video service. Therefore the clips need to be streamed from the actual live network and may
not be streamed from a dedicated server.
It should be considered that the rendering of the video and finally the quality of its reproduction depends on the
buffering and decoding strategy of the player, as well as on the operating system and available system resources.
ETSI
---------------------- Page: 8 ----------------------
9 ETSI TR 101 578 V1.3.1 (2018-10)
1 Scope
The present document focuses on Quality of Service (QoS) measurements for IP-based video services with reliable
transport where downloading and viewing takes place in parallel. In principle the presented measurement approach can
be used for all video services, where the video is embedded in a HTML context as of video on demand services like
e.g. YouTube™. Similar applications are also available on social networks.
In the following, QoS parameters to be used for such video service measurements are presented. The underlying
procedure consists of two phases: first requesting a control script containing among other information a link to the
content, and second, requesting this content. In the present document, YouTube™ serves as the default example but the
described QoS parameters can easily be applied to other IP-based video services based on reliable transport.
Furthermore, the present document also offers practical guidance for measurement execution and evaluation of
HTTP/HTTPS streaming QoS measurement.
The present document covers the video request and playout of the video. Other services offered by content providers
such as e.g. uploading video or managing the private account are not covered.
2 References
2.1 Normative references
Normative references are not applicable in the present document.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI TS 102 250-2: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 2: Definition of Quality of Service parameters and their
computation".
[i.2] ETSI TS 102 250-5: "Speech and multimedia Transmission Quality (STQ); QoS aspects for
popular services in mobile networks; Part 5: Definition of typical measurement profiles".
3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
CPU Central Processing Unit
DASH Dynamic Adaptive Streaming over HTTP
DNS Domain Name System
FLV Flash Video
FTP File Transfer Protocol
GPU Graphics Processing Unit
HDD Hard Disk Drive
HLS HTTP Live Streaming
HTML HyperText Markup Language
HTTP HyperText Transfer Protocol
ETSI
---------------------- Page: 9 ----------------------
10 ETSI TR 101 578 V1.3.1 (2018-10)
HTTPS HTTP Secure
IP Internet Protocol
LAN Local Area Network
NDIS Network Driver Interface Specification
OS Operating System
PC Personal Computer
PEC Performance Enhancement Client
QoS Quality of Service
QUIC Quick UDP Internet Connection
RTP Real-time Transport Protocol
RTSP Real Time Streaming Protocol
SYN TCP synchronize flag
TCP Transmission Control Protocol
UDP User Datagram Protocol
URL Uniform Resource Locator
WLAN Wireless Local Area Network
4 Quality of Service measurements for IP-based video
services like YouTube™
4.0 General
Many video services offer the videos in several resolutions and allows the viewer to select the resolution quality
manually. However, most services apply an adaptive mode (called 'automatic' for YouTube™), where based on the
transport channel performances or other information the chosen resolution (bitrate) and the Pre-Playout buffering time
is adjusted adaptively to an optimum regarding video quality, avoiding freezing and long waiting time before the video
reproduction starts. The maximum available resolution is defined by the video provider and often by the display of the
user device.
4.1 Phases of IP-based video services
Most IP-based video services, like the YouTube™ video service, are comprised of several phases which are mainly the
set-up of a HTML context including the request for the actual video server location and the download of the video itself.
It has to be considered that there is no sub-sequent download of HTML content and video rather a parallel set-up of
many connections.
Figure 1 shows typical phases of IP-based video services, like YouTube™.
ETSI
---------------------- Page: 10 ----------------------
11 ETSI TR 101 578 V1.3.1 (2018-10)
Figure 1: Typical phases of IP-based video services
In principle each video service can be divided into the setup of the context until the media server links are received and
the phase where the video data are retrieved and the video is playout.
If approaching a video service, typically a video identifier is requested and not the specific URL to download the video
from the media servers. Therefore, a typical streaming service can be separated in two phases, in a first phase – Video
Access Phase - the hosting server is contacted (e.g. YouTube™), the video identifier is transmitted. The server usually
provides specific URLs from where the media content can be retrieved. These URLs are usually selected in accordance
to the operating system of the device, its display resolution, the access technology, the region and the possibility to have
audio and video contents split in different servers. In this last case, more media URLs are provided, one for the video
and another one for the audio.
After reception of the media URLs, the media content is requested from these locations, the related media content
servers are approached and an initial Pre-Playout buffering phase follows. After this initial buffering (often made visible
to the user by an animation on the screen) the video starts to play. This is visibly the start of the video. At this point in
time the second phase starts: the Video Playout. Usually, in the background the video is further downloaded and
buffered, even while the display has already started. In live video streams or long video, the buffer is kept filled by
regular retrieving and downloading of media content. The Video Playout phase ends with the end of the video playout.
This end can be either the end of the video or an initiated end of displaying (video stopped by the user).
In practice usually these two phases can be separated by feedback of the application to the user. The underlaying
actions, as e.g. start and end buffering are not always indicated to a user and also not to be retrieved from encrypted
and/or proprietary protocol layers.
It should also be noted that most video services provide more information than only the video itself. There is
complementary HTML content e.g. transporting comments, advertisements, lists of most watched video and more. Like
in a web-browser, this additional information is requested in parallel and typically directly after launching the service.
The download of this content can even last in to the play-out phase of the video and can also populate the player or page
during the display. As a consequence, during requesting and downloading video, other content is downloaded in parallel
connections/threads.
ETSI
---------------------- Page: 11 ----------------------
12 ETSI TR 101 578 V1.3.1 (2018-10)
4.2 QoS aspects of IP-based video services
4.2.0 Scope of aspects
When looking at impairments for a video streaming service, this clause focuses on objectively
...