|
TECHNICAL REPORT
Universal Mobile Telelecommunications System ( (UMTS);
LTE;
Video telephony robubustness improvements extetensions;
Perforformance evaluation
(3GPP TR 26.9.922 version 13.0.0 Release 13 13)
---------------------- Page: 1 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 1 ETSI TR 126 922 V13.0.0 (2016-03)
Reference
DTR/TSGS-0426922vd00
Keywords
LTE, UMTS
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
The present document can be downloaded from:
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.
© European Telecommunications Standards Institute 2016.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are Trade Marks of ETSI registered for the benefit of its Members.
TM
3GPP and LTE™ are Trade Marks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
---------------------- Page: 2 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 2 ETSI TR 126 922 V13.0.0 (2016-03)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI 3rd Generation Partnership Project (3GPP).
The present document may refer to technical specifications or reports using their 3GPP identities, UMTS identities or
GSM identities. These should be interpreted as being references to the corresponding ETSI deliverables.
The cross reference between GSM, UMTS, 3GPP and ETSI identities can be found under
.
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
ETSI
---------------------- Page: 3 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 3 ETSI TR 126 922 V13.0.0 (2016-03)
Contents
Intellectual Property Rights . 2
Foreword . 2
Modal verbs terminology . 2
Foreword . 4
1 Scope . 5
2 References . 5
3 Definitions and abbreviations . 6
3.1 Definitions . 6
3.2 Abbreviations . 6
4 Background . 6
5 Overview of video robustness improvements extensions (VTRI_EXT) tools . 7
5.1 Introduction . 7
5.2 Retransmission . 7
5.3 Forward error correction . 7
5.4 Reference picture selection. 7
6 Test cases and conditions . 7
6.1 QoS requirements for conversational video services . 7
6.2 Channel conditions . 9
6.3 Error profiles . 10
6.3.1 Introduction. 10
6.3.2 QoS LTE . 10
6.3.3 LTE-OTT . 10
6.3.4 WiFi . 10
6.3.5 Summary . 10
6.4 Test Content . 11
7 Evaluation criteria . 12
7.1 Testing configuration . 12
7.2 Performance metrics . 12
8 Results . 13
8.1 Test cases. 13
8.2 Simulation (RTT= 100 ms) . 13
8.3 Simulation (RTT= 200 ms) . 15
8.4 Simulation (RTT= 300 ms) . 16
8.5 Simulation (RTT= 400 ms) . 18
8.6 Summary . 20
9 Conclusions and recommendations . 20
Annex A: Error patterns . 22
A.1 IMS-QoS . 22
A.2 VT-LTE OTT . 22
A.3 VT-Wifi . . 24
Annex B: Change history . 28
History . 29
ETSI
---------------------- Page: 4 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 4 ETSI TR 126 922 V13.0.0 (2016-03)
Foreword
rd
This Technical Report has been produced by the 3 Generation Partnership Project (3GPP).
The contents of the present document are subject to continuing work within the TSG and may change following formal
TSG approval. Should the TSG modify the contents of the present document, it will be re-released by the TSG with an
identifying change of release date and an increase in version number as follows:
Version x.y.z
where:
x the first digit:
1 presented to TSG for information;
2 presented to TSG for approval;
3 or greater indicates TSG approved document under change control.
y the second digit is incremented for all changes of substance, i.e. technical enhancements, corrections,
updates, etc.
z the third digit is incremented when editorial only changes have been incorporated in the document.
ETSI
---------------------- Page: 5 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 5 ETSI TR 126 922 V13.0.0 (2016-03)
1 Scope
The present document reports the study on video telephony robustness improvements extensions in Multimedia
Telephony Service for IMS (MTSI) and provides recommendation on their applicability for MTSI video telephony
applications.
2 References
The following documents contain provisions, which, through reference in this text, constitute provisions of the present
document.
- References are either specific (identified by date of publication, edition number, version number, etc.) or
non-specific.
- For a specific reference, subsequent revisions do not apply.
- For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP document (including
a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same
Release as the present document.
[1] 3GPP TR 21.905: "Vocabulary for 3GPP Specifications".
[2] 3GPP TS 22.105: "Services and service capabilities".
[3] 3GPP TS 26.114: "IP Multimedia Subsystem (IMS); Multimedia telephony; Media handling and
interaction".
[4] IETF RFC 4588: "RTP Retransmission Payload Format", July 2006.
[5] IETF RFC 6865: "Simple Reed-Solomon Forward Error Correction (FEC) Scheme for
FECFRAME", February 2013.
[6] IETF RFC 5109: "RTP Payload Format for Generic Forward Error Correction", December 2007.
[7] IETF RFC 4585: "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based
Feedback (RTP/AVPF)", July 2006.
[8] K. Yamagishi, T. Hayashi, "Parametric Packet-Layer Model for Monitoring Video Quality of
IPTV Services", IEEE ICC 2008, pp. 110-114, May 2008.
[9] Q. Huynh-Thu, M. Ghanbari, "Impact of Jitter and Jerkiness on Perceived Video Quality", Proc. of
the Second International Workshop on Video Processing and Quality Metrics for Consumer
Electronics (VPQM), 2006.
[10] C. Wang, X. Jiang, Y. Wang, "Video Quality Assessment Models for IPTV Services", JDCTA,
April 2013.
[11] Pierre Ferre, Dimitris Agrafiotis, Tuan Kiang Chiew, Angela Doufexi, Andrew Nix, David Bull,
"Packet Loss Modelling for H.264 Video Transmission over IEEE 802.11g Wireless LANs", IEEE
WIAMIS 2005.
[12] S. Holmer, M. Shemer, M. Paniconi, "Handling Packet Loss in WebRTC", pp. 1860-1864, ICIP,
2013.
ETSI
---------------------- Page: 6 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 6 ETSI TR 126 922 V13.0.0 (2016-03)
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in TR 21.905 [1] apply.
3.2 Abbreviations
For the purposes of the present document, the abbreviations given in TR 21.905 [1] and the following apply.
AV Audio Video
AVC Advanced Video Coding
AVPF Audio-Video Profile with Feedback
ER Error Resiliency
FPS Frames Per Second
HEVC High Efficiency Video Coding
IMS-VT IP Multimedia Subsystem Video Telephony
KB Kilo Byte
MTSI Multimedia Telephony Service for IMS
OTT Over The Top
PLI Picture Loss Indication
PLR Packet Loss Rate
QVGA Quarter Video Graphics Array
RPS Reference Picture Selection
RPSI Reference Picture Selection Indication
RTT Round Trip Time
VGA Video Graphics Array
VT Video Telephony
VTRI_EXT Video Robustness Improvements Extensions
Wifi Wireless Fidelity
Note: Wifi is synonymous with Wi-Fi as defined by the Wi-Fi Allicance
4 Background
The present document reports the study on video telephony robustness improvements extensions in Multimedia
Telephony Service for IMS and provides recommendation on their applicability for MTSI video telephony applications.
These extensions target error robustness for higher bitrate MTSI video telephony as well as inter-working with WLAN
use cases where error resiliency is more important. In order to be technically competitive, e.g. to some proprietary
systems, MTSI should have the capability to employ mechanisms that can offer different trade-offs between rendering
delay, video rendering jitter (smoothness) and video quality that can adapt to varying channel conditions for better user
experience. Retransmission, Forward Error Correction (FEC), and complementary reference picture selection indication
(RPSI) AVPF feedback mechanisms offer these trade-offs. The present document first provides an overview of the
additional error resiliency (ER) tools that could improve the performance of the Multimedia Telephony Service for IMS
(TS 26.114 [3]). Then test conditions representative of error conditions experienced in IMS Video Telephony are
presented. Following the description of the test conditions, evaluation criteria for determining the benefits of proposed
tools and mechanisms is presented. Performance of the proposed ER tools is evaluated under the defined testing
conditions that take into account packet loss rate/pattern, end to end delay, bitrate overhead and video smoothness
(dropped frames, rendering jitter). Based on the performance results, conclusions are made in terms of
recommendations for support of proposed ER tools and mechanisms for Multimedia Telephony Service for IMS.
ETSI
---------------------- Page: 7 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 7 ETSI TR 126 922 V13.0.0 (2016-03)
5 Overview of video robustness improvements
extensions (VTRI_EXT) tools
5.1 Introduction
Multimedia Telephony Service for IMS (MTSI 3GPP TS 26.114 [3]) defines MTSI clients" sender and receiver
behaviour utilizing IETF RFC 4585 [7] AVPF Generic NACK and Picture Loss Indication (PLI) feedback messages for
ER. Current error correction scheme provides basic error correction through codec level error resiliency (ER)
mechanisms. Transport and application level error resiliency schemes such as Retransmission (NACK), Forward Error
Correction (FEC) along with advanced codec level ER schemes such as Reference Picture Selection (RPS) provide
alternative error correction mechanisms that offer different performance trade-offs. The performance of error correction
schemes varies with end-to-end delay, channel bandwidth and packet loss rate.
5.2 Retransmission
Retransmission (NACK) scheme [4] provides efficient error correction in terms of bandwidth under short round-trip-
time (RTT) cases with low packet loss rates. The efficiency of retransmission scheme becomes more pronounced at
higher bitrates since selective retransmission of lost packets instead of entire pictures are needed. Under low RTT
scenarios it can provide low video rendering jitter dependent on the de-jittering mechanism at the cost of additional
delay. If additional delay cannot be accommodated, then retransmission can still provide recovery from error with video
freezes during recovery similar to the existing error resiliency scheme in TS 26.114.
5.3 Forward error correction
Forward Error Correction (FEC) schemes [5] and [6] provide a mechanism that balances video quality and end-to-end
delay. FEC schemes can adapt to varying channel error conditions. FEC is suitable for high RTT channels with high
packet loss rates where retransmission leads to high video rendering delay and codec based recovery mechanisms like
RPSI, PLI lead to frequent video freezes and/or corruptions. FEC schemes are complemented by retransmission
(NACK) or RPSI, PLI feedback mechanisms to address FEC failure cases.
5.4 Reference picture selection
Reference picture selection indication (RPSI) feedback message in AVPF [7] that is currently not supported in
TS 26.114 offers establishment of common reference point for recovery between the sender and the receiver. In essence
it provides codec level ER mechanism similar to the transport layer ER mechanism supported by the generic NACK
message in TS 26.114.
6 Test cases and conditions
6.1 QoS requirements for conversational video services
Specification TS 22.105 [2] defines the range of QoS requirements and end user QoS requirements for conversational
video services. According to TS 22.105, the following requirements should be supported.
ETSI
---------------------- Page: 8 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 8 ETSI TR 126 922 V13.0.0 (2016-03)
Table 6.1-1: Range of QoS requirements copied from TS 22.105 (clause 5.4)
Real Time (Constant Delay) Non Real Time (Variable Delay)
Operating BER/Max Transfer Delay BER/Max Transfer Delay
environment
Satellite Max Transfer Delay less than 400 ms Max Transfer Delay 1200 ms or more
(Terminal
(NOTE 2)
relative speed to BER 10-3 - 10-7
ground up to (NOTE 1) BER = 10-5 to 10-8
1000 km/h for
plane)
Rural outdoor
Max Transfer Delay 20 - 300 ms Max Transfer Delay 150 ms or more
(Terminal (NOTE 2)
relative speed to
BER 10-3 - 10-7
ground up to 500 (NOTE 1) BER = 10-5 to 10-8
km/h) (NOTE 3)
Urban/ Suburban Max Transfer Delay 20 - 300 ms Max Transfer Delay 150 ms or more
outdoor (Note 2)
(Terminal
BER 10-3 - 10-7
relative speed to (NOTE 1) BER = 10-5 to 10-8
ground up to 120
km/h)
Indoor/ Low
Max Transfer Delay 20 - 300 ms Max Transfer Delay 150 ms or more
range outdoor (NOTE 2)
(Terminal BER 10-3 - 10-7
relative speed to
(NOTE 1) BER = 10-5 to 10-8
ground up to 10
km/h)
NOTE 1: There is likely to be a compromise between BER and delay.
NOTE 2: The Max Transfer Delay should be here regarded as the target value for 95% of the data.
NOTE 3: The value of 500 km/h as the maximum speed to be supported in the rural outdoor environment
was selected in order to provide service on high speed vehicles (e.g. trains). This is not meant
to be the typical value for this environment (250 km/h is more typical).
And the requirements for end user QoS as performance expectations for conversational/real-time services is shown in
table 6.1-2.
ETSI
---------------------- Page: 9 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 9 ETSI TR 126 922 V13.0.0 (2016-03)
Table 6.1-2: End-user performance expectations (copied from TS 22.105 clause 5.5)
Medium Application Degree of Data rate Key performance parameters and target
symmetry values
End-to-end One- Delay Information loss
way Variation
Delay within a call
Audio Conversational
voice Two-way 4-25 kb/s <150 msec < 1 msec < 3% FER
preferred
<400 msec limit
NOTE 1
Video Videophone Two-way 32-384 < 150 msec < 1% FER
kb/s preferred
<400 msec limit
Lip-synch: < 100
msec
Data Telemetry Two-way <28.8 kb/s < 250 msec N.A
- two-way control Zero
Data realtime games Two-way < 60 kb/s < 75 msec N.A < 3% FER
preferred preferred,
NOTE 2 < 5% FER limit
NOTE 2
Data Telnet Two-way < 1 KB < 250 msec N.A Zero
(asymmetric)
NOTE 1: The overall one way delay in the mobile network (from UE to PLMN border) is approximately 100msec.
NOTE 2: Thesevalues are considered the most demanding ones with respect to delay requirements (e.g.
supporting First Person Shooter games). Other types of games may require higher or lower data rates
and more or less information loss but can tolerate longer end-to-end delay
QoS test conditions used to evaluate the proposed tools should follow the service requirements described in TS 22.105.
In addition to QoS networks, test conditions addressing interworking with non-QoS networks should be considered for
the following reasons:
- Interworking with non-QoS networks is a relevant deployment use case and may result in losses in the non-
managed part of the delivery.
- Despite QoS, there may be circumstances for which the QoS guarantees fail and service continuity is relevant.
6.2 Channel conditions
Channels conditions from QoS LTE, best effort over the top (OTT) LTE and WiFi channels are logged from video
telephony calls for video configurations defined in clause 6.4. Packet captures are conducted on video telephony (VT)
calls under mobile and stationary test conditions. Sending and receiving rates, delay (RTT/2), packet loss patterns are
derived from captures sending and receiving times, timestamps and sequence numbers. The sources of the packet losses
are from the physical channel as well as congestion. During the channel capturing process, the operating rate of the VT
calls targeted rates below the available bandwidth for avoiding congestion. It is not always possible to avoid congestion
during the capturing process. Logs exhibiting frequent large variations in rate due to congestion are filtered out.
is observed in the loss pattern
Packet losses are characterized by the burst patterns. A packet loss-free burst of order k
0
when at least k consecutive packets are correctly received. A packet loss burst order k starts and finishes with a
0 0
missing packet ("1") and is composed of at most k -1 consecutive received packets [11]. In the analysis presented in the
0
present document, k =1 is used for simplicity. Sequences of m (total number of logged packets) loss indicators are
0
divided into p alternating loss-free burst (X ) and packet loss bursts (Y ). Average packet loss rate PLR , average loss
j j avg
free duration X and average loss duration Y are computed as:
avg avg
p−1
Y
∑
j
j=0
, (6.2-1)
PLR =
avg
p−1
(X + Y )
∑ j j
j=0
ETSI
---------------------- Page: 10 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 10 ETSI TR 126 922 V13.0.0 (2016-03)
p−1
1
, (6.2-2)
X = X
avg ∑ j
p
j=0
p−1
1
. (6.2-3)
Y = Y
avg ∑ j
p
j=0
6.3 Error profiles
6.3.1 Introduction
Error profiles representing guaranteed QoS and best effort (non-QoS) cases are used for evaluation. A number of real
channel capture logs from QoS and non-QoS services are provided for emulation of channel conditions and/or
derivation of channel models for simulation of channel conditions. Captured channel logs are used in the simulations of
channel conditions for evaluation of proposed error resiliency tools.
6.3.2 QoS LTE
IMS-VT QoS calls conducted under low speed mobile conditions covering near cell and edge cell conditions were
logged for analysis. QVGA (320x240), 15 fps, 350 kbps (maximum bitrate) H.264 video is used during the IMS-VT
call. 17 MO to MT and 17 MT to MO logs selected from ~100 short duration calls (less than 1 minute) are used. In
Table 6.3-1, MO to MT (IMS-QoS Test1) and likewise MT to MO (IMS-QoS Test2) call statistics are consolidated into
one due to short duration of the calls. Packet loss statistics are tabulated in Table 6.3-1. Clause A.1 provides packet loss
patterns for the consolidated logs.
6.3.3 LTE-OTT
Video telephony calls over LTE-OTT were conducted under driving conditions. One of the UEs is positioned in a
stationary office environment with good LTE signal and the other UE in a moving vehicle. VGA (640x480) 30 fps 600
kbps (VT-LTE OTT Test1 & Test2) and QVGA 15 fps 300 kbps (VT-LTE OTT Test3 & Test4) videos were used for
collecting channel logs. Packet loss statistics are tabulated in Table 6.3-1. Clause A.2 provides packet loss patterns for
LTE-OTT tests.
6.3.4 WiFi
Video telephony calls over WiFi are conducted in office environment. Stationary office to office call and office to
walking UE calls are logged. 720p (1 280x720) 30 fps 1 000 kbps video is used for collecting channel logs. Total of 8
logs (VT-Wifi Test1-8) are collected. Packet loss statistics are tabulated in Table 6.3-1. Clause A.3 provides packet loss
patterns for WiFi tests.
6.3.5 Summary
Table 6.3-1 summarizes error profiles used during the evaluation process.
ETSI
---------------------- Page: 11 ----------------------
3GPP TR 26.922 version 13.0.0 Release 13 11 ETSI TR 126 922 V13.0.0 (2016-03)
Table 6.3-1: Summary of error pattern statistics
Test Condition Bit - Frame Resolution Duration No. of Avg loss Avg. loss Avg
rate Rate (sec) packets free duration PLR (%)
(kbps) (fps) duration (pkts)
(pkts)
IMS-QoS
Test1 Low mobility 350 15 320x240 309 12032 2 007 1,5 0,07%
IMS-QoS
Test2 Low mobility 350 15 320x240 309 11870 627 4,1 0,66%
VT-LTE
OTT Test1 High mobility 600 30 640x480 2 291 158 699 1 521 4,6 0,30%
VT-LTE
OTT Test2 High mobility 600 30 640x480 2 290 145 352 1 305 5,7 0,43%
VT-LTE Walk & High
OTT Test3 mobility 300 15 320x240 982 40 305 2 672 15,1 0,56%
VT-LTE Walk & High
OTT Test4 mobility 300 15 320x240 981 39 222 2 440 11,8 0,48%
VT-Wifi
Test1 Stationary 1 000 30 1 280x720 766 93 771 1 801 1,9 0,10%
VT-Wifi
Test2 Stationary 1 000 30 1 280x720 765 92 795 1 685 1,9 0,11%
VT-Wifi
Test3 Stationary 1 000 30 1 280x720 715 53 698 292 2,7 0,92%
VT-Wifi
Test4 Stationary 1 000 30 1 280x720 717 72 244 36 1,9 5,02%
VT-Wifi
Test5 Stationary 1 000 30 1 280x720 620 75 946 1 724 2,2 0,13%
VT-Wifi
Test6 Stationary 1 000 30 1 280x720 620 75 472 1 477 3,2 0,21%
VT-Wifi
Test7 Walk 1 000 30 1 280x720 381 24 045 607 9,8 1,60%
VT-Wifi
Test8 Walk 1 000 30 1 280x720 381 37 093 67 3,4 4,75%
VT-Wifi
Test9 Walk 1 000 30 1 280x720 913 54 260 39 2,7 7,19%
VT-
Random Random 1 000 30 1 280x720 1 013 98 634 - - 10,04%
6.4 Test Content
For evaluation of ER tools, the two main factors that have impact on the overall performance is the video bitrate and the
frame rate. It is assumed that the video is coded in low delay configuration, i.e. IPPPPP… or IBBBB…. configuration.
The video resolution, content, and codec type (AVC, HEVC) have minimal impact since as described in clause 7, the
corrupted pictures will be considered as non-rendered pictures. The following video resolutions, bitrate and frame rates
are used during the evaluation process.
Table 6.4-1: Test content configuration
Resolution Bitrate (kbps) Frame rate (fps)
320x240 300 & 350 15
640x480 600 30
1 280x720 1 000 30
ETSI
...