|
TECHNICAL SPECIFICATION
Speech and multimedia Transmission Quality (STQ);
Reference benchmarking,
background traffic profiles and KPIs;
Part 1: Reference benchmarking, background traffic profiles
and KPIs for VoIP and FoIP in fixed networks
---------------------- Page: 1 ----------------------
2 ETSI TS 103 222-1 V1.3.1 (2019-03)
Reference
RTS/STQ-280-1
Keywords
KPI, QoS
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
The present document can be downloaded from:
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
If you find errors in the present document, please send your comment to one of the following services:
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.
© ETSI 2019.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.
TM TM
3GPP and LTE are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and
of the oneM2M Partners.
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI
---------------------- Page: 2 ----------------------
3 ETSI TS 103 222-1 V1.3.1 (2019-03)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
Introduction . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 7
3 Definition of terms, symbols and abbreviations . 8
3.1 Terms . 8
3.2 Symbols . 8
3.3 Abbreviations . 8
4 Management Summary. 10
4.1 Introduction . 10
4.2 Scope of functionality . 10
5 Technical concept . 11
5.1 Voice over IP . 11
5.2 Call set-up delay and Session initiation call set-up delay . 12
5.3 Call set-up time (post dialling delay) . 14
5.4 Premature release probability (call failure rate; telephony service non accessibility) . 16
5.5 Telephony Cut-off Call Ratio [%] (Call drop rate) . 16
5.6 Media establishment delay . 17
5.7 Level of active speech signal in receive direction . 17
5.8 Noise level in receive direction . 18
5.9 Signal to noise ratio in receive direction . 18
5.10 Speech signal attenuation (or gain) . 18
5.11 Talker echo delay . 18
5.12 Double talk performance . 18
5.13 Interrupted voice transmission . 19
5.14 Listening speech quality . 20
5.14.1 General aspects of Listening Speech Quality. 20
5.14.2 General aspects of voice channel test calls . 20
5.14.3 Connections without parallel data transfer . 22
5.14.3.1 Connections with one voice channel . 22
5.14.3.2 Multiple voice channel access . 24
5.14.4 Connections with parallel data transfer . 27
5.14.4.0 Introduction . 27
5.14.4.1 Quality measurement of one voice channel and parallel data transfer . 27
5.14.4.2 Parallel quality measurement of one voice channel and data transmission speed . 31
5.14.4.3 Quality measurement of multiple voice channels and data transfer . 33
5.14.4.4 Parallel quality measurement of multiple voice channels and data transmission speed . 34
5.15 Listening speech quality stability . 37
5.16 End-to-end audio delay . 37
5.17 End -to-end audio delay variation . 38
5.18 Frequency response in receive direction . 39
5.19 Fax transmission with Recommendations ITU-T T.30 and T.38 . 39
5.19.1 General considerations . 39
5.19.2 Fax set-up duration . 41
5.19.3 Fax transmission duration . 41
5.19.4 Fax failure ratio . 42
5.19.5 Test case descriptions . 42
5.19.5.1 Quality measurement of one fax channel . 42
5.19.5.2 Quality measurement of one fax channel and parallel data transfer . 43
ETSI
---------------------- Page: 3 ----------------------
4 ETSI TS 103 222-1 V1.3.1 (2019-03)
5.20 Early media listening speech quality . 50
5.20.1 Introduction. 50
5.20.2 Early media generated by the called party . 50
5.21 Jitter Buffer and IP prioritization response time. 51
5.21.1 Jitter Buffer and IP prioritization response time without data transfer . 51
5.21.2 Jitter Buffer and IP prioritization response time with data transfer . 52
5.22 Stability of the de-jitter buffer delay adjustments for VBD calls during IP data transfer. 53
6 Internet related load generation methods for VoIP and FoIP tests . 56
6.1 Introduction to Load generation alternatives . 56
6.2 TCP load generation with flow-control enabled . 56
6.3 TCP load generation with no flow-control . 56
6.4 Monitoring the generated load . 57
6.5 QoS IP transmission layer parameter tests . 57
6.5.0 Introduction. 57
6.5.1 VoIP (Voice over IP) . 57
6.5.2 TCP Parameters . 58
6.5.3 UDP Parameters . 58
6.5.4 Web Page . 59
6.5.5 Unmodified content . 59
6.5.6 Transparent connection . 59
6.5.7 DNS . 60
6.5.8 Traceroute . 60
Annex A (informative): Bibliography . 61
History . 62
ETSI
---------------------- Page: 4 ----------------------
5 ETSI TS 103 222-1 V1.3.1 (2019-03)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This Technical Specification (TS) has been produced by ETSI Technical Committee Speech and multimedia
Transmission Quality (STQ).
The present document is part 1 of a multi-part deliverable covering the Reference benchmarking, background traffic
profiles and KPIs, as identified below:
Part 1: "Reference benchmarking, background traffic profiles and KPIs for VoIP and FoIP in fixed
networks";
Part 2: "Reference benchmarking and KPIs for High speed internet";
Part 3: "Reference benchmarking, background traffic profiles and KPIs for UMTS and VoLTE";
Part 4: "Reference benchmarking for IPTV, Web TV and RCS-e Video Share".
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
The present document describes possible key performance indicators for VoIP and FoIP as well as framework
requirements for reference benchmarking particularly with regard to background traffic. The latest version replaces
clause 6 with load generation methods that are appropriate for the agreed scope of the present document.
ETSI
---------------------- Page: 5 ----------------------
6 ETSI TS 103 222-1 V1.3.1 (2019-03)
1 Scope
The present document:
• identifies and defines possible key performance indicators for voice and fax telephony services;
• defines benchmarking methods for the spectrum of potential applications.
The offer of new NGN services requires new KPIs, QoS measurement and benchmarking methods which are needed to
ensure the quality of new services. To ensure the comparability of test results, reference benchmarking methods and
background traffic load profiles are needed.
The scope of the defined testing procedures is the evaluation of the network access by VoIP and FoIP fixed-network
services. The measurements are conducted stationary between a subscriber access-point to a measurement point
emulating an idealized termination point in the core network. All access technologies offered by the operator under test
are considered. In this context the measurements and key performance indicators determinations are performed by
analysing signals accessible on the network.
The present document is the first part of the multi-part deliverable which consists of four parts.
The present document contains possible KPIs for VoIP and FoIP as well as framework requirements for reference
benchmarking particularly with regard to background traffic profiles.
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference/.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] Recommendation ITU-T E.800 (09-2008): "Definitions of terms related to quality of service".
[2] Recommendation ITU-T P.863 (03-2018): "Perceptual objective listening quality prediction".
[3] ETSI TS 101 563 (V1.3.1): "Speech and multimedia Transmission Quality (STQ);
IMS/PES/VoLTE exchange performance requirements".
[4] Recommendation ITU-T Q.543 (03-1993): "Digital exchange performance design objectives".
[5] ETSI ES 202 765-2 (V1.2.1): "Speech and multimedia Transmission Quality (STQ); QoS and
network performance metrics and measurement methods; Part 2: Transmission Quality Indicator
combining Voice Quality Metrics".
[6] Recommendation ITU-T G.131 (11-2003): "Talker echo and its control".
[7] ETSI ES 203 021-3 (V2.1.2): "Access and Terminals (AT); Harmonized basic attachment
requirements for Terminals for connection to analogue interfaces of the Telephone Networks;
Update of the technical contents of TBR 021, EN 301 437, TBR 015, TBR 017; Part 3: Basic
Interworking with the Public Telephone Networks".
[8] ETSI TBR 003 (Edition 1) (11-1995): "Integrated Services Digital Network (ISDN); Attachment
requirements for terminal equipment to connect to an ISDN using ISDN basic access".
ETSI
---------------------- Page: 6 ----------------------
7 ETSI TS 103 222-1 V1.3.1 (2019-03)
[9] ETSI TBR 004 (Edition 1) (11-1995): "Integrated Services Digital Network (ISDN); Attachment
requirements for terminal equipment to connect to an ISDN using ISDN primary rate access".
[10] ETSI EN 300 175-8: "Digital Enhanced Cordless Telecommunications (DECT); Common
Interface (CI); Part 8: Speech and audio coding and transmission".
[11] Recommendation ITU-T O.41 (10-1994): "Psophometer for use on telephone-type circuits".
[12] Recommendation ITU-T P.56 (12-2011): "Objective measurement of active speech level".
[13] Recommendation ITU-T P.501 (03-2017): "Test signals for use in telephonometry".
[14] ETSI ES 202 737 (V1.4.1): "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for narrowband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[15] ETSI ES 202 739 (V1.4.1): "Speech and multimedia Transmission Quality (STQ); Transmission
requirements for wideband VoIP terminals (handset and headset) from a QoS perspective as
perceived by the user".
[16] Recommendation ITU-T P.340 (05-2000): "Transmission characteristics and speech quality
parameters of hands-free terminals".
[17] Recommendation ITU-T P.502 (05-2000): "Objective test methods for speech communication
systems using complex test signals".
[18] Recommendation ITU-T P.863.1 (09-2014): "Application guide for Recommendation ITU-T
P.863".
[19] Recommendation ITU-T E.458 (02-1996): "Figure of merit for facsimile transmission
performance".
[20] Recommendation ITU-T E.453 (08-1994): "Facsimile image quality as corrupted by transmission-
induced scan line errors".
[21] ETSI TS 102 250-2 (V2.3.1): "Speech and multimedia Transmission Quality (STQ); QoS aspects
for popular services in mobile networks; Part 2: Definition of Quality of Service parameters and
their computation".
[22] Recommendation ITU-T Y.1541: "Network performance objectives for IP-based services".
[23] Recommendation ITU-T G.711: "Pulse code modulation (PCM) of voice frequencies".
[24] Recommendation ITU-T V.34: "A modem operating at data signalling rates of up to 33 600 bit/s
for use on the general switched telephone network and on leased point-to-point 2-wire telephone-
type circuits".
[25] Recommendation ITU-T V.17: "A 2-wire modem for facsimile applications with rates up to
14 400 bit/s".
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] Void.
ETSI
---------------------- Page: 7 ----------------------
8 ETSI TS 103 222-1 V1.3.1 (2019-03)
[i.2] ETSI ETR 138 (12-1997): "Network Aspects (NA); Quality of service indicators for Open
Network Provision (ONP) of voice telephony and Integrated Services Digital Network (ISDN)".
[i.3] ETSI EG 202 425 (V1.1.1): "Speech Processing, Transmission and Quality Aspects (STQ);
Definition and implementation of VoIP reference point".
[i.4] ETSI EG 202 057-2: "Speech and multimedia Transmission Quality (STQ); User related QoS
parameter definitions and measurements; Part 2: Voice telephony, Group 3 fax, modem data
services and SMS".
[i.5] ETSI TR 103 138: "Speech and multimedia Transmission Quality (STQ); Speech samples and
their use for QoS testing".
[i.6] IEC 61260:1995: "Electroacoustics - Octave-band and fractional-octave-band filters".
[i.7] Recommendation ITU-T T.30 (09-2005): "Procedures for document facsimile transmission in the
general switched telephone network".
[i.8] Recommendation ITU-T T.38 (09-2010): "Procedures for real-time Group 3 facsimile
communication over IP networks".
[i.9] Recommendation ITU-T T.24: "Standardized digitized image set".
[i.10] Recommendation ITU-T G.168 (04-2015): "Digital network echo cancellers".
[i.11] Recommendation ITU-T V.25: "Automatic answering equipment and general procedures for
automatic calling equipment on the general switched telephone network including procedures for
disabling of echo control devices for both manually and automatically established calls".
[i.12] IETF RFC 8337 (March 2018): "Model-Based Metrics for Bulk Transport Capacity", M.Mathis
and A.Morton.
NOTE: Available from https://tools.ietf.org/html/rfc8337.
[i.13] IETF RFC 4122 (July 2005): "A Universally Unique IDentifier (UUID) URN Namespace".
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the following terms apply:
benchmark: evaluation of performance value/s of a parameter or set of parameters for the purpose of establishing
value/s as the norm against which future performance achievements may be compared or assessed
NOTE: The definition is taken from Recommendation ITU-T E.800 [1].
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
AB Direction of call establishment User B to user A
ACK ACKnowledgement
AGCF Access Gateway Control Function
AGW Access GateWay
AIMD Additive Increase, Multiplicative Decrease
ETSI
---------------------- Page: 8 ----------------------
9 ETSI TS 103 222-1 V1.3.1 (2019-03)
ANS ANswer Tone
AS Application Server
BA Direction of call establishment User A to user B
BRI Basic Rate Interface
BST Broadband Speed Test
CED CallED station identification tone
CFR Call Failure Rate
CI Common Interface
CLI Calling Line Identification
CNG CalliNG tone
CPE Customer Premises Equipment
CSCF Call Session Control Function
CSS Composite Source Signal
DL DownLink
DNS Domain Name System
DSS1 Digital subscriber Signalling System No. 1
DTMF Dual-Tone Multi-Frequency signalling
ECM Error Correction Mode
ERP Ear Reference Point
FE Functional Entity
FM Feature Manager
FoIP Fax over IP
FOM Figure Of Merit
FTP File Transfer Protocol
HTTP Hypertext Transfer Protocol
HTTPS Hypertext Transfer Protocol Secure
IAD Integrated Access Device
IEC International Electrotechnical Commission
IETF Internet Engineering Task Force
IMAP Internet Message Access Protocol
IMAPS Internet Message Access Protocol Secure
IMS Internet Multimedia Subsystem
IP Internet Protocol
IPTV Internet Protocol TeleVision
ISDN Integrated Services Digital Network
ITU-T International Telecommunication Union - Telecommunication standardization sector
IVR Interactive Voice Response
KPI Key Performance Indicator
LQO Listening Quality Objective
MGC Media Gateway Controller
MGW Media GateWay
MMTel MultiMedia Telephony service
MOS Mean Opinion Score
MRP Mouth Reference Point
MSAN Multi-Service Access Nodes
NGN New Generation Network
NTP Network Time Protocol
OVL OVerLoad point
PCMA Pulse Code Modulation A-law
P-CSCF Proxy - Call Session Control Function
PES PSTN Emulation Subsystem
PRI Primary Rate Interface
PSTN Public Switched Telephone Network
PT ProTocol
QoS Quality of Service
RDP Remote Desktop Protocol
RFC Request For Comments
RTCP Real Time Control Protocol
RTP Real Time Protocol
RTSP Real Time Streaming Protocol
S-CSCF Service - Call Session Control Function
SDP Session Description Protocol
ETSI
---------------------- Page: 9 ----------------------
10 ETSI TS 103 222-1 V1.3.1 (2019-03)
SIP Session Initiation Protocol
SMTP Simple Mail Transfer Protocol
SMTPS Simple Mail Transfer Protocol Secure
SNR Speech signal level/Noise level
SSH Secure SHell
SSL Secure Sockets Layer
SWB Super Wide Band
TCP Transmission Control Protocol
TELR Talker Echo Loudness Rating
TLS Transport Layer Security
TOR Terminal Owning Region
TR Technical Report
TV TeleVision
UA User Agent
UAS User Agent Server
UDP User Datagram Protocol
UE User Equipment
UL UpLink
UMTS Universal Mobile Telecommunications System
UNiA User Network interface A
UNiB User Network interface B
UUID Universally Unique IDentifier
NOTE: As described in IETF RFC 4122 [i.13].
VBD Voice Band Data
VGW Voice GateWay
VoIP Voice over IP
VoLTE Voice over Long Term Evolution
4 Management Summary
4.1 Introduction
The spectrum of potential applications of a benchmarking platform requires measurements including but not limited to
the following: analogue (a/b), ISDN, VoIP (including SIP trunking) and high-speed internet.
The performance data which are collected will be relevant for a real-world environment encompassing a mix of
technologies. The scope of the defined testing procedures is the evaluation of the network access by VoIP and FoIP
fix-network services. The measurements are conducted stationary between a subscri
...