|
GROUP SPECIFICATION
KPIs for Next Generation Protocols:
Basis for measuring benefits of NGP
Disclaimer
The present document has been produced and approved by the Next Generation Protocols (NGP) ETSI Industry Specification
Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.
---------------------- Page: 1 ----------------------
2 ETSI GS NGP 012 V1.1.1 (2018-12)
Reference
DGS/NGP-0012
Keywords
performance, security
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
The present document can be downloaded from:
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the only prevailing document is the
print of the Portable Document Format (PDF) version kept on a specific network drive within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
If you find errors in the present document, please send your comment to one of the following services:
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.
© ETSI 2018.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.
TM TM
3GPP and LTE are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and
of the oneM2M Partners.
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI
---------------------- Page: 2 ----------------------
3 ETSI GS NGP 012 V1.1.1 (2018-12)
Contents
Intellectual Property Rights . 4
Foreword . 4
Modal verbs terminology . 4
1 Scope . 5
2 References . 5
2.1 Normative references . 5
2.2 Informative references . 5
3 Definition of terms, symbols and abbreviations . 5
3.1 Terms . 5
3.2 Symbols . 6
3.3 Abbreviations . 6
4 Overview . 6
5 Methodology . 6
6 Key Performance Indicators for network protocols . 7
6.1 KPIs for naming and addressing . 7
6.2 KPIs for performance . 8
6.3 KPIs for mobility . 8
6.4 KPIs for buffering . 9
6.5 KPIs for multihoming . 9
6.6 KPIs for protocol efficiency . 9
6.7 KPIs for security and privacy . 11
6.8 KPIs for traffic management . 11
6.9 KPIs for interoperability . 12
7 Assessment of return on investment . 12
7.0 About this clause . 12
7.1 Deployment effort . 12
7.2 Revenue opportunities . 13
Annex A (informative): Guidance on weighting of KPIs . 14
A.0 Rationale for weighting . 14
A.1 Weighting KPIs within a KPI category . 14
A.2 Weighting for a network. 14
A.3 Weighting for a scenario . 14
Annex B (informative): Authors & contributors . 15
Annex C (informative): Bibliography . 16
Annex D (informative): Change History . 17
History . 18
ETSI
---------------------- Page: 3 ----------------------
4 ETSI GS NGP 012 V1.1.1 (2018-12)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Next Generation
Protocols (NGP).
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
ETSI
---------------------- Page: 4 ----------------------
5 ETSI GS NGP 012 V1.1.1 (2018-12)
1 Scope
The scope of the present document is to specify Key Performance Indicators (KPIs) that can be used to compare the
efficiency, performance and security of Next Generation Protocols (NGPs) against current networking protocols.
The relative importance of each KPI depends on the scenario in which protocols are being compared. Therefore, this
document provides guidelines for weighting the KPIS to help arrive at a meaningful comparison. Scenarios of particular
relevance are detailed in ETSI NGP GS 001 [1], with resulting requirements listed in ETSI NGP GS 005 [2].
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
[1] ETSI GS NGP 001: "Next Generation Protocol (NGP); Scenario Definitions".
[2] ETSI GS NGP 005: "Next Generation Protocol (NGP); Next Generation Protocol Requirements".
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI TR 121 905: "Digital cellular telecommunications system (Phase 2+) (GSM); Universal
Mobile Telecommunications System (UMTS); LTE; Vocabulary for 3GPP Specifications
(3GPP TR 21.905)".
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the following terms apply:
Key Performance Indicator (KPI): measurable property that significantly impacts business operations as its value
changes
ETSI
---------------------- Page: 5 ----------------------
6 ETSI GS NGP 012 V1.1.1 (2018-12)
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the abbreviations given in ETSI TR 121 905 [i.1] and the following apply:
TM rd
3GPP 3 Generation Participation Project
CERT Computer Emergency Response Team
IoT Internet of Things
IP Internet Protocol
ISG Industry Specific Group
KPI Key Performance Indicator
NAT Network Address Translation
NGP Next Generation Protocols
PDU Protocol Data Unit
ROI Return On Investment
TCP Transmission Control Protocol
4 Overview
Next Generation Protocols aim to improve on existing protocols in various ways. Any improvement shall be
demonstrable and measurable. Hence the need for a set of KPIs with which to measure, and compare, Next Generation
Protocols against the protocols they intend to replace.
5 Methodology
Each KPI consists of several characteristics:
• An ID for reference.
• A definition and rationale, to explain why this is a Key Performance Indicator.
• A metric, to indicate the unit of measurement.
• Desired value.
For the most accurate comparison, only the protocols being tested should vary, and other elements (CPUs, network
paths, access media, etc.) should remain fixed. This does not apply for hardware-only processing comparisons against
software processing.
ETSI
---------------------- Page: 6 ----------------------
7 ETSI GS NGP 012 V1.1.1 (2018-12)
6 Key Performance Indicators for network protocols
6.1 KPIs for naming and addressing
Table 6.1
ID Definition and rationale Metric Desired value
Add1 Scalability: the number of entities that Integer A higher number of addressable entities.
can be uniquely addressed by the
scheme. An address scheme should
scale to support the projected
addressable entities of the network.
The measurement is the count of
addressable entities supported by the
address scheme itself without
external mappings (e.g. NAT).
Add2 Allocation and reuse: the efficiency of Time (ms) A shorter time taken to allocate n addresses to n entities;
allocating an address to an a shorter time taken to reallocate n addresses.
addressa
...