|
GROUP SPECIFICATION
Network Functions Virtualisation (NFV) Release 3;
Testing;
NFVI Compute and Network Metrics Specification
Disclaimer
The present document has been produced and approved by the Network Functions Virtualisation (NFV) ETSI Industry
Specification Group (ISG) and represents the views of those members who participated in this ISG.
It does not necessarily represent the views of the entire ETSI membership.
---------------------- Page: 1 ----------------------
2 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
Reference
RGS/NFV-TST008ed321
Keywords
metrics, network, NFV, NFVI, testing
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE
Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16
Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88
Important notice
The present document can be downloaded from:
The present document may be made available in electronic versions and/or in print. The content of any electronic and/or
print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any
existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI
deliverable is the one made publicly available in PDF format at www.etsi.org/deliver.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
If you find errors in the present document, please send your comment to one of the following services:
Copyright Notification
No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying
and microfilm except as authorized by written permission of ETSI.
The content of the PDF version shall not be modified without the written authorization of ETSI.
The copyright and the foregoing restriction extend to reproduction in all media.
© ETSI 2019.
All rights reserved.
TM TM TM
DECT , PLUGTESTS , UMTS and the ETSI logo are trademarks of ETSI registered for the benefit of its Members.
TM TM
3GPP and LTE are trademarks of ETSI registered for the benefit of its Members and
of the 3GPP Organizational Partners.
oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and
of the oneM2M Partners.
GSM and the GSM logo are trademarks registered and owned by the GSM Association.
ETSI
---------------------- Page: 2 ----------------------
3 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
Contents
Intellectual Property Rights . 5
Foreword . 5
Modal verbs terminology . 5
Introduction . 5
1 Scope . 6
2 References . 6
2.1 Normative references . 6
2.2 Informative references . 6
3 Definition of terms, symbols and abbreviations . 7
3.1 Terms . 7
3.2 Symbols . 7
3.3 Abbreviations . 7
4 Time and Time Intervals for Metrics. 7
5 Framework for Metric Definitions . 8
6 Compute Metrics . 9
6.1 Background . 9
6.2 Name . 9
6.3 Parameters . 9
6.4 Scope . 9
6.5 Units of Measure . 9
6.6 Definition . 10
6.7 Method of Measurement . 10
6.8 Sources of Error . 10
6.9 Discussion . 10
7 Network Metrics . 11
7.1 Background . 11
7.2 Name . 11
7.3 Parameters . 11
7.4 Scope . 11
7.5 Units of Measure . 11
7.6 Definition . 12
7.7 Method of Measurement . 12
7.8 Sources of Error . 12
7.9 Discussion . 12
8 Memory Metrics . 13
8.1 Background . 13
8.2 Name . 13
8.3 Parameters . 13
8.4 Scope . 14
8.5 Units of Measure . 14
8.6 Definition . 14
8.7 Method of Measurement . 15
8.8 Sources of Error . 15
8.9 Discussion . 15
9 Follow-on Activities . 15
Annex A (informative): Examples of Scope Specification for Metrics . 16
A.1 Description . 16
A.2 Memory Scope . 17
ETSI
---------------------- Page: 3 ----------------------
4 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
Annex B (informative): Authors & contributors . 19
History . 20
ETSI
---------------------- Page: 4 ----------------------
5 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
Intellectual Property Rights
Essential patents
IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (https://ipr.etsi.org/).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Trademarks
The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners.
ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no
right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does
not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks.
Foreword
This Group Specification (GS) has been produced by ETSI Industry Specification Group (ISG) Network Functions
Virtualisation (NFV).
Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and
"cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of
provisions).
"must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation.
Introduction
Although many metrics for the performance and utilization of the Network Function Virtualisation Infrastructure
(NFVI) components have been in wide use for many years, there were no independent specifications to support
consistent metric development and interpretation. The present document provides the needed specifications for key
NFVI metrics.
ETSI
---------------------- Page: 5 ----------------------
6 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
1 Scope
The present document specifies detailed and vendor-agnostic key operational performance metrics at different layers of
the NFVI, especially processor usage and network interface usage metrics. These metrics are expected to serve as
references for processed and time-aggregated measurement values for performance management information that
traverses the Or-Vi and/or Vi-Vnfm reference points of the NFV architectural framework. The present document
contains normative provisions.
2 References
2.1 Normative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
Referenced documents which are not found to be publicly available in the expected location might be found at
https://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are necessary for the application of the present document.
Not applicable.
2.2 Informative references
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the
referenced document (including any amendments) applies.
NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee
their long term validity.
The following referenced documents are not necessary for the application of the present document but they assist the
user with regard to a particular subject area.
[i.1] ETSI GS NFV-INF 003 (V.1.1.1) (12-2014): "Network Functions Virtualisation (NFV);
Infrastructure; Compute Domain".
TM
[i.2] Linux /UNIX system programming training, Linux man-pages: "TOP(1)".
NOTE: Available at http://man7.org/linux/man-pages/man1/top.1.html#2._SUMMARY%C2%A0Display.
TM
[i.3] O'Reilly Linux Dev Center: "Exploring the /proc/net/ Directory".
NOTE: Available at https://www.linuxtoday.com/infrastructure/2000112300806NWHLSW.
[i.4] RHEL™ 6.8 Deployment Guide: "E.2.18. /proc/meminfo".
NOTE: Available at https://access.redhat.com/documentation/en-
US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s2-proc-meminfo.html.
[i.5] ETSI GS NFV 003: "Network Functions Virtualisation (NFV); Terminology for Main Concepts in
NFV".
[i.6] IETF RFC 7348: "Virtual eXtensible Local Area Network (VXLAN): A Framework for
Overlaying Virtualized Layer 2 Networks over Layer 3 Networks".
ETSI
---------------------- Page: 6 ----------------------
7 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
[i.7] "free(1) Linux User Commands", published by man7.org.
NOTE: Available at http://man7.org/linux/man-pages/man1/free.1.html.
[i.8] collectd codebase, file: "/src/memory.c", published by GitHub, Inc.
NOTE: Available at https://github.com/collectd/collectd/blob/collectd-5.7/src/memory.c#L325.
[i.9] ETSI GS NFV-IFA 003 (V2.4.1): "Network Functions Virtualisation (NFV) Release 2;
Acceleration Technologies; vSwitch Benchmarking and Acceleration Specification".
[i.10] Debian Wiki: "Hugepages".
NOTE: Available at https://wiki.debian.org/Hugepages.
3 Definition of terms, symbols and abbreviations
3.1 Terms
For the purposes of the present document, the terms given in ETSI GS NFV 003 [i.5] apply.
3.2 Symbols
Void.
3.3 Abbreviations
For the purposes of the present document, the abbreviations given in ETSI GS NFV 003 [i.5] and the following apply:
CPU Central Processing Unit
CRC Cyclic Redundancy Check
HZ Hertz of the system clock, an operating system parameter
KiB Kibibytes
NFVI Network Functions Virtualisation Infrastructure
OS Operating System
RAM Random Access Memory
VIM Virtual Infrastructure Manager
VXLAN Virtual eXtensible Local Area Network
NOTE: IETF RFC 7348 [i.6].
4 Time and Time Intervals for Metrics
Coherent compute domains [i.1] usually need access to a clock with accurate time-of-day (or simply time) and sources
of periodic interrupts. Time sources are accessed to provide timestamps for events and log entries that document the
recent history of the compute environment. Periodic interrupts provide a trigger to increment counters and read current
conditions in the compute and networking environments. The compute domain may contain a very large number of
NFV compute nodes [i.1], and each node needs to execute a process to synchronize its hardware and system clocks to a
source of accurate time-of-day, preferably traceable to an international time standard.
With the foundation of time, date, and periodic interrupts, a measurement system can determine the beginning and end
of time intervals, which is a fundamental aspect of metrics that involve counting and collecting events.
Table 4-1 specifies requirements applicable to time, date, and periodic interrupts.
ETSI
---------------------- Page: 7 ----------------------
8 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
Table 4-1: Requirements applicable to time, date and periodic interrupts
General-Time-01 Each node in the compute domain shall be able to take readings from (or access) a clock with
accurate time-of-day and calendar date.
General-Time-02 Each node in the compute domain shall have a source of periodic interrupts available which are
derived from the time-of-day clock, with configurable period (a parameter of metrics that use this
feature).
When the results from measurement systems are collected and reported by management systems, the management
systems may provide an additional time and date reading associated with the operation to collect the results, using their
own time source.
5 Framework for Metric Definitions
The metric definitions in the present document are primarily based on the fact that the resources of the NFVI have
utilization and performance characteristics that can be assessed by measurement processes. The resources may be
implemented in hardware, software (such as virtual resources), or a combination of both. The measurement processes
are primarily implemented in software (such as in the kernel or user space), but may be assisted by features of the
hardware.
The measured NFVI resources and the measurement processes shall be completely specified in the dimensions of model
numbers, firmware versions, software versions, and any other aspects that influence the results (such as physical
location of the components within a datacentre's racks and shelves). For example, the fixed frequency of the physical
CPU clock in Hz, which governs the rate that the CPU executes instructions, is one important descriptor of the NFVI.
Clock Speed may depend on other CPU settings, such as energy-saving power control. For one list of NFVI platform
descriptors, see clause 5.1 of ETSI GS NFV-IFA 003 [i.9].
For each metric it specifies, the present document provides the following elements:
• Background
• Name
• Parameters (input factors)
• Scope of coverage
• Unit(s) of measure
• Definition
• Method of Measurement
• Sources of Error
• Discussion
NOTE: The present document specifies well-known metrics, and assumes that Virtual Infrastructure Managers
(VIM) will control and expose the metrics as specified here, or will be enhanced to collect and convey the
metrics with the required framework elements, which are Name, Parameters, Scope, Units of measure,
and the source of the measurement (where the metric was measured, which may be synonymous with the
Scope).
ETSI
---------------------- Page: 8 ----------------------
9 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
6 Compute Metrics
6.1 Background
The Central Processing Unit (CPU) is an essential component of every coherent compute domain. Each CPU is a
limited resource in terms of the instructions per second it can execute. It is valuable to monitor the utilization of the
CPU resource to fulfil the goals of maintaining continued and efficient operations, and for troubleshooting abnormal
behaviour to find root causes. For many uses, it is helpful to categorize the CPU's execution time into multiple
execution contexts, such as system and user contexts. A compute node may include additional processors beyond the
main CPU; the metrics specified in this clause can also be used to measure and report the usage of such processors.
VNFs also have a view of CPU resources in terms of execution time they have used during a measurement interval.
However, the configured instantiation of the VNF determines how to map its view of virtual processor resource usage to
actual hardware CPU resources available and used. For example, a VNF's processes may be pinned to one or more CPU
cores, or the VNF may be sharing the resources of many CPU cores with other VNFs.
6.2 Name
There are two variants of this metric:
• Processor Usage
• Processor Utilization
The two variants allow reporting this metric as a percentage. The metric is a function of the scope, set of reported
contexts, measurement interval and other factors.
6.3 Parameters
The following parameters shall be supported for this metric:
• Tick interval: the period of timed interrupts when the processor's execution context can be recorded. Note that
this parameter is an integral part of the method of measurement. The tick interval is sometimes called a "jiffy".
The tick interval is controlled by a system parameter, "HZ", whose default value shall be 250 Hz for
measurements complying with the present document.
• Set of execution contexts: the desired set of processor states with reported utilization. For example, the
simplest set includes two states: active and idle. Time in the active context can be calculated as a sum of states
with more specific definitions, such as the active states user and system. A commonly chosen set of four states
is user, system, wait, and idle. See [i.2] for a list of seven states available in Linux OS.
• End time: the termination of the measurement interval (time and date).
• Measurement interval: the duration of the observation by the measurement system to assess the metric.
6.4 Scope
The list of one or more compute resources which shall be included in the values reported, and whether the resource is
physical or virtual. Annex A gives examples of the scope usage for physical and virtual processor metrics.
6.5 Units of Measure
Processor usage results shall be reported as time in nanoseconds, and utilization shall be reported as the ratio of total
time in one or more execution contexts to the total time in the measurement interval, expressed as a percentage.
ETSI
---------------------- Page: 9 ----------------------
10 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
6.6 Definition
The compute resource usage at time T (for a given scope, context, and measurement interval) shall be assessed as
indicated in table 6.6-1.
Table 6.6-1: Requirements for Processor Metric variants
Requirement number Metric Variant Name Requirement
Compute-01 Processor Usage Reported values of this metric variant shall quantify the total time
that one or more compute resources (according to the defined
scope) execute instructions in the specified execution context during
the measurement interval.
Compute-02 Processor Utilization Reported values of this metric variant shall quantify the ratio of time
of Processor Usage to the time in the measurement interval,
expressed as a percentage.
6.7 Method of Measurement
The measurement process shall interrupt the processor periodically, determine the execution context which was present
when the timed interrupt occurred, and allocate the entire time interval since the previous interrupt to the usage total of
that execution context. The total times for each desired context shall be accumulated throughout the specified
measurement interval. On the completion of a measurement interval, the measured times shall be summed and the usage
and/or the utilization shall be reported.
6.8 Sources of Error
The sources of error for this metric are listed below:
1) The method only reads the execution context when the timed interrupt occurs. Therefore, a context which is
present only briefly between successive timed interrupts will not be observed and accumulate no measured
usage.
2) The interval between successive time interrupts (between ticks, or the length of a jiffy) is a configurable
system control setting in most operation systems. Use of a long interval with respect to processor context
swapping will result in usage assessment with large quantization and to frequently miss contexts that have
short duration.
3) The measurement process itself uses processor resources, and the resources consumed increase with the
number of timed interrupts, or ticks per measurement interval. In Linux systems, the system parameter that
controls the tick interval or duration of a jiffy is called HZ.
4) The accurate generation of timed interrupts on the intended interval boundaries is a function of many factors,
including the activity of other interrupts and virtualisation of the process that generates the tick interval. There
can be slightly less of slightly more than the configured number of ticks in each second, and this causes error
in both the processor usage and utilization measurements.
5) Synchronization of the time-of-day clock with an external reference usually ensures sufficiently accurate
timestamps and measurement intervals, but loss of synchronization for an extended period will cause time
accuracy to suffer.
6.9 Discussion
The processor usage measurement for virtual processors is a topic of ongoing study and advanced development.
Methods which produce more accurate usage measurements are expected in the future.
ETSI
---------------------- Page: 10 ----------------------
11 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
7 Network Metrics
7.1 Background
Network Interfaces exist at many points on the communication paths through the NFVI. It is valuable to monitor the
traffic load of arbitrary interfaces, both the physical interfaces with hardware resources and their virtual counterparts.
Additional information can assist troubleshooting and resolution, so interfaces also indicate problems with attempted
transmission or reception of packets.
7.2 Name
There are four fundamental metrics of traffic load and communication issues:
1) Packet count.
2) Octet count.
3) Dropped Packet count.
4) Errored Packet count.
These four metrics apply to a single direction of transmission. Therefore, transmit and receive versions of the four
metrics make a total of eight for each interface.
7.3 Parameters
The following parameters shall be supported for the four metrics:
• Interface: the name (or other identifier) of a single interface where communication metrics are monitored and
shall be unique within the Scope of measurement.
• Measurement time: the point in time when the counter(s) was (were) read (time and date).
• Interface Speed: the nominal frequency of the physical interface bit clock in bits per second, which governs
the rate that the interface operates, and may be reported using a prefix multiplier. Virtual interfaces may not
have a meaningful value for this parameter.
• Interface Status: the operational state of the interface indicating readiness for use, usually expressed as "up"
or "down".
7.4 Scope
The list of one or more interface resources which are included in the values reported, and whether the resource is
physical or virtual.
7.5 Units of Measure
The Packet interface metrics shall be reported in units of packets as defined by the layer at which the interface is
operating, such as the link layer. Examples include the Ethernet layer and VXLAN.
The Octet interface metric shall be reported in units of 8-bit bytes contained in successfully communicated packets as
defined by the layer at which the interface is operating, such as link layer. Examples include the Ethernet layer and
VXLAN.
ETSI
---------------------- Page: 11 ----------------------
12 ETSI GS NFV-TST 008 V3.2.1 (2019-03)
7.6 Definition
The values of e
...