Active and Passive Metrics and Methods (and everything in-between, or Hybrid)

The adjectives "active" and "passive" have been used for many years to distinguish two different classes of Internet performance assessment. The first Passive and Active Measurement (PAM) Conference was held in 2000, but the earliest proceedings available on-line are from the second PAM conference in 2001 [https://www.ripe.net/ripe/meetings/pam-2001]. The notions of "active" and "passive" are well-established. In general: An Active metric or method depends on a dedicated measurement packet stream and observations of the stream. A Passive metric or method depends *solely* on observation of one or more existing packet streams. The streams only serve measurement when they are observed for that purpose, and are present whether measurements take place or not. As new techniques for assessment emerge it is helpful to have clear definitions of these notions. This memo provides more detailed definitions, defines a new category for combinations of traditional active and passive techniques, and discusses means to evaluate new techniques as they emerge. This memo provides definitions for Active and Passive Metrics and Methods based on long usage in the Internet measurement community, and especially the Internet Engineering Task Force.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

The scope of this memo is to define and describe Active and Passive versions of metrics and methods which are consistent with the long-time usage of these adjectives in the Internet measurement community and especially the Internet Engineering Task Force. Since the science of measurement is expanding, we provide a category for combinations of the traditional extremes, treating Active and Passive as a continuum and designating combinations of their attributes as Hybrid methods. Further, this memo's purpose includes describing multiple dimensions in which to evaluate methods as they emerge.

This section defines the key terms of the memo. Some definitions use the notion of "stream of interest" which is synonymous with "population of interest" defined in clause 6.1.1 of ITU-T Recommendation Y.1540. The definitions are consistent with .

The standard definition of a quantity, produced in an assessment of performance and/or reliability of the network, which has an intended utility and is carefully specified to convey the exact meaning of a measured value. (This definition is consistent with that of Performance Metric in RFC 2330 and RFC 6390).

The procedure or set of operations having the object of determining a Measured Value or Measurement Result.

See section 2 of for this definition (a location in the network where packets can be observed), and related definitions. The comparable term defined in IETF literature on Active measurement is Measurement Point, see section 4.1 of . Two terms have come into use describing similar actions at the identified point in the network path.

Active measurement methods have the following attributes: Commonly, the packet stream of interest is generated as the basis of measurement. Another packet stream may be generated to increase traffic load, but the loading stream itself may not be measured. The packets in the stream of interest have fields (or are augmented or modified to include fields) which are dedicated to measurement. Since measurement usually requires determining the corresponding packets at multiple measurement points, a sequence number is the most common information dedicated to measurement. The Source and Destination of the packet stream of interest are usually known a' priori. The characteristics of the packet stream of interest are known at the Source at least, and may be communicated to Destination as part of the method. When adding traffic to the network for measurement, Active Methods influence the quantities measured to some degree, and those performing tests should take steps to quantify the effect(s) and/or minimize such effects.

An Active Metric incorporates one or more of the aspects of Active Methods in the metric definition. For example, IETF metrics for IP performance (developed according to the framework) include the Source packet stream characteristics as metric input parameters, and also specify the packet characteristics (Type-P) and Source and Destination IP addresses (with their implications on both stream treatment and interfaces associated with measurement points).

Passive measurement methods are based solely on observations of undisturbed and unmodified packet stream of interest dependent on the existence of one or more packet streams to supply the stream of interest dependent on the presence of the packet stream of interest at one or more designated observation points. Some passive methods simply observe and collect information on all packets that pass Observation Point(s), while others filter the packets as a first step and only collect information on packets that match the filter criteria, and thereby narrow the stream of interest. It is common that passive methods are conducted at one or more Observation Points. Passive methods to assess Performance Metrics often require multiple observation points, e.g., to assess latency of packet transfer across a network path between two Observation Points. In this case, the observed packets must include enough information to determine the corresponding packets at different Observation Points. Communication of the observations (in some form) to a collector is an essential aspect of Passive Methods. In some configurations, the traffic load associated with results export to a collector may influence the network performance. However, the collection of results is not unique to Passive Methods, and the load from management and operations of measurement systems must always be considered for potential effects on the measured values.

Passive Metrics apply to observations of packet traffic (traffic flows in ). Passive performance metrics are assessed independent of the packets or traffic flows, and solely through observation. Some refer to such assessments as "out-of-band". One example of passive performance metrics for IP packet transfer can be found in ITU-T Recommendation Y.1540, where the metrics are defined on the basis of reference events as packet pass reference points, so the metrics are agnostic to the distinction between active and passive when the necessary packet correspondence can be derived from the observed stream of interest when required.

Methods of Measurement which use a combination of Active Methods and Passive Methods, to assess Active Metrics, Passive Metrics, or new metrics derived from the a' priori knowledge and observations of the stream of interest. ITU-T Recommendation Y.1540 defines metrics are applicable to the hybrid category, since packet correspondence at different observation/reference points could be derived from "fields which are dedicated to measurement", but otherwise the methods are passive. With respect to the stream of interest, Hybrid methods fit in the continuum as follows, in terms of what happens at the Source(or Observation Point nearby): If you generate the stream of interest => Active If you augment of modify a stream of interest => Hybrid If you solely observe a stream of interest => Passive

This section illustrates the definitions and presents some examples.

If we compare the Active and Passive Methods, there are at least two dimensions on which methods can be evaluated. This evaluation space may be useful when a method is a combination of the two alternative methods. The two dimensions (initially chosen) are: The degree to which the stream of interest effects overall network conditions experienced by that stream and other streams. This is a key dimension for Active measurement error analysis. (Comment: There is also the notion of time averages - a measurement stream may have significant effect while it is present, but the stream is only generated 0.1% of the time. On the other hand, observations alone have no effect on network performance. To keep things simple, we consider the stream effect only when it is present.) The degree to which stream characteristics are know a' priori. There are methodological advantages of knowing the source stream characteristics, and having complete control of the stream characteristics. For example, knowing the number of packets in a stream allows more efficient operation of the measurement receiver, and so is an asset for active measurement methods. Passive methods (with no sample filter) have few clues available to anticipate what the protocol first packet observed will use or how many packets will comprise the flow, but once the standard protocol of a flow is known the possibilities narrow (for some compliant flows). Therefore this is a key dimension for Passive measurement error analysis. There are a few examples we can plot on a two-dimensional space. We can anchor the dimensions with reference point descriptions.

We recognize that method categorization could be based on additional dimensions, but this would require a different graphical approach. For example, "effect of stream of interest on network conditions" could easily be further qualified into: effect on the performance of the stream of interest itself: for example, choosing a packet marking or DSCP resulting in domain treatment as a real-time stream (as opposed to default/best-effort marking. effect on unmeasured streams that share the path and/or bottlenecks: for example, an extremely sparse measured stream of minimal size packets typically has little effect on other flows (and itself), while a stream designed to characterize path capacity may effect all other flows passing through the capacity bottleneck (including itself). effect on network conditions resulting in network adaptation: for example, a network monitoring load and congestion conditions might change routing, placing some flows to alternate paths to mitigate the congestion. At present, we have combined 1 and 2 on one axis, as examination of examples indicates strong correlation of affects on this pair, and network adaptation is not addressed. As suggestions emerge we will re-examine the possibilities. It is apparent that different methods of IP network measurement can produce different results, even when measuring the same path at the same time. The two dimensions of the graph help to understand how the results might change with the method chosen. For example, an Active Method to assess throughput adds some amount of traffic to the network which might result in lower throughput for all streams. However, a Passive Method to assess throughput can also err on the low side due to unknown limitations of the hosts providing traffic, competition for host resources, limitations of the network interface, or private sub-networks that are not an intentional part of the path, etc. And Hybrid Methods could easily suffer from both forms of error. Another example of potential errors stems from the pitfalls of using an Active stream with known bias, such as a periodic stream defined in . The strength of modelling periodic streams (like VoIP) is a potential weakness when extending the measured results to other application whose streams are non-periodic. The solutions are to model the application streams more exactly with an Active Method, or accept the risks and potential errors with the Passive Method discussed above.

In , an IPv6 Option Header for Performance and Diagnostic Measurements (PDM) is described which (when added to the stream of interest at strategic interfaces) supports performance measurements. This method processes a user traffic stream and adds "fields which are dedicated to measurement". Thus: The method may have a small effect on the measured stream and other streams in the network. The measured stream has unknown characteristics until it is processed to add the PDM Option header. We conclude that this is a Hybrid method, having at least one characteristic of both active and passive methods.

Draft , proposed to color packets by re-writing a field of the stream at strategic interfaces to support performance measurements. This method processes a user traffic stream and inserts "fields which are dedicated to measurement". Thus: The method may have a small effect on the measured stream and other streams in the network (smaller than PDM above). The measured stream has unknown characteristics until it is processed to add the coloring in the header, and the stream could be measured and time-stamped during that process. We note that proposes a method similar to , and ippm-list discussion indicates may be covered by the same IPR as . We conclude that this is a Hybrid method, having at least one characteristic of both active and passive methods.

When considering privacy of those involved in measurement or those whose traffic is measured, there is sensitive information communicated and observed at observation and measurement points described above. We refer the reader to the privacy considerations described in the Large Scale Measurement of Broadband Performance (LMAP) Framework , which covers active and passive measurement techniques and supporting material on measurement context.

This memo makes no requests for IANA consideration.

Thanks to Mike Ackermann for asking the right question, and for several suggestions on terminology. Brian Trammell provided key terms and references for the passive category. Tiziano Ionta reviewed the draft and suggested the classification for the "coloring" method of measurement. Nalini Elkins identified several areas for clarification following her review. Bill Jouris reviewed 01 editorially and suggested several improvements.