A critical part of any test or set of tests in the 2G/3G/4G/LTE and IMS environment involves measurements that can provide insight into the operation of the individual nodes and the networks that connect and surround them. A comprehensive set of Operational Measurements (OMs) should be an integral part of any test tool, so that a broader understanding of the capacity and capabilities of the network entities under test is provided, as well as the necessary information for operators and vendors to judge network and node capacity and adequately plan future requirements and upgrades.
Testing today’s sophisticated mobile networks requires measurements beyond the usual messages sent/received/error counts and standard statistics, so it is important that the set of measurements available provide awareness of more complex concepts such as transaction minimum, maximum, average and standard deviation of duration. For stateful interfaces, transaction session measurements should be provided. Measurements such as message retries, retransmissions, duplicates, invalid, unsupported and timeouts should be available as should measurements from multiple levels of the involved protocol stacks, and interface resources, such as sockets and queues.
This information is required for both conformance testing and performance testing and the test tool should provide access to such information under load.
dsTest provides such a set of Operational Measurements that are collected at configurable intervals.
All dsTest OMs can be displayed with the dsClient CLI. With dsClient GUI, OMs can be graphically displayed, saved, and replayed. Refer to Graphing Measurements.
The measurements collected by dsTest are stored in a SQLite database file. Measurement database files are not overwritten by the next test run but are rolled over to a new file when the next run starts. The maximum number of files to retain can be configured.
A conversion utility that generates CSV files from the SQLite database is provided so that the OMs can be imported into a spreadsheet program.
Types of dsTest OMs
OMs consist of two main types: static and dynamic. Static OMs are always defined, with a fixed identifier, and created when the dsTest configuration is loaded. For example, see the PCEF Gx Interface Application OMs. Dynamic OMs are created as needed when a configuration file is loaded based upon the needs of the specific configuration. For example, specific dynamic OMs are created for aSmartEvents™ state machine based upon the states and event handlers defined in the state machine.
Categories of dsTest OMs
Categories of OMs are:
- Event Accumulators;
- Duration Accumulators;
- Value Logs;
- Error Accumulators.
Event accumulators provide an individual count of a specific event, including:
- Number of transaction initiated;
- Number of transactions successfully completed;
- Count of each specific (application interface dependent) message type sent and received;
- Dictionary (Diameter interfaces) validation success.
Duration accumulators provide information on:
- Elapsed time for successful transactions;
- Elapsed time from request to answer;
- Transaction Duration Variance, which gives the cumulative square of the difference each transaction’s duration is from the average duration. From this accumulator, the standard deviation of transaction duration can be calculated.
Value logs are collected for items such as:
- Minimum Transaction Duration;
- Maximum Transaction Duration;
- Available transactions, transactions that are available to use in new transactions;
- Transmit and receive queue depths.
Error accumulators track error events, including:
- Number of transaction that failed to complete successfully. In the case of Diameter messages, these accumulators track each of the defined Diameter Result Codes received and generated. Refer to ourDiameter Result Codes Reference page for more details.
- Transactions reset;
- Unsupported messages received;
- Error messages received;
- No transactions available;
- Transaction timeouts;
- Requests retransmitted;
- Duplicate requests received and transmitted;
- Dictionary invalid;
- Transaction retries.
dsTest Operational Measurements
The following outline lists the specific OMs sets that are collected by dsTest. Each item is linked to thedsTest Help description for listed OM set.
Mobile Application Part (MAP) Interfaces
C, E, D/D’, Gc, Gd, Gf, Gr/Gr’
GPRS Tunneling Protocol (GTP)
GTP Protocol, GTP’/Ga
CC/DCCA, Cx, Gx, Gxx, Rf/Gz, Ro/Gy, Rx, S13, S6a/S6d, S6b, S9, Sd, Sh, SLg,SLh, STa, SWa, SWm, SWx/Wx, Sy, Diameter Agent
RADIUS Application, RADIUS port