Asterisk - VoIP Testing - A Practical Guide
Asterisk - VoIP Testing - A Practical Guide
com
www.radcom-inc.com www.protocols.com
________________________________________________________________
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
Table of Figures
Figure 1 - Typical VoIP architecture .............................................................................................................. 4 Figure 2 - Test Strategy .................................................................................................................................. 5 Figure 4 Gateway testing ............................................................................................................................. 9 Figure 5 PAMS provides objective MOS results....................................................................................... 10 Figure 6 - Gatekeeper testing........................................................................................................................ 11 Figure 7 - IVR testing ................................................................................................................................... 13 Figure 8 - DTMF frequencies ....................................................................................................................... 13 Figure 9 VoIP call analysis and packet statistics ....................................................................................... 14 Figure 10 - Billing/Prepaid system testing.................................................................................................... 15 Figure 11 - NMS testing ............................................................................................................................... 16
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
1. Introduction
Voice over IP networks are complex! They represent the converging worlds of tele- and data communications, and therefore present myriad implementation and testing challenges: Integration to traditional telecom infrastructure Integration to billing systems Many add-on services Large variety of protocols Quality is an issue Network specialists are expensive and scarce Reliability is a must Multiple High Quality Services: voice, fax, video, unified messaging, call centers, etc. This white paper presents a typical VoIP architecture and then suggests a framework for testing VoIP networks. The test strategy is presented as well as a detailed discussion of the actual testing required for each network element. Finally, a list of Voice over IP specifications is provided as an appendix as well as a list of acronyms. The main objective of this paper is to provide insight into the intricacies of architecting Voice over IP networks of carrier grade quality. It is intended for network design and test engineers.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
2. VoIP Architecture
A typical VoIP network includes the following components: Media gateways Signalling gateways Gatekeepers Class 5 switches SS7 network Network management system Billing systems
All of these network elements communicate with each other using a plethora of protocols, as can be seen in Figure 1. A detailed list of protocols and specifications can be seen in Appendix I.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
3. Test Strategy
Testing VoIP networks is a tri-fold task: Functionality verification Standards compliance Performance verification
A successful pre-deployment testing strategy must address each of these three facets:
Functionality
Fault-Insertion Test
Long-Term Stability
Verify that all functions work properly and consistently via long term stability testing
Performance Test
Phase 2
Figure 2 - Test Strategy Changes such as software or hardware version upgrades can cause degradation in functionality, quality and performance. Therefore, it is very important to repeat this test cycle after every change made to the VoIP network.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
4. VoIP Testing
Following are VoIP network components that must be tested prior to deployment: be Gateway (GW) and Media Gateway (MG) Gatekeeper (GK) and Media Gateway Controller (MGC) Signaling Gateway Interactive Voice Response (IVR) and Voice Mails Billing and Prepaid system Network Management System (NMS)
environment minimize
deployment,
When functional tests fail there is no way of avoiding the dive into the detailed protocol implementation to verify the conformance of the VoIP devices. This
requires detailed decoding capabilities of all VoIP protocols. H.323 protocols use the ASN.1 notation while protocols such as SIP and Megaco use plain ASCII messages. Figure 3 shows the signalling decodes of a VoIP call and Appendix I includes a complete list of all VoIP protocols and their specifications. Effective pre-deployment testing follows a well-defined methodology that addresses the variety of issues that can impact the networks adherence to specifications in a real world environment. Special consideration should be given to the expected behavior of the VoIP network. This includes parameters such as the number of anticipated users and the estimated amount of traffic per user. Existing network infrastructure should also be taken into account what type of network is used: Frame Relay, ATM, VSAT, xDSL,
______________________________________________________________________________________ VoIP Testing - A Practical Guide.doc Page 6 6/14/01
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
WLL etc. The expected network performance including parameters such as latency, packet loss and available bandwidth is also of significant importance. The test engineer should also consider implementation specific parameters such as the compression methods that will be used, the packet structure of the packetized voice and more. The Poisson statistical model, a generally accepted tool to predict end user behavior, should be incorporated in the pre-deployment test plan. Using this model and based on the assumption that the average call duration is 180 sec, the VoIP network specifications can be defined using the following parameters: 1. Blocking - defined as the percentage of calls that get a busy signal because all lines are in use. This can be calculated as,
Blocking =
Or in other words,
Blocking =
2. Busy Hour Traffic -This is the amount of call traffic handled by a group of phone lines during the busiest hour of the busiest day for your system. Busy Hour Traffic is defined in units of Erlangs or CCS. It can be typically calculated as,
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
N(E1) =
Simultaneous calls can be made according to number of trunks i.e. 24/23/30 (for T1CAS/T1-PRI/E1-PRI respectively), but the limitation will be derived from two other factors: Compression method Guaranteed bandwidth
After the Voice over IP network has been proven for functionality, a series of stress tests should be conducted. It is important to have a consistent definition of stress. The recommended criteria for a stressed network dictate the configuration of the test devices and are as follows: A. Pre-define number of calls per session and 100 setup calls per second. B. Create Jitter, Packet-loss, Packet out of sequence and Latency in Uniform mode. C. The VAD and the silence suppression mechanism should be activated. D. The RTP packets should consist of 1 frame per packet and 3 frames per packet.
The foregoing reflects general requirements involved in VoIP network testing. following will address specific tests of the various components: Gateway testing Gatekeeper testing IVR testing Billing system testing Network management system testing
The
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
5. Gateway Testing
GW Testing
the convergence VoIP network the connection between the packet side and the circuit side. One has to test the
323Sim
GW
MediaPro
QPro
Signalling performance is measured as the Grade of Service (GoS) and media performance is measured as Quality of Figure 3 Gateway testing
Service (QoS). The tests include the generation of a large volume of calls from the circuit side and analysis of the signalling and media performance of these calls on the packet side. A second stage includes the generation of a large volume of calls from the packet side and analysis of the performance of these calls on the circuit side. Finally, it is recommended that the complete system be tested using an end-to-end test scheme, like the one displayed in Figure 3. Two gateways are connected through an Internet cloud passing calls that are generated on the circuit side. This is the most ubiquitous configuration in current VoIP networks. The scenario includes performance
measurement on both the circuit side and the packet side to provide a complete picture of the capability of the network under test. The tests should include a variety of aspects: Compression and De-compression Bandwidth utilization Silence suppression and VAD DTMF detection and Generation Jitter suppression and Echo cancellation Fall-back to PSTN mechanism Alternative re-routing mechanism IVR for 2-Stage Dialing
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
Moreover, evaluating Quality is testing the and Voice
extremely
important. The algorithm most commonly used for these purposes was developed by British Telecom and it is called PAMS Analysis (Perceptual Measurement
System). A speech signal is generated on one side of the network and the degraded signal is captured at the other side. A quality prediction is made on Figure 4 PAMS provides objective MOS results
the received signal based on a mathematical comparison to a stored reference file. The PAMS algorithm implements a model of the human hearing and transforms the speech signal to a three-domain representation time, frequency and amplitude. It is important to be able to perform this test from the circuit network to the packet network and from the packet network to the circuit network. Finally, in a real converged network voice and data are not the only types of traffic. Fax is also very common on VoIP networks. When considering fax transmissions the most important thing to test is the packet loss recovery mechanism. This includes the T.38 redundant packet transmission, the TCP retransmission sliding window mechanism and the FEC (Forward Error Correction). Furthermore, the switching mechanism between fax and voice needs to be tested. All of these tests can be performed by sending fax traffic through a simulated packet network with a variety of different network conditions emulating the loss of packets and measuring the quality of the fax received.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
6. Gatekeeper Testing
The Gatekeeper is the traffic controller of the Voice over IP network. It determines
GK Testing
GK
RAS
RAS
operation
under
stressful
network
GW
register VoIP elements. Privacy and security are an important aspect of any network and are of particular concern on a VoIP network. Therefore, it is also important to test the Admission and Authorization mechanism on the Gatekeeper. The Gatekeeper communicates with both the VoIP terminals and the Gateway, and the language it uses is H.225 and more specifically RAS (Registration, Admission, Status). To properly test the compliancy of the Gatekeepers implementation of RAS, emulation of a VoIP terminal performing RAS negotiation with the Gatekeeper under a stressed network is required. Once the Gatekeeper accepts a terminal, it can make calls and use the Routing Directory Service that the Gatekeeper provides. This routing can be done in two ways least cost routing or best cost routing. Least cost routing means that the least costly route will be selected. Best cost routing means that the best BPS (Bit Per Second) route will be selected. In other words, the Gatekeeper will choose a route that provides the best combination of performance and cost. Some Gatekeepers support RSVP
(Resource ReSerVation Protocol) and can assign a route to a call based on the resources available toward the receiving end. Gatekeepers have two modes of operation - direct mode and routed mode. The routed mode is more commonly used. When the gatekeeper performs address translation, the gatekeeper provides endpoints with the transport address for the call signaling channel
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
destination. In the direct mode, the gatekeeper provides the endpoints with the address of the destination endpoint and directs them to the call-signaling channel so that all messages can be exchanged directly between the two endpoints without gatekeeper involvement. The Gatekeeper test procedure should include tests for both modes of call control routing. The Gatekeeper can also control bandwidth allocation. Through H.225.0 signaling, the gatekeeper is able to limit the bandwidth of the call to less than what was requested as well as reject calls from a terminal if it determines that there is insufficient bandwidth available on the network to support the call. The testing scenario should include several tests with calls generated asking for a bandwidth that is just below the allocated bandwidth and just above it to verify the operation of the bandwidth allocation mechanism on the Gatekeeper. This should be performed with a variety of bandwidth settings on the Gatekeeper.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
7. IVR Testing
IVR (Interactive Voice Response) is an integral part of any business phone system. Practically every call center
QPro GW HUB RTP InterSim HUB
IVR Testing
GW
implements some sort of an IVR system because it reduces operational and human resource costs. For VoIP
323Sim
MediaPro
IVR/Voice-Mail
systems to be used in a business environment they must support IVR, which also means that they have to be tested to ensure their correct operation in real world applications. Both functionality and performance under stress need to be tested. IVR systems use DTMF (Dual Tone Multi Frequency) tones to transfer user requests to the system. DTMF tones are the same tones used for tone dialing. The DTMF tones are sums of two sine wave tones at the following frequencies: 1209 Hz 1336 Hz ABC 697 Hz 1 GHI 770 Hz 4 PRS 852 Hz 7 2 JKL 5 TUV 8 1477 Hz DEF 3 MNO 6 WXY 9
TP-00xx, Date 2000, Slide 20
941 Hz *
oper 0 #
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
Testing the capability of VoIP networks to deal with IVR systems must include a DTMF integrity test that passes all combinations of DTMF tones on the VoIP network and verifies the correct transmission over the packet network. But verifying correct
transmission alone is not sufficient, careful attention should be given to ensure that the transmission would remain correct even when the network is under stress traffic. Of paramount importance to IVR systems is the ability to record the users voice. Voice mail is the most common application. Testing this capability of the IVR system requires the ability to play back the voice mail and measure voice quality on the recorded audio stream. Voice recognition is another mechanism of IVR systems and it should be tested to ensure its functionality and reliability under stressed network conditions. Finally, all of the above mentioned tests must be conducted under rather severe network conditions since Latency, jitter, packet loss and out of sequence packets are common occurrences in a real world packet network.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
(Call Detail Record) integrity when the network is operational which means 24*7*365. the CDR integrity consists of transmission of the and
TP-00xx, Date 2000, Slide 22
GW
correct
MediaPro
measurement parameters:
following
CLID (Calling Line Identification) Call duration Called ID PIN (Personal Identification Number)
When the network is used for both voice and data traffic, the billing system should also be able to measure bandwidth used by the customer, as well as the Quality of Service provided. Prepaid calling cards allow mobile users to place inexpensive phone calls. This service employs a combination of an IVR system and the billing system and, as such, should also be tested for functionality.
The billing system is automatically connected to the charging system automatically charging a customers account (service provider account or credit card account) upon usage of the network. This is another aspect of the billing system that needs to be verified to ensure that there is no lost revenue. Once again, it is important to perform all of these tests under stressed network conditions.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
9. NMS Testing
The Network Management System will typically have connections to the
GK
NMS
NMS Testing
Gateway and the Gatekeeper of the Voice over IP network. It will aggregate and report on network alarms such as over utilization of the and assigned network
MediaPro 323Sim HUB RTP InterSim
RAS
RAS
HUB
GW
bandwidth,
bottlenecks
This is usually
TP-00xx, Date 2000, Slide 24
status report will be generated every pre-configured period of time. Breakdown maintenance alarms will be sent when a specific failure has
occurred. The testing should include alarms verification when specific failures occur. This can be accomplished by emulating the types of errors that might occur in the real world: Jitter exceeds a certain threshold a typical number would be 5 mSec. Packet loss percentage exceeds a certain threshold a typical number would
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
10. Conclusions
Since VoIP enables provisioning of enhanced telephony services, many service providers and infrastructure vendors are aggressively focusing on this technology. Service providers eye global expansion as a means of achieving economies of scale and increasing their subscriber base. Toward that end, many are engaged in building POPs on international markets and/or entering partnerships with local players. However, in order to attract and maintain customers, VoIP networks must deliver a successful combination of functionality, performance and quality. This paper offers a guideline to pre-deployment testing methodology that will help ensure consistent and reliable delivery of the carrier-grade customer experience demanded by mission-critical applications.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
MGCP/SGCP
Megaco
SDP
SIP
RTP RTCP
RSTP
RSVP
Internet Protocol Device Control Media Gateway Control Protocol MEdia GAteway COntrol Session Description Protocol Session Initiation Protocol Real Time Protocol Real Time Control Protocol Real Time Streaming Protocol Resource ReSerVation Protocol
http://www.alternic.org/drafts/drafts-t-u/draft-taylor-ipdc00.txt
http://www.ietf.org/rfc/rfc2705.txt?number=2705
RFC 3015
http://www.ietf.org/rfc/rfc3015.txt
RFC 2327
http://www.ietf.org/rfc/rfc2327.txt?number=2327
RFC 2543
http://www.ietf.org/rfc/rfc2543.txt?number=2543
http://www.ietf.org/rfc/rfc1889.txt?number=1889
http://www.ietf.org/rfc/rfc1889.txt?number=1889
RFC 2326
http://www.ietf.org/rfc/rfc2326.txt?number=2326
RFC 2205
http://www.ietf.org/rfc/rfc2205.txt?number=2205
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
GoS Grade of Service - The probability of a call being blocked or delayed more than a specified interval, expressed as a decimal fraction. Grade of service may be applied to the busy hour or to some other specified period or set of traffic conditions. Grade of service may be viewed independently from the perspective of incoming versus outgoing calls, and is not necessarily equal in each direction. H.245 The H.245 control channel is responsible for control messages governing operation of the H.323 terminal. H.323 This standard defines a set of call control channel set up and CODEC Specifications for transmitting real time voice and video over networks that dont offer guaranteed service or high quality of service. H.323 is comprised of a number of standards. IE IP Information Element a field within a signalling message. Internet protocol - The IP part of the TCP/IP protocol, which routes a message across networks. Each entry on the Internet has a unique IP address for purposes of routing. IPDC (Internet Protocol Device Control) A protocol for controlling media gateways developed by the Technical Advisory Committee, which was convened by Level 3 and others. It analyzes incoming data signals, in band control signals and tones and sets up and controls the appropriate gateways. It also handles management and reporting. ISP ITSP IVR Internet Service Provider Internet Telephony Service Provider (Interactive Voice Response) An automated telephone answering system that responds with a voice menu and allows the user to make choices and enter information via the keypad. IVR systems are widely used in call centers as well as a replacement for human switchboard operators. The system may also integrate database access and fax response. Jitter The Jitter of an audio stream is defined as the variation (calculated as standard deviation) of the inter arrival times of the audio RTP packets. For each pair of successive RTP packets the difference in arrival time at
______________________________________________________________________________________ VoIP Testing - A Practical Guide.doc Page 20 6/14/01
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
the receiver is divided by the difference in the transmission time at the transmitter. These ratios are accumulated for the whole audio stream and the standard deviation of these values provides the jitter of the stream. Kbps KHz LIM Mbps Megaco Kilo bits per second. KiloHertz Line Interface Module Million bits per second (MEdia GAteway COntrol) An IP telephony protocol that is a combination of the MGCP and IPDC protocols. It is simpler than H.323 MGCP Media Gateway Control Protocol. Used for controlling telephony gateways from external call control elements called media gateway controllers or call agents. MOS Mean Opinion Score a method for measuring voice quality. Provides a means of evaluating the subjective performance of voice and/or video transmission equipment using procedures as set out in ITU-T P.800 Packet A frame or block of data used for transmission over communication channels. PAMS PDD Perceptual Analysis Measurement System Post Dialing Delay - The time between punching in the last digit of a telephone number and receiving a ring or busy signal. PGAD Port QoS Post Gateway Answer Delay A communications connection to the PC or to a device Quality of Service - The ability to define a level of performance in a data communications system. RTCP RTP Real time control protocol, used for control of RTP. Real Time protocol, used by RSVP to establish communication between user and network. RTP Real time protocol, IETF specification for audio and video signal management.
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________
Silence Suppression Transmission where silence during the voice conversation is filled with other transmission such as data, video etc. SIP Session Initiation Protocol, an application layer control simple signaling protocol for VoIP implementations. SSRC UDP VoD VoIP A unique identifier of the audio stream, part of the RTP header. User datagram protocol, the transport layer above IP. Voice over Data Voice over Internet Protocol
RADCOM Equipment, Inc. 6 Forest Ave. Paramus, NJ 07652 TEL: 800-RADCOM-4 FAX: 201-556-9030 E-Mail: [email protected] [email protected]
www.radcom-inc.com www.protocols.com
________________________________________________________________