Voice Delivery
- Voice monitoring metrics
- Assessing a voice service provider
- Assessing endpoints on a LAN
- Voice quality assessments
- Basic voice assessments
- Create a Basic voice assessment
- Run a Basic voice assessment
- View Basic voice assessment results
- Download Basic voice assessment results
- Share Basic voice assessment results
- Edit a Basic voice assessment
- Delete a Basic voice assessment
- Pause a Basic voice assessment
- Resume a paused Basic voice assessment
- Stop a Basic voice assessment
- Advanced voice assessments
- Voice tests
- Create a Voice test
- Create a Voice test from a template
- Run a Voice test
- View Voice test results
- Download Voice test results
- Share Voice test results
- Edit a Voice test
- Delete a Voice test
- Create a Voice test template
- View a Voice test template
- Edit a Voice test template
- Delete a Voice test template
- Create a Voice test schedule
- View a Voice test schedule
- Stop a scheduled Voice test
- Copy a Voice test schedule
- Delete a Voice test schedule
AppNeta Performance Manager (APM) has monitoring capability designed specifically for ensuring good voice quality. It enables you to assess your network’s ability to handle voice traffic with three tools: Basic voice assessments, Advanced voice assessments, and Voice tests. In order to use these tools, the source monitoring point needs to be licensed for voice.
Basic voice assessments provide a way to assess the ability of multiple network paths to carry voice traffic. This testing happens at a point in time and relies on diagnostic techniques to infer voice quality. Basic voice assessments do not require a monitoring point as a target and are typically used to:
- Provide a quick voice assessment of your network in advance of a VOIP deployment
- Check an existing voice deployment for issues
- Assess network paths using a large number of concurrent calls (up to 250)
- Assess endpoints (e.g., VOIP handsets) on a LAN
- Assess a voice service provider across a WAN (no MP required at provider)
- Validate QoS
Advanced voice assessments also provide a way to assess the ability of multiple network paths to carry voice traffic. The difference from Basic assessments is that this testing happens continually over a specified period of time and relies on both diagnostic and continuous monitoring techniques to infer voice quality. Like basic voice assessments, Advanced voice assessments do not require a monitoring point as a target. They are typically used to:
- Provide a quick voice assessment of your network in advance of a VOIP deployment
- Check an existing voice deployment for issues
- Assess network paths (up to 25) over a period of time in order to capture transient conditions
- Assess endpoints (e.g., VOIP handsets) on a LAN
- Assess a voice service provider across a WAN (no MP required at provider)
- Determine how varying call loads affect the ability to handle voice traffic using the call ramp-up feature
Voice tests provide a way to simulate multiple simultaneous voice calls using the actual voice codecs and protocols. They allow you to more accurately measure of how your network would treat voice traffic, but at the expense of greater bandwidth consumption. A voice test is comprised of one or more sessions; each session specifies a path, a number of concurrent calls, and QoS settings. Tests are structured this way so that you can evaluate a variety of call scenarios, all within the same test. Voice tests require a monitoring point as the target (though the target monitoring point does not require a voice license) and are typically used to:
- Create voice traffic load to expose network problems. Other tools are used to diagnose the issues found.
- Test a network’s ability to handle actual voice traffic (for example, prior to deploying additional voice endpoints)
- Test a network’s ability to handle inter-office voice traffic (for example, HQ to branch)
Note: If you do not have a voice license, you can use PathTest as a replacement for Voice test, though it does not use voice protocols (SIP/RTP/RTCP) to simulate voice traffic so results are less accurate than using Voice test. PathTest is also useful when larger call loads are required.
Features include:
Feature | Basic voice assessments |
Advanced voice assessments |
Voice tests |
---|---|---|---|
Concurrent calls | 250 | 25 | Depends on monitoring point capacity |
Target monitoring point required | No | No | Yes (target does not require a voice license) |
Protocols used | ICMP | ICMP | UDP (SIP/RTP/RTCP) |
Network load | Light | Light | True call loads |
Targets assessed/tested | One at a time | One at a time, round robin, for period specified |
Simultaneously |
Compares QoS and non-QoS | Yes (if QoS selected) |
No | No |
Uses Diagnostics | Yes | Yes | No |
Uses CPA | No | Yes | No |
Individual test results available | Yes | No | No |
Schedule assessment/test | No | Yes | Yes |
Call ramp-up | No | Yes | No |
Downloadable report | Yes | Yes | Yes |
Statistics generated | |||
- Readiness | Yes | Yes | No |
- MOS | Yes | Yes | Yes |
- Voice packet loss | Yes | Yes | Yes |
- Voice jitter | Yes | Yes | Yes |
- Latency | No | Yes | Yes |
- Availability | No | Yes | No |
- Total capacity | No | Yes | No |
- Packet discards | No | No | Yes |
- Packet reordering | No | No | Yes |
Voice monitoring metrics
The following describes the metrics available on voice assessment and voice test reports:
- Readiness - A representation of MOS that helps you understand how well a path is handling voice traffic:
- 4.2 - 5.00 - Excellent
- 3.8 - 4.19 - Good
- 3.4 - 3.79 - Marginal
- 2.8 - 3.39 - Poor
- 1.0 - 2.79 - Very poor
- MOS - An estimate of the rating that a typical user would give to the sound quality of a call. It is expressed on a scale of 1 to 5, where 5 is perfect. It is a function of loss, latency, and jitter. It also varies with voice codec and call load. If audio codec G.722.1 is selected for a session, a MOS score will not appear.
- Voice packet loss - See Data and voice loss.
- Voice jitter - See Data and voice jitter.
- Latency - See Round-trip time and latency.
- Availability - The percentage of time a connection exists between the monitoring point and the target during the period of the assessment.
- Total capacity - See Capacity
- Packet discards - Packets arriving at the destination too late, as determined by the jitter buffer size, are discarded. For example, when the default jitter buffer size of 40ms is used, if a packet arrives more than 40ms later than expected, it is discarded.
- Packet reordering - Packets within a burst that arrived out of their original sequence are considered reordered. Because packets travel through a network path independently, they may not arrive in the order in which they were sent. In multi-link, load-balanced networks, data can become reordered, especially during high-traffic periods.
Assessing a voice service provider
If the voice quality you are receiving from a cloud-based VOIP service is not acceptable, there are a few steps you can take to determine where the problem is:
- Test the WAN link - Create a dual-ended path from the location experiencing the problem to an AppNeta WAN Target and let it run for a while. If all looks good, continue.
- Test the connection to the VOIP server - Create a single-ended path from the location experiencing the problem to the service provider’s PBX (you may need to get in touch with the service provider to determine its IP address). Apply the default voice WAN alert profile to capture any events and initiate diagnostics tests. Let this run for a while. If all looks good, continue.
- Run a basic voice assessment - Use the single-ended path to create and run a basic voice assessment against the service provider’s PBX. Review the report generated. If all looks good, continue.
- Run an advanced voice assessment - Use the single-ended path to create and run an advanced voice assessment against the service provider’s PBX. Let it run for a while and review the report generated. If all looks good, continue.
- Contact AppNeta Support - If you are still unable to determine where the voice quality issues are originating from, contact AppNeta Support.
Assessing endpoints on a LAN
When you are assessing voice endpoints on a LAN, make sure you are targeting the correct devices. Devices such as printers will fail a voice assessment. This issue is particularly evident when using the “Discover My Network” method to create single-ended paths. To avoid the issue, narrow the address range to just your VOIP subnet.
Also note that not all VOIP handsets will respond to test packets. This results in a low MOS even though the audio sounds fine. If you need help, contact AppNeta Support.
Voice quality assessments
Voice quality assessments are based on four metrics that affect voice quality: bandwidth utilization, packet loss, latency, and jitter. When human voice is converted from analog to digital, it is sampled thousands of times per second, using one of several techniques called ‘codecs’ that not only converts the sampled voice, but also compresses it. The amount of compression, the number of samples taken, and the number of samples packed into each IP packet all directly affected how much bandwidth is consumed by the call. When your call is traversing a network with insufficient bandwidth for the VOIP configuration, the call will experience higher latency and possibly packet loss. Packet loss is important because VOIP traffic uses the UDP protocol. With UDP, lost packets are not re-transmitted, resulting in broken audio on the listener’s end.
Basic voice assessments
The actions available for Basic voice assessments include:
- Create a Basic voice assessment
- Run a Basic voice assessment
- View Basic voice assessment results
- Download Basic voice assessment results
- Share Basic voice assessment results
- Edit a Basic voice assessment
- Delete a Basic voice assessment
- Pause a Basic voice assessment
- Resume a paused Basic voice assessment
- Stop a Basic voice assessment
Create a Basic voice assessment
To create a Basic voice assessment:
- Navigate to Delivery > Voice Assessments.
- Click + New Basic Voice Assessment.
- In the Assessment Name field, enter a name for the assessment.
- In the Select Paths section, specify the paths to include in the assessment.
- Data only target types will not be available for selection.
- Paths with a target type of Auto are converted to target type Voice Handset.
- Update the remaining fields as appropriate.
- Call Load - If the call load is >1 and you specify a call load target type that matches the path target type, the path will be tested up to five times with different call loads. Otherwise, it will be tested once.
- Quality of Service - If this is set, two tests will be performed for each call load - one with QoS set and one without.
- Click Save Assessment.
- The assessment is saved.
Note: For paths targeting an OS X sequencer, note that OS X rate limits ICMP responses by default. There are two ways to disable this behavior:
- To disable it temporarily, enter sysctl –w net.inet.icmp.icmplim=0 from the command line. The limit will be reset when the system restarts.
- To permanently remove it, create /etc/sysctl.conf and add the following line: net.inet.icmp.icmplim=0.
Run a Basic voice assessment
To run a Basic voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the assessment you want to run, select > Start (or Run Assessment Again).
- The assessment starts (or you will be given a chance to edit the assessment before starting it). The status shows: .
View Basic voice assessment results
To view Basic voice assessment results:
- Navigate to Delivery > Voice Assessments.
- Click the completed assessment you are interested in (status shows: ).
- The assessment results are displayed.
- To view all tests on a path, click to the left of the path.
- The individual tests are listed.
- To view test details, click the test you are interested in.
Download Basic voice assessment results
To download Basic voice assessment results:
- Navigate to Delivery > Voice Assessments.
- Click the completed assessment you are interested in (status shows: ).
- The assessment results are displayed.
- Select > Download PDF.
- Fill in the PDF report settings.
- Click Generate.
- A PDF of the assessment results is downloaded to your computer.
Share Basic voice assessment results
To share Basic voice assessment results:
- Navigate to Delivery > Voice Assessments.
- Click the completed assessment you are interested in (status shows: ).
- The assessment results are displayed.
- Select > Share Link.
- Fill in the required information (for example, the recipient’s email address).
- Click Share.
- A link to the assessment results is emailed to the specified recipient. They will need access to APM in order to view the results.
- Click Close.
Edit a Basic voice assessment
To edit a Basic voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the assessment you want to edit, select > Edit.
- Update fields as appropriate.
- Click Save Assessment.
- The assessment is saved.
Delete a Basic voice assessment
To delete a Basic voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the assessment you want to delete, select > Delete.
- Click OK.
- The assessment is deleted.
Pause a Basic voice assessment
To pause a Basic voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the assessment you want to pause (status shows: ), select > Pause.
- The assessment is paused. The status shows: .
Resume a paused Basic voice assessment
To resume a paused Basic voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the paused assessment you want to resume (status shows: ), select > Resume.
- The assessment resumes. The status shows: .
Stop a Basic voice assessment
To stop a Basic voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the currently running assessment you want to stop (status shows: ), select > Stop.
- The assessment stops. The status shows: .
Advanced voice assessments
The actions available for Advanced voice assessments include:
- Create an Advanced voice assessment
- View an Advanced voice assessment
- Download Advanced voice assessment results
- Share Advanced voice assessment results
- Edit an Advanced voice assessment
- Delete an Advanced voice assessment
- Stop an Advanced voice assessment
- Re-run an Advanced voice assessment
Create an Advanced voice assessment
To create an Advanced voice assessment:
- Navigate to Delivery > Voice Assessments.
- Click + New Advanced Voice Assessment.
- In the Assessment Name field, enter a name for the assessment.
- In the Select Paths section, specify the paths to include in the assessment.
- Data only target types will not be available for selection.
- Paths with a target type of Auto are converted to target type Voice Handset.
- Update the remaining fields as appropriate.
- Assessment Schedule - Schedule the start and end times for testing.
- Call Load Analysis - If the call load is >1 and you specify a call load target type that matches the path target type, the path will be tested up to five times with different call loads. Otherwise, it will be tested once. You can also ramp-up the number of concurrent calls as the test proceeds and have the ramp-up repeat daily.
- Click Start Assessment.
- The assessment is saved and run.
View an Advanced voice assessment
To view Advanced voice assessment results:
- Navigate to Delivery > Voice Assessments.
- Click the completed assessment you are interested in (status shows: ).
- The assessment results are displayed.
Download Advanced voice assessment results
To download Advanced voice assessment results:
- Navigate to Delivery > Voice Assessments.
- Click the completed assessment you are interested in (status shows: ).
- The assessment results are displayed.
- Select > Download PDF.
- Fill in the PDF report settings.
- Click Generate.
- A PDF of the assessment results is downloaded to your computer.
Share Advanced voice assessment results
To share Advanced voice assessment results:
- Navigate to Delivery > Voice Assessments.
- Click the completed assessment you are interested in (status shows: ).
- The assessment results are displayed.
- Select > Share Link.
- Fill in the required information (for example, the recipient’s email address).
- Click Share.
- A link to the assessment results is emailed to the specified recipient. They will need access to APM in order to view the results.
- Click Close.
Edit an Advanced voice assessment
To edit an Advanced voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the assessment you want to edit, select > Edit.
- Update fields as appropriate.
- Click Save Assessment.
- The assessment is saved.
Delete an Advanced voice assessment
To delete an Advanced voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the assessment you want to delete, select > Delete.
- Click OK.
- The assessment is deleted.
Stop an Advanced voice assessment
To stop an Advanced voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the currently running assessment you want to stop (status shows: ), select > Stop.
- The assessment stops. The status shows: .
Re-run an Advanced voice assessment
To re-run an Advanced voice assessment:
- Navigate to Delivery > Voice Assessments.
- For the assessment you want to run, select > Run Assessment Again.
- You will be given a chance to edit the assessment before starting it.
- Click Start Assessment.
- The assessment is run. The status shows: .
Voice tests
You can create a Voice test directly or you can create it from a template. The test can be run once or it can be scheduled to run repeatedly.
The actions available for Voice tests include:
- Create a Voice test
- Create a Voice test from a template
- Run a Voice test
- View Voice test results
- Download Voice test results
- Share Voice test results
- Edit a Voice test
- Delete a Voice test
The actions available for Voice test templates include:
- Create a Voice test template
- View a Voice test template
- Edit a Voice test template
- Delete a Voice test template
The actions available for Voice test schedules include:
- Create a Voice test schedule
- View a Voice test schedule
- Stop a scheduled Voice test
- Copy a Voice test schedule
- Delete a Voice test schedule
Create a Voice test
To create a Voice test:
- Navigate to Delivery > Voice Tests.
- Click + New Voice Test.
- Update fields as appropriate.
- Click Add new Session….
- Select the network path to run the test on.
- Update fields as appropriate.
- Click OK.
- The source and destination are checked for applicability. Four checkmarks are required in each row to continue.
- Add more test sessions if required.
- Click Start Voice Test.
- The Voice test starts.
Note: The maximum number of calls or sessions you can have in a Voice test is subject to device load. APM will reject any test configuration that will exceed device load for either the source or the target. An error message indicates which device would be overloaded and by how much.
Create a Voice test from a template
To create a Voice test from a template:
- Navigate to Delivery > Voice Tests.
- Click the Templates tab.
- For the template you want to use, select > Create Test.
- Update fields as appropriate.
- Click OK.
- The source and destination are checked for applicability. Four checkmarks are required in each row to continue.
- Click Start Voice Test.
- The Voice test starts.
Run a Voice test
To run a Voice test:
- Navigate to Delivery > Voice Tests.
- For the Voice test you want to run, select > Run Test Again.
- Click Start Voice Test.
- The Voice test starts.
View Voice test results
To view Voice test results:
- Navigate to Delivery > Voice Tests.
- Click the completed test you are interested in (status shows: ).
- The test results are displayed.
Download Voice test results
To download Voice test results:
- Navigate to Delivery > Voice Tests.
- Click the completed test you are interested in (status shows: ).
- The test results are displayed.
- Select > Download PDF.
- A PDF of the test results is downloaded to your computer.
Share Voice test results
To share Voice test results:
- Navigate to Delivery > Voice Tests.
- Click the completed test you are interested in (status shows: ).
- The test results are displayed.
- Select > Share Link.
- Fill in the required information (for example, the recipient’s email address).
- Click Share.
- A link to the test results is emailed to the specified recipient. They will need access to APM in order to view the results.
- Click Close.
Edit a Voice test
To edit a Voice test:
- Navigate to Delivery > Voice Tests.
- For the Voice test you want to edit, select > Edit.
- Update fields as appropriate.
- Click Start Voice Test.
- The Voice test starts.
Delete a Voice test
To delete a Voice test:
- Navigate to Delivery > Voice Tests.
- For the Voice test you want to delete, select > Delete.
- Click OK.
- The Voice test is deleted.
Create a Voice test template
Voice test templates are used to create Voice tests with similar criteria.
To create a Voice test template:
- Navigate to Delivery > Voice Tests.
- Click + New Voice Test.
- Update fields as appropriate.
- Click Add new Session….
- Select the network path to run the test on.
- Update fields as appropriate.
- Click OK.
- The source and destination are checked for applicability. Four checkmarks are required in each row to continue.
- Add more test sessions if required.
- Click Save as Template.
- Click Submit.
- Update fields as appropriate.
- Click Save Template.
- The configuration is saved as a template.
- Click the Templates tab to confirm the template was created.
View a Voice test template
To view a Voice test template:
- Navigate to Delivery > Voice Tests.
- Click the Templates tab.
- Click the template you want to view.
- The template is displayed.
Edit a Voice test template
To edit a Voice test template:
- Navigate to Delivery > Voice Tests.
- Click the Templates tab.
- Click the template you want to edit.
- Update fields as appropriate.
- Click Save Template.
- The updated template is saved.
Delete a Voice test template
To delete a Voice test template:
- Navigate to Delivery > Voice Tests.
- Click the Templates tab.
- For the template you want to delete, select > Delete.
- Click OK.
- The template is deleted.
Create a Voice test schedule
Voice test schedules are used to repeatedly execute a Voice test at a given interval for a specified period of time.
To create a Voice test schedule:
- Navigate to Delivery > Voice Tests.
- Click + New Voice Test.
- Update fields as appropriate.
- Click Add new Session….
- Select the network path to run the test on.
- Update fields as appropriate.
- Click OK.
- The source and destination are checked for applicability. Four checkmarks are required in each row to continue.
- Add more test sessions if required.
- Click Schedule Test.
- In the Schedule Name field, enter a schedule name.
- Update the remaining fields as appropriate.
- Click Create Schedule.
- The schedule is saved and a test is scheduled.
- Click the Schedules tab to confirm the schedule was created.
View a Voice test schedule
To view a Voice test schedule:
- Navigate to Delivery > Voice Tests.
- Click the Schedules tab.
- Click the schedule you want to view.
- The schedule is displayed.
Stop a scheduled Voice test
To stop a scheduled Voice test:
- Navigate to Delivery > Voice Tests.
- Click the Schedules tab.
- For the schedule you want to stop, select > Stop.
- The schedule is stopped.
Copy a Voice test schedule
To copy a Voice test schedule:
- Navigate to Delivery > Voice Tests.
- Click the Schedules tab.
- For the schedule you want to copy, select > Copy.
- Click Schedule Test.
- Change the Schedule Name field.
- Update the remaining schedule fields as appropriate.
- Click Create Schedule.
- The new schedule is created.
Delete a Voice test schedule
To delete a Voice test schedule:
- Navigate to Delivery > Voice Tests.
- Click the Schedules tab.
- For the schedule you want to delete, select > Delete.
- If the Delete action is not present, stop the scheduled Voice test first.
- The schedule is deleted.