As part of Continuous Path Analysis (CPA), APM regularly traces the route for each network path, hop-by-hop, from source to target. This information can be used to confirm whether traffic is traveling through expected hops and networks.

The primary tool for route analysis is the Route Visualization on the Network Paths page as it provides a visual of historical routes taken for up to 20 paths at the same time.

Screenshot of the Route Visualization.

The most recently measured route for a path can also be seen on the Route chart.

When to use the route visualization

The Route Visualization provides a way to view traceroutes taken on multiple paths in a single view and to see how the routes taken by packets on those paths change over time. This enables you to:

  • see routes beyond your private network
  • review historical routing changes
  • watch how routes change over time
  • compare routes of different paths to find common/different hops between them

Use the Route Visualization in cases where a routing issue is suspected. These include:

  • a significant change in RTT (Round-trip time) (due to packets taking a non-optimal route, potentially because of a network failure or non-optimal DNS configuration)
  • a significant change in capacity (due to packets taking a non-optimal route, potentially because of a network failure)
  • seeing unexpected route changes
  • connectivity failures

We have included a number of examples showing how to interpret results from the Route Visualization tool to diagnose network issues.

View network path route details

To view network path route details using the Route Visualization:

  1. Navigate to Delivery > Network Paths.
  2. Search for or filter the paths you are interested in.
    • Search/filter results of 20 or fewer paths is required. Consider filtering on Network Path Name, Monitoring Point, Target, and/or Location.
  3. If the Route Visualization (Routes pane) is not visible, click the Show Routes button.
    • The Route Visualization is displayed.
      Screenshot of the Route Visualization within Delivery > Network Paths.
  4. Drag the timeline cursor along the timeline to see how the routes change over time.

Note: The granularity of measurements is approximately every five minutes so routes may change more than what is shown. The route changes shown indicate that the route observed at the time of the measurement is different than the last time it was measured.

See Route Visualization legend for descriptions of the elements on the Route Visualization.
See Route Visualization actions for descriptions of the actions available.

Route visualization legend

Page Element Description
Screenshot of the Hops radio button on the Route Visualization. Hops View
Display the hops traversed by traceroutes from source to target on the filtered paths in the selected time range.
Screenshot of the Networks radio button on the Route Visualization. Networks View
Display the public networks traversed by traceroutes from source to target on the filtered paths in the selected time range. The networks can be expanded to see the hops within.

Note: APM Private requires access to AppNeta’s IP2ASN service.
Screenshot of the Export element on the Route Visualization. Export information about the filtered routes to a text file.
Screenshot of the Center Time selector on the Route Visualization. Center time selection
Specify the date and time of interest. Click “Now” for the current time. Data is available for a period of time and depends on whether you are using APM Public or APM Private.
Screenshot of the Range selector on the Route Visualization. Time range selection
Specify the time range to review. The Center Time is at the center of the range unless “Now” is selected. In that case, the Center Time is at the end of the range.
Screenshot of the Protocols radio buttons on the Route Visualization. Protocol
Selecting a Protocol shows traceroutes taken by ICMP, UDP, or TCP. Auto shows the protocol used by Continuous Path Analysis (CPA) for each path. For single-ended paths this is ICMP. For dual-ended paths this is UDP.
Screenshot of the timeline on the Route Visualization. Timeline cursor
Drag the timeline cursor to see how routes change over the selected time range.
Screenshot of the data bar on the Route Visualization. Data bar
Hovering over or clicking an object (source, hop, target, or link between them) highlights all traceroutes traversing the object. It also reveals details about the object in the data bar. Clicking the object locks the data in the data bar until it is unlocked by clicking the lock icon or another object.
Screenshot of the data bar focused on hops actions. Data bar - Hops
In Hops view, when you click a source, a hop, or a target, actions are available.
- Filter: Show only routes that pass through the selected object.
- Unlock: Unlock data bar selection.
Screenshot of the data bar focused on networks actions. Data bar - Networks
In Networks view, when you select a network, actions are available.
- Expand/collapse: Expand/collapse selected network.
- Filter: Show only routes that pass through the selected network.
- Unlock: Unlock data bar selection.
Screenshot of traceroutes on the Route Visualization. Traceroutes
The visualization shows all routes taken on the selected paths over the selected time range. The line thickness indicates how often the route was used. Hover over or click any of the elements in the visualization to see information about it in the data bar.
Screenshot of a grey line from a traceroute. Traceroutes - Grey line
Traceroutes observed within the selected time range but not at the selected time (using the Center Time cursor) are shown in grey.
Screenshot of a black line from a traceroute. Traceroutes - Black line
Traceroutes active at the selected time (using the Center Time cursor) are shown in black.
Screenshot of a blue line from a traceroute. Traceroutes - Blue line
When you hover over a line, all traceroutes that passed across that link at some point during the selected time range appear blue.
Screenshot of a hop (larger grey dot) from a traceroute. Traceroutes - Hop
A point on the traceroute at which routing occurs.
Screenshot of a path source (blue circle with white diamond inside) from a traceroute. Traceroutes - Source
The origin of traceroutes on a path.
Screenshot of a path target (blue circle with white circle inside and a blue dot inside that) from a traceroute. Traceroutes - Target
An IP address that is a target of traceroutes.
Screenshot of a path source/target (half source icon and half target icon) from a traceroute. Traceroutes - Source/Target
An IP address that is both a source and a target of traceroutes.
Screenshot of an incomplete hop (red circle with white circle inside) from a traceroute. Traceroutes - Incomplete hop
Red outline indicates that one or more traceroutes terminate here instead of reaching the target. The incomplete traceroutes are listed in the Incomplete routes object (see below).
Screenshot of a unknown hop (smaller grey dot) from a traceroute. Traceroutes - Unknown hop
A hop where IP address information is not available (hop did not respond to traceroute requests).
Screenshot of the incomplete routes dropdown on the Route Visualization. Incomplete routes
Shows the number of traceroutes that did not make it to their target. Clicking the object shows the targets in question.

Route visualization actions

Once the network path of interest are filtered (max 20 paths), there are a number of actions available on the Route Visualization:

See routes change over time

Drag the timeline cursor along the timeline to see how routes change over the selected time range.

Set center time and time range

The Center Time and Range elements are used to specify the window of time to review. By default, the time range is 10 minutes and the center time is the current time.

Change the center time to a specific date and time
  1. In the Center Time element, click the date and time.
  2. Select Date and specify the date.
  3. Select Time and specify the time.
  4. Click Close.
    • The time cursor is on the time specified in the center of the time range.
    • The routes that were active at the specified time are in black.
    • Slide the time cursor to see routes active at different times.
Change to the current date and time

In the Center Time element, click Now.
* The time range is the last 10 minutes and the time cursor is in the center of that range.

Change the time range (up to 24 hours)

In the Range element, click the desired time range.
* The time range is adjusted accordingly.

Filter by hop

To view all routes that pass through a given hop on the Route Visualization:

  1. Select Hops.
  2. Click the hop you are interested in.
  3. Click the filter icon ().
    • Only routes that pass through the selected hop are shown.
    • The filter is displayed at the top left of the pane.

Filter by network

To view all routes that pass through a given network on the Route Visualization:

  1. Select Networks.
  2. Click the network you are interested in.
  3. Click the filter icon ().
    • Only routes that pass through the selected network are shown.
    • The filter is displayed at the top left of the pane.

Filter by protocol

To view routes taken by packets of a specific protocol:

  1. Select either Hops or Networks.
  2. In the Protocol field, select the protocol to filter on.
    • Auto shows the protocol used by Continuous Path Analysis (CPA) for each path. For single-ended paths this is ICMP. For dual-ended paths this is UDP.

Expand/collapse a network

You can see all the hops within a network by expanding the network and dismiss them by collapsing it.

To expand or collapse a network:

  1. Select Networks.
  2. Click to the left of a network name to expand the network.
  3. Click to the left of a network name to collapse the network.

View hop information

Click or hover over a hop to see information about a hop and all traceroutes that pass through it. This can include:

  • Hop hostname.
  • Hop IP address.
  • Network the hop belongs to.
  • Traceroutes passing through the hop including protocol, number, and percentage of all traceroutes sent during the selected time range.
  • Average Round-trip time (RTT) over the selected time range.

Click or hover over a link to see information about a link and all traceroutes that pass through it. This includes:

  • The hostname or IP address of the hops on either side of the link.
  • Traceroutes passing across the link including protocol, number, and percentage of all traceroutes sent during the selected time range.

Click or hover over a link or hop to view traceroutes with a common link or hop. All traceroutes that pass through the selected link or hop become blue.

Label a hop

To call out points of interest on the diagram when sharing it (for example, via a screenshot), you can label a hop with up to two labels from data in the Route Visualization data bar.

To label a hop:

  1. Select Hops.
  2. Click the hop you want to label.
  3. In the data bar, select the hop data you want to show on the diagram.

Export route information

You can export a text file of hop data for all active (black) traceroutes for a given point in time.

To export route information:

  1. Click Export.
    • A dialog showing the data to be exported appears.
  2. Click Export.
    • A text file containing the hop data is downloaded to your computer.

Examples

Highly recommended - Watch a video describing some of the features of the Route Visualization as well as walkthroughs that use the tool to diagnose real-life problems.

The following are real-life examples using the Route Visualization to diagnose network issues.

Variable browsing experience

Symptom

Several users at two sites (one on the west coast of North America, the other on the east coast) complained of a highly variable browsing experience. Sometimes pages loaded quickly and other times the load times were quite slow.

Investigation

It was determined that when the problem occurred, it seemed to happen when accessing the same set of web sites. As these sites were not normally monitored, web paths were configured to monitor them from both sites. Network Admins then reviewed the corresponding network path performance data and noticed highly variable round-trip times (RTT) for the network paths in question.
Screenshot of the RTT chart showing highly variable RTT.
The End-user Experience chart confirmed the issue on the web paths during the same time period (script completion varied from about 8 seconds to over 30 seconds).
Screenshot of the End User Experience chart showing highly variable script completion times.

The Route Visualization was then used to explore the issue further. One of the network paths exhibiting the problem was selected and the time range being reviewed was specified. The first thing Network Admins noticed was that a single target hostname was shared by multiple hosts. This is typical for Content Delivery Networks (CDNs) where the same content is distributed from a variety of locations. The idea is that clients can access content at the host closest to them in order to reduce network delays.
GIF of the Route visualization showing the same target with multiple IP addresses.

By hovering over each of the target hosts, the Network Admins could see that three of the four hosts showed relatively high average RTT. For example, this one showed an average RTT of 90.03ms:
Screenshot of a target host with an average RTT of 90.03ms.
One host showed a relatively low average RTT of 1.83ms:
Screenshot of a target host with an average RTT of 1.83ms.
Using Networks view, it was clear that the “fast” host was closer to the source than the “slower” hosts. Packets had fewer networks/hops when traveling between the source and the “fast” host than they did traveling between the source and the “slower” hosts.
Screenshot of Network view with a target host with an average RTT of 90.03ms.
Screenshot of Network view with a target host with an average RTT of 1.83ms.
So why wasn’t content always being delivered by the host closest to the client? Network Admins knew that the CDN would provide the address of the host closest to the DNS resolver requesting it, and they had one resolver on the west coast and one on the east coast. As long as clients made their requests to their local resolver, all should work as expected.

Discovery

After reviewing the DNS configuration on client PCs it was determined that all were configured for both west coast and east coast DNS resolvers. Clients would sometimes be using a west coast resolver and at other times they’d use an east coast resolver. The DNS resolvers would respond properly with the IP addresses of the hosts closest to them, but this meant that west coast users could access content housed on the east coast and east coast users could access content housed on the west coast, causing longer round-trip times and longer script completion times.

Resolution

To resolve the issue, Network Admins updated the DNS configurations on the clients at each site so that they all used local DNS resolvers. Content was then delivered from the closest/fastest target host to the client, resolving the slowness issue.

Note: When remote offices use a central corporate DNS service, the same issue can occur. Browsing experience may not be optimal when accessing sites serviced by a CDN as remote office clients will access the target host closest to the DNS service, not necessarily the one closest to the client.

Service outage

Symptom

Users were suddenly unable to connect to a major SaaS service.

Investigation

Network Administrators reviewed network paths to the service and noticed that all showed connectivity violations (Red circle with white X). They then reviewed the Route Visualization to see if the problem was at the service or somewhere in the network.

Discovery

Network paths were filtered by the target service with the connectivity violations. In the Hops view it was clear that traceroutes on all paths were unable to make it to the service for approximately 45 minutes starting at about 1:17pm (black lines stopped part way to target).
GIF of the issue in Hops view showing all traceroutes to the target disconnecting part way to the target at about 1:17pm and connecting again about 45 minutes later.
But it wasn’t clear what all these routes had in common. They all seemed to be ending at seemingly unrelated hops. Changing from Hops view to Networks view it became clear that all routes ended at networks immediately prior to the target network.
GIF of the issue in Networks view showing all traceroutes to the target disconnecting from the target's network at about 1:17pm.
By expanding the service’s network to see the hops inside it, it was clear that the traceroutes were being stopped at the edge of their network.
GIF of the issue in Networks view with the service's network expanded showing all traceroutes to the target disconnecting from the target's network at about 1:17pm.
The issue was clearly located at the service provider’s network.

Resolution

After 45 minutes, connectivity was restored. The service provider had resolved the issue in their network.

Common issue on multiple network paths

Symptom

Problems were noticed when uploading data to a specific site.

Investigation

Network Administrators reviewed the network path performance data and could see significant and ongoing Data Loss on the path.
Screenshot of the Data Loss chart for a path showing data loss occurring.
They checked out other paths from the same source and to the same target. They found some from the same source that had a similar Data Loss pattern occurring at the same time …
Screenshot of the Data Loss chart for a path showing data loss occurring.
… and others that didn’t.
Screenshot of the Data Loss chart for a path showing data loss not occurring.
They then reviewed the paths on Route Visualization to find the hops common to all problem paths and not on any non-problem paths.

Discovery

By selecting the network paths that had the problem (labelled 2 and 3 in the diagram) as well as one that did not have the problem (labelled 1 in the diagram), it was clear that there were two hops in common (identified by the red box in the diagram) on the “problem” paths that weren’t shared with the “non-problem” path. Using the Networks view on the Route Visualization, they could see the network provider that was responsible for the offending routers.
Screenshot of the Route Visualization showing two hops in common on paths with data loss issues (labelled 2 and 3) that are not in common with paths with no data loss (labelled 1).

Resolution

After a call to the network provider explaining the issue, they were able to investigate further and fix the problem.