Use the following steps to install a v35 virtual monitoring point in your network. The monitoring point runs on a host machine that has the KVM hypervisor installed. KVM runs on one of the following Linux versions: CentOS-7.x, RHEL 7.x or Ubuntu 16.04.3 LTS.

Once KVM is available and the necessary software and configuration information has been downloaded to the KVM host, the VM can be created and configured to connect to APM. After making the physical connection between the KVM host and your network the v35 will connect back to APM and can be used for monitoring. Once the setup is completed, additional features can be set up as required.

  1. Prerequisites
  2. Set up a KVM host
    1. Set up a CentOS or RHEL KVM host
    2. Set up an Ubuntu KVM host
  3. Load install scripts
  4. Load base and config images
  5. Create a v35 virtual machine
    1. Create a v35 with 1Gbps
    2. Create a v35 with 10Gbps
  6. Configure for web proxy if required
  7. Assign licenses
  8. Set the monitoring point location
  9. Set up monitoring
  10. Configure additional features if required
    1. Set up v35 with additional interfaces
    2. Set up v35 for Usage monitoring
    3. Set up v35 with VLAN
    4. Set up v35 with additional 10Gbps
    5. Set up v35 with 10Gbps and VLAN
  11. Helpful commands

Prerequisites

The following are the minimum requirements for the KVM host system:

  • Capable of running a guest virtual machine with at least the v35 requirements.
  • Runs CentOS-7.x, RHEL 7.x or Ubuntu 16.04.3 LTS.
  • Has the following virtualization management packages installed:
    • CentOS-7.x, RHEL 7.x - qemu-kvm qemu-img libvirt libvirt-python libvirt-client virt-install bridge-utils
    • Ubuntu 16.04.3 LTS - qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virtinst
  • Has up to four Ethernet NICs installed on the KVM host. These are mapped to interfaces on the v35 as follows:
v35 interface Required or Optional Supported NIC speeds Function
eth0 Required 1Gbps or 10Gbps Network connectivity, Delivery monitoring, and Experience monitoring
eth1 Optional 1Gbps Usage monitoring
eth2 Optional 1Gbps or 10Gbps Delivery monitoring and Experience monitoring
eth3 Optional 1Gbps or 10Gbps Delivery monitoring and Experience monitoring
  • For 1Gbps: A Linux bridge or an OVS bridge to bind the interface on the v35 to the physical interface on the host.
    • The OVS bridge is required only if you want to use a VLAN. If this is the case, Open vSwitch (OVS) software must be installed. Open a support ticket for any assistance.
  • For 10 Gbps: A 10Gbps NIC with SR-IOV support. Supported 10Gbps NICs include:
    • Intel X540
    • Intel X550
    • Intel 82599

Set up a KVM host

If you have not already done so, set up either a CentOS, a RHEL, or an Ubuntu KVM host.

Set up a CentOS or RHEL KVM host

To set up a CentOS-7.x or RHEL 7.x KVM host:

  1. Install CentOS-7.x or RHEL 7.x.
  2. Install the required packages:

    sudo yum -y install qemu-kvm qemu-img libvirt libvirt-python libvirt-client virt-install bridge-utils
    
  3. Reboot.
  4. Verify the installation to see that all is okay and no guests are running at this point:

    virsh list --all
    

Set up an Ubuntu KVM host

To set up an Ubuntu 16.04.3 LTS KVM host:

  1. Install Ubuntu 16.04.3 LTS.
  2. Install the required packages:

    sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virtinst
    
  3. Verify the installation to see that all is okay and no guests are running at this point:

    virsh list --all
    

Load install scripts

There are two scripts used for v35 setup that must be downloaded and copied to the KVM host:

  • vk35tool.py - Script to create a v35 and configure its interfaces.
  • vk35hook.sh - Hook script to automatically connect a mirror port to an OVS bridge.

To download and copy the script files:

  1. Click this link to download the .tar file containing the scripts.
  2. Copy the .tar file to the KVM host.
    • Copy it to the directory you want to run the vk35tool.py script from.
  3. Unzip the .tar file:

    tar xvzf vk35-tools.tar.gz
    
  4. Make a directory for the vk35hook.sh script:

    mkdir -p /var/lib/appneta/
    
  5. Copy the vk35hook.sh script to the directory:

    cp vk35hook.sh /var/lib/appneta/
    

Load base and config images

The AppNeta software required for the v35 includes a base image (.qcow2) and a config image (.iso). These must be downloaded and then copied to the KVM host.

To download the base and config images and copy them to your host:

  1. Log in to APM.
  2. Make sure you are using the correct organization
  3. Navigate to > Manage Monitoring Points > + Add Monitoring Points > Virtual Monitoring Point > Download the v35 for KVM Platform.
  4. Click Download KVM Base Image.
    • The KVM base image (.qcow2) is downloaded to your computer.
  5. Click Continue.
  6. Follow the instructions provided to specify the v35 configuration.
    • Note the Hostname as it will be used later.
    • A .zip file containing the config image (.iso) is downloaded to your computer.
  7. Copy the KVM base image (.qcow2) and the config image (.iso) to /var/lib/libvirt/images/ on the host system.
    • It is good practice to rename these files to match the Hostanme but leave the .qcow2 and .iso extensions. This is especially important if you are setting up more than one v35 on the KVM host as the .qcow2 and .iso files cannot be shared by virtual machines.

Create a v35 virtual machine

Once the software and configuration files are available on the host, the v35 virtual machine that uses them can be created.

You can create a v35 that uses a 1Gbps NIC or a 10Gbps NIC for connectivity to APM as well as Delivery and Experience monitoring:

Create a v35 with 1Gbps

eth0 on the v35 is mapped to the Linux bridge interface on the KVM host you will be using for network connectivity as well as Delivery and Experience monitoring. By default, the v35 is created with 4096MB RAM and two virtual CPUs.

You’ll need the following information to complete the setup:

Parameter Description
<v35_name> v35 virtual machine name. For simplicity, use the Hostname noted in a previous step.
<base_image> Name of the downloaded base image (e.g., <Hostname>.qcow2).
<config_image> Name of the downloaded config image containing initial monitoring point configuration (e.g., <Hostname>.iso).
<interface_type> Interface type on the KVM host (determined below).
<interface_name> Interface on the KVM host (determined below).

To create the v35:

  1. Log on to the KVM host.
  2. Go to the directory you copied the install scripts (vk35tool.py and vk35hook.sh) to.
  3. Determine which interfaces are available to the v35.

    ./vk35tool.py scan
    
  4. Install the v35.

    ./vk35tool.py install <v35_name> \
    --image /var/lib/libvirt/images/<base_image> \
    --config /var/lib/libvirt/images/<config_image> \
    --eth0 type=<interface_type>,source=<interface_name>
    
  5. Verify that the v35 is “running”.

    virsh list --all
    

Connect to the network via 1Gbps

To physically connect the monitoring point to the network via 1Gbps:

  1. Verify that the switch you’ll be connecting to has its default port speed and duplex set to auto-negotiate.
  2. Using a standard Ethernet cable, connect the physical interface on the KVM host associated with the bridge port configured above to the switch.
  3. Verify that the v35 is connected to APM.
    1. In APM, navigate to > Manage Monitoring Points.
    2. Wait for the new monitoring point to show up in the list (you may need to refresh your screen).
      • The monitoring point status should show “Connection Established” (). At this point it is configured properly.
      • If it doesn’t appear after a few minutes, you’ll need to troubleshoot the problem.

Create a v35 with 10Gbps

There are a few steps required in order to set up the v35 to use a 10Gbps interface:

  1. Configure host for virtual functions support.
  2. Isolate CPU cores and configure host for SR-IOV support.
  3. Apply changes and verify them.
  4. Create a virtual function network.
  5. Create v35 using 10Gbps interface.
  6. Connect to the network via 10Gbps.

Note: This procedure should be completed by a system administrator with good knowledge of hypervisor management and tuning. To configure the system for optimal performance, they should also understand the CPU architecture of the KVM host. Those without this expertise should contact AppNeta Support for help.

Configure host for virtual functions support

To configure the KVM host for virtual functions support:

  1. Edit /etc/modprobe.d/ixgbe.conf on the KVM host.
  2. Add the following lines:

    options ixgbe max_vfs=8
    blacklist ixgbevf
    

Isolate CPU cores and configure host for SR-IOV support

You can improve performance by running the v35 on at least two dedicated CPU cores. The following points should be used to determine which CPU cores on the KVM host to isolate for use with the v35:

  • If you have a multi-socket (NUMA) system:
    • The isolated cores should be on the same node so they access the same local memory.
    • Use cores on the node that is connected to the PCI slot containing the 10Gbps NIC.
  • If you have a hyper-threaded system (where a single physical processor core can act like two logical cores), make sure the isolated cores are not logical cores on the same physical core.

To configure the KVM host to isolate CPU cores and for SR-IOV support:

  1. Edit /etc/default/grub.
  2. Locate the line containing GRUB_CMDLINE_LINUX.
  3. Specify isolated CPU cores by adding isolcpus=<cpus> and nohz_full=<cpus> to it.
    • For example, “GRUB_CMDLINE_LINUX … isolcpus=4-6,8-10 nohz_full=4-6,8-10 …” isolates cores 4, 5, 6, 8, 9, and 10.
  4. Configure for SR-IOV support by adding intel_iommu=on (or amd_iommu=on if the KVM host has an AMD processor) to it.
    • For example, “GRUB_CMDLINE_LINUX … intel_iommu=on …”.
  5. Regenerate the grub.cfg file and rebuild initramfs.
    • For CentOS or RHEL, use:

       sudo grub2-mkconfig -o /boot/grub2/grub.cfg
       sudo dracut --regenerate-all -f
      
    • For Ubuntu, use:

       sudo update-grub
       sudo update-initramfs -k all -u
      

Apply changes and verify them

To apply the changes made in the previous steps and verify them:

  1. Reboot the KVM host to apply the changes.
  2. Verify that virtual functions (vf) have been created.

    ip link show
    
  3. Verify that CPU cores are isolated and that SR-IOV support is configured.

    cat /proc/cmdline
    
    • Look for isolcpus=<cpus> and nohz_full=<cpus>.
    • Look for intel_iommu=on (or amd_iommu=on).

Create a virtual function network

Once the KVM host is configured with virtual function (vf) support and SR-IOV you can create a virtual function network for the v35 to use. In this example we create enp5s0f1-vf based on interface enp5s0f1.

To create a virtual function network:

  1. Create an .xml network definition file. For example:

    $ cat enp5s0f1-vf.xml
    <network>
      <name>enp5s0f1-vf</name>
      <forward mode='hostdev' managed='yes'>
        <pf dev='enp5s0f1'/>
      </forward>
    </network>
    
  2. Define the network.

     virsh net-define enp5s0f1-vf.xml
    
  3. Start the network.

     virsh net-start enp5s0f1-vf
    
  4. Configure the network to start each time the system is started.

     virsh net-autostart enp5s0f1-vf
    

Create v35 using 10Gbps interface

eth0 on the v35 is mapped to the 10Gbps interface on the KVM host you will be using for network connectivity as well as Delivery and Experience monitoring. By default, the v35 is created with 4096MB RAM and two virtual CPUs.

This procedure assumes that a 10Gbps NIC is already installed in the KVM host and that the previous steps have been completed.

You’ll need the following information to complete the setup:

Parameter Description
<v35_name> v35 virtual machine name.
<base_image> Name of the downloaded base image (e.g., <Hostname>.qcow2).
<config_image> Name of the downloaded config image containing initial monitoring point configuration (e.g., <Hostname>.iso).
<10g_interface_name> The name of a 10Gbps Virtual Function (-vf) network on the KVM host (e.g., enp5s0f1-vf).
<cpus> The list of isolated CPU cores (e.g., 4-6,8-10).

To create the v35:

  1. Log on to the KVM host.
  2. Go to the directory you copied the install scripts (vk35tool.py and vk35hook.sh) to.
  3. Find the name of the 10Gbps interface you want to use (e.g., enp5s0f1-vf).

    ./vk35tool.py scan
    
  4. Install the v35.

    ./vk35tool.py install <v35_name> \
    --image /var/lib/libvirt/images/<base_image> \
    --config /var/lib/libvirt/images/<config_image> \
    --eth0 type=network,source=<10g_interface_name> \
    --cpuset <cpus>
    
  5. Verify that the v35 is “running”.

    virsh list --all
    

Connect to the network via 10Gbps

To physically connect the monitoring point to the network via 10Gbps:

  1. Verify that the switch you’ll be connecting to is 10Gbps capable has its default port speed and duplex set to auto-negotiate.
  2. Using the appropriate cable, connect the 10Gbps port to the switch.
  3. Verify that the v35 is connected to APM.
    1. In APM, navigate to > Manage Monitoring Points.
    2. Wait for the new monitoring point to show up in the list (you may need to refresh your screen).
      • The monitoring point status should show “Connection Established” (). At this point it is configured properly.
      • If it doesn’t appear after a few minutes, you’ll need to troubleshoot the problem.

Configure for web proxy if required

If Internet traffic on your network is forwarded by a web proxy, you must configure the monitoring point with the proxy details.

Assign licenses

Every monitoring point is sold with a base license that allows it to monitor a certain number of applications. The base license must be assigned to the monitoring point in APM before it can be used. Optionally, add-on licenses can be assigned.

To license a monitoring point:

  1. Return to the browser tab where you are running APM.
  2. Navigate to > Manage Monitoring Points.
  3. Wait for the new monitoring point to show up in the list (you may need to refresh your screen).
    • The monitoring point status should show “Connection Established” ().
    • If it doesn’t appear after a few minutes, you’ll need to troubleshoot the problem.
  4. Assign a base license to the monitoring point.
  5. Optionally, assign add-on licenses as required.

Set the monitoring point location

When adding a new monitoring point to APM, you need to specify its location. This setting provides essential geographical context for the data that the monitoring point collects.

To set the location of the monitoring point:

  1. Enter the monitoring point location if prompted during the setup procedure.

Set up monitoring

At this point the monitoring point setup is complete. Continue the APM setup procedure at Set up monitoring.

Configure additional features if required

Once the basic v35 is set up and is connecting back to APM, you can install additional features as required. These include:

Set up v35 with additional interfaces

In addition to eth0, you can configure up to two additional interfaces for Delivery and Experience monitoring (eth2 and/or eth3). This procedure assumes that the v35 is already set up and running.

You’ll need the following information to complete the setup:

Parameter Description
<v35_name> v35 virtual machine name.
<v35_interface> v35 interface to use (either eth2 or eth3).
<interface_type> Interface type on the KVM host (determined below).
<interface_name> Interface on the KVM host (determined below).

To update the v35 to use an additional interface:

  1. Determine which interfaces are available to the v35.

    ./vk35tool.py scan
    
  2. Shutdown the v35.

    virsh shutdown <v35_name>
    
  3. Update the v35 configuration to use an additional interface.

    ./vk35tool.py update <v35_name> \
    --<v35_interface> type=<interface_type>,source=<interface_name>
    
  4. Start the v35.

    virsh start <v35_name>
    
  5. Verify that the v35 is “running”.

    virsh list --all
    
  6. Verify that the v35 is connected to APM.

    1. In APM, navigate to > Manage Monitoring Points.
    2. Wait a couple minutes for the new monitoring point to show up in the list (you may need to refresh your screen).
      • The monitoring point status should show “Connection Established” (). At this point it is ready to monitor traffic.
      • If it doesn’t appear after a few minutes, you’ll need to troubleshoot the problem.

Set up v35 for Usage monitoring

This procedure assumes that the v35 is already set up and running and that Open vSwitch (OVS) software is installed on your KVM host and an OVS bridge has been created. You’ll need the following information to complete the setup:

Parameter Description
<v35_name> v35 virtual machine name.
<ovs_net_span> Virtual network based on the OVS bridge configured for mirroring traffic
<hostport> Port on host to be mirrored

There are a few steps required in order to set up the v35 for Usage monitoring:

  1. Configure the v35 for Usage monitoring.
  2. Connect to a mirror port.
  3. Set up the vk35hook.sh script.

Configure the v35 for Usage monitoring

To update the v35 configuration for Usage monitoring:

  1. Shutdown the v35.

    virsh shutdown <v35_name>
    
  2. Update the v35 configuration for Usage monitoring. Map eth1 to the OVS port configured for capturing mirrored traffic. Mirror traffic from the specified host port.

    ./vk35tool.py update <v35_name> \
    --eth1 type=network,source=<ovs_net_span>, \
    portgroup=<hostport>
    
  3. Start the v35.

    virsh start <v35_name>
    
  4. Verify that the v35 is “running”.

    virsh list --all
    
  5. Verify that the v35 is connected to APM.

    1. In APM, navigate to > Manage Monitoring Points.
    2. Wait for the new monitoring point to show up in the list (you may need to refresh your screen).
      • The monitoring point status should show “Connection Established” ().
      • If it doesn’t appear after a few minutes, you’ll need to troubleshoot the problem.

Connect to a mirror port

If you have not already done so, see Deploying Your Monitoring Point for information on where to deploy your monitoring point in you network for Usage monitoring.

To connect the v35 to a mirror port:

  1. Using a standard Ethernet cable, connect the OVS source port on the KVM host configured for capturing mirrored traffic to the port on your switch configured to mirror uplink traffic.

Set up the vk35hook.sh script

The vk35hook.sh script maps the physical port that is capturing mirrored traffic to the OVS bridge (which is mapped to eth1 on the v35). On some systems, whenever the v35 is shut down this mapping is removed, so the vk35hook.sh script needs to be run whenever the v35 is started to reset this mapping.

To have the vk35hook.sh script run automatically when the v35 is started:

  1. With the v35 running, run vk35tool.py (as superuser) using the “genhook” parameter:

    sudo ./vk35tool.py genhook <v35_name>
    

Set up v35 with VLAN

This procedure assumes that the v35 is already set up and running and that Open vSwitch (OVS) software is installed on your KVM host and an OVS bridge has been created. You’ll need the following information to complete the setup:

Parameter Description
<v35_name> v35 virtual machine name.
<v35_interface> v35 interface to use (either eth0, eth2, or eth3).
<ovs_net_lan> Virtual network based on the OVS bridge configured to forward VLAN traffic.
<vlan_portgroup> Port group that defines a set of tagged and untagged VLANs the traffic is received from.

To update the v35 to use VLAN:

  1. Shutdown the v35.

    virsh shutdown <v35_name>
    
  2. Update the v35 configuration to use VLAN.

    ./vk35tool.py update <v35_name> \
    --<v35_interface> type=network,source=<ovs_net_lan>, \
    portgroup=<vlan_portgroup>
    
  3. Start the v35.

    virsh start <v35_name>
    
  4. Verify that the v35 is “running”.

    virsh list --all
    
  5. Verify that the v35 is connected to APM.

    1. In APM, navigate to > Manage Monitoring Points.
    2. Wait a couple minutes for the new monitoring point to show up in the list (you may need to refresh your screen).
      • The monitoring point status should show “Connection Established” (). At this point it is ready to monitor VLAN traffic.
      • If it doesn’t appear after a few minutes, you’ll need to troubleshoot the problem.

Set up v35 with additional 10Gbps

10Gbps support is only available for network connectivity (access to APM), Delivery monitoring, and Experience monitoring. It is not available for Usage monitoring. This procedure assumes that a v35 is already set up and running and that it has been configured for 10Gbps network connectivity. It also assumes that you are planning to add a second 10Gbps port for Delivery or Experience monitoring and that the 10Gbps NIC for this is already installed.

There are a few steps required in order to set up an additional 10Gbps interface:

  1. Create an additional virtual function network.
  2. Configure v35 to use 10Gbps interface.
  3. Connect to the network via additional 10Gbps port.

Create an additional virtual function network

This procedure assumes that virtual functions support and SR-IOV support have been configured on the KVM host. This allows you to create a virtual function network for the v35 to use. In this example we create enp5s0f2-vf based on interface enp5s0f2.

To create a virtual function network:

  1. Create an .xml network definition file. For example:

    $ cat enp5s0f2-vf.xml
    <network>
      <name>enp5s0f2-vf</name>
      <forward mode='hostdev' managed='yes'>
        <pf dev='enp5s0f2'/>
      </forward>
    </network>
    
  2. Define the network.

     virsh net-define enp5s0f2-vf.xml
    
  3. Start the network.

     virsh net-start enp5s0f2-vf
    
  4. Configure the network to start each time the system is started.

     virsh net-autostart enp5s0f2-vf
    

Configure v35 to use 10Gbps interface

You’ll need the following information to complete the setup:

Parameter Description
<v35_name> v35 virtual machine name.
<v35_interface> v35 interface to use (either eth2 or eth3).
<10g_interface_name> The name of a 10Gbps Virtual Function (-vf) network on the KVM host (e.g., enp5s0f2-vf).

To update the v35 configuration for 10Gbps:

  1. Find the name of the 10Gbps interface you want to use (e.g., enp5s0f2-vf).

    ./vk35tool.py scan
    
  2. Shutdown the v35.

    virsh shutdown <v35_name>
    
  3. Update the v35 configuration for 10Gbps support. Map an interface on the v35 to the 10Gbps interface on the KVM host.

    ./vk35tool.py update <v35_name> \
    --<v35_interface> type=network,source=<10g_interface_name>
    
  4. Start the v35.

    virsh start <v35_name>
    
  5. Verify that the v35 is “running”.

    virsh list --all
    

Connect to the network via additional 10Gbps port

To physically connect the monitoring point to the network via 10Gbps:

  1. Verify that the switch you’ll be connecting to is 10Gbps capable has its default port speed and duplex set to auto-negotiate.
  2. Using the appropriate cable, connect the 10Gbps port to the switch.
  3. Verify that the v35 is connected to APM.
    1. In APM, navigate to > Manage Monitoring Points.
    2. Wait for the monitoring point to show up in the list (you may need to refresh your screen).
      • The monitoring point status should show “Connection Established” ().
      • If it doesn’t appear after a few minutes, you’ll need to troubleshoot the problem.

Set up v35 with 10Gbps and VLAN

This procedure assumes that the v35 is already set up and running and that it has been configured for 10Gbps.

You’ll need the following information to complete the setup:

Parameter Description
<v35_name> v35 virtual machine name.
<v35_interface> v35 interface to use (either eth0, eth2, or eth3).
<10g_interface_name> The name of a 10Gbps Virtual Function (-vf) interface on the KVM host (e.g., enp5s0f1-vf).
<vlan_id> VLAN ID to use.

To update the v35 configuration to use VLAN:

  1. Shutdown the v35.

    virsh shutdown <v35_name>
    
  2. Update the v35 configuration to specify the VLAN to use on the 10Gbps interface.

    ./vk35tool.py update <v35_name> \
    --<v35_interface> type=network,source=<10g_interface_name>,vlan=<vlan_id>
    
  3. Start the v35.

    virsh start <v35_name>
    
  4. Verify that the v35 is “running”.

    virsh list --all
    
  5. Verify that the v35 is connected to APM.

    1. In APM, navigate to > Manage Monitoring Points.
    2. Wait for the new monitoring point to show up in the list (you may need to refresh your screen).
      • The monitoring point status should show “Connection Established” (). At this point it is ready to monitor VLAN traffic on the 10Gbps interface.
      • If it doesn’t appear after a few minutes, you’ll need to troubleshoot the problem.

Helpful commands

The following commands can be used on the host to manage your KVM virtual machines.

Command Description
virsh list –all List all virtual machines on your host.
virsh start <v35_name> Start a v35 instance that has been shut down.
virsh shutdown <v35_name> Shutdown a v35 instance (graceful).
virsh destroy <v35_name> Shutdown a v35 instance (force).
virsh undefine <v35_name> Delete a v35 instance.
virsh -h virsh help.

The following commands can be used to create and manage v35 virtual machines.

Command Description
vk35tool.py install <v35_name> Create a v35 VM.
vk35tool.py update <v35_name> Update a v35 VM configuration.
vk35tool.py show <v35_name> Show network interface on a v35 VM.
vk35tool.py scan List Linux interfaces available to a v35 VM.
vk35tool.py -h vk35tool help.