Experience monitoring is all about your end users, and they don’t just stop after hitting the endpoint: they log in, click links, search, etc. AppNeta Performance Manager (APM) lets you test how well your application responds to a series of user actions like these using a script. You’ll use a Firefox plug-in to record the base script, inevitably there’ll be some troubleshooting involved, then finally, you’ll logically divide your script into sections called ‘milestones’. APM will return timing data, alerts, and an Apdex score for both the web path and each milestone so that you know exactly when and which actions are holding things up.
Two scripting languages are supported: AppNeta and Selenium. While APM currently supports mixing and matching both types of scripts within the same app definition, we encourage you to create all new scripts in Selenium if your monitoring point supports it. Currently Selenium is supported on Global Monitoring Points as well as several Enterprise Monitoring Points. See the Monitoring Point Comparison page for details.
This page covers scripting in Selenium.
HTML elements only! APM supports interaction with html elements only. It does not support 3rd-party browser add-ons or multimedia players. This includes things like Flash, ActiveX, Java applets, browser extensions, and video/audio players.
Unsupported sites: APM does not support scripting against the following sites: Google Analytics, Fullstory, and Segment.io.
- Prepare your environment
- Download the Selenium IDE plug-in
- Record a script
- Clean up
- Have a maintenance plan
Prepare your environment
There are several points you should consider before you start scripting
- Script against the target url. Make sure that you script against the same version of the site that the web monitor will access.
- Obtain the login credentials. We recommend that you set up a dedicated account for the AppNeta script to log into the app being tested. Give it the bare permissions required for your script to run.
- Use a clean browser. Remember that each time it runs, your script will be starting from a blank slate, so you’ll need to make sure the browser you are scripting in is equally clean:
- Close all unnecessary tabs.
- Clear out your browser’s cache and cookies.
- Disable any plugins you don’t need or that could interfere with the script, particularly script blockers and ones that automatically enter text into fields, such as password managers.
The following additional points are recommended, but not compulsory:
- Use two different browsers. Firefox and Chrome for example. Have the site you’re testing against open in Firefox, and APM open in the other. This ensures you don’t lose your work if the site your testing against hangs or crashes Firefox. Plus, you can restart Firefox and clear your cookies without affecting your AppNeta session.
- IP address logging/alerting. If you log authentication requests, be sure to alert on any attempts to authenticate on IP addresses other than your own workstation and the public IP addresses of the web monitors.
- Save your work. The Selenium plug-in allows you to save your script. Save a copy and regularly ctrl+s just like you would any other document.
- Plan ahead. As with writing any coding endeavor it’s best to plan out what you’re trying to do ahead of time. Spend some time thinking about what you want to test, and what results you expect to see from it.
- Use a dual-monitor setup. Not compulsory but helpful. While recording have AppNeta open in one browser and the app you’re testing against open in the other. While troubleshooting, perform an action on one screen to observe the expected result, and then edit to your code accordingly on the other screen.
Download the Selenium IDE plug-in
The Selenium IDE recorder is a Firefox plug-in that makes the scripting like recording a macro: open up the recorder, interact with your app, copy/paste the transcript into APM. But first things first, download the plug-in.
Version support: You must use the Firefox Extended Support Release combined with Selenium IDE version 2.9. Download the Selenium IDE as instructed below to ensure you have the correct version.
Plug-in conflict: You cannot have the AppNeta plug-in and the Selenium plug-in installed at the same time. Installing either type overwrites the currently installed type.
- Open Firefox and log into APM.
- Navigate to Experience > Web App Groups.
- Click Configure for the web app group containing the workflow you want to edit.
- In the Workflows… pane, click the edit icon next to the workflow you want to edit.
- In the Script pane, select Selenium Script.
- Click Selenium Firefox Plug-in Download and follow the on-screen install instructions.
Record a script
As with writing any script or program, it’s best to plan out what you’re trying to do ahead of time. Take some time to think about what you want to test, and what results you expect to see from it. Start out small. Record a few lines and verify that they playback successfully before moving on. Note that APM supports interaction with html elements only. It does not support 3rd-party browser add-ons or multimedia players. This includes things like Flash, ActiveX, Java applets, browser extensions, and video/audio players.
- In any supported browser, preferably not Firefox, log in to AppNeta and navigate to Experience > web applications.
- Click user actions for the app you’ll be scripting against. You might need to first click ‘configure’ to reveal the user action panel.
- Open up Firefox and navigate to the url you specified in your web app definition.
- Start the script generator from Tools > Selenium IDE.
- Enter the same url at the top of the plug-in.
- Execute the workflow you planned out, and as you do, monitor the script recorder to make sure that all your actions are recorded. You might fumble a bit. It’s normal to try a few times to get a clean transcript. If you find that an action can’t be recorded, move on and tackle it in the troubleshooting phase.
- When you’re done recording, copy the contents of the source tab to the source tab of the add user action page and click ‘save’.
If an action can’t be recorded move on and tackle it in the troubleshooting phase.
Hotkeys: The plug-in supports cut, paste, copy, delete and undo operations in the table view using hotkeys or the ctrl+click (Windows) or cmd+click (OS X) menu.
Selenium short list
You’ll find the following commands helpful:
- Opens the specified url.
- Performs a click operation, and optionally waits for a new page to load.
- Pauses execution until the expected new page loads. Called automatically when clickAndWait is used.
- Pauses execution until the expected html element is present.
- Verifies the page title.
- Verifies that the expected text is somewhere on the page.
- Verifies that the expected html element is present on the page.
- Verifies that the expected text and its corresponding html tag are present on the page.
- Verifies the contents of a table.
- Official Selenium command reference
- 3rd-party Selenium command reference
- Firefox Selenium tutorial
- Newcircle Selenium tutorial
The script recorder gives you the base for your script, but inevitably there will be some troubleshooting and clean up involved. This is part and parcel of scripting. APM provides a few tools to help you work though any issues.
The script editor will mark any syntax errors with . There may be some commands that are supported by the Selenium IDE but not supported by APM. APM will explicitly flag these commands.
Click ‘preview’ in the upper right corner of the script editor to run the script. APM will capture a screenshot approximately every 1 second. At minimum it’s important to run the preview to see if the script completes. Also You can compare the screenshots to your expected results, and of course it’ll be obvious if the script fails to complete.
A common source of error is the attribute that at the script is using to identify the web page element with which it’s attempting to interact. There are a number of ways to specify an element:
- It’s important that the id remains consistent; some applications dynamically generate the id attribute and this will lead to unpredictable results.
- This attribute is meant for form elements. Like id, it’s best if they are static.
- css path
- In other contexts this is called ‘css selector’. It is the standard css syntax you would use to identify an element or set of elements by walking the DOM. Because css path doesn’t rely on a static id or name, they can be used on pages that dynamically generate links or elements, such as search engine results. You can grab the css path for an element in google chrome like this; and test a css path you wrote with this plug-in.
- xpath is similar to css path. You specify the position of an element in the DOM by walking the DOM hierarchy. When it comes to xpath versus css path, generally there’s no reason to choose one over the other as long as you can get to the element you’re trying to target. But it’s worth mentioning that xpath can go up and down the DOM and has a facility for text recognition. Here are some good examples. You can test an xpath you wrote like this.
Finally, borrow ideas from our example script. You’ll see that while there are a lot of commands at your disposal, even the most popular services just need a few of them.
Once you can get the script to reliably play back in the Selenium IDE, you can copy and paste the contents of the source tab in the plug-in to the source tab in APM. Then, there are two remaining clean up operations.
If you captured user credentials in your script:
- Define those variables in APM. You can define variables per target or per script).
- Replace your actual credentials with the variables you defined.
Mark your milestones
The highest granularity for timing data is ‘milestone’. A ‘milestone’ is a collection of one or more actions that you the coder decide is a meaningful unit of work that deserves to be measured individually. Milestones exist only as a facility for you breakdown complex scripts that would otherwise be opaque. For example, if a script consisted of 10 actions and had a high completion time, how would you determine which action was the culprit? By splitting those actions into multiple milestones, you can get monitoring data for each and quickly narrow down the culprit. A script can have up to 20 milestones.
The easiest way to mark a milestone is to use the table tab. Generally, commands that result in a page transition like
open have an optional milestone field. Try to follow the form:
Milestone <number> <description>.
Have a maintenance plan
Don’t let your passwords expire. Set yourself regular reminders the change the password.
Include APM in your change control process. A change to your app, even a minor one, could change the flow of your script or cause an id attribute to change.
Run your APM script against your staging server as well. This ensures that your script remains valid across changes and has the added bonus of allowing you to benchmark changes in your application performance before they’re made in production.