Experience monitoring is all about your end users, and they don’t just stop after hitting the endpoint: they log in, click links, search, etc. AppNeta Performance Manager (APM) lets you test how well your application responds to a series of user actions like these using a script. You’ll use a Firefox plug-in to record the base script, inevitably there’ll be some troubleshooting involved, then finally, you’ll logically divide your script into sections called ‘milestones’. APM will return timing data, alerts, and an Apdex score for both the web path and each milestone so that you know exactly when and which actions are holding things up.
Two scripting languages are supported: AppNeta and Selenium. While APM currently supports mixing and matching both types of scripts within the same app definition, we encourage you to create all new scripts in Selenium if your monitoring point supports it. Currently Selenium is supported only on Global Monitoring Points.
This page covers scripting in the AppNeta language.
- Prepare your environment
- Download our plug-in
- Record a base script
- Clean up and troubleshoot
- Mark your milestones
- Have a maintenance plan
Prepare your environment
First, some suggestions:
- Use a dual monitor setup. During the recording phase you can have AppNeta open in one browser, and your app open in the other. During the troubleshooting phase, you can performing an action on one screen to get a clear sense of the expected result, and then on the other screen edit to your code accordingly.
- Use a different web browser in each window. For example, use Firefox on one and Chrome on the other. That way, if the site hangs or crashes your browser, you won’t lose your work. Plus, you can close, re-open, and clear your cookies on the secondary browser with impunity.
- Use dedicated test credentials. When using usernames and passwords in your scripts, don’t use your everyday account. Set up a dedicated account just for the script, and only give it read-only and the bare permissions required for your script to run. If you log authentication requests, be sure to alert on any attempts to authenticate on IP addresses other than your own workstation and the IP addresses of the web monitors.
- Keep a backup. Keep open a text editor like Notepad++ or Sublime Text, and regularly copy/paste your script there. Use a reasonably advanced text editor with an undo/redo facility, so you can track your changes.
- As with writing any script or program, it’s best to plan out what you’re trying to do ahead of time. Spend some time thinking about what you want to test, and what results you expect to see from it. Take note that APM supports interaction with html elements only. It does not support 3rd-party browser add-ons or multimedia players. This includes things like Flash, ActiveX, Java applets, browser extensions, and video/audio players.
Now the compulsory:
- Make sure that you’re scripting against the same version of the site that the web monitor will access, and obtain the login credentials, if necessary.
- Remember that each time it runs, your script will be starting from a blank slate, so you’ll need to make sure the browser you are scripting in is equally clean:
- Close all unnecessary tabs.
- Clear out your browser’s cache and cookies.
- Disable any plugins you don’t need or that could interfere with the script, particularly script blockers and ones that automatically enter text into fields such as password managers.
Download our plug-in
The script recorder is a Firefox plug-in that makes the scripting like recording a macro: open up the recorder, interact with your app, copy/paste the transcription into APM. But first things first, download the plug-in.
Version support: You must use the latest version of Firefox combined with Selenium IDE version 2.9. Download the Selenium IDE as instructed below to ensure you have the correct version.
Plug-in conflict: You cannot have the AppNeta plug-in and the Selenium plug-in installed at the same time. Installing either type overwrites the currently installed type.
- Open up Firefox and log into AppNeta.
- Navigate to APM > web applications.
- Click user actions for any app. You might need to first click ‘configure’ to reveal the user action panel.
- Click the orange download button in the script section, and follow the on-screen install instructions.
Record a base script
As with writing any script or program, it’s best to plan out what you’re trying to do ahead of time. Take some time to think about what you want to test, and what results you expect to see from it. Note that APM supports interaction with html elements only. It does not support 3rd-party browser add-ons or multimedia players. This includes things like Flash, ActiveX, Java applets, browser extensions, and video/audio players.
- In any supported browser, preferably not Firefox, log in to AppNeta and navigate to APM > web applications.
- Click user actions for the app you’ll be scripting against. You might need to first click ‘configure’ to reveal the user action panel.
- Open up Firefox and navigate to the endpoint you specified in your web app definition.
- Start the script generator using the shortcut CTRL+ALT+S (Windows) or CTRL+OPT+A (OS X).
- Execute the workflow you planned out, and as you do, monitor the script recorder to make sure that all your actions are recorded. You might fumble a bit, but it’s normal to try a few times to get a clean transcription. If you find that an action can’t be recorded, move on and tackle it in the troubleshooting phase.
- When you are done recording, copy the contents of the source tab to the script section of the add user action page and click ‘save’.
Selenium short list
You’ll find the following commands helpful:
- Opens the specified url.
- Performs a click operation, and optionally waits for a new page to load.
- Pauses execution until the expected new page loads. Called automatically when clickAndWait is used.
- Pauses execution until the expected html element is present.
- Verifies the page title.
- Verifies that the expected text is somewhere on the page.
- Verifies that the expected html element is present on the page.
- Verifies that the expected text and its corresponding html tag are present on the page.
- Verifies the contents of a table.
Clean up and troubleshoot
The script recorder gives you the base for your script, but inevitably there will be some troubleshooting and clean up involved. This is part and parcel of coding. APM provides a few tools to help you work though any issues.
First, the script editor will mark any syntax errors with . For help with command syntax, use the command dictionary built into the script editor. Find your command and use the ‘add’ button paste a new line with the full command syntax.
The second tool that the script editor gives you is the preview tool. Click ‘preview’ in the upper right corner of the script editor to run the script. APM will capture a screenshot for each action executed. You can compare the screenshots to your expected results, and of course it’ll be obvious if the script fails to complete. It’s worth repeating that APM supports interaction with html elements only. It does not support 3rd-party browser add-ons or multimedia players. This includes things like Flash, ActiveX, Java applets, browser extensions, and video/audio players. After editing your script you need to click ‘preview’ again.
Did you declare your variables? If you defined variables on the Edit Workflow page, you must declare them at the top of your script. See the examples below.
The next place to look for the source of errors is the IDs that at the script is using to identify the elements a web page that the script will manipulate. These elements are commonly those that are designed for interaction like links, buttons, checkboxes, and text fields. There are a number of ways to specify a web page element:
- It’s important that the id remains consistent; some applications dynamically generate the id attribute and this will lead to unpredictable results.
- This attribute is meant for form elements. Like id, it’s best if they are static.
- css path
- In other contexts this is called ‘css selector’. It is the standard css syntax you would use to identify an element or set of elements by walking the DOM. Because css path doesn’t rely on a static id or name, they can be used on pages that dynamically generate links or elements, such as search engine results. You can grab the css path for an element google chrome like this; and test a css path you wrote with this plug-in.
- xpath is similar to css path. You specify the position of an element in the DOM by walking the DOM hierarchy. When it comes to xpath versus css path, generally there’s no reason to choose one over the other as long as you can get to the element you’re trying to target. But it’s worth mentioning that xpath can go up and down the DOM and has a facility for text recognition. Here are some good examples. You can test an xpath you wrote like this.
Finally, borrow ideas from our system-defined templates. You’ll see that while there are a lot of commands at your disposal, even the most popular services just need a few of them. The following scripts are provided for example only, log in to AppNeta to see the most current versions.
declare $username $username declare $password $password open '/' 'Homepage' exitIf appliance='m22' exitIf appliance='m20' type id='username' $username password id='password' $password clickAndWait xpath='/html/body/div/section/div/div/div/form/div/div/button' 'Login' type id='quickSearchInput' 'page' clickAndWait css='input.hidden' 'Search page' click id='header-details-user-fullname' clickAndWait id='log_out' 'Logout' clickAndWait id='logout' 'Confirm logout'
Mark your milestones
The highest granularity for timing data is ‘milestone’. A ‘milestone’ is a collection of one or more actions that you the coder decide is a meaningful unit of work that deserves to be measured individually. Milestones exist only as a facility for you breakdown complex scripts that would otherwise be opaque. For example, if a script consisted of 10 actions and had a high completion time, how would you determine which action was the culprit? By splitting those actions into multiple milestones, you can get monitoring data for each and quickly narrow down the culprit. A script can have up to 20 milestones.
There are several ways to declare a milestone:
The milestone action records the current timing data as a milestone. It can be inserted into the script at any point.
Similarly, you can mark a milestone inline with any of the following actions: clickAndWait, typeAndWait, open, and wait. They all follow the form: keyword <variable> [‘milestone name’].
open '/' type id='Email' 'firstname.lastname@example.org' click id='next' password id='Password' 'password1234' clickAndWait id='signIn' **'Milestone: Log In'** click 'home'
open '/' **startMilestone** type id='Email' 'email@example.com' click id='next' password id='Password' 'password1234' **endMilestone 'Entering credentials'** clickAndWait id='signIn'
Have a maintenance plan
Don’t let your passwords expire, set yourself regular reminders the change the password. Include APM in your change control process. A change to your app, even a minor one, could change the flow of your script or cause an id attribute to change. Run your APM script against your staging server as well. This ensures that your script remains valid across changes and has the added bonus of allowing you to benchmark changes in your application performance before they’re made in production.