Synthetic Monitoring: When Bad things Happen to Good Checks
Running synthetic monitoring thinking it will match up with a user’s reality throw for throw is a fool’s game. While you can test in prod, your testing parameters are limited by an insider’s knowledge of the transaction’s pathways – making true objectivity challenging to achieve in testing. Yet still, every transaction tells a story.
We use synthetic monitoring to emulate user’s requests; creating a set number of steps to generate automated recordings that check, and also alert if the steps fail or the script times out before a certain threshold.
The more checks we script that test events, the more insight we gain into the functionality of our applications. As can be the case, the checks we create may miss the mark or fail altogether because they are not optimized.
Synthetic Monitoring | Time
A synthetic transaction performs it’s steps the same regardless of its intervals. Where Real User Monitoring (RUM) is a live performance, synthetic monitoring is listening to a song on repeat.
The goal with building durable synthetic monitoring is to create smart Transaction Checks that perform smoothly – like a downloaded song – vs. the sensitivity of The Beatles, Revolver album that you snapped into your portable cd player on a bumpy drive in 1997.
It’s better for your bottom line, and your on-call 3am response team, if your Transaction checks fail when an element is truly down, instead of at the hand of misplaced steps or avoidable timeout. Hours expended by your IT team trying to figure out why a check failed only to find that the downtime was due to something in the check’s creation isn’t ideal.
Here are a pair of the more notable time bandits:
Testing Long Routes
All roads may lead to Rome but not all of them are short.
Sounds like common sense: long scripts take longer to run.
Also sounds like common sense: a single check should test a single pathway from start to finish.
Sometimes common sense seems to contradict itself. With synthetic monitoring the goal is to test processes to support user experience. It’s ok to segment those tests into multiple checks to increase your testing precision. Running 50 miles without stopping is the same as running a 50 step check – maybe it’s doable, but it’s likely you’ll timeout on the road.
Synthetic tests retrieve data sequentially. First, a page, then it’s elements. A Transaction check parses the page and relays the elements. If you have created a check step that waits an allotted amount of time for an element, the check could fail early on if the element doesn’t exist within a 1 second window.
Use the right commands. If you’re wanting to wait for an element’s existence, use Wait for. If you’re wanting to wait an extra second at the top of the page load, you can do that too but you should have a logical testing strategy for the construction of each check.
An Orderly Fashion
The counterpart to timing is order. We mentioned that a page will load, followed by its elements, resulting in a directory of components you can pull from to build your check. Part of strategic building is deciding where to place your steps.
It’s pretty obvious that your first step is Go to your URL, from there you need to navigate as close to the user’s behavior as possible. To make this process more natural, you might use the Transaction Recorder as a visual aid for your check’s construction.
Feature Info: Our transaction recorder uses a chrome extension, (which works across multiple OS), to establish two way communication between the recorder and a page’s script.
Larger companies are likely to have a dedicated IT force to design checks and monitor user pathways. For smaller companies where your resident IT guy also moonlights in sales, the transaction recorder can be a time saver in creation that doesn’t compromise the integrity of the check.
Another time sucker is the duplication of efforts in steps. Once you’ve got confirmation that an element exists, you don’t need to add steps that re-validate that existence before performing an action. Cleaning up your checks so they are as streamlined as possible isn’t just about increasing efficiency, but also stability.
In this example, we wait for the element that will lead us to additional navigation, hover over the element, and then fill in the field. While we’re not re-validating the element or adding arbitrary time intervals, Hover isn’t needed. If the element doesn’t exist, the check won’t be able to fill it.
Dynamic Synthetic Monitoring
Customization is a massive deal. Users like to feel individually recognized, and that requires dynamic content.
Welcome back, Karen, goes from a welcome message to a potential obstacle when it comes to constructing a successful check and monitoring personalized user pathways.
Synthetic Monitoring is Complicated
Dynamic content in general makes finding errors a complex process – it’s like the truth – everyone’s version is different. Because dynamic elements often don’t make up the entire page, certain aspects may fail while the rest of the page renders normally. After all, the check may return successfully if it completes the process and lands on an apology page that’s OK 200. So how does a Transaction check know that it’s achieved valid success when every version of the page is different?
Do, or Do Not
There are two approaches to testing dynamic routes; test for elements that exist, or test for elements that do not exist.
Not every bit of a dynamic page changes every time. Often, changes are kept to a DIV tag, or specific header. To make this work in synthetic testing, look for strings to expect that are ever present, and that load after, or are contingent upon the dynamic elements so the check validates what you need it to.
Check for content that shouldn’t be there. This is a slightly riskier method in that any errors preventing you from finding a string won’t be detected – so you may miss the errors altogether. Even when part of the site is broken, much of the page will still render properly and only one aspect will indicate failure.
Synthetic Monitoring: Best Tool for the Job
Go to, wait for, click it, fill it,
hover, focus, check, uncheck it
There are many efficient methods to set up synthetic monitoring with steps, and optimizers like constants, and variables. Once you’ve composed your script, there are a few last details to review before you unleash your check on your product.
How Much & How Often
How much is too much when it comes to Transaction checks? There’s no need to overload your dashboard with a dozen different checks, but you should be testing every revenue-generating pathway of your site.
It would be easy to throw up your hands and say Check it all! Check everything that results in an outcome! But it’s overkill and it’ll cost you. One way to save money is by using a third-party monitoring platform, *wink, wink*, that has a subscription system in place for creating and managing synthetic monitoring in a cost-effective way.
Another way to save is to use a variety of checks aimed at testing various facets of infrastructure. If you have 5 check types all reviewing a URL for DNS, SSL, intermittent heartbeats, HTTP(S) checks, and Ping checks, then your transaction check doesn’t need to focus on uptime. As your account grows, consider adding other check types that are functional and fit into your overall purpose. A little well-placed judgement when it comes to synthetic monitoring will help you optimize.
You want your synthetic monitoring to be ongoing – to run in the background like an ad jingle stuck in your brain. Continuous testing plus a game plan supports quick response time for an incident.
The Monitoring world talks a lot about the differences between RUM and synthetic monitoring. Why use synthetic monitoring at all if RUM gives you actual user data? One key reason is in the data itself.
The information you get from synthetic monitoring isn’t about the demographics of your userbase or their live-action habits, it’s tactical data for your team to use to break down and solve problems. For our IT guy moonlighting in sales, synthetic data may not tell him who to market to but it will tell him if a newly released update works that a client has been waiting for.
Regardless of the data’s application, synthetic monitoring is a pretty critical element of success and, with a few helpful tips, can be easy to implement.
Minute-by-minute Uptime checks.
Start your 21-day free trial with no credit card required at Uptime.com.