‘Bots’ often cause problems for website owners. Among many negative effects, they often damage the accuracy & precision of web analytics data, and cause website owners to make faulty decisions.
Here’s a quick Google Analytics ‘Custom Alert’ to help you spot some bots before they’ve caused lots of damage. It doesn’t solve the problem, but it helps flag when it may be happening, allowing you to delve in & investigate further, and to then filter out the traffic if it is indeed a problem.
Here’s an example of what bot traffic looks like, when isolated from all of the rest of the traffic on a site:
Just there, it’s a robot sending around 375 visits per day. That’s not huge, but it’s roughly 11,000 visits over a period of a month, none of which converts, all of which has a 100% bounce rate. That causes the following issues:
- It totally skews our bounce stats
- It skews our conversion stats too.
- All of the surrounding metrics like ‘per visit value’, ‘% new visits’ become misleading.
- Stats by region/browser/etc are often messed up, as bots tend to favour one particular region/browser.
- It does all of the above in an unpredictable manner, and is time consuming to look for.
The Solution: How to alert yourself when this is happening?
One solution to alert yourself when this may be happening is as follows:
- Set up an ‘advanced segment’ to spot ‘new, direct, bouncing’ traffic.
- Set up an alert so that if that ‘new, direct, bouncing’ traffic ever increases massively, Google Analytics sends you an email.
Here are those 2 steps:
Step 1: Create An Advanced Segment
Usually, but not always, bots follow this pattern:
- They visit the site ‘direct’.
- They don’t store cookies, so are identified as ‘New’ visitors.
- They record as 100% bounce rate.
Not all of that traffic will be bots, but we know if it jumps considerably there is a much higher likelihood that it’s a bot than if any other type of traffic jumps.
In order to isolate those people, we’ll set up an advanced segment:
(if you’re lazy, you can simply click this link to create the above: http://bit.ly/maybebots)
Step 2: Set up the Alert
Following that, set up an alert to fire off an email when traffic leaps from your new ‘may be a bot’ segment:
To set that up, go to the following in Google Analytics: Intelligence Events > Overview > Custom Alerts > Manage Custom Alerts > Create new alert.
Once there, copy the following:
That essentially says “Please pay attention only to Direct Traffic, where that traffic is New to the site, and where it Only Views One Page. While you’re paying attention to that – if it’s more than double what it was last week on any given day – send me an email.”
Depending on your site’s traffic pattern, you may want to increase/decrease that ‘100%’ value.
Not perfect, and it won’t catch everything, but better than not spotting anything at all.
That’s it. Set that live & – when that ‘new, direct, bouncing’ traffic that is often caused by bots doubles – you’ll get an email telling you. From there you can investigate further & filter it from your google analytics data if it is indeed a bot.
Do post any thoughts you have on this, or any other solutions.
You can also follow me below on twitter, or share this with others.