Bots Account for 10% of Web Traffic
Written by Lucy Black   
Monday, 01 October 2012

Bot traffic appears to be growing and is a problem for programmers, publishers and advertisers as this infographic shows.

Solve Media has reviewed a monthly average of 100 million identity authentications across 5,000 web publishers for a 20-month period form January 2011 to August 2012 and has revealed a 400 percent rise in aberrant traffic across registration, voting, commenting and contact services during that period.

Solve Media's key findings are:

  • The majority of bot traffic comes from the U.S, where bots account for 16% of total traffic
  • Singapore (56%) and Taiwan (54%) have the highest percentage of bot traffic 

 (Click in infographic to open in new window)

thebotstopsheresmall

Source: Solve Media

 

Websites are geared to human visitors. When the proportion of non-human visitors increases it negatively impacts all concerned.

In this infographic the focus is on advertisers, who pay for marketing to entities who are not consumers. However, the problem is equally pressing for both consumer-facing and content-providing websites - in general bot visits represent a waste of resources and if they steal content or reduce the security of a site they are even more harmful.

One question raised by this infographic is why is there such wide variation in the estimates of bot traffic - ranging from 4% according to comScore to 31% according to Incapsula. It is most probably due to the variations on how bots are identified - after all not all announce there identity.

Personally I'm inclined to believe in Incapsula's figure but note that this includes entities that Incapsula characterizes as "Good Bots", namely the crawlers sent out by Google, Bing, Baidu and other major websites to index webpages and make your content available to search engines.

So web developers need to find ways to lock out the rogue bots that are harmful, such as the CAPTCHA-type solutions advocated by Solve Media while at the same time making sure that the bots that bring you more business are made welcome.

The problem is that a CAPTCHA hard enough to tell a bot from a human often annoys the human and only slow the bot down.

This xkcd  cartoon indicates the scale of the problem:

botscartoon

We still need a solution to this growing problem.

 

thebotstophereicon

More Information

Solve Media Uncovers $1.5 Billion in Wasted Ad Spend

Know your Top 10 Bots (Incapsula)

Related Articles

Google Now Searches JavaScript

 

blog comments powered by Disqus

 

To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin,  or sign up for our weekly newsletter.

 

Banner


Microsoft Releases WinJS 2.1 As Open Source
03/04/2014

WinJS is Microsoft's JavaScript library for developing WinRT or Windows Store apps. One of the lesser announcements at this year's Build is that it is being released as open source.



Brendan Eich New CEO Of Mozilla
25/03/2014

Brendan Eich, the inventor of JavaScript and one of the co-founders of Mozilla has been named as its new Chief Executive Officer.


More News

 

Last Updated ( Monday, 01 October 2012 )
 
 

   
RSS feed of news items only
I Programmer News
Copyright © 2014 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.