If you hit the interwebs, don’t be surprised when it hits back. (The anger of the crowds)

There are plenty of stories on how numbers of people use the web for righting wrongs; catching petty thiefs, freeing a reporter from prison in a foreign land, fighting a cult (or religion depending on how you look at it). Well, this is not as cool an any of those stories, but if you have a popular blog or web site, you may have noticed your web traffic bills going up and perhaps the traffic logs on your server not quite matching up with Google analytics or other JavaScript based counting systems. If you haven’t, well, you should really pay attention to those things. This is a story of a company that decided it knew best how to protect us, regardless of whom pays for that attempt at protection. Who is guarding the guards?

AVG 8 by Grisoft, the Anti-Virus, Anti-Spyware, Anti-Spam, (but pro DoS) company bought and bundled a new product called LinkScanner from Exploit Prevention Labs to its software package. Basically the program will check every hyperlink on a search return, from a Google search for example, to see if it is a valid site or not. If your blog or web page is in that return, linkScanner will send a ‘click’ to your page. If it determines your site is valid, it will put a nice green check mark by the link.
Suddenly, you have WAY more traffic then ever! Your site is more popular! Because your good enough, your smart enough and doggone it, people like you. The only problem is this is all FAKE traffic. And with near 20 million installs, (70 million total users) that is a LOT of fake traffic. At Mashable, where I currently work, we enjoyed this mess just as many other popular blogs and sites. At first we did not know what was going on, in fact our first thought was a denial of service (DoS) attack. However, we discovered that this spike in traffic had a common tread. The header looked like this..
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;1813.
After a bit a research, we saw that other web masters were seeing the same issue and it was sourced to AVG’s toolbar. To deal with the flood of traffic, yet still insure we get the nice green check mark, we sent back to AVG a single page. Other webmasters redirected the traffic back to AVG as a form of protest. A few people climbed trees in Berkeley as protest, but that did not have much effect.
The outcry however, started to build. Sites like forums.whirlpool.net.au and theregister.co.uk became exchange central in the fight to pressure AVG to stop this practice. And AVG heard the outcry. However, instead of stopping, chief of research Roger Thompson made the statement that was the equivalent of the shot hear around the world: “I don’t want to sound flip about this, but if you want to make omelettes, you have to break some eggs.” Just who’s eggs are we talking about here? Together with the pros at Media Temple, we were able to keep the traffic under control using various techniques. But for how long? There was talk that AVG was going to change the header so that no one can tell the difference between it and a real user. The web masters of the world shuttered.

And then they did it.

Now the webmasters of the world were Angry. And you do not to make them angry. A movement began to recommend removal of AVG software from all client machines, header and GET calls were examined to continue filtering, discussions of redirecting and flagging AVG as spam were banted about. Revolution was in the air. You could smell it… And just as fast as it started, it ended..

from the whirlpool forums http://forums.whirlpool.net.au/forum-replies.cfm?t=1007329&r=15942208#r15942208

Peter Cameron, Managing Director of AVG Australia / New Zealand here again.

As promised, I am letting you know that the latest update for AVG Free edition has addressed and rectified the issue that Simon and other members of Whirlpool (and others) have brought to our attention. This update has now been released to users and has also been built into the latest installation package for AVG Free.

It took a few days, but we saw our junk traffic drop and systems returning to normal. There were great hazaa’s and high fives across the web. The anger of the crowds forced a change in the system.

Although this was a battle won, the l33t war has just begun. The idea that a product with enough distribution can take upon itself to enforce their ‘view’ of how systems should work, and disregard the pain and trouble it can cause others on the system violate the whole ethos of trying to make the web work for everyone. From my point of view, linkscanner was a form of zombie DoS agent. By cloaking itself and forcing high loads on other systems for its own ends places a cost that must be born by all service providers. What if Microsoft decides one day to do the same thing? Or Google? Giants like these may not be so quickly turned. For now, wiskey for my men and beer for my horses, for together, we dined in hell!