Security firms have battled botnets for many years, but only in the last 18 months, with takedowns of rogue ISPs (especially McColo), has the security community been more aware of opportunities to disrupt botnets, and grown more confident that this online menace can be successfully fought.

Botnets are distributed networks of “zombie‟ or “bot‟ PCs, infected by malware which enables them to be marshalled by cyber criminals primarily to distribute enormous volumes of spam and other malware and launch phishing attacks via email.

Botnets have been around for years. Back in 2003 – ancient history in internet terms – security firms often saw thousands of PCs infected with the Sobig mass-mailing virus, and later in that same year, when the Fizzer malware logged thousands of computers into internet relay chat (IRC) rooms. At that time, this was perceived only as a problem for IRC admins. In fact, the infected PCs were being connected to chat rooms by their human bot-masters, waiting to receive command and control (C&C) instructions. It was only when MessageLabs Intelligence correlated the malware and spam traffic from each spam sending IP address, that the bigger picture was later revealed.

Over the years, security firms kept botnets under observation, doing their best to inhibit their ever-increasing output of spam, but not knowing how to tackle their C&C infrastructure. Then in October 2008, the security community realized that the best way to disrupt the botnets was to take down the ISPs that hosted them.

The first to go at the end of September 2008 was Intercage (aka Atrivo), linked to the infamous Russian Business Network. However, the most widely publicized was McColo, a San Jose-based ISP, which was found to deal almost exclusively with cyber gangs. McColo, set up by a 19 year-old Russian, was host to a botnet called Srizbi controlling 1.3 million IP addresses, as well as the Mega-D, Rustock, Asprox, Bobax and Gheg botnets.

In November 2008, community action resulted in McColo’s peering ISPs disconnecting it from the internet, largely because of an article written by Brian Krebs in the Washington Post []. Taking down McColo came as a shock to the botnet gangs and spam levels dropped instantly by as much as 80 per cent, which represented an enormous victory against the spammers.

Srizbi was crippled, never to return and the other botnets were badly disrupted. Over the following two months, spam gradually recovered to previous levels, as the surviving botnets relocated their command and control channels and the criminals spawned several new botnets as well.

The operations behind many of these botnets were forced to re-evaluate how they functioned and sought to put more protection in place to prevent a repeat of the huge disruption caused by the take-down of a single ISP. Take-downs like McColo must cost bot herders and spammers hundreds of thousands of dollars - perhaps millions - in lost revenues.

When Srizbi disappeared, activity from the surviving botnets increased dramatically, seeking to fill the huge gap in the market left behind. At that time, Srizbi had been responsible for as much as 50 percent of all spam. After losing their botnet of choice, spammers rented capacity from other botnet operators in order to keep up their spam campaigns.

However, when the next major take-down of a dubious ISP took place, it was clear that the cyber criminals had already learned from the strike against McColo.

This time the security community’s target was another California-based ISP called 3FN (aka APS Telecom and Pricewert) which was hosting command and control channels for Cutwail (aka Pandex), one of the oldest botnets which had been spewing out malware since January 2007. By June 2009 Cutwail had swollen to more than 1.5 million active IP addresses in an aggressive recruitment drive.

3FN was taken down on Friday 5 June 2009 and Cutwail went with it. But within a couple of days, by Monday morning, Cutwail was back online and as strong as before.

Botnet gangs had made many refinements to their creations in the six months since the McColo take-down. Thus, the organizations behind Cutwail were able to quickly reorganize after losing an important part of their botnet infrastructure. The fact that the technology was now much more flexible and robust allowed them to review the status of the botnet and return to business as usual in just a few days. It was clear that botnets now had a business continuity or disaster recovery plan of their own.

The botnet C&C mechanisms had shifted away from IRC towards HTTP. Instead of receiving instructions from one place, algorithms were built into the bots so they would look for random-looking domain names, which are purchased by the botnet gang each day, and from which the bots receive their commands. This ensures that the botnets aren't so reliant on one ISP.

But as the botnet controllers evolved their tactics, so did the security firms. One botnet in particular had grown significantly in the wake of the McColo take-down; a botnet called Mega-D (aka Ozdok). By November 2009, the algorithms behind the C&C mechanism used to issue the botnet with new instructions were broken by FireEye, a security company that specializes in combating botnets. It was now possible to predict which domain names were to be used by the botnet and to register them in advance of the botnet controllers. It was almost like cracking the Enigma code; and for the first time it was possible to know the botnet’s next move and to register these domains faster than the botnet controllers.

Mega-D appeared to be crippled, its spam-sending days seemed to be over, until a few days later, MessageLabs Intelligence identified large volumes of Mega-D spam being distributed from IP addresses that had not been used to send spam previously. This suggested that the botnet controllers had enacted their business continuity plan, seemingly with inactive sleeper bots or perhaps even a whole parallel backup botnet.

Disaster Recovery isn’t the only tenet that botnet controllers have borrowed from the world of corporate IT; they also use a technique called “fast-flux” hosting, which dynamically distributes resources across a number of continually changing IP addresses using a form of “round-robin” DNS. In the hands of a botnet controller, fast-flux can hide the true location of websites used to host malware, spam and phishing content, by hiding them behind the IP addresses of compromised, botnet-controlled computers, each acting as a web server or proxy.

Another technique that botnet controllers use to hide their botnets from the prying eyes of security firms is to expose only a small proportion of their zombies at any one time, cycling their use over a period of several days and limiting the amount of spam sent from each to minimize the risk of them appearing in blacklists of known spam-sending IP addresses.

Until recently, botnet controllers had to recruit one PC at a time. But with the advent of “generic droppers” like Bredolab, and perhaps Conficker, larger botnets can be assembled to order, whether for a spam campaign or something even more sinister.

Cyber criminals can now purchase the control of thousands – even tens of thousands - of ready-compromised PCs, recruited en masse to their own botnet. For example, cyber criminals may pay for malware, spyware or botnet Trojans to be installed by the Bredolab operators who would then instruct the Bredolab botnet to “drop” the malware onto however many computers were needed, using computers that were already under their control. This takes botnet recruitment from a random, scattergun approach where PCs are infected at random, to a more commoditized recruitment campaign. The only limitation to the size of the botnet is how much the criminals are prepared to spend. Now cyber criminals with enough cash can set up a botnet operation and be in business virtually overnight.

As botnets grow in sophistication and number, there is a danger of them becoming an extension of the hidden world of international and industrial espionage. The authorities in many countries are now concerned that attacks on government and business resources will become the next battleground in cyber warfare. The botnet could become the weapon of choice to disrupt infrastructure, and a lot less expensive - or traceable - than a ballistic missile.

More worrying perhaps is that such an attack could come from within a country’s own borders and as such would be difficult to prevent. Globally, India tops the chart, with at least 106,500 active spam-sending bots in an average week. In other parts of Asia, Vietnam comes in second with over 87,000 active bots tracked by MessageLabs Intelligence, while in Singapore, this figure dips to the 5,200 range and in Malaysia, 3,200. One way to look at this phenomenon is as a potentially massive sleeper cell. It would take hundreds of thousands of zombie PCs to launch a successful DDoS attack against a typical web server; however, cyber criminals often prefer to spread the workload across several thousand computers to better avoid detection.

The zombie PCs that make up botnets are recruited largely from inadequately protected domestic PCs, but there are also numerous compromised business networks too. Conventional firewalls that don’t inspect HTTP streams –the preferred C&C mechanism for many contemporary botnets- are not sufficient protection, nor is conventional antivirus software on its own.

Businesses should minimize the risk of becoming part of a botnet by ensuring that they are protected by filtering internet traffic for spam and other malicious or harmful content before these reach their corporate network. Many ISPs provide such value-add services now, as more pressure is put on the industry to tackle this problem as close to the source as possible.

Service Level Agreements (SLAs) are also critical here, for example, during a major spam run, how many false positives does the SLA allow for before too much genuine mail is be quarantined along with all of the spam?

All eyes are now on the cloud, whether private or public, to fight the spammers. Traditional desktop appliances are no longer flexible or strong enough to keep defences running around the clock, 365 days of the year, which is what is now required from businesses and individuals. Botnets are essentially “private clouds‟ working together, distributing the infected software to PCs. Botnet operators are like SaaS providers, with all the will in the world it will be almost impossible to fully eradicate the problem using technology alone. The most effective way to alleviate the botnet burden is by turning the Internet against them using the fabric of the cloud as a catalyst to kill the botnets, making the internet a safer neighbourhood to stroll once again.

By Dan Bleaken, Malware Data Analyst, Symantec Hosted Services