webbotsWe have always to discern between good and evil, right? Well, our digital and technology-driven world is no exception. Here, we have to differentiate between good and evil bots: Some are helpful and facilitate our daily lives; others are not to be trifled with. The bad bots are often machines that are designed for only one purpose - causing as much damage as possible.

The same goes for the several million strong army of web bots that pay websites a visit across the globe on a daily basis. Again, the bad ones here outnumber the good ones. Overall, the small search or attack programs are responsible for more than half of all web traffic, according to Incapsula, increasingly confronting website operators with serious problems.

Generally, bots are computer programs or scripts that automatically analyze and evaluate information from websites. Among the good bots are, for example, so-called web crawlers from Internet search engines that evaluate website content to create a meaningful ranking. Many companies are unaware of such steady bot traffic, just as of the effects "evil bots" can have on their web applications.

The cloud-based application delivery service from Imperva aims to protect websites and increases their performance, besides improving end user experiences and safeguarding web applications and their data from attack. Its service includes a web application firewall to thwart hacking attempts, DDoS mitigation to ensure attacks do not affect online business assets, a content delivery network to optimize and accelerate web traffic, and a load balancer to maximize the potential of web environments.

The company’s prior annual reports have already shown that bots are the Internet’s silent majority, stating: “Behind the scenes, billions of these software agents shape our web experience by influencing the way we learn, trade, work, let loose, and interact with each other online. However, bots are also often designed for mischief. In fact, many of them are used for some kind of malicious activity—including mass-scale hack attacks, DDoS floods, spam schemes, and click fraud campaigns.”

The expert published in the meantime its annual “Bot Traffic Report” for the third time, which is a statistical study examining the typically-transparent flow of bot traffic on the web. This year, Incapsula is actually digging deeper into its database to reveal an even more substantial data sample, thereby providing new insights into bot activity.

According to Incapsula’s report, bot traffic volumes decreased to 56% of all web visits—a reversal of the upward trend that has been observed in the past two years.

However, the company noticed that the bulk of the decline in bots reflects a drop in so-called good bot activity, mainly those associated with RSS services. The analysts’ initial assumption was that the shift was related to the Google Reader service shutdown. Upon further inspection however, Incapsula saw that the Feedfetcher bot—associated with the Google Reader service—was still as active as ever, while the decline in RSS bot activity was across the board and just another indication of the slow demise of RSS services.

Key findings of the report include:

More than half of all Web traffic is "not human": It is a fact that a large and steadily growing proportion of website visitors comes from bots, states Incapsula, accounting for the mentioned 56% of all web traffic. In short, the report reveals that websites are visited more often by bots than humans.

Number of malicious bots grows: Around half of all bots mean ill, but what exactly do they do during their site visits? Almost a quarter of them performs automated Distributed Denial-of-Service (DDoS) attacks. The aim is to suspend online stores and other web-based services by artificially overloading the website traffic. Besides the fact that DDoS attacks hamper users’ access to sites and the purchase of products, the bots are increasingly focusing on getting payment information or other customer data.

As in the science fiction movies, the evil bots multiply with enormous speed and simultaneously continue to evolve explains Barry Shteiman, Director Security Strategy at Imperva. Besides that, it is alarming that DDoS bots are so highly developed that they remain undetected by common anti-DDoS solutions and do economically harm.

Effective protection is possible: However, there is also some good news, since there is Imperva’s ThreatRadar Bot Protection Service available as an add-on for the SecureSphere Web Application Firewall (WAF). With this tool, the incoming traffic on websites can be classified. In addition, human users or bots are recognized, and at the same time categorized as good-natured or malicious information collector. SecureSphere uses this information to find ways for the defense of the unwanted bots before they can access sensitive data.

Incapsula scales on demand to block even the largest DDoS attacks, allowing you to stop massive network DDoS attacks before they reach your network. It mitigates all types of DDoS attacks including those targeting web applications, DNS servers, and direct to IP address attacks. With advanced application-layer protection, Incapsula inspects SSL traffic and spots low-data rate attacks and application exploits that can bypass other solutions. Unlike other DDoS mitigation solutions, which create negative web-viewing experiences when activated (e.g. CAPTHCAs, or delay screens), Incapsula makes use of progressive client interrogation solutions, which block DDoS attacks transparently, preventing user interruption.

As malicious bots evolve, Incapsula is driven to think beyond common signature-based and challenge-based security measures, explaining that their analysts “know that bots can falsify their identities, but not their intentions. By examining the context of website visits, Incapsula’s goal is to stay ahead of the evolving bot threat and to thwart the large number of malicious impersonators.”

Since marketers are obviously constantly battling against an unseen enemy that tries to sabotage, terrorize, steal, or damage the company’s reputation, I think the the concluding infographic from Incapsula on how to protect your organization from external threats just comes in handy. (Infographic)

By Daniela La Marca