Big data and predictive technologies – sometimes called artificial intelligence (AI) – are changing the world. Everyone from Google to your supermarket and hospital are leveraging the power of data to transform the way they operate fundamentally. While the traditional security industry has tended to take a reactive approach towards prevention (rather than a data-driven one) innovative practitioners are now beginning to show the transformative impact big data can have when used intelligently.
The security of yesterday
In this age of custom malware and targeted advanced persistent threats (such as Stuxnet), slow, resource-intensive security is a big problem. One example was the recent attack on the New York Times. Chinese intruders installed 45 different pieces of malware on the media company’s systems over a period of four months. All this happened despite the presence of traditional endpoint protection, including up-to-date antivirus from a traditional security player which, according to cyber security firm Mandiant, detected just one piece of malware.
We can’t say for certain what was at play in this instance, but it does point to some fundamental challenges with traditional, non-predictive security systems. The faster a bad guy can act, the further behind the good guys will end up. This leads to long delays in identifying new threats and in protecting vulnerable systems. With more time to plan and execute their attacks – and aided by the delay to push out protection – cyber criminals are able to compromise many more systems than ever before. In the absolutely worst cases, this gap becomes something the most sophisticated attackers can hide indefinitely.
Data is the DNA of security
Security is the science of detecting and preventing attacks against our electronic systems. It’s an industry that’s swimming in classifiable data, and one that had been crying out for a big data overhaul. Until very recently, security systems were limited to searching in small chunks of that data for a few tell-tale bytes – those being a pattern of data or an antivirus signature unique to a specific attack. Found some matching bytes? Bad guy possibly detected. Didn’t find the bytes? Not conclusive.
Security systems built this way are brittle and resource-intensive. They spend all their time looking for the proverbial byte-sized needle in a data haystack, only to start the whole cycle from scratch when some of those bytes get changed by an attacker looking to evade detection.
Making sense of data to improve security
This is where big data and machine learning (systems that can learn from data) can level the playing field. We can use machines to identify more complex signals and relations in datasets far bigger than any human could analyse. Consider the digital profile that advertisers are able to build by collecting and correlating multiple sources of data. The same approach can also apply to security. Imagine hundreds, even thousands of sources of data – system logs, code, attacker behaviour, preferred target types, method of operation – all being collected and correlated to produce a profile on steroids.
Instead of just creating a signature for each piece of malware, we can build a database of all the malware and everything associated with the people who use it, from their development accounts through to the servers they use and how they plan to monetise.
But why stop there? By going even further, by breaking down the malware itself and reducing it to its fundamental building blocks, we can create a database of all the bad code in the world. Once you start to analyse code use across applications you realise just how institutional code reuse is. Everything evolves from everything else, and suddenly we have a way to detect new threats long before they get found on a victim’s computer.
In the same way Google shines a spotlight on every single web page, big data can allow security companies to do the same thing to the threat landscape, turning over every stone looking for any sort of link to previously seen threats or incidents. Suddenly it’s the malware authors who have to do the hard work. Now, evading detection is more than changing a few key parameters – they have to ensure no code is reused, no server is reused and no development accounts are reused. They have to start from scratch each and every time.
Combining big data with machine learning allows the security industry to develop agile, machine-driven analytics that can identify threats before they do harm. It’s another example of the practical benefits that a big data approach can have on an industry. While this isn’t going to solve the problem of advanced attacks and malware alone, for the first time the tables are turned and the good guys are gaining an advantage.
Marc Rogers is principal security researcher at Lookout