Prepare yourself for big data and the promise of big security

Churning through massive amounts of data in near real-time and identifying anomalies as they occur is the holy grail of IT security.

One of the great promises, and more obvious use cases for big data, is in IT security. Security-related data clearly fits the definition of "big," with nearly every device and application on the network generating reams of logging and performance data. Churning through massive amounts of data in near real-time and identifying anomalies as they occur is the holy grail of IT security. There's also the interesting aspect of environmental data. Perhaps economic conditions, news reports, or even the weather in certain geographies might affect the probability of a security incident, presumably allowing big data and predictive analytics to predict a security breach before it even occurs.

The small reality

Much like flying cars or dehydrated beer, a "drop in" big data security application that patiently scans every iota of internal and environmental data, and then quietly drops an appointment on your calendar for the denial of service attack that will happen next week, is not yet ready for mass consumption. While conceptually simple, the data gathering, storage, and analytic technology required to pull off such a feat are still in the juvenile stages at best. Furthermore, the cost for these technologies and the integration required for true predictive security are significant. Unless your business is highly sensitive to security concerns, at this point, the cost likely puts it out of reach.

 

The good news is that predictive security has a compelling and obvious benefit, one that's captured the attention of CIOs and, in turn, spurred investment by the large big data and IT security companies. While none of the "usual suspects" in the vendor pool have a prepackaged and easily installed big data security offering, there are several things you can do to get ready for predictive security.

Instrumentation

Instrumentation is consultant-speak for establishing logging and data capture on relevant devices and services. While your firewalls and package software may do a fine job with logging out of the box, third party applications or custom code that's accessible to the outside world may have minimal logging, or "orphaned" logging that's not integrated into your overall security and monitoring infrastructure. Even if a low-cost, drop-in solution for predictive security existed, if your IT shop doesn't have every application and device properly instrumented and centrally monitored, big data simply won't help.

---

Security'sweakest link: Technology no match for social engineering

---

Practice and procedure

Even in organizations that have a well-managed security infrastructure, once a breach is identified there are often befuddled looks and no clear lines of reporting or responsibility, costing precious hours during an attack. While it's easy for IT to do security drills, what happens in a real incident where your back-end transactional or financial system is compromised? Can IT unilaterally shut it down, essentially pressing "pause" on your company's ability to market, ship goods, or record, and manage cash? Who needs to be notified, and who has ultimate decision making authority? The main promise of predictive security is buying an organization time; however, well-planned procedures that include all elements of your business, not just IT, can do the same in the short term.

Awareness

Thanks to the US Government and its NSA antics, IT security is on the forefront of many executives' minds. Not only may some remote hacker be snooping around your network, but now government actors from around the world may be siphoning data for their own mercurial purposes. It may be tempting to fuel the fear and speculation to capture a fattened security budget, but now is the time for IT and data experts to bring some calm and rationality to discussions around security. This is a concern that's been elevated to the Board level at many organizations, and one that could use some sound, technically-grounded advice. Conveying what's currently possible, and what's coming down the road in terms of big data-driven predictive security and forensics, is a great start regardless of whether that technology is available today.