Big data is being touted as a next-generation security solution.

And there’s no reason to be surprised about it. After all, hardcore IT people with network security responsibilities are often very keen on making sure that network devices log all traffic, and configure their monitoring tools to inform them of any unexpected changes. So the very nature of Big data analytics seems tailor-made for analyzing the huge amounts of data that can be captured in these logs and getting useful information from them. Big data security analytics (BDSA) need to be on the radar for every IT shop.

But the issue isn’t quite so simple.

It is easy to configure network devices to generate log data. But even in a moderately-sized network, it is possible that logging everything can generate a mess of log files equaling hundreds of gigabytes of data every week.

The ‘big’ mess of big data

On one hand, it is really easy to configure network devices to generate log data. Analyzing that data can be exceptionally useful for your business (and not just for security). But even in a moderately-sized network, it is entirely possible that logging everything can generate a mess of log files equaling a couple of hundred gigabytes of data every week.

What does that mean for your IT department? You will need to factor in the storage and processing requirements necessary to properly analyze large data sets in a timely manner. In order to determine the behavior patterns that indicate something is amiss in your network, you will be storing months and months of data.

Over time, you will likely be able to narrow the data collection matrix down as you determine what is relevant or not — but when you start out, it won’t be that straightforward. Log files and other information that may not have initially seemed relevant to your data set may seem so later. For example, you might later elect to add building access controls or employee monitoring systems to detect internal unauthorized intrusions as a standard part of your security analysis process.

Incorporating BDSA into your network

It’s likely that the IT professionals in your organization responsible for security are already swamped with work. This means that adding big data analytics for security requires as much automation as possible. Invest in an appropriate tool or vendor. Whichever you choose, it is imperative to smoothly integrate these tools into your existing security management process to limit negative impact on IT workflow.

It is unlikely that BDSA will be replacing intrusion detection and intrusion prevention systems any time soon. But, the potential for having them work together remains significant. For starters, the addition of BDSA to the heuristic modeling for IPS and IDS can address some of the more common complaints that IT has about these systems. The number one complaint that vendors of these technologies hear from users is the prevalence of false positives. For example, generating intrusion alerts when there is no actual issue has a cascade effect on IT, as employees must rush to solve a non-existent problem. But with the addition of BDSA, the incidences of false positives should be significantly reduced.

Get a handle on big data

Without a combined effort, strengthening IT security isn’t just difficult, it’s virtually impossible. Start the conversation on security now.

Learn more

A final caveat

BDSA cannot entirely prevent a hack. The trick to utilizing big data security analytics effectively will be to properly deploy the right data acquisition and analysis tools that allow new technology to fit seamlessly into your network and security management processes.

Why your IT department needs big data security analytics

​It is easy to configure network devices to generate log data. But even in a moderately-sized network, it is possible that logging everything can generate a mess of log files equaling hundreds of gigabytes of data every week.

The ‘big’ mess of big data

Incorporating BDSA into your network

Get a handle on big data

A final caveat

It is easy to configure network devices to generate log data. But even in a moderately-sized network, it is possible that logging everything can generate a mess of log files equaling hundreds of gigabytes of data every week.