Federation Security Analytics: A Data Science Approach

Too many alerts with little to no context, is the state of today’s information security landscape. For example, it’s common for an enterprise who has been breached to have received an alert from a security tool, only to have it lost in the noise of many other threats coming in at the rate of hundreds per day. To add to the flurry of alerts, security threats are constantly changing and getting more complex due a changing and complex IT environment, making it difficult to map out a single attack across all of the different infrastructure touch points. And as security teams and tools get wise to the tactics, the threats will continue to evolve to thwart them.

The key is to develop a security analytics infrastructure facilitating data science techniques that can evolve as the threats evolve. Additionally, taking a Data Science approach to security threats aims to reduce the flurry of alerts, as well as provide more context to the alert so they can be prioritized, do more efficient root cause analysis, and be quickly resolved. This is the goal of Federation Security Analytics, as it combines the technology power of a Data Lake with proven Data Science applications to:

-See and understand everything happening in your environment

-Detect and prioritize the most advanced attacks, including long and slow attacks that happen over time

-Investigate and remediate incidents with unprecedented precision and speed

federation_sa

I spoke with RSA Senior Manager David Mitchell to discuss how Federation Security Analytics can better spot today’s attacks, plus provide an adaptable infrastructure to protect the organization as attacks evolve and become more sophisticated.

1.  What are the biggest problems faced by Security Operations Centers and how does traditional SIEM fail to address these challenges?

Real threats today are more advanced and targeted, some aimed at locating specific information through an individual or use case. They are also constantly changing, targeting an environment that is not owned by the enterprise. Applications in the cloud, public networks, and mobile devices now contribute to threats outside a well-defined enterprise perimeter. The perimeter is now more porous; therefore, traditional SIEM tools that are signature or perimeter-based cannot effectively identify many of today’s attacks.

2.  How does Big Data with Data Science change the game to address these problems more effectively?

It does two things – it allows you to collect everything through an engineered big data infrastructure and enrich this data to identify high-value, high-risk assets. Once you have determined what your high-value, high-risk assets are, you prioritize them and collect everything around those assets – logs, network packets, endpoint data, and more.

To spot an advanced threat or a threat that has not advertised itself (or an unknown threat), you cannot use traditional signature-based techniques since you cannot create a signature of a threat that has never existed before. Using security data science you can remove the hay in the environment, extract information that does not make sense, and flag it to determine if a threat is real. This approach can also reduce the flurry of alerts and false positives, and provides more context to the alert so they can be prioritized, have more efficient root cause analysis performed on them, and be quickly resolved.

3.  What is Federation Security Analytics solution and what makes it unique?

Federation Security Analytics includes technology and expertise across RSA, Pivotal, and EMC II. It is unique in that it is not just a suggested architecture, but an engineered and field-tested solution that enables you to simultaneously collect required security data, analyze it, and create alerts. You can reliably collect all your data and send real-time alerts without being impacted by interacting with the data and vice versa.  The solution is also packaged with services to install and configure in your environment by the RSA Global Services organization.

4.  Can you describe a use case addressed by Federation Security Analytics?

The use of covert channel activity is one use case. These are long, advanced, persistent threats that are difficult to detect. It requires monitoring of inbound and outbound connections and being able to detect internal hosts with strange outbound communication patterns (beaconing) and spot those external hosts that are most likely to be compromised (high risk suspicious domains). Being able to detect beaconing and suspicious domains will then allow you to identify the source of the attack. From this point, analysts can immediately pivot to identify the users that are under attack. The method for uncovering covert channel activity or malicious behavior requires the collection and analysis of multiple pieces of data over an extended time period so you can identify normal behavior and apply a weighted probability risk score to all subsequent behaviors.

5.  One of the biggest barriers to getting value from Big Data is the skills shortage. How does EMC address this issue?

Because this is an engineered solution, it removes the infrastructure skills necessary to architect a reliable, high-performing big data system that provides visibility to all of the data that is collected, analysis of both real-time and historical data, and generation of actionable results through real-time and long and slow alerting.

It also helps to remove up front security Data Science skills, as this solution also provides three security analytics applications or use cases using Data Science techniques out of the box. Through the community and security expertise from RSA, we will continue to develop and provide additional use cases to our customers. As threats continue to evolve, the enterprise can be better positioned to adapt to changes in threat strategy, as well as easily scale and modify its infrastructure without having to reinvent the solution.

All Paths Lead To A Federation Data Lake

Is your organization constrained by 2nd platform data warehouse technologies with limited or no budget to move forward towards 3rd platform agile technologies such as a Data Lake? As an EMC customer you have the advantage of leveraging existing EMC investments to develop a Federation Data Lake at minimal cost. Additionally, the Federation Data Lake will generate healthy returns, as it is packaged up with the expertise needed to immediately execute on data lake uses cases such as data warehouse ETL offloading and archiving.

Data Lake

With the release of William Schmarzo’s Five Tactics to Modernize Your Existing Data Warehouse, I wanted to explore whether the Dean of Big Data views data warehouse modernization tactics or paths ultimately leading to a Federation Data Lake.

1.  What is a Data Lake and who should care?

Continue reading

Don’t Accept The Status Quo For Hadoop

Hadoop is Everywhere – 99% companies will deploy/pilot Hadoop in 18-24 months according to IDC.  These environments will largely be based around standalone servers resulting in added management tasks due to data being spread out across many disk spindles across the data center.  With Hadoop clusters quickly expanding, organizations are starting to experience the typical growing pains one can compare to adolescence.  This begs the question- should DAS server configuration be the accepted status-quo for Hadoop deployments?

idcisilon

Whether you are getting started with Hadoop or growing your Hadoop deployment, EMC provides a long-term solution for Hadoop through shared storage and VM’s, delivering distinct value to the business in lower TCO and faster time-to-results.  I spoke with EMC Technical Advisory Architect Chris Harrold to explain why organizations are now turning to EMC to help transition Hadoop environments into adulthood.

1.  Almost every Hadoop deployment is based around the accepted configuration of standalone servers with DAS.   What have you seen as issues with this configuration with your customers?

Continue reading

Can Big Data Shape A Better Future? Quid is Paving the Way

World hunger, political conflict, business competition and other complex problems cannot be solved with mathematical algorithms measuring probabilities alone. However, by combining together human intelligence with the best artificial intelligence, the company Quid has built software that experts are calling the worlds first augmented intelligence platform. Using superior speed and storage capacity of computation, the process by which human beings typically acquire the deep pattern recognition of expertise is accelerated. The software does more than run simple prediction algorithms, it allows users to interact with data in an immersive, visual environment to better understand the world at a high resolution so that they can ultimately shape it and change it.

Founded in 2010, Quid is addressing a new class of problems to help organizations make strategic decisions around business innovation, public relations, foreign policy, human welfare, and more. Through advanced visualizations that interpret massive amounts of diverse internal and publicly accessible external data sets, Quid tells a unique and compelling story about the complexity of our world – trends, comparisons, multi-dimensional relationships, etc. – to change the direction of decision making.

For Quid, it’s not about man battling it out with machines, but rather, man working with machines when entering a new level of complex problem solving. For example, military intelligence may one day be able to change the direction of future conflicts by working with Quid software to analyze millions of data points from war logs and reports, news articles, and social media about the most recent casualties of war. The intelligence teams plugged into Quid would be able to see the war unfold as it happens across multiple data dimensions and uncover the mathematical patterns hidden in the data that are shaping the direction of the conflict.

Physics_explore

photo

I spoke with Quid Co-founder and CTO Sean Gourley to explain how Quid is helping organizations leverage Big Data and augmented intelligence to tackle the Bigger Problems they are facing in a fast moving world.

1.  Quid applies Data Intelligence to Big Data – a very different concept than applying Data Science to Big Data. Please explain.

Continue reading