Innovating The Marketing Process With A Data Lake

Like many global companies, EMC depends heavily on a CRM to manage sales and purchasing data about its vast global installed base. Over time, we realized that without big data analytics, this customer data was trapped inside our systems and providing limited value.


EMC decided that offering analytical capabilities through a data lake architecture would substantially increase the value of this data. To get there, EMC hired Todd Forsythe, EMC Vice President, Corporate Marketing, to create the Marketing Science Lab. I spoke to Todd about why he is so excited about the impact of big data and the Marketing Science Lab on sales and marketing:

1. What is the Marketing Science Lab and what were the key business drivers?

The EMC Marketing Science Lab funnels CRM data into EMC’s data lake solution for predictive modeling, segmentation analysis, and customer profiling. By marrying our internal data with external, unstructured data sources, such as social media conversations, we’re able to find individuals who are talking about progressive views on IT and business transformation.

Mix in analytics and we’re suddenly talking about targeted up-sell and cross-sell programs that drive far more engagement and sales than ever before.

With a 360 view of our customers using all of these digital breadcrumbs, our campaigns are getting to a higher level of personalization and precision targeting. Ultimately, we’re lowering acquisition costs and improving conversion rates—all music to the ears of anyone in marketing or sales.

2.  What were the key business drivers leading to the creation of the Marketing Science Lab?

As we’ve moved to a digital, social world, we found that highly predictive customer data often does not sit within the walls of our company. Instead, it resides in conversations across social media, community forums and so on. This has meant that the volume of data available to marketing has exploded. We needed a big data solution to store, analyze and visualize all of these social digital touch points with customers.

3.  What excites you most about the data lake and Marketing Science Lab?

Early in my career, I realized that data, especially customer data, was a magical lever that you can pull to improve marketing effectiveness and gain a richer understanding of your customers. Marrying modern social data with legacy data is exciting to marketing because it opens up a completely new way of looking at and interacting with our customers.

Traditionally, marketing organizations create a message, design a creative concept, identify the target market and use data to execute and measure. We now realize that big data should come first. It completely flips the marketing process upside down. Your customers’ behavior should drive your messaging, creative, and execution.

4.  What other organizational changes were necessary?

We’ve needed build up completely new skill sets in marketing. big data experts and PhD statisticians who can apply mathematical methodologies to large data sets have joined our team. We hired data scientists to manipulate and visualize large data sets. Having this layer of visualization is key so that marketers can easily consume and understand the findings and insight.

5.  What challenges did your team face with creating the Marketing Science Lab?

When we embarked on the Marketing Science Lab project, the biggest challenge was speed. We went from tumbleweeds to a full team with an operational big data solution in six months. Finding data scientist talent with experience in marketing also was hard to come by and took longer than we anticipated. And when marketing got wind of our new analytics capability, demand quickly outstripped our resources. So we needed to revamp our process to focus on the biggest priorities.

6.  How is EMC’s business benefitting from the Marketing Science Lab?

The results have been tremendous. When we apply models that identify customers with the highest propensity to buy in our marketing campaigns, we see response and conversion rates up to 10 times higher than before. Delivering more relevant offers to our customers improves sales productivity because better conversion rates means that the selling process is more effective. And it increases ROI because we’re spending less while creating greater yield.

7.  What are some of the best practices you would recommend to other organizations embarking on a big data project?

If your organization wants to tap into the power of big data, you first need to look closely at your end process and understand clearly how big data analytics would affect your process at every step. Second, determine how to scale insights to create the greatest impact. For example, as we build these predictive models, we embed them into our CRM environment in a way that makes it easy for marketers across the globe access segmentation and propensity to buy information and engage in data-driven decision-making.

EMC CIO Takes On Big Data Problems With Big Data Analytics

Every second of every day, IT generates enormous amounts of data around operational activity – system behavior, application performance, user actions, security activity, and more. Instead of viewing this data explosion as a Big Data problem, IT views it as opportunity to use Big Data solutions such as IT Operations Analytics to improve the quality of their services.


For example, 75% IT professionals surveyed recently said that they believe that IT Operations Analytics are able to transform data into relevant insights into actionable plans for improvement. I spoke with EMC CIO Vic Bhagat to describe how EMC is embracing Big Data for IT Operations Analytics to solve critical problems affecting EMC IT Operations and customers.

1.  What are the biggest problems faced by IT Operations Management at EMC and how were these problems addressed before the world of Big Data?

IT generates enormous amounts of data when monitoring complex, rapidly growing and changing IT infrastructures and the applications. The challenge for IT Operations Management is to leverage this data to build an adaptive system that is more proactive, and less reactive. The more the system can learn from the data, the better it can identify variances and problems areas in a timely manner to help IT fix issues before it negatively impacts the business such as downtime or poor performance.

In the past, we relied on traditional business intelligence and data warehousing systems to gain intelligence or insight based on historical trends. Now, with analytics, we can uncover important variables and modify them to predict an outcome. And, the more data we collect at a detailed level, the more accurate we can be.

2.  How does Big Data analytics change the game to address these problems more effectively?

It cuts down the time to gain insight. The most heavily used word after ‘selfie’ is now ‘data lake’. Everyone wants to build a data lake since it provides the right architecture and capabilities to cut down the cycle time in deriving newer, predictive insight, and then continuously integrating these results back into our business processes and decision-making. At EMC, we are moving away from data warehouses to a data lake architecture enabling us to not only gain faster insight, but also gain newer insight by bringing together and analyzing both structured and unstructured data.

For example, in a data warehouse you manage structured data such as part numbers, bay numbers, disk numbers, chassis numbers, and more. In a data lake you can manage all of this structured data in addition to unstructured data such as user manuals for each system and component. Let’s now apply this data lake solution to a use case – we continuously monitor the health of a customer’s infrastructure with our call home systems. We can now leverage a data lake with more data sets to not only make more accurate component failure predictions, but we can also provide the relevant information needed from user manuals to fix the problem in a timely manner so the customer experiences no downtime.

3.  What is EMC’s IT Operations Analytics solution leveraging Big Data technologies and techniques?

We are leveraging the entire Pivotal Big Data Suite to ingest and store all of the structured and unstructured data – Pivotal Gemfire XD, Pivotal HD, Pivotal HAWQ, and Pivotal Greenplum Database. Our Data Scientists are then able to apply advanced analytic techniques to the data they need using their choice of tools which are MadLib, R, and Python. This Big Data environment will be part of a wider business data lake strategy, where all enterprise data will be managed, accessed, and used equally by all business applications, not just IT Operations. Only a few legacy or specialized applications will standalone.

4. What benefits has EMC gained from this Big Data solution?

The benefits are enormous and can be extracted from both business and technical benefits. Building predictive models and predicting imminent system failure reduces downtime and the number of alerts and enables us to identify the real issues faster, reducing the cycle for decision making and taking corrective action. This improves our performance, productivity and value we gain from Big Data.

But we are only scratching the surface. The more we can optimize our Big Data environment so that it is elastic and accessible, the faster and more precise Data Scientists will be in solving problems. For example, we can now predict MS Exchange outages two hours in advance.

5. One of the biggest barriers to getting value from Big Data is the skills shortage. How does EMC IT Operations address this issue?

EMC had the foresight to build Centers of Excellence (COE) around the globe, producing the expertise and skills needed to transition into the realm of Data Science. We are fortunate to leverage talent within the company, but also leverage the COE to attract and acquire new Data Science talent outside the company.

6. What books are you currently reading on your Kindle or if you are still paper based like me, what books are stacked on your nightstand?

I’m Kindle based, so I read periodicals such as Techmeme and Engadget. Since we are a company that is data and digital driven, I am reading a book called ‘Leading Digital’. I want help lead this digital revolution at EMC and this book provides great examples of how digital makes significant changes in how a company operates and kills bureaucracy.

Big Data Pains & Gains From A Real Life CIO

What does it take to make CIO Magazine’s Top 100 List? Big Data victory is one of them.
Michael Cucchi, Sr Director of Product Maketing at Pivotal, had the privilege to speak with one of the winners – EMC CIO Vic Bhagat. Discussing the pains and gains of EMC’s Big Data initiative, I have put together a summary of this interview below.  EMC IT’s approach to Big Data is exactly what the EVP Federation enables organizations to do – first collect any and all data in a Data Lake, deploy the right analytic tool that your people know how to use to analyze the data, and finally learn agile development so you can take those insights and build applications rapidly.

1. Why is Big Data important to your business?

Continue reading

Can Big Data Shape A Better Future? Quid is Paving the Way

World hunger, political conflict, business competition and other complex problems cannot be solved with mathematical algorithms measuring probabilities alone. However, by combining together human intelligence with the best artificial intelligence, the company Quid has built software that experts are calling the worlds first augmented intelligence platform. Using superior speed and storage capacity of computation, the process by which human beings typically acquire the deep pattern recognition of expertise is accelerated. The software does more than run simple prediction algorithms, it allows users to interact with data in an immersive, visual environment to better understand the world at a high resolution so that they can ultimately shape it and change it.

Founded in 2010, Quid is addressing a new class of problems to help organizations make strategic decisions around business innovation, public relations, foreign policy, human welfare, and more. Through advanced visualizations that interpret massive amounts of diverse internal and publicly accessible external data sets, Quid tells a unique and compelling story about the complexity of our world – trends, comparisons, multi-dimensional relationships, etc. – to change the direction of decision making.

For Quid, it’s not about man battling it out with machines, but rather, man working with machines when entering a new level of complex problem solving. For example, military intelligence may one day be able to change the direction of future conflicts by working with Quid software to analyze millions of data points from war logs and reports, news articles, and social media about the most recent casualties of war. The intelligence teams plugged into Quid would be able to see the war unfold as it happens across multiple data dimensions and uncover the mathematical patterns hidden in the data that are shaping the direction of the conflict.



I spoke with Quid Co-founder and CTO Sean Gourley to explain how Quid is helping organizations leverage Big Data and augmented intelligence to tackle the Bigger Problems they are facing in a fast moving world.

1.  Quid applies Data Intelligence to Big Data – a very different concept than applying Data Science to Big Data. Please explain.

Continue reading