What Are The Real Effects Of Climate Change? EMC Utilizes Data Science To Find Out.

Mona Patel

Mona Patel

Senior Manager, Big Data Solutions Marketing at EMC
Mona Patel is a Senior Manager for Big Data Marketing at EMC Corporation. With over 15 years of working with data at The Department of Water and Power, Air Touch Communications, Oracle, and MicroStrategy, Mona decided to grow her career at EMC, a leader in Big Data.

Contributing to social good is now literally at everyone’s fingertips. That is why EMC and Earthwatch Institute have teamed up to encourage citizens to become data collectors, or citizen scientists. Through the collection of more data sources, data scientists can better uncover how climate change is affecting plants and animals by altering the timing of key natural events.

This collaboration is called the Whenology project, with the first study underway to investigate how climate change is affecting raptor migrations at Acadia National Park. To create awareness and encourage more participation, EMC launched a microsite that provides educational materials, track progress, and report insights.


I spoke with EMC Distinguished Engineer John Cardente about the Whenology project and it’s potential to provide a powerful citizen science platform for collaboratively tackling virtually any large-scale, high impact societal issue.

1.  What is the Whenology project and what are your major objectives?

The Whenology project was born out of collaboration between the EMC Corporate Sustainability Office and Earthwatch Institute. The project’s name is a play on Phenology, a field of science that studies how climate change affects the seasonal timings of plant and animal life cycles.

The goal is to help scientists bring a variety of data sets together for the first time, analyze them to better understand how climate change may be disrupting complex interactions between life-cycle events (phenophases), and improve collaboration with citizen scientists. EMC’s comprehensive portfolio of Big Data solutions and technologies puts us in a unique position to help this important scientific endeavor.

We’re kicking off Whenology with a pilot project to study phenophase changes related to raptor migrations at Acadia National Park. Acadia is an important waypoint along the Eastern Seaboard migration route and scientists are worried that changes there may prevent migrating birds from getting the nutrition needed to complete their journey.

This project relies heavily on observational data collected by citizen scientists. It would be impossible without the participation of citizen science organizations like eBird, the Hawk Migration Association of North America, and the USA National Phenology Network who have all provided data for the project.

2.  The key to insight is not only about building the right model, but also about asking the right questions. What questions are you hoping to get answers for with big data?

The interactions between species in nature are varied and complex. But they all work off a common “clock”, climate patterns. As climate change perturbs this clock, scientists are unsure how those interspecies relationships are being affected. The fear is that a tipping point will be reached after which sudden, drastic changes will occur. That’s pretty scary. We’re hoping that by assembling a wide variety of data sets and providing Big Data tools, scientists will be able to uncover the relationships, develop models capable of forecasting phenophase changes, and initiate societal changes to prevent bad outcomes.

From a technology perspective, we’re very interested in learning more about enabling large-scale data science collaborations. Building environments to enable citizen scientists from across the globe share data, analytics, visualizations, and insights will yield valuable lessons that EMC can in turn use to help its customers.

3.  What stage are you at with this project and what obstacles are you facing?

The team has done a lot of great work to get the pilot project going. We’ve worked with the citizen science organizations mentioned above to bring multiple data sets together in a single environment for the first time. In addition, we’ve developed a suite of analytics software to collaboratively combine, analyze, and visualize the data. We’re starting to uncover interesting insights but want to be responsible about reporting any findings and therefore are waiting until the rigorous analyses are completed.

Obtaining the data was a challenge, as we had to ensure any agreements the participating organizations had with their users were not violated. But perhaps the biggest challenge has been determining the right balance between maintaining scientific rigor and sharing findings with the public in a timely manner so that we can start to influence behaviors.

Publishing findings too early or late could be equally harmful. We want to get that right and we’re fortunate to have great scientists involved to make sure we do. What hasn’t been a challenge is finding people to participate in the project. A lot of people at EMC care deeply about the environment and are excited by the opportunity to use their technical skills to make a difference. It’s been a profound experience.

4. What is your role with the project? EMC?

As a Distinguished Engineer in the Corporate CTO Office, I often get tasked with bootstrapping new initiatives. My role focuses on Big Data and Data Science so it seemed natural to support this project. To that end, I developed the initial suite of software to process, analyze, and visualize the data. I also created the dynamic data visualizations for the microsite. I’ve had a tremendous amount of fun working on this project. More importantly, I’ve developed a strong interest in data science for social good and plan to do more projects like Whenology in the future.

5.  What technologies, tools, and skills are required for this project and what are the gaps?

This project is a great example of exploratory analytics, we’re not sure what insights are hidden inside the data or how to find them! This situation requires the usual data science skills like data wrangling, feature engineering, applying machine learning techniques, and visualizing data. It also requires a lot of applied curiosity, and experimentation. That’s the part of data science that I really enjoy.

The pilot project’s limited scope makes it possible to use open source tools like R, Python, Spark, and D3js. As the project expands, however, we’re going to need more capable technologies like those provided by EMC’s Federation Business Data Lake.

Our goal is to expand the Whenology project to cover not only a wider geographic region but also other climate change related topics. If successful, we might even expand to hosting social good projects related to other “grand challenges” like healthcare. Accomplishing that will likely require the full compliment of EMC Federation Big Data technologies.

6.  For people reading this blog story, how can they help or participate?

To start, readers can checkout the “Participate” section of the Whenology microsite. It provides profiles and contact information for the citizen science organizations participating in the Whenology project. Or, check out SciStarter.com and search for a citizen science project that matches your interests and capabilities. We need your help! Together, we can make a big difference.

Hadoop Summit 2015 Reflections

Chris Harrold

Chris Harrold

CTO Big Data Solutions at EMC
Chris is responsible for the development of large-scale analytics solutions for EMC customers around emerging analytics platform technologies. Currently, he is focused on EMC Business Data Lake Solutions and delivering this solution to key EMC customer accounts.


Before the ink has even really dried on HS15 in San Jose I am sitting down in a rare moment of peace to write out some reflections from my experience and what I have seen from the sessions, keynotes, partners, and users here at the show.

Hadoop Gets Real

The most lasting impression I got from the overall theme of the show and the people in attendance was that Hadoop is not an “emerging tool” anymore. The momentum, use cases, and indeed the buzz of attendees was that there is massive adoption and momentum built up in the marketplace. Behind this wave of early adoption is a lot of pent-up demand that is waiting for things to stabilize and become more enterprise ready. Once the tooling around the Hadoop ecosystem is more robust, and the platforms that it runs on are more operational, there is no limit to the demand that this ecosystem can produce.

In counterpoint to this fact, there is another countercurrent of theme that Hadoop is not “all things to all people”, and so there is a lot of discussion around the emergence of the logical successor to Hadoop as the analytics tool of record. Certainly the buzz around Spark is indicative that this is the way of the future and ties into the second theme of the show that I observed in numerous conversations and sessions.

Continue reading

Is It All About The Data Scientist?

Mona Patel

Mona Patel

Senior Manager, Big Data Solutions Marketing at EMC
Mona Patel is a Senior Manager for Big Data Marketing at EMC Corporation. With over 15 years of working with data at The Department of Water and Power, Air Touch Communications, Oracle, and MicroStrategy, Mona decided to grow her career at EMC, a leader in Big Data.

The answer is no. It is a holistic, team effort that involves expanding the mind and skill set of executives, business users, IT implementers, data scientists, and application developers to all work collectively to define a strategy and derive newer insight from big data.

And that is why EMC is so heavily focused on breaking down organizational silos and training professionals to become data scientists or at least think like data scientists, transforming these individuals into data savvy professionals working towards the same goal – competitive advantage.

I spoke to Louis Frolio, Advisory Technical Ed Consultant for EMC Big Data Solutions, how as part of a team in EMC Education Services is creating a massive professional transformation through a MOOC – Massive Open Online Course. Data Lakes for Big Data MOOC gives you an opportunity to become a data savvy professional and take on a big data or data science role in your organization at absolutely no cost.

The course kicked off May 11, but you still have plenty of time to enroll and complete the course to earn a certificate before June 8. The top 500 students (based on cumulative grade for the MOOC) will receive an electronic copy of the Data Science book just released by EMC Education Services.

1.  What is a MOOC and what is the goal of this education format? Why was it used for this course?

Continue reading

When Big Data Becomes More Valuable Than Your Products/Services

Mona Patel

Mona Patel

Senior Manager, Big Data Solutions Marketing at EMC
Mona Patel is a Senior Manager for Big Data Marketing at EMC Corporation. With over 15 years of working with data at The Department of Water and Power, Air Touch Communications, Oracle, and MicroStrategy, Mona decided to grow her career at EMC, a leader in Big Data.

A recent global study across 1000 executives conducted by EMC and Capgemini reports the following: “64% of respondents said that big data is changing traditional business boundaries and enabling non-traditional providers to move into their industry, and over half (53%) expect to face increased competition from start-ups enabled by data.”

My take: Eventually any company expecting to compete effectively must become a software company, where data is the primary asset driving business strategy and revenue. Going a step further, by monetizing big data, companies are creating new revenue streams that will actually eclipse the value of a company’s existing products or services over time. This is supported by the EMC and Capgemini study as well: “Among our respondents, 63% consider that the monetization of data could eventually become as valuable to their organizations as their existing products and services.”


The question is how to find gold in the flood of data flowing in and out of the organization to compete effectively, especially against new digital startups. To answer this question, I spoke to Capgemini’s Global Vice President for Big Data Steve Jones who strongly believes the answer lies within the power of a business data lake.

1.  As an industry leader in big data, what is so exciting about a data lake solution?

Continue reading