Want To Build A Data Science Team? EMC Offers a Holistic Approach

Many of our customers invest in big data solutions to target their sales prospects better, explore advanced medical research, and make their internal processes more efficient. The biggest obstacle to getting these initiatives out of the gate is the shortage of big data skills within their own firms and across the industry.

To address this skills gap, EMC has developed a thorough data science and big data analytics curriculum for our customers. EMC was one of the first companies to offer data science education with rigorous, live instruction using free and open source tools. As of today, more than 10,000 customers, partners, and college students have attended the training.


I spoke with EMC’s David Dietrich, who leads this unique program to discuss his approach to data science education, which differs from more traditional product-oriented education. What I found most interesting is that in addition to David’s work at EMC, he has also helped design big data analytics curricula for Babson College and other universities.  More recently,  David has published a book, Data Science and Big Data Analytics, to help further develop data science skills and expertise in the industry.

1.  Why is EMC pushing so hard to educate and develop data scientists?

As an information company, we’re extremely attuned to the value of big data, which is exploding in both the sheer amount and how organizations in virtually every field and industry are using it to solve critical problems. When EMC acquired our first big data company, Greenplum, several years ago, we quickly became aware that there was a shortage of people who had the data science and business skills to help companies utilize big data.

2.  How is EMC taking a holistic approach to data science education?

We recognize that learning how to use big data technology alone does not ensure success. Senior management must make sure that appropriate people and processes are in place to drive the change and innovation necessary for valuable big data results to occur. To help companies on their journey, we offer courses for data scientists, who execute big data projects, and business executives who sponsor, run and manage them.

Our goal is to educate all levels of an organization so that data scientists and business people understand one another. That way, the organization is able to roll out big data projects with greater adoption and success. In addition to offering courses to our customers, we also work closely with universities and educational institutions to help them develop their own curriculum and programs.

3.  Please describe some of the important skills for aspiring data scientists.

Working in strategy and analytics for the past 20 years, I’ve always been drawn to experimenting with data to solve problems, which is exactly is the mindset you need to tackle big data. Companies often ask me how to go about using massive amounts of structured and unstructured data to solve business problems. How do they know what to choose and ignore? How do they know what algorithms to apply? Our courses encourage a culture of experimentation that leads to answering these questions. We teach our students how to test an idea with data, measure it quantitatively, learn from it and iterate. This test and learn mindset is critical to becoming a talented data scientist and data-driven organization.

4.  What are some of the challenges with evolving into a data-driven organization?

There can be a substantial divide between data scientists and business people who manage and work with them on big data projects. Many business people lack the technical background to understand how the algorithms apply to the problem and how to test ideas with data. And some data scientists may not understand the business context. We’re trying to educate each side so they can get a clearer picture and drive toward common goals. Once you bridge that gap, you can start driving real change, and solving old problems with big data or new information sources that were once unusable.

5.  What should companies expect after they have successfully made the leap to big data?

We’re educating them in how to train and staff a big data team, as well as build processes to be effective and successful. With this approach, companies can more effectively define the business problem, acquire the right data sets, experiment, communicate the results, and finally, operationalize the new processes.

Revolution Analytics Boosts the Adoption of R in the Enterprise

The path to competitive advantage is being able to make predictions from Big Data. Therefore, the more you can build predictive analytics into your business processes, the more successful your organization will become. There is no doubt that open-source R is the programming language of choice for predictive analytics, and thanks to Revolution Analytics, R has the enterprise capabilities needed to drive adoption across the organization and for every employee to make data-driven decisions.

Revolution Analytics is to R what the vendor RedHat is to the Linux operating system—a company devoted to enhancing and supporting open-source software for enterprise deployments. For example, Revolution Analytics recently released R Enterprise 7 to meet the performance demands of Big Data whereby R now runs natively within Hadoop and data warehouses. I spoke with David Smith, VP of Marketing at Revolution Analytics to explain how Revolution Analytics has accelerated the adoption of R in the enterprise.

1.  What benefits do Revolution Analytics provide to organizations over just using open-source R?

Continue reading

Dear BI Users: Your Hadoop SQL Wish Has Finally Come True

To accelerate the value of Big Data, many products have been developed to make data managed in Hadoop much easier to access and analyze through SQL.  First there was Hive, which provides a SQL query abstraction layer by converting SQL queries into MapReduce jobs.  More recently, Cloudera announced Impala which bypasses MapReduce to enable interactive queries on data stored in Hadoop using the same variant of SQL that Hive uses.  And today, EMC Greenplum announced Pivotal HD, the only high performing, true SQL query engine on top of Hadoop.  Don’t be confused by these approaches, as there is a common thread – to leverage Hadoop as a Big Data platform for running SQL queries.  The major difference with Pivotal HD is that now there is a single, scalable, flexible, and cost-effective data platform for all of your analytic needs.



I spoke with Greenplum Chief Scientist Milind Bhandarkar to explain this breakthrough SQL interface to Hadoop.

1. How does Pivotal HD provide a true, high performing SQL interface to Hadoop?

Continue reading

OpenChorus Project: The Dawn of The Data Science Movement

OpenChorus Project is the first real attempt to help companies succeed with Big Data. How? We all know that the barrier to success has been a lack of available data science talent and the tools needed to address Big Data analytic challenges. Open sourcing Greenplum Chorus is an attempt to rapidly grow the data science community by giving them a rich analytic platform to easily gain insight, grow and share their skills, and ultimately deliver value with Big Data projects.

Partners, startups, and even individual developers can download the source code and deliver new Chorus-integrated Big Data applications and tools needed for the diverse requirements across industries and business functions. For example, the release of Greenplum Chorus 2.2 at the end of this quarter will include valuable contributions from partners Gnip, Tableau, and Kaggle, enabling Data Scientists to correlate Twitter data into their analysis, leverage advanced Tableau visualizations, and gain access to Kaggle expert Data Scientists.

Check out the interview with Logan Lee, Director of Product Management at Greenplum, about the company’s reasons for releasing the Chorus code and the types of contributions that are expected to create a much needed Data Science movement.