Can Big Data Shape A Better Future? Quid is Paving the Way

World hunger, political conflict, business competition and other complex problems cannot be solved with mathematical algorithms measuring probabilities alone. However, by combining together human intelligence with the best artificial intelligence, the company Quid has built software that experts are calling the worlds first augmented intelligence platform. Using superior speed and storage capacity of computation, the process by which human beings typically acquire the deep pattern recognition of expertise is accelerated. The software does more than run simple prediction algorithms, it allows users to interact with data in an immersive, visual environment to better understand the world at a high resolution so that they can ultimately shape it and change it.

Founded in 2010, Quid is addressing a new class of problems to help organizations make strategic decisions around business innovation, public relations, foreign policy, human welfare, and more. Through advanced visualizations that interpret massive amounts of diverse internal and publicly accessible external data sets, Quid tells a unique and compelling story about the complexity of our world – trends, comparisons, multi-dimensional relationships, etc. – to change the direction of decision making.

For Quid, it’s not about man battling it out with machines, but rather, man working with machines when entering a new level of complex problem solving. For example, military intelligence may one day be able to change the direction of future conflicts by working with Quid software to analyze millions of data points from war logs and reports, news articles, and social media about the most recent casualties of war. The intelligence teams plugged into Quid would be able to see the war unfold as it happens across multiple data dimensions and uncover the mathematical patterns hidden in the data that are shaping the direction of the conflict.



I spoke with Quid Co-founder and CTO Sean Gourley to explain how Quid is helping organizations leverage Big Data and augmented intelligence to tackle the Bigger Problems they are facing in a fast moving world.

1.  Quid applies Data Intelligence to Big Data – a very different concept than applying Data Science to Big Data. Please explain.

Continue reading

Addicted To Analytics – EMC’s Marketing Science Lab

Our CMO Jeremy Burton is a visionary, always at least three steps ahead of the pack. I identified this gift early on, recalling Jeremy’s ability to excite a very calm and composed Technology Marketing team at Oracle 12 years ago through innovative ideas. I knew he was on the rise – Big and Fast. Yes, pun intended. Today, Jeremy’s vision at EMC is not only to harness Big Data to build a data-driven Marketing organization, but also to inspire other organizations to do the same in their Marketing departments through EMC’s Marketing Science Lab.

The Marketing Science Lab, built on EMC Big Data technology, provides a 360 degree view of customers in order to better understand their behavior and sentiment and improve marketing effectiveness. The Marketing Science Lab is not only accessible to EMC Marketers, but also presents itself through a highly visual dashboard application at EMC’s Executive Briefing Center in Santa Clara, CA. EMC’s Michael Foley leads the Marketing Science Lab and provides us with more insight into how this Big Data project is transforming EMC’s business and making everyone in Marketing addicted to analytics.


1.  Can you describe the Big Data technology behind the Marketing Science Lab?

Continue reading

How Will Twitter’s API Restrictions Affect Big Data Applications? DataSift Helps Clarify What’s Going On

Earlier this month, Twitter stepped up its enforcment of an API restriction that does not allow the building of ‘client apps that mimic or reproduce the mainstream Twitter consumer client experience’. Around the same time, LinkedIn announced that its users would no longer be able to display their tweets on LinkedIn. Will LinkedIn’s user experience be negatively impacted? What might happen to other applications? !

Fortunately, for every door that closes, a new one opens. DataSift is one of a select few companies that Twitter has entrusted to resyndicate and provide access to the full Twitter feed for use in internal analytics applications. Essentially, Twitter is leaving the door open, but through a process managed by third parties. There is one catch – you have to pay a fee for access. The good news is that the fee comes with value added platform services from DataSift, targeted at your specific industry and business need.

What I found most interesting about DataSift is their vision. DataSift believes that every entity (small, medium, or large) should have the ability to take advantage of Big Data, and especially Social Data. As DataSift Founder & CTO Nick Halstead puts it, “We are trying to help democratize the Big Data industry to enable entrepreneurs and enterprises to easily create socially-intelligent applications. No data-scientists required, no Hadoop expertise needed.”

I was so moved by DataSift’s mission, that I used my social network to contact Rob Bailey, CEO of DataSift, to learn more on how the company can bring Big Data even to the little guy for use in analytics platforms.

1.  Anyone can access and manipulate Twitter data using publicly available APIs. Why would an organization pay for access to Twitter data through DataSift?

Continue reading

How Valuable is Facebook Data?

Facebook goes IPO this Friday. This milestone event has me thinking. With 900 million active users generating terabytes of data each day, how valuable is Facebook data?

I do believe Facebook advertisers and application developer gain real value from Facebook data to in order to grow and stay competitive. For example, a Facebook game developer can improve customer loyalty because its customers are Facebook users who grant permission to access their private data such as posts and comments to perform sentiment analysis.

But unless you are a Facebook advertiser or application developer, how can you accurately measure sentiment from Facebook data? And if used, can it lead to bad decision making? I believe so.  Most user profiles on Facebook are not public; therefore, you obtain a very small sample size of sentiment from your customers. Not all Facebook users post or comment on public group or fan pages; therefore, you again obtain a small sample size of sentiment.

Let me give you an example. A retailer uses Facebook to capture customer sentiment and discovers negative sentiment for its Mother’s Day Spa Basket through conversations on its public fan page and other related public conversations. To keep customers loyal and happy, the retailer sends a $10 gift card to all customers who had purchased the Mother’s Day Spa Basket. In reality, perhaps only a small percentage of customers are unhappy. As a result, it costs more for the retailer to offer gift cards than it costs to lose a small percentage of its customers.

Enough about my perspective on the value of Facebook “Big Data”. Instead, lets hear from Facebook directly, the master of leveraging Big Data to successfully grow its business. Greg Dingle, Software Engineer at Facebook, provides a glimpse into how analysts at Facebook leverage Big Data to deliver a value to Facebook users.

What is your role at Facebook?

Continue reading