Fast forward 2018 and today every CEO, CIO, CDO, CMO is seeking answers to questions that don't exist yet.
Data scientists solve complex data problems through employing deep expertise in some scientific discipline.
Why do you need Data Scientists in 2018?
Until a few years ago, the work of data scientists was isolated and mattered primarily for research and/or R&D purposes. The industry has been extremely thankful forthe contributions of these clever individuals, but we need them now in the mainstream! This needis creating a huge demand in the industry and it's radically transforming the hunt for big data and machine learning talent.
Typical questions in the minds of the leaders could be...
#1 How do I drive innovation within my company, industry or community in a dramatic fashion?
#2 How do we disrupt the industry with our products and services?
#3 How do we change and challenge our workforce's mindset by providing insightful “out-of-the-box” data analytics?
#4 How do we launch a new LOB (Line of Business), product and/or service using these thin/thick/lazy/idle datafied analytics with machine and deep learning?
#5 For Governmental or CSR-driven businesses - How do we dramatically drive social change, reduce crime using data? For instance, see how Memphis reduced its crime rate by 30% using data.
Think about it from your own industry perspective.
An Example: The Oil & Gas sector. How do we enhance and optimize exploration and production (E&P)? With conventional technologies becoming commoditized and depleting traditional resources, one has to competitively use unconventional methods such as deepwater, tight oil or shale gas. I won't get deeper into aspects such as seismic plotting, WAZ and NATS, etc. as that is only generating more data waiting to be exploited!
Think what it can mean for your industry if you had the “right eyes” to spot the gold!
No industry today can say that it is not data driven. With vast volume, speed and variety of data coming from external and internal sources, the need to scientifically approach data is paramount forcompetitive intelligence of organization. That is what is keeping CxOs awake at odd hours.
Say Hello to Data Science & Data Scientists
A Wikipedia definition of data science...
Data science is the study of the generalizable extraction of knowledge from data,  yet the key word isscience. It incorporates varying elements and builds on techniques and theories from many fields, including signal processing, mathematics, probability models, machine learning, statistical learning, computer programming, data engineering, pattern recognition and learning, visualization, uncertainty modeling, data warehousing and high performance computing with the goal of extracting meaning from data and creating data products. Data Science is not restricted to only big data, although the fact that data is scaling up makes big data an important aspect of data science.
And who is a Data Scientist?
It is generally expected that data scientists are able to work with various elements of mathematics, statistics and computer science, although expertise in these subjects are not required. However, a data scientist is most likely to be an expert in only one or two of these disciplines and proficient in another two or three. This means that data science must be practiced as a team, where across the membership of the team there is expertise and proficiency across all the disciplines.
Why and Why Now?
Okay, so having addressed the boringwhatpart, let's get on to thewhyof it. What is really happening in the industry?
I was having an interesting conversation with one of the co-founders of my previous management consulting startup, and we were discussing why the consulting industry was being disrupted so dramatically. If you didn't notice, many consulting firms such asBooz, Monitor Group, etc. have succumbed already to their traditional “ways of working,” and many will follow suit if they don't change the way they work!
The key challenge was that folks did (and still do!) template-based consulting. If you are employed by those firms or have been a client working with those firms then you know exactly what I am talking about.
Customers are increasingly asking for the 'what-if' approach; they don't care anymore about your diagnostic/prescriptive approach. They can do that themselves!
Customers & consumers today are more advanced and data savvy than you can imagine. They can search, collate, splice/dice and come with standard and even relatively advanced options before you can. What they need is expertise, insight and powerful “what-if” thinking! If you don't provide for that, you're out!*
Similar change is happening in the enterprise data management landscape. We are increasingly moving away from a “descriptive or prescriptive, causal, “tell-me-what-happened” model” to “predictive, correlational, all-structured, what-if model.”
We have officially entered the era of data-intensive computing. Two factors are encouraging this trend. First, vast amounts of data is becoming available in more and more application areas, and second it is affordable infrastructures (cloud, etc.) allowing for persistently storage/retrieval/sharing of data that was previously unthinkable and/or inaccessible.
This is only getting more intense as application of Artificial Intelligence into IoT (Internet of Things) - which is touted as the next platform, is using emerging technologies such as near-field communications, real-time localization, and embedded sensors to convert dumb things into smarter things.
This tectonic shift of application of theory, experiments and simulations into mainstream business allows unifying knowledge from scientific research with vast amounts of multidisciplinary data and thereby creating real business wisdom that was previously unthinkable!
The following diagram should make it clear what I mean. Where do you see your organization in this picture?
Harnessing this Opportunity
The massive amount of personalized information being pushed out by mankind is a huge opportunity to spot, sort and commercialize on the patterns applicable to your business. Whether it is analyzing billions of transactions to detect fraud (risk management) or forecast/predict consumer behavior to plan for growth via customer acquisition/retention or development, you need more than just a cool data platform. You need a solid team and a big data management plan.
To build and run successful big data teams and projects you need data scientists within your teams. Big data is not just about Hadoop or some cool new technology, it is about doing the “what-if” before your competitor does. And then move on to the next “what-if.”
Instead of only focusing on using a consistent set of data to measure past performance and report for business planning, businesses must also focus on a combination of analytics and machine learning techniques so they can draw inferences and insights out of the massive sea of data. This will help you solve the higher order questions and derive far greater business value that you could have ever imagined!
*We have built a successful business with that mindset and there have been customers who kicked out the “template-based, prescriptive” big consulting guys to choose to work with us. Repeatedly!
Tarry Singh, columnist, is CEO, Founder and AI Neuroscience Researcher of AI startup https://deepkapha.ai. deepkapha.ai focuses on the following three pillars: 1) breakthrough AI research that intends to knit the world of neuroscience, models and frameworks around deep learning for the future, 2) build AI Solutions for corporate customers and train engineering teams to holistically build AI solutions with hands-on, market-relevant advanced AI projects, and 3) AI Philanthropy initiative “givebackAI" to train the world that cannot afford an expensive education. We have already trained over 10.000 learners worldwide and expect to have trained* about 50.000 by mid 2019.