Current Data Scientist Craze Can’t Last
Randy Bean
NewVantage Partners

Randy Bean is an industry thought-leader and author, and CEO of NewVantage Partners, a strategic advisory and management consulting firm which he founded in 2001.  He is a contributor to Forbes, Harvard Business Review, MIT Sloan Management Review, and The Wall Street Journal.   

By Randy Bean, CEO and Managing Partner, NewVantage Partners

This article is by Featured Blogger Randy Bean from his LinkedIn page.

Yogi Berra once remarked, “Nobody comes here anymore, it’s too crowded.” Fashion, restaurants, political candidates, academic theories, and business trends come and go. Would you believe the same forces apply to the data professional?

Today every company wants a data scientist... or 20. The current demand is driven by a proliferation of data, resulting in a matching demand for the expertise required to organize, analyze, navigate, and sift through large sets of data to identify patterns, unearth insights, and do this all much faster than the competition. In the now glamorized world of national intelligence, the sine qua non for data analytics, the mission is to strike them before they strike us.

Yet while it must be rewarding, both professionally as well as financially, for members of the data science profession to now receive the recognition they have long felt overdue, there are reasons for caution and humility as well.

Why caution and humility? I frequently listen as executives gripe or commiserate about their inability to uncover fresh or breakthrough insights fast enough. Common complaints include an “inability to see the forest from the trees”, being “stuck in the weeds”, or suffering from “analysis paralysis”. Notable successes are often less publicized, and in many cases remain secret, because to tell the story would be to “give it away” or reveal the “secret sauce.”

When intelligence failures occur, in business or in national security, these failures are rarely attributed to a lack of data or analysis, but to a lack of coordination or interpretation. While data expertise is valuable, effectiveness dictates that it be coupled with business expertise, judgement, and communication results.

The data scientist has not always been perceived as an essential member of the business team. As a profession, data analysts have fought a long battle for recognition and respect. While I am not a data science professional myself, I frequently tell a story at cocktail events, when people ask me what line of work I am in. These days, if I make some reference to “Big Data”, people hover around. This was far from the case in the past. Usually, you mentioned the term “data” and people would flee. Tastes change.

The fact is that data analysts have been in abundance for decades — in different guises. In business, they have occupied positions such as actuary, market researcher, statistician, informatics analyst, and predictive modeler. There have been periods in the past when data analysts have been celebrated as representing “the new science of business.” Other times, statisticians and analysts have been relegated to the sidelines —perceived as non-essential, non-mission critical, or not tied directly to revenue growth. I recall in the wake of the 2008 financial crisis, seeing data analyst positions eliminated, and market research and statistics groups disbanded in a wholesale wave of downsizing, consolidation, and belt tightening.

Does this mean that data scientists are just a fad? Yes and no. It’s hard to see demands diminish for the type of skills that data scientists represent. Yet, like other fads, emotional associations will diminish over time leading to, one hopes, a clearer understanding of the movement. For the data profession, this could be a good thing.

Let me explain how this could look. A few years ago, I met with a large financial services firm which was proposing to hire “thousands” of Big Data scientists, in spite of having a vast and well-regarded organization of PhD statisticians and data modelers that had been in place for decades. I was curious why it was necessary to hire so many new people. Couldn’t the old people be “trained”? No, I was told, the “cultural divide” was too great. Fresh perspectives and new statistical approaches were needed. “We’ll find them in Silicon Valley!”

Early this year, I checked back to see how all the new data scientists were faring. “Oh”, the chief executive replied, “we hired some, but determined it was more effective to train our existing people”. It turned out that an understanding of their business, in all its complexity and nuance and culture, actually mattered. That “feel for the business” trumped the new data analytics skills. Human judgment counts after all. Panaceas rarely exist.

Originally published on LinkedIn