Love it or hate it, big data is here to stay. As data volumes and sources of data proliferate at ever increasing rates, leading companies will be forced to plan for a data-driven future. Data is pervasive. Businesses operate in an Age of Data. Rapid access to the latest data can accelerate innovation and disrupt traditional markets. Businesses are finding new ways to do business that serve their customers more effectively and responsively. Businesses can adapt or risk burying their heads in the sand. Should there be any doubt about the prevalence of data, consider just a few “data points”:
IDC estimates that by 2020, business transactions on the Internet will reach 450 billion per day
Walmart handles more than 1 million customer transactions per hour
More than 5 billion people are calling, texting, tweeting, and browsing on mobile phones worldwide
More than 30 billion pieces of content are shared on Facebook each month
It is estimated that the volume of business data worldwide doubles every 1.2 years, or sooner
Data production will be 44 times greater in 2020 that it was in 2009.
A Disruptive Force in the Digital Economy
Big data is transforming businesses across industry sectors — from industrial systems to financial services, from media to health care delivery, from drug discovery to government services, from national security to professional sports. The opportunity to deploy data and analytics has accelerated the speed at which companies can enter new markets, with new solutions, and quickly challenge or displace traditional competitors and market leaders. Consider some of the firms which are at the forefront of the Digital Economy – Amazon, Google, eBay, Facebook, Uber, and Airbnb. These are firms that are rooted in big data and analytics, and have leveraged new data-driven business models to disrupt and transform traditional industries such as retailing, media, and travel. For innovative firms such as these, big data brings speed, agility, experimentation, iteration, and the ability to fail fast, learn from experience, and execute smarter. To borrow from Samuel Beckett: “Ever tried? Ever failed? No matter. Try again. Fail again. Fail better.” This has become the mantra of the big data economy.
Transforming the Mainstream Economy
We are at the formative stages of the Big data transformation – Big Data 1.0. The next battleground in the progression of Big data will be transformation of the mainstream economy, comprised of large global firms that maintain massive amounts of data, and make massive investments in data assets. Most traditional businesses are hamstrung by legacy systems, decades old data warehouses, and embedded cultures and skills sets that predate the new big data approaches. These corporations represent the lion’s share of investment in data solutions and services. For most of these firms, big data remains waters that are largely unchartered, and an opportunity that has yet to be capitalized. While most mainstream firms have invested in big data initiatives, and have undertaken big data proof-of-concepts, these firms have lagged behind in their efforts to integrate big data-driven initiatives into their core processes and operations. Few have yet to demonstrate transformative or disruptive success with quantifiable results.
Leveraging Data as an Asset
Data is an enterprise asset, which cuts across products, services, and organizational units of a company. This makes data hard to manage and data initiatives difficult to organize. The big data mindset is driven by experimentation, discovery, agility, and a “data first” approach, characterized by analytical sandboxes, centers of excellence, and big data labs This mindset often runs counter to, or can complement, traditional hypothesis-driven approaches to data management. Business must now ask new sets of questions. How can we “monetize” new sources of data to new create new products and services? Can we leverage digital technologies – mobile, social media, machine learning, and the Internet of Things (IOT) — to better connect with our clients and constituents, and better and faster services to our customers? Can we use data to transform internal and external business processes? Can we find creative new uses for the data we have – new opportunities for insight, new markets or ways of delivering our services? Can we use the data that we have to be better members of our community, and leverage data for social responsibility? A dynamic and rapidly-changing business environment dictates flexibility and agility.
Forging a Big Data Strategy
Businesses need a clear data strategy if they hope to effectively leverage big data as a core business asset. An effective data strategy must be a highly dynamic roadmap for the future, and a work in progress to be adapted. It must recognize and account for the rapid evolution and proliferation of Big data and its potential business impact, as well as new business approaches which are altering traditional thinking about how data is managed and deployed. For the industry giants, transformation is commonly measured in years or the better part of a decade. The best strategies identify “quick wins” that can be used to demonstrate success, show results, build credibility, enlist support, counter skepticism, and create momentum. Many firms are implementing data governance programs and processes to address data issues that cut across organizational boundaries. Becoming a data-driven company is a continuous journey characterized by change and flux.
Making the Cultural Shift
Mainstream businesses face ongoing challenges in adopting big data and analytic practices. While some firms pay lip service to the notion of forging a data culture, fewer firms undertake the hard work, and demonstrate the commitment to creating a shared vision which becomes ingrained in the corporate culture and values. The benefits provided by new technology approaches and solutions, such as Hadoop, are having a transformational impact on the application of data and analytics. However, the greatest business challenge for most mainstream corporations is not about technology; it is the process of cultural change Cultural change represents a business problem, requiring a business solution and business approach. Business adoption of big data requires addressing issues of organizational alignment, change management, business process design, coordination, and communication. These are issues that involve people and communication and understanding. Businesses must start by identifying and asking the critical business questions that will drive business value, and identify and address the critical human and organizational issues that will ensure successful business adoption. Technology is a critical consideration that will follow.
Not all companies will emerge as winners in the digital future. Mainstream firms need to ask themselves how they can leverage big data and analytics to drive innovation, deliver new products and services, and compete successfully in an emerging digital world. Because large traditional companies control the bulk of spending on data programs and initiatives, the greatest impact of big data has yet to be fully realized. Will mainstream companies be able to culturally adapt and transform in response to the opportunity and challenges presented by big data, or will they risk falling behind and becoming relics of the past, in dying and displaced industries? This is where the biggest investments and bets on big data will be made, and the grounds on which these battles will be fought.
Originally published on Forbes.com