Companies are well underway populating data lakes, deploying analytics platforms and experimenting with AI and machine learning as part of a universal push towards data-driven business. Yet with a plethora of data and tools at their disposal, why are most enterprises still struggling to derive meaningful business insights at scale?
C-level executives and business experts are hungry to leverage data to create business advantage, whether that means capturing insights on customer preferences to enhance the buying experience and increase sales, or optimizing order fulfillment and sourcing logistics to reduce inventory on hand. While the potential for data as a strategic asset has top management transfixed, IT and business leaders are scrambling to make good on the promise. Gartner estimates that between 60% to 85% of big data projects will fail and contends that only 20% of analytics insights will deliver their intended business outcomes through 2022.
Much of the disconnect lies with the current state of enterprise data maturity. While the NewVantage Partners (NVP) 2021 Big Data and AI Executive survey found nearly all responding companies accelerating the pace of big data and AI investments (92%, which is a 40% spike from the prior year’s survey), less than a quarter (24%) of companies have successfully built a data-driven organization with slightly less than half (48.5%) able to drive innovation using data and 41.2% competing on analytics. Only 29.2% are experiencing transformational business outcomes thanks to effective use of data, NVP found.
The problem isn’t that organizations aren’t capturing enough data or even the right data to generate business insights. Rather, the issue is organizations don’t have the right contextual framework to draw predictive conclusions or analysis of the data without requiring significant manual effort and reliance on a select group of individuals to make the connections.
In an era where companies are having difficulty sourcing specialized data analytics and business talent, dependence on a core group of individuals to drive insights is a recipe for major bottlenecks. It also doesn’t help that data science experts are typically not immersed in day-to-day operations, which makes them much more removed from determining what constitutes a valuable insight compared to mainstream business users.
(metamorworks/Shutterstock)
Let’s take a company that wants to understand whether it’s able to meet certain customer requirements by shipping product on time. To draw the right conclusions, it might require customer data from a CRM as well as inventory and logistics data from different core business systems. The exercise requires expertise to extract data from the right sources in an attempt to answer the question, but even that doesn’t go far enough. What’s really required is knowledge of business operations translated in such a way that it connects the dots and draws conclusions across disparate data while doing so without the need for expert intervention.
Lost Without Translation
To avoid the risk of big data initiatives becoming a “dumping ground” for data without delivering any real business value, organizations need to zero in on two things:
An ability to translate elements across various data sources into a common language business users can understand;
And self-service capabilities, so those same in-the-trenches users can pursue insights on their own, without the help of data scientists.
Let’s start with the idea of a translation layer.
Data coming into a data lake is big and complex and varied, which means it needs to be harmonized across systems so that it is mapped consistently—that’s the same requirement of traditional data warehouses. Beyond integration and harmonization, however, there needs to be another layer to make the insights magic happen. What’s required is some form of a contextual data model that describes the data elements flowing into the data lake so it can be meaningful to a broader audience. Think about two foreign nations trying to negotiate an agreement in their own natural languages using translators. If the transcript of the meeting doesn’t include both the recording of the conversation and the translation record, the dialog is lost to anyone who doesn’t speak both languages.
Turning raw data into insight is not easy (Peshkova/Shutterstock)
Now consider the translation layer as it relates to a real-world metric like “on-time delivery.” While most companies track this closely, the metric could have different meaning for different business users, even within the same company. There is added complexity when you factor in the realities of the business. For example, it might be relatively straightforward to determine that an organization shipped out an order on time, but that metric could reveal itself very differently if a third-party logistics company engaged in the transaction. As a result, an organization would need context across multiple data sources at a customer, order and line-item level to determine the more layered insight depicting “on-time delivery.” The more efficient way to achieve that distinction is through a context-rich data model, which clearly defines data elements and provides meaning to the insight without a lot of data science and modeling heavy lifting.
Equally important is a self-service capability, which allows business users to frame the questions in everyday business language and search for information using familiar terms. Facilitating user engagement in this manner masks the complexity of cross-process analytics and surfaces insights by making connections across data stored in heterogeneous and siloed systems. It also empowers business users to continually ask questions of the data on their own as the need arises, uncovering more effective insights that in turn, facilitate better decision making and expedite decisive action.
Turning Insights Into Results
Companies able to pivot attention to the quality of insights, not just the quantity of data collected, are starting to reap the rewards of data-driven business. A prominent oil and gas company that spent more than five years trying to wrangle traditional analytics solutions to get insights on common metrics like on-time and full deliveries or days payable outstanding (DPO) was able to move beyond forensic insights to predictive analysis. Specifically, it was able to achieve a greater than 40% reduction in inventory on-hand carrying costs by linking inventory use data with actual planning parameters using the tools of a context-rich data model.
Similarly, a major manufacturer was able to improve its on-time delivery metrics from the low 80th percentile to the mid-90th percentile by connecting the dots between production capabilities and shipment results, and making the necessary adjustments based on the insights. In the retail space, companies could categorize the effective window for seasonal or perishable goods—each with limited shelf life—to dramatically reduce obsolete inventory.
These examples are just the tip of the iceberg of what’s possible when actual business users in sales, procurement or production departments have the tools to ask questions and explore big data within the context of their roles and using a language they understand. Those organizations that can get past the idea that more data is better and instead shift focus to adding meaning to data will be best positioned to drive insights that deliver demonstrable and measurable value. By embracing a new approach, organizations can gain competitive advantage and steer a course to true data-driven business.
About the author: Bas Kamphuis is chief growth officer at Magnitude Software. With a 20+-year background in corporate strategy and alliances across hardware, software, services and consulting organizations, Bas is focused on the development of multi-level partnerships based on core technology integration, full solution design and delivery. Previously, Bas served as General Manager, Strategic ISVs at Amazon Web Services, where he was responsible for the global commercial relations between AWS and its most strategic ISV partners, such as Microsoft and SAP, and their respective partner ecosystems. Bas is a native of the Netherlands and is currently based in Los Altos, CA.
Related Items:
In Search of the Modern Data Stack
Pandemic Driving Analytics Use Globally, Survey Saysd
One Deceptively Simple Secret for Data Lake Success
- Home page
- Content Marketing
- Digital Marketing Strategy
- Digital Marketing Strategy
- Digital Marketing Strategy
- PPC
- SEO
- Social maketing
- WordPress web development
- Data science
