The BI & Big Data Analysis Conundrum

Why do so many BI & big data analytics solutions fail to deliver business value?

It’s common knowledge that in today’s digital world, every move leaves a digital trace that can be turned into actionable insights and actions. That’s why so many organizations are investing in BI tools to support decision-making, and in big data analytics solutions to maximize customer experience and optimize business results.

However, the reality is different. Many organizations who have purchased such tools/solutions have turned to me after failing to get sufficient business value from them. In my opinion, and based on my experience from meeting with them, there are a number of reasons to explain this conundrum. Here are my tips on how to improve:

"Jack of all trades, master of none”: To save on resources, many organizations combine the business and implementation aspects of BI and big data analytics solutions in one person. However, these two roles require very different skill sets, and their combination may result in missed business value in BI implementation and big data analysis. Just as data analyst skills are different to data scientist skills, the same goes for business knowledge vs. analytic capabilities. The result may be the difference between turning data into valuable business insights and simply playing with the data in a big data analytics implementation, and between achieving valuable decision-making solutions and a negligible solution in BI implementation. So, irrespective of whether organizations hire a service provider or choose to implement in-house, two resources are essential for success - one with the business understanding to ask the right questions and the knowledge to turn the data into business insights and actions, and the other for the technical/analytics execution.

"If you think it's expensive to hire a professional, wait until you hire an amateur": Cutting costs through the use of cheaper, and often less-professional suppliers, inevitably results in costing more in the long run. In particular, implementation tends to take longer and the business value is negligible. Furthermore, organizations must ensure that their service suppliers have an appropriate resource who understands the business, as well as another resource with the necessary technical capabilities to ensure successful and valuable implementation and analysis.

“Think before you act”: Many organizations rush to purchase and quickly implement a BI or big data analysis tool without initially defining their actual business requirements and the results they want to achieve. There is a huge difference between the two, and it's very easy to fail through incorrect specification of requirements or through focusing on incorrect, non-valuable business challenges in the sea of data available. Avoid being over-agile and plan well.

“Differentiate between business and IT ownership”: Because of unclear ownerships, some organizations fail to create a unified language for business value. Business specification and data source mapping are two critical steps in BI and the data analysis process. Each KPI requires a defined business owner who understands what is required and knows how to accurately define it, as well as an IT owner who understands the business KPI and knows how to map it correctly to the data source.

“Be relevant to the organization strategy”: Every organization faces the same challenge – no measurement means no management, while too many measurements can backfire and create chaos. Every BI and big data analysis solution must be correlated with, and relevant to, the organization strategy, as well as support the creation of an organization business language; otherwise, they will lack relevance.

“What you see is what you get?”: Optimal data visualization is critical to enable organizations to understand the data and make decisions. However, while today’s market offers a plethora of tools with extensive flexibility in the ranges of charts and views that can be generated, many require an understanding of graphics and data visualizations and the abilities of a UI/UX designer in order to correctly perceive the information that will drive to actions. Furthermore, many tools lack the provision of best practices, benchmarks and directions on how to plan and define the visual delivery of actionable insights in an intuitive way. So, while organizations may be enticed by attractive, impressive and flexible interfaces, they still need to ensure that their users have a simple, intuitive and easy understanding of what they see.

“Reliability of data provided by business owners": Many organizations are managed through personal Excel reports run by different business owners. In many cases, these reports are not presented in a unified organizational language and with cross-organization processing of numbers. Since the success of any BI and big data analysis solution is based on the integrity and credibility of the data, organizations need to establish a process to ensure unified collation of all data in a single organizational language.

“Garbage in, garbage out”: Any data analysis system can only be as good and correct as the data it receives. BI systems help to raise problems in the actual data and in the processes performed in the operational systems. Typical examples include typing errors, logical problems in procedures, and incompatibility between processes in financial departments, customer service departments or between systems in various departments. Consequently, data optimization - cleansing of data - is crucial to ensure relevant, quality data and achieve business value.