Enterprises are drowning in data. The sheer volume of data about customers, suppliers, products, and business partners has never been so extensive and yet so crucially important. BI tools matter precisely because they promise to make sense of the tsunami of data and uncover the insights within that are vital to winning in the marketplace.
The development of BI technology has largely been focused on fulfilling that elusive goal. This article explains the major shifts in approaches to business intelligence and outlines the advantages of the most recent evolution of this technology based on automation through machine learning and AI. As BI vendors move towards this new approach, buyers need to understand this shift as they evaluate products.
The First Generation of BI: IT as Central Guardian
The first generation of BI technology was largely managed by the IT department as the central guardian of all enterprise data. The Extract, Transform, Load (ETL) paradigm combined data from multiple systems to a single database, data store, or warehouse for legacy storage or analytics. Once stored, data was normalized—removing redundancy and duplication—to make it easier to run queries and retrieve data for reporting. IT staff would run queries on behalf of business users who did not necessarily have expertise in query languages.
Ultimately, the IT organization would deliver a static report to the business owner. The entire process could take days or even weeks due to dependence on skilled IT staff. Also, answers provided often provoked additional questions that had to go through the same inefficient process.
Emergence of Data Discovery and Visualization
Given these drawbacks, this approach was eclipsed by a more agile approach that favored self-service capabilities. A new set of BI products emerged that eliminated the technical stack designed for IT users and focused on providing data discovery and visualization tools to business users.
Business users were empowered to use BI tools to access their data, even without advanced technical skills. The focus of these tools is to provide business analysts the ability to conduct an ad-hoc analysis of multiple data sources. These tools provide data analysts with an intuitive way to sift through large volumes of disparate data to expose hidden patterns and outliers. They replace the rows and columns of traditional data presentations with graphical pictures and charts.
The emergence of data discovery and visualization tools, and the emancipation of business analysts from old-school Extract, Transform, Load (ETL) processes and data modeling, was highly successful. This revolution democratized data and greatly accelerated the speed of data analysis to help companies make data-driven decisions in competitive environments.
As a result, IT-centric BI vendors quickly started to build these data discovery and visualization capabilities into their product portfolios. Conversely, data discovery and visualization vendors were quickly pushed by customers to provide enterprise licensing and features, like data governance, security, data preparation and report generation.
The Next Evolution: Augmented Analytics
As data sets become increasingly large, capabilities like data preparation require users with some technical skills to make sense of potentially huge datasets generated by, for example, IoT devices. After all, data preparation and integration were typically handled by IT specialists as part of the ETL process. In today’s organizations, data scientists have the primary responsibility of focusing their advanced skills to turn massive quantities of raw data into insight that can have a direct impact on the business.
The problem is that there is a chronic shortage of people with the requisite analytics skills to do this. Data scientists are relatively thin on the ground. The result is the emergence of a third approach to business analytics. Essentially, it’s an evolution of the self-service approach towards automation usually referred to as Augmented Analytics.
Insight generation is the primary reason that organizations invest in BI. But if the people with the right skills to use these tools to get from raw data to actionable insight are scarce, a different approach is needed. Augmented analytics promises to reduce dependence on highly skilled data scientists by automating insight generation through the use of machine learning and artificial intelligence algorithms. Today, many well-known BI platforms have built-in capabilities to automate data preparation and detect some correlations and anomalies in large data sets. Some platforms are capable of analysis and even insight generation and action plan generation, but there is still a considerable amount of hype to muddy the waters.
The Three Ages of BI
BI tools have evolved from pixel-perfect reporting engines to analytics platforms powered by powerful artificial intelligence technology.
- Owned and managed by IT
- No business user autonomy over-performing data analysis > Limited self-service capabilities
- Focus on reporting and KPIs
- Business user-driven
- Some data analysis autonomy for business users
- Focus on ease-of-use for benefit of broad non-technical user base > User-driven insights to aid business decision-making
- Business users don’t require IT assistance to define data models
- Data preparation and analysis aided by artificial intelligence
- Ease-of-use greatly increased by AI automation
- Data correlation and insights automatically generated to drive decision-making
Buyer Next Steps
While the traditional BI focus on reports and dashboards is not going away any time soon, companies should start looking at vendor capabilities in the area of augmented analytics. Most BI vendors are building augmented analytics into their platforms, but it is easy to get swept up in the hype. The TrustRadius Buyer’s Guide to Business Intelligence is a good place to gain an understanding of the concepts that are driving this new approach, along with other emerging trends in the domain.