Managed Self-service BI

… and the journey ahead.

It’s interesting to watch technology vendors and consulting companies conjure new buzz words to sell products and services. Sometimes they just try too hard. They should try to focus on practical realities we can relate to. Self-service BI is a classic example of a term that’s used as if self-service BI is something completely new, to the extent that it’s often referred to as the second wave of BI.

Really? Is it just not a fresh look at self-service BI and an attempt to improve the approach through new technology and better processes? In other words, is it not just an attempt to shift from an unmanaged self-service mode to a managed self-service mode? In my opinion, it certainly is, but it is also much more than that.

Shadow BI

Self-service BI has many guises. I am not sure about the rest of the world, but in South Africa the most popular incarnation is an MIS (Management Information Systems) capability. Self-service BI or Shadow BI exists as a response to challenges that IT business intelligence teams have failed to solve. This generally boils down to taking too long to deliver answers and not always getting the answers right. The single most important point to acknowledge is that Shadow BI teams are answering critical business questions, and hence immensely valuable to the business.

But Shadow BI comes with its own challenges. This includes the infamous “spread-mart” hell which breaks every rule in the book around data management and governance. It is also labor intense (repeated effort and high maintenance) and prone to human error (business rules live in the minds of individuals). So the business ends up with serious risk exposure around business continuity and accuracy of process measurement, and hence decision quality could be compromised.

Technology Evolution

Meanwhile in the world of technology things are moving forward exponentially. Scale of computing, spectrum of functional possibilities and ease of use drive massive potential change. Just like we’ve packed super computers into smart phones, business intelligence desktop tools start to deliver capabilities at scale without the complexities of the business intelligence developers’ traditional toolkit. All of a sudden technology vendors have made available DIY versions of ETL tools, analytical data modelling tools (“cubes”) and data visualisation tools. Now the world is your oyster … or not.

Process Evolution  

A new process is required that must resolve two key challenges. We have already pointed out that traditional IT BI methods have failed to deliver accurate and timeous results. Here we need to use the new technology to build prototypes before we embark on lengthy IT BI projects. Prototyping quickly establishes the viability and correctness of a solution. Compare this to creating requirements documents, developing a solution and finding gaps and problems during user testing. Scale of DIY BI tools also provide a means to leverage the prototype as an interim solution while the more robust solution is being developed.

The second challenge we need to resolve, is to find a permanent home for self-service solutions. If we simply adopted the new technology without rethinking the self-service process, then we will be responsible for a new breed of super spread-marts and a more chaotic world of Shadow BI. To address this, we need to establish a self-service solution industrialization process. This effectively means promoting self-service solutions to enterprise class platforms.

Continuous Collaboration

But it’s not just collaboration within teams that’s required. Our new process will also require improved collaboration between IT and business. More than ever do we need to combine business process (domain) knowledge with IT design and technical skills. Even with the most advanced technology in the world, success is not a given unless there is serious Business-IT collaboration. Collaboration is assumed to be ongoing and encouraged through iterative and agile delivery techniques.

Driving Change

Technology and process adoption is not a switch you flick. The historical reasons for the existence of Shadow BI makes it all the more difficult to drive change. At least it is a known and acknowledged change that is required. It gets more complicated though because the world has changed. Consumer culture has changed. Businesses are changing and the elevated importance of the data driven organisation adds pressure to the equation.

Bigger questions emerge. We have to go beyond our need for a managed self-service BI model. Our organisation needs a new data and analytics strategy. So we go to the drawing board and ask:

• Have we appropriately prioritized and leveraged data/analytics in our organisation? Is it a C-suite priority? (we can’t drive change without the right levels of support).
• How do we drive a shift in analytical focus? (looking back must only be for the sake of looking forward).
• Have we structured our people/teams accordingly? Are we trying to solve new problems based on old team structures and dated skill sets?
• Have we got the right systems and building blocks in place? Is our analytics platform modern enough? How do we retro-fit master data management into our existing systems?

Big questions indeed. And there are many more we could ask. So while we focus on moving from Shadow BI to managed self-service BI, we have to keep in mind that it requires a broader strategic outlook and road-map.

This post was motivated by the challenges I see BI teams facing and their attempt to solve these challenges with a bottom-up approach. Too few of these teams are succeeding and it is not a lack of skill or motivation, but a failure at the very top of these organisations to respond (or acknowledge) the importance of data and analytics in the face of an economic paradigm shift.