Python & Power BI: Load from SQL Server

This is part 4 of the Python integration in Power BI series. We are going to explore sourcing data from SQL Server (including Azure SQL DB). If you have been following the series, you would be familiar with the the basics of Pandas dataframes. You saw how easy it is to load data from a CSV file into a dataframe. Loading data from SQL Server is just as easy … but the hard work is setting up database connection strings.

This post will cover how to connect to SQL Server with a library called SQLAlchemy, and how to load data from SQL Server into a Pandas dataframe. We cover a few other things along the way i.e. using Windows environment variables, multi-line strings and working with string parameters.

Protecting sensitive information

In the previous post we worked with the os (operating system library) to access files. We will use the os library again, but this time to protect sensitive information. During the development process I like to keep keys, passwords, server names, etc outside of the script in a place that is not too easy to access. I store this info in User Environment Variables so that are only accessible to me. (Any other ideas to protect sensitive info welcome).

The os library has a getenv (get environment variable) method. We retrieve the value from the environment variable and assign it to a Python variable as follows:

401.png

Here you can see that two values are read from the environment variables and we see that sqldb has a value ‘Sandbox’. In this way we ensure that our Python scripts don’t store sensitive information, but will still run successfully in our development environment. If you have problems with the environment variables, just use plain text for now.

SQLAlchemy connection strings

SQLAlchemy is quite sophisticated, but we will using it in a fairly limited way. The “engine” or connection is covered in the documentation, but rarely will we have a standard or basic connection. Most of the time we will have a non-standard connection, in which case we have to pass the connection parameters in a connection string. SQLAlchemy expects this string in URL format.

We will start by connecting to a SQL instance on a local machine. First we add the sqlalchemy library to our script. Then we store the connection string in a variable. 402.png

To convert the connection string to URL format, we make use of the urllib library. Here we use the quote_plus method to convert spaces and special characters to URL format.

403

Maintaining connection strings in the above format is error prone and cumbersome. Python allows multi-line strings through the use of triple quotes. This can be either single quotes or double quotes.

404

Now we want to include the variables we created earlier. We are not going to construct the string from parts using concatenation. We are rather going to use the % placeholder for parameters in the string. We are only working with strings, so we will use %s as the placeholder. Note that they are interpreted in sequence.

405

 

Establishing a connection to SQL Server

We use the SQLAlchemy create_engine method to create an engine object that allows us to interact with the database e.g. the interaction could be as basic as establishing a query connection.

406

By adding an echo parameter to the create_engine method, we get more connection info.

407

Only turn echo on if you are troubleshooting, otherwise its just a lot of noise.

Loading SQL results to a dataframe

This is the easy part. We have a choice between read_sql, read_sql_query and read_sql_table. Lets start with the table.

408

The schema (defaults to dbo) and columns (defaults to all columns) parameters are optional. There are quite a few other parameters. Have a looking at the documentation.

To load results from a SQL query is only slightly more complex. We will make use of the multi-line string again for readability and maintenance.

409

Connecting to Azure SQL DB

Hopefully by now you realize the benefits of using Azure SQL DB. Our company uses Azure SQL DB as a data management solution for Power BI. Yes, there is huge benefit in persisting and managing your data outside of Power BI! Perhaps I’ll cover this Power BI solution architecture in a future post. (hint … have a look at dataframe.to_sql)

Azure SQL DB makes use of the latest SQL Drivers. SQL Alchemy will moan if you try to adapt the connection string we have above which uses the standard SQL driver. Also note the additional settings such as encryption and trust certificates.

410

The rest works the same as the local SQL instance once you have created the engine.

Conclusion

By now you know that once your data is in a dataframe, The Python script in the Power Query editor will recognize your data.

Of course it is much easier to do this in Power Query without a line of code. But you may have scenarios where you perform data cleansing, transformations, combining, transposing, etc on the dataframe before loading your data into Power BI.

The key idea that drew my interest in the first place was the ability to re-use code across platforms and applications.

SQLSaturday #793 Cape Town

I’ll be speaking about Python integration in Power BI at SQLSaturday #793 Cape Town this coming Saturday, 8 September. If you are in Cape Town, come and join us. There are many great sessions to pick from.

 

Posted in Power BI, Python

Load files in a folder using Python

This is part 3 of the Basics of Python in Power BI series and builds on from the previous post about Python dataframes. In this post we show you how to import files from a folder using Python. Combining files from a folder is currently an easy process in Power Query, but I mentioned in my first post that I am interested in the re-use of existing Python scripts.

For readability, I am switching between Spyder and JupyterLab, depending on what I need to show. I’ll also cover some foundation concepts so that an understanding is gained, rather than just an answer. Three concepts are covered in this post: working with files using the operating system (os) package, working with lists and a bit more dataframe functionality.

Working with files in Python

Python has a built-in os (operating system) library. This gives us access to methods such as getcwd (get current working directory), chdir (change directory) and listdir (list directory).

b2py1

While getcwd returns the current working directory (or current path), chdir changes the path to the newly specified one and this becomes the current working directory. When we use listdir with a path, it only returns the list of files in that path, but does not change the current working directory.

Lists

The listdir method returns the result as a Python list data structure. The outputs 4 and 5 above can be identified as lists because they are enclosed in square brackets and items are comma separated. The items in the lists in these results are all of data type string. When lists have the same data types the items are referred to as homogeneous. A list can even hold items of a mix of data types (heterogeneous).

The items in a list are indexed starting at 0. We can either find the content by providing an index, or we can find the index of the first occurrence of  an item.

b210

We are even able to change the content (mutable) of the list item by specifying the index, as seen above (that index position must already exist).

We can iterate over the items in a list using a for loop construct.

b212

The syntax is equivalent to saying “for i = 0 to 3 multiply each item by 2”.

Since listdir returns a list,  we may loop through the list to display each file name.

 

b213

More dataframes

We are going to loop through the files as above and add each file to a dataframe. We use the dataframe head method display the first n rows.

 

b214

Our first attempt unfortunately keeps overwriting the previous loop iteration so we end up with the last file in the list only. To solve the problem, we are going to create an empty list, then append each dataframe as a list item. Note that the items are of type dataframe.

 

b215

Our final script above has one critical step and a few desired additions. In the second last line we concatenate all the dataframe items in the list into a single dataframe. Pandas ensures that headers are not duplicated. In addition, we parsed dates & data types and added a source file column.

 

Power BI Query Editor

For the curious folk, lets take a look at the M code. I stripped comments from the final script to eliminate distractions.

b216

 

 

The first thing you will notice as that df_one and df_all are available for selection.

b217

Instead of loading the query, select Edit to open the Power Query Editor window.

 

b218

Now open the Advanced Editor to have a look at the M script. In the Source step we can see the Python.Execute command which allows us to run Python script.

 

b219

I am still deciding if I am happy or disappointed to discover that Python.Execute accepts a single string parameter which contains the entire Python script. At least I was happy that it was easy to edit. I made a dummy (but valid) change to see what happens.

 

At this point I was prompted about execute permissions which I suppose makes sense as there is really no control over the content of the Python script.

b220

b221

 

When you click on Run the Python script is executed again. Click on Close & Apply will take you back to Power BI as usual.

That is it for this post. I’ll continue with Python script for the Query Editor as I see that there is more focus on the Py visuals currently.

 

 

 

Posted in Power BI, Python

Basics of Python in Power BI: Part 2

The is a follow-on post to the Basics of Python in Power BI post. The idea behind the first post was to illustrate the most basic of Python integration in Power BI by keeping the Python code to a bare minimum. In this post I want to try and cover the basics of Pandas dataframes as this appears to be the primary way for Power BI to recognize data delivered via a Python script. Keep in mind that when I say “basics”, I am referring to dataframes in relation to Power BI. Also the results are from a discovery process with the preview, not from any Power BI reference material.

As a data science tool, the Python fundamentals to really master are data structures. DataCamp has a great post on the basics of data structures. In my brief journey with Python thus far, the built-in data structures I have needed to get to grips with are lists and dictionaries. I’ll cover the basics of lists and dictionaries in future posts. For now we will focus on Pandas dataframes, not just because it is supported in Power BI, but because the data science universe says so!

Dataframes

Pandas is a very popular Python package for data science because of its ability to manipulate and analyze data. This is possible because of data structures such as dataframes. The good news for Microsoft data platform folk is that Pandas shares many similarities with SQL, so much so that the Pandas documentation includes a comparison with SQL.

DataCamp defines a dataframe as “two-dimensional labeled data structures with columns of potentially different types” and “you could say that the Pandas DataFrame consists of three main components: the data, the index, and the columns”. For now, in order to get to grips with dataframes, lets think of it as a table with an index.

Recall how we read the CSV file into a Pandas dataframe in the previous post:

b302

The original file had 2 columns. When we used the read_csv method to add the file content to the dataframe, the index was automatically added.

Note: throughout the post I am using the head(5) method to display the top 5 records. You don’t need this in Power BI. In fact Power BI ignores this as it does not modify the dataframe.

Mind the index

When we use the Python script in Power BI, it does not display the index.

b303

If we have our own index column in the data e.g. ValueDate, we could specify the index column as a parameter in the read_csv method as follows:

b304

As we have noted, Power BI does not display the index … even if it is one of the data columns!

b305

It may be of interest that the to_sql method includes the index column when loading to SQL. I’ll blog about this in a future post as a reference for those who want to persist their data to SQL.

Watch the data types

While I am unable to recreate the scenario, I had a case where Load vs Edit displayed different behavior in the Query Editor. In this scenario, Edit did not show the Change Type step. In this case I manually changed the data types to date and decimal.

I tried to overcome this by forcing the data types to be set during the read_csv method by parsing the date and setting the data type for TheValue as float. Data types for read_csv can only be set to string, float or int (integer). If we compare the raw date vs parsed date, we will see that the raw date is treated as a string.

b310

Although TheValue is correctly detected as float, we can ensure that the column is cast as float by adding the dtype parameter to the read_csv method.

b311

It appears that Power BI still applies a Change Type step regardless of this.

b307

This is not really an issue, but it is useful to know that the read_csv method can handle data type changes including decimals with a comma or dates in other formats. Perhaps this will change in Power BI in the future.

Sort for compression

I am not the expert about the technicalities of compression, but I read a blog post by Matt Allington discussing the impact of unsorted data on large data models. Since its easy to sort data with a dataframe, we may as well apply this to our code to avoid potential pitfalls.

b308

Or we could write it like this:

b309

Summary

Dataframes are powerful data structures. We have scratched the surface by touching on specifics that may affect Power BI. This includes dataframe indexes, dealing with data types and sorting data.

For more info on handling CSV files (headers, skipping rows, missing values, etc) I’ll have to send you to DataCamp.

Disclaimer: I am in no way affiliated to DataCamp, but find the content incredibly useful for getting started with Python.

 

 

 

 

 

Posted in Power BI, Python

Basics of Python in Power BI

Disclaimer: My main area of expertise is SQL Server Business Intelligence and Power BI. I am a novice at Python. There are probably a million better ways to write the Python code, but I am impressed by the language and what I have been able to achieve with it.

I started using Python a bit more than 6 months ago because of the projects I am working on where data services are predominantly made available through rest APIs. The libraries available in Python made life easy and I knew it was inevitable for me to go down this road when Machine Learning Services (MLS) was released in SQL Server 2017.

To shorten a long story, I started using Python as a data staging solution using the Anaconda distribution (for machine learning capability and alignment to MLS). My focus was retrieving data from APIs into Pandas dataframes and then writing the results to SQL Server tables. Python proved to be surprising easy to learn (although I had some .NET background), although I knew my focus was incredibly narrow.

When Microsoft announced a preview of Python integration (for both Power Query and data visualization) I was so excited that I got up early on a bank holiday to try it out for myself. There was literally no examples to follow other than the data visualization bits from the blog post announcing the August release of Power BI Desktop.

I made some assumptions that the Pandas dataframes would be recognized by Power BI and got the Power Query bits working quickly. I struggled a little bit with a basic Python data visualization because it was the first time I ever tried to use Python for visualization. The end result was 3 simple steps to enable Python in Power BI and 3 more steps to get started with Python in Power BI.

Step A: Install the August 2018 release of Power BI Desktop

Step B: Enable Python support from the Preview featurespy1

 

Step C: Restart Power BI Desktop and set your Python home directory
py2

You may leave the default directory if you don’t have Python installed. Otherwise choose the path of the Python distribution you want Power BI to use when running your report.

Now 3 steps to get started with Python in Power BI

Step 1: Test your Python script in a Python editor

My sample data is a daily historical exchange rate for South African Rand (ZAR) against the US Dollar (USD). Its a simple CSV file containing the date and the ZAR amount for $1.

With the Anaconda distribution of Python, the Pandas data manipulation and analytics library is already installed. If you are using a different Python distribution, you may need to install Pandas.

The code has been kept fairly simple to get the basics right:

import pandas
df_USD = pandas.read_csv('D:\\data\\csv\\ZARUSD201x.csv')
df_USD.plot(kind='line', x=['ValueDate'], y=['TheValue'])

‘The first line of code adds a reference to the pandas library. The second line of code uses Pandas to open a CSV file and load it into a data structure called a dataframe. The third line of code plots a line chart using the date as the x-axis and the value as the y-axis.

pp1

The result may look good in the image above, but it appears much smaller in the editor. The CSV file also adds one small issue viz. the data is sorted descending. We need two small changes to the code to improve the result.

import pandas
df_USD = pandas.read_csv('D:\\data\\csv\\ZARUSD201x.csv').sort_values('ValueDate')
df_USD.plot(kind='line', x=['ValueDate'], y=['TheValue'], figsize=(25,10))

Line 2 makes use of the Pandas dataframe sort_values function to re-order the data. Line 3 adds the figsize parameter to control the display size of the chart. The result is much better.

pp2

Step 2: Get data into the Power BI data model using Python script

Now that we have a working Python script we can add it to Power BI. We do this in two parts. In this step we add the data to the data model via Power Query and in step 3 we will add the code for the visualization.

py4

Copy the first two lines of code into the Python script dialogue.

py5

The Navigator will show all available dataframes. In this example it will only be df_USD.

py6

A preview of the data will be displayed when selecting the dataframe. The data types were correctly detected in this example so there is no need to edit in Power Query. Select Load to add the data to the data model.

py7

Step 3: Plot the data in Power BI using the “Py” visual

Add a “Py” visual to the canvas. You will need to enable script visuals. Add the value and date fields to the visual. Change the date field to show the actual ValueDate (not the hierarchy). To make the Py visual respond to slicers, add year and month slicers from the ValueDate hierarchy.

py10

The “Py” visual will be blank initially and a Python script window appears below that. For the Python visual the data is required as a Pandas dataframe. The Power BI data model fields that are selected are converted to a dataframe (dataset) and the dataset is de-duplicated. This part of the script is generated by Power BI and appears in grey. The code to plot the chart goes below this.

 

py8

Now copy the last line of code from the Python editor and modify it to use the Power BI dataframe.

dataset.plot(kind='line', x=['ValueDate'], y=['TheValue'], figsize=(25,10))

Unfortunately this does not render anything. If we look at the details of the error message we see that nothing is rendered (even though the Spyder editor was able to render the plot)

py9

To fix this we have to use the matplotlib Python library to render the plot.

from matplotlib import pyplot
dataset.plot(kind='line', x=['ValueDate'], y=['TheValue'], figsize=(25,10))
pyplot.show()

Now the visual appears, but there is still one problem.

py11

Its not impossible, but highly unlikely that the exchange rate is just climbing on a daily basis for July 2018. The problem here is that a Pandas dataframe has an index and automatically sorts the data on this index. To change the sort order we change the sorting of the dataframe in a similar way to the way we sorted the dataframe in step 1.

dataset.sort_values('ValueDate').plot(kind='line', x=['ValueDate'], y=['TheValue'], figsize=(25,10))

Now we get the correct result.

py12

I hope this post helps you get started with Python in Power BI. While I’m a novice at Python, I thought it would be worth sharing with others who are also starting out.

Posted in Uncategorized

Managed Self-service BI

… and the journey ahead.

It’s interesting to watch technology vendors and consulting companies conjure new buzz words to sell products and services. Sometimes they just try too hard. They should try to focus on practical realities we can relate to. Self-service BI is a classic example of a term that’s used as if self-service BI is something completely new, to the extent that it’s often referred to as the second wave of BI.

Really? Is it just not a fresh look at self-service BI and an attempt to improve the approach through new technology and better processes? In other words, is it not just an attempt to shift from an unmanaged self-service mode to a managed self-service mode? In my opinion, it certainly is, but it is also much more than that.

Shadow BI

Self-service BI has many guises. I am not sure about the rest of the world, but in South Africa the most popular incarnation is an MIS (Management Information Systems) capability. Self-service BI or Shadow BI exists as a response to challenges that IT business intelligence teams have failed to solve. This generally boils down to taking too long to deliver answers and not always getting the answers right. The single most important point to acknowledge is that Shadow BI teams are answering critical business questions, and hence immensely valuable to the business.

ShadowBI

But Shadow BI comes with its own challenges. This includes the infamous “spread-mart” hell which breaks every rule in the book around data management and governance. It is also labor intense (repeated effort and high maintenance) and prone to human error (business rules live in the minds of individuals). So the business ends up with serious risk exposure around business continuity and accuracy of process measurement, and hence decision quality could be compromised.

Technology Evolution

Meanwhile in the world of technology things are moving forward exponentially. Scale of computing, spectrum of functional possibilities and ease of use drive massive potential change. Just like we’ve packed super computers into smart phones, business intelligence desktop tools start to deliver capabilities at scale without the complexities of the business intelligence developers’ traditional toolkit. All of a sudden technology vendors have made available DIY versions of ETL tools, analytical data modelling tools (“cubes”) and data visualisation tools. Now the world is your oyster … or not.

Process Evolution  

A new process is required that must resolve two key challenges. We have already pointed out that traditional IT BI methods have failed to deliver accurate and timeous results. Here we need to use the new technology to build prototypes before we embark on lengthy IT BI projects. Prototyping quickly establishes the viability and correctness of a solution. Compare this to creating requirements documents, developing a solution and finding gaps and problems during user testing. Scale of DIY BI tools also provide a means to leverage the prototype as an interim solution while the more robust solution is being developed.

The second challenge we need to resolve, is to find a permanent home for self-service solutions. If we simply adopted the new technology without rethinking the self-service process, then we will be responsible for a new breed of super spread-marts and a more chaotic world of Shadow BI. To address this, we need to establish a self-service solution industrialization process. This effectively means promoting self-service solutions to enterprise class platforms.

SelfServiceProcess

Continuous Collaboration

But it’s not just collaboration within teams that’s required. Our new process will also require improved collaboration between IT and business. More than ever do we need to combine business process (domain) knowledge with IT design and technical skills. Even with the most advanced technology in the world, success is not a given unless there is serious Business-IT collaboration. Collaboration is assumed to be ongoing and encouraged through iterative and agile delivery techniques.

Driving Change

Technology and process adoption is not a switch you flick. The historical reasons for the existence of Shadow BI makes it all the more difficult to drive change. At least it is a known and acknowledged change that is required. It gets more complicated though because the world has changed. Consumer culture has changed. Businesses are changing and the elevated importance of the data driven organisation adds pressure to the equation.

Bigger questions emerge. We have to go beyond our need for a managed self-service BI model. Our organisation needs a new data and analytics strategy. So we go to the drawing board and ask:

  • Have we appropriately prioritized and leveraged data/analytics in our organisation? Is it a C-suite priority? (we can’t drive change without the right levels of support).
  • How do we drive a shift in analytical focus? (looking back must only be for the sake of looking forward).
  • Have we structured our people/teams accordingly? Are we trying to solve new problems based on old team structures and dated skill sets?
  • Have we got the right systems and building blocks in place? Is our analytics platform modern enough? How do we retro-fit master data management into our existing systems?

Big questions indeed. And there are many more we could ask. So while we focus on moving from Shadow BI to managed self-service BI, we have to keep in mind that it requires a broader strategic outlook and road-map.

This post was motivated by the challenges I see BI teams facing and their attempt to solve these challenges with a bottom-up approach. Too few of these teams are succeeding and it is not a lack of skill or motivation, but a failure at the very top of these organisations to respond (or acknowledge) the importance of data and analytics in the face of an economic paradigm shift.

Posted in Self-Service BI

What is Power BI?

… and why you should be interested.

Technology is a business enabler. Our mind-set must be one of solving business problems and then determine how technology will help solve the problems. The first pass for evaluating technology requires an initial understanding of the key concepts rather than platforms and architecture. In this post I’ll try to demystify what Power BI 2.0 is. It has its origins in Power BI version 1 which was preceded by Power Pivot, and this may have left a trail of confusion. As a rapidly evolving platform the information in this post may be out of date, or simply incorrect because I have misunderstood some concepts or features.

Power BI is labelled as a Business Analytics suite. There is much confusion and variation around the term Business Analytics, so I am surprised that Microsoft used this label. While I am not going to attempt to define business analytics, it is sufficient for the purpose of this post to say that it encompasses self-service BI and also refers to a continuous and iterative analytics process that matches the business need for speed, agility and a forward/future focused analytics approach.

Power BI Solar System

I like to describe Power BI as a solar system. The Power BI Service is the sun in this solar system. This solar system belongs to the Cortana Analytics galaxy in the Microsoft universe.

Rafal Lukawiecki provided a very simple but elegant explanation of Cortana Analytics during a keynote at SQLSaturday in Johannesburg (slides available here). Also have a look at the example of Cortana Analytics in healthcare. But back to Power BI …

The Power BI Service is the glue between the user and the data. This analytics function is supported by content builders who create reports, dashboards and content packs. The content is further enhanced through features such as natural language query, data driven alerts, annotations and sharing.

To gain a better understanding of this over-simplified view, we have to navigate the map of the Power BI solar system. The original idea of this layout came from a Jen Underwood slide, but I have adapted this with in an attempt to pack more detail and context into a single image.

Power BI Map

Navigating the map is reasonably simple. There are 4 areas to review:

  • Source Data (entire bottom half of the image)
  • Development Tools (top left hand corner)
  • Power BI Service (top center)
  • Data Presentation and Exploration (top right hand corner)

Source data is the data you want to surface insights from. This data may reside in files, databases or accessible via services. The data may be on-premises or in the cloud. The data may be public or private. The data may be raw (transactional system data), curated (data warehouse data) or packaged (content packs) for end user consumption. A personal gateway lets you connect to data on your laptop/device. While a connector is available for on-premises SQL Server Analysis Services, it is the much anticipated enterprise gateway that we patiently are waiting for. But even more exciting are the possibilities with the Power BI Rest APIs that allow organisations to create custom applications that can push data directly to the Power BI service.

PowerBIDesktop

To author content for Power BI we need a combination of the Power BI Desktop tool (or Excel with the Power add-ins) and the Power BI Service. Power BI Desktop and Excel are self-service BI tools that are more than capable of delivering “power user” features for data integration (ETL), data modelling and report building. Officially Excel is not part of the Power BI suite, but Amir Netz has promised much love for the Excel crowd and Office 2016 is proof of this (currently in preview and rolling out from 22 September). Excel 2016 preview sports the same version of the tabular model engine that is currently available in Power BI Desktop. Transition between the tools is not what users are hoping for yet, but this is outside of the scope of this post.

PowerBIDashboard

Once a Power BI Desktop (or Excel) solution has been created, the content can be published to the Power BI service. The service becomes the host for the queries, data (model) and reports. The only parts of the solution that can only be created in the service are the dashboards and content packs (prepacked solutions including dashboards). These content packs could be commercially packaged solutions or custom content packs created within your organisation. The service includes a natural language query feature at dashboard level that lets you type in questions that are answered through data or data visualizations. The service also manages data refresh, alerts and annotations.

Finally there is the end user experience. Content can be accessed via browsers (yes, Chrome and Safari are supported), via the Power BI application (in the Windows Store) on Windows 8 & 10, or via mobiles devices (phones and tablets … it appears that Apple devices are a priority). The tablet experience is richer than the phone in that it can drill down to underlying reports which the phone app can’t do.

Overall the Power BI offering from Microsoft ranks high in completeness of vision both as a standalone business analytics suite and as a solar system within the Microsoft universe that offers companion platforms such as SQL Server, HDInsight, Machine Learning, etc. As a new product, ability to execute is reasonable with a demonstrated commitment to incremental delivery. The service is currently updated weekly and Power BI Desktop monthly. The Power BI team in Microsoft has a keen ear for customer input which bodes well for future expansion of this solar system.

More compelling though is that Microsoft are opening up parts of the platform to developers. Custom visualization is a very clever response to the data viz reputation of products like Tableau. REST APIs are a powerful means to embed analytics in your organisations application & data architecture, and entrenches the Microsoft integration capability.

While the marketing material talks about any data, anywhere, any way … it is the lesser emphasized platform to extend and build on (rather than replace) that I find attractive.

Tagged with:
Posted in Power BI

First thoughts

Having threatened to blog for many years, there is no better time than the present to actually do so.

We live in an era that creates more data about events or activities than we can make sense of. New technologies can help us harvest, shape and analyse data. But finding patterns and determining & actioning insight requires both analytical & practical intelligence. So technology is only an enabler if we leverage what’s possible and know how to put the tools to good use.

As a consultant, I get exposure to a variety of business challenges that require analytical solutions. Using new age self-service BI tools (mostly Power BI) has made it possible to convert a conceptual solution to a tangible proof of concept with ease and within a very short space of time. My background as a BI professional with tools such as Analysis Services, Integration Services and Reporting  Services has made the transition trivial. In contrast, business adoption of self-service BI tools are slow and challenging for many reasons that may be a future topic of discussion.

In this space I’ll attempt to blog about common, useful and interesting patterns that I encounter during my labour of love in the world of data. Although it has a technology focus, expect some random posts about BI concepts, analytical thinking and practical intelligence.

Posted in Other