2018 Gartner Magic Quadrant for Analytics and BI Platforms

Here is the 2018 Gartner magic quadrant for Analytics and BI. Microsoft and Tableau continue their dominance as leaders.

For a comparison with previous year, please check my post here 

EN-CNTNT-Gartner-MQAnalytics_BI2018.jpg

Advertisements

Advanced Tableau Analytics – Multi Condition Triggers and Alerts in Bollinger Bands

We have explored Bollinger Bands implementation in my previous post. In this article, we will explore deep on implementing advanced features such as multi-conditions and alerts on Bollinger Bands.

As we visualize the trend of data (e.g. stock price movement),  don’t you think it is beneficial to have alerts in visualization if the price violates upper or lower control limits?  Using Tableau, that can be straightforward implementation with a calculated field for condition and using a Mark Label on the chart as shown belowTableau_Bollinger_Multi_Chart_Conditions

Tableau_bollinger_band3

Tableau_Bollinger_Multi_Chart_Conditions_2

But what if the requirement is to show this alert based on one more condition that was to meet/trigger in another sheet or  based on two or three such conditions from different sheets?

Multi-Chart conditions

Going a step further, imagine that you want to have a dashboard that shows alerts based on combination of conditions from different sheets. Here the implementation gets tricky as Tableau by default is not designed for inter-chart interaction as each sheet generates visualizations based on its underlying data and data source.

One way of implementing is to use multiple data sources and link them in your chart. This is possible if you can link (join) data source of ChartA with that of ChartB with a common dimension. The resulting implementation can have scenarios like :

  • Trigger condition in ChartA has been met
  • Trigger condition in ChartB has been met
  • In Chart C, check for its own Trigger condition, check for trigger in ChartA and ChartB display an Alert, which can be a strategy or aid a decision ( for example, in Stock Trading it can be a BUY/SELL/HOLD decision)

In real-lime, this can be extended further for advanced features to generate Dynamic Dashboards and email alerts. With analytics evolving smarter, you don’t want to view all data visualizations everyday and will be interested only in those visualizations where the triggers have been met.

The final implementation is below

Tableau_Bollinger_Multi_Chart_Conditions_3.PNG

I will update this post with source and implementation steps shortly.

Bollinger Bands analysis using Tableau

Download source files used in this article

Volatility analysis charts using Bollinger Bands are often used for trading decisions. Those who track stock price movements must be familiar with charts types shown below

Tableau_bollinger_band1

Bollinger Bands are intervals drawn on price chart at standard deviation levels above and below the corresponding moving average.

Bollinger Bands consist of :

  • an N-period moving average (MA)
  • an upper band at K times an N-period standard deviation above the moving average (MA + Kσ)
  • a lower band at K times an N-period standard deviation below the moving average (MA − Kσ)

Let us see how can we develop this chart using Tableau.

In this example, I connect to a database table which has daily price information of a stock. Simple data source – dimension as Date and measure as price.

Step one is to create a line chart, as shown below

Tableau_bollinger_band2

Next we need to add 4 calculated files

  • Moving Average
  • Standard Deviation
  • Upper Band
  • Lower Band

Tableau_bollinger_band3

Note: The above formulas have used 20 as N-period (look back period) and 2 as K-times to multiply since these are  typical values used in real world. If we need to change these dynamically, we can consider using them as parameter fields.

Add ‘Measure Names‘ as filter to your sheet (select MA, LB and UB in the filter selection). Then drag and drop ‘Measure Values’ to Rows in the chart.

Tableau_bollinger_band4.PNG

Choose ‘Dual Axis’ for Measure Values and synchronize the axis as next step.

Tableau_bollinger_band5.PNG

We have the Bollinger Bands generated; however the color schemes are not intuitive. So lets modify that a little.

Here is the final chart

Tableau_bollinger_band6.PNG

Download source files used in this article

Cluster Analysis using Tableau and R – Part-2

Related article : Cluster Analysis using Tableau and R – Part-1

We have performed clustering analysis from both Tableau and R in my previous post. Tableau uses k-means algorithm for cluster analysis which partitions the data into k clusters with a center or mean value of all the points in each. Clustering is based on the distance each measure lies from the center.

Let’s look into that in detail.

First generate a cluster scatter plot in Tableau as we did in part -1 using Iris data set.

clustering_tableau_R10

Right click on the cluster that we added and choose Describe Clusters option

clustering_tableau_R11

This provides you the clustering details given below

clustering_tableau_R12

Now lets perform the k-means clustering from R and print the cluster

#copying iris to myiris variable
myiris<-iris
#Remove Species column
myiris$Species <- NULL
#clustering
kmeans.result <- kmeans(myiris,3)

#print the cluster data
kmeans.result

clustering_tableau_R13

Check the cluster means against that of Tableau cluster centers. Aren’t these comparable? However Tableau clustering analysis is limited and the default one is k-means as compared to the number of packages and functions available in R to perform various types of clustering. I will dedicate a future article to cover cluster and fpc packages in R.

In this case, what options that a Tableau user has for extended/advanced clustering? The answer is R integration by calling R packages from Tableau using similar steps that I explained in this article.

How to integrate R with Tableau

We have seen how R can be integrated to your data science project using Power BI or Visual Studio(RTVS). Now its time to look at R integration with Tableau.

Before we get started with the steps, let us discuss how this is beneficial.

Tableau is a great visualization tool which helps you to understand your data, provide interactivity and assist in making business decisions. R integration is going to bring the capabilities of to your Tableau visualizations – such as statistical functions predictive analysis. The advantage of interactive visualizations in Tableau powered by the complex statistical analysis behind the scenes using R presents a strong case for data scientists to go for this integration.

I have added a high level representation below of this implementation. You can call R functions from Tableau and it passes the result back to Tableau which can be used to generate visualizations. You can utilize all packages (difficult to accomplish using Tableau scripting alone) that are running in R Server and generate visualizations using the resultant data (complex to accomplish using R alone).

 

R_Tableau

As first step, make sure that RServe is running as a service that you can connect to. The screenshots below shows how to install RServe from RStudio.

R_Tableau_Integration

Start the service

R_Tableau_Integration_1

Now RServe is ready for connections. Go to Tableau and choose the option Help–> Settings and Performance–>Manage External Service Connection

R_Tableau_Integration_2

In this case, my RServe is running on the same PC. So I selected localhost as server. Default RServe port is 6311. Leave that as is and test your connection as below

R_Tableau_Integration_3

Above message confirms that you have established the connectivity with R service.

Next, we will look at an example calling R scripts from Tableau.

On-Premises Data Gateway for Power BI

We have seen different Power BI implementation options in my article here. If you do not want to move your database infrastructure to cloud, but looking for for generating visualizations from Power BI service(cloud) connecting to on-premises data source, that is possible using Data Gateway. Power BI visualizations on cloud use gateway to get data from on premise data (database such as SQL Server, Oracle or even structured files such as spreadsheets or csv files.

Note : Complete on-premises solution is available with the release of Power BI Report Server which is part of Power BI Premium. This article assumes the Power BI service implementation as below and gateway component is highlighted.

Power BI Gateway 1

On-premises data gateway allows Power BI service to connect to your database to refresh dataset. Gateway allows both scheduled refresh and live/direct query.  Time for looking at the steps on how to install and configure the gateway.

Download the standalone data gateway and start the service in on-premises. To download, use the option below after logon to your accountPower BI Gateway Installation_1

After installation, you have to sign in using your Azure account and specify a name to your gateway so that Power BI service tie it to your Power BI account.

Power BI Gateway Installation_3

Power BI Gateway Installation_4

Make sure that On-Premises Data Gateway is running as a windows service in your PC. Ideally this will be one of your on-premises server with connectivity to your database.

Go to Power BI web login, select ‘Manage Gateway’ option as shown below

Power BI Gateway Installation_5

Note that your Data Gateway is already listed there. You need to create a data source and provide authentication details for database connectivity (Windows User details in case of a file data source such as excel spreadsheet)

Power BI Gateway Installation_6

Next, go to your workspace, select the data set and go to ‘settings’ as shown below

Power BI Gateway Installation_7

Under ‘Gateway connection’ section, select your data gateway and click ‘apply’

Power BI Gateway Installation_8

Under ‘Scheduled refresh’ section, turn on ‘Keep your data up to date’ and specify refresh frequency as below

Power BI Gateway Installation_9

That completes it. Your dataset is ready for scheduled refresh connecting to your on-premises data source!

For verifying, you can update data in SQL Server database (local server) and wait for the refresh to happen.

Power BI Gateway Installation_11

Once complete, launch your visualization which uses the dataset and see your updated data reflecting there!

Power BI Gateway Installation_10

Data Analytics – The Power BI way

In my previous articles related to BI, we have explored extensively on Tableau and SAP Business Objects. While I continue the series, it would be unfair if I don’t cover another ’emerging’ solution – Microsoft Power BI (in fact it is a market leading BI solution already, refer to my post for 2017 BI trends here Gartner Magic Quadrant for BI and Analytics)

Those who are familiar with Tableau will find themselves home with Power BI in many ways. However Power BI is unique in terms of features, the publish options, pricing and visualization options such as custom visualizations.

Power BI was completely cloud based solution until recently – i.e, until the release of Power BI Report Server in this month (June 2017).  In the cloud based solution, you have to publish your visualizations to Power BI Service on cloud. You have alternative to Power BI desktop for development in the form of Power BI Service Web Interface which provides similar options for creating and customizing visualizations.

I have sketched a high level cloud based (Power BI Service) implementation architecture below  connecting to on-premises database to understand the different components better.

Power BI Architecture

Implementation architecture using Power BI Service on cloud connecting to on-premises database

The implementation architecture below shows the on-premises solution using Power BI Report Server. In this implementation, development and publishing do not require cloud. Note that the development require installation of custom release version of Power BI Desktop (Power BI Desktop optimized for Power BI Report Server)

 

PowerBI Architecture_OnPremise

On-Premises implementation architecture using Power BI Report Server

In terms of connectivity, Power BI can connect to almost every data source. However the type and location of data should decide your design and type of connectivity.

On-Premises Database connectivity from Power BI Service : If you have an on-premises database, Power BI service allows the connectivity using gateways. The options are (a) data imported to Power BI service or (b) use direct query or live connections. We will explore these implementation models in detail in the upcoming articles.

PowerBI_DataSourcePowerBI_DataSource1

Custom Visuals : This is really great feature that Microsoft lets Power BI developers to create and publish their custom visualizations which can be added to your visualization tray and integrate with your reports/dashboards. Thanks for the support to creativity from developer community.

R Integration to Power BI :  Similar to Tableau R integration, we can connect to R for performing complex predictive analysis.

PowerBI_R_Integration

I will cover R integration details and many more in upcoming posts.