Cluster Analysis using Tableau and R – Part-2

Related article : Cluster Analysis using Tableau and R – Part-1

We have performed clustering analysis from both Tableau and R in my previous post. Tableau uses k-means algorithm for cluster analysis which partitions the data into k clusters with a center or mean value of all the points in each. Clustering is based on the distance each measure lies from the center.

Let’s look into that in detail.

First generate a cluster scatter plot in Tableau as we did in part -1 using Iris data set.

clustering_tableau_R10

Right click on the cluster that we added and choose Describe Clusters option

clustering_tableau_R11

This provides you the clustering details given below

clustering_tableau_R12

Now lets perform the k-means clustering from R and print the cluster

#copying iris to myiris variable
myiris<-iris
#Remove Species column
myiris$Species <- NULL
#clustering
kmeans.result <- kmeans(myiris,3)

#print the cluster data
kmeans.result

clustering_tableau_R13

Check the cluster means against that of Tableau cluster centers. Aren’t these comparable? However Tableau clustering analysis is limited and the default one is k-means as compared to the number of packages and functions available in R to perform various types of clustering. I will dedicate a future article to cover cluster and fpc packages in R.

In this case, what options that a Tableau user has for extended/advanced clustering? The answer is R integration by calling R packages from Tableau using similar steps that I explained in this article.

How to integrate R with Tableau

We have seen how R can be integrated to your data science project using Power BI or Visual Studio(RTVS). Now its time to look at R integration with Tableau.

Before we get started with the steps, let us discuss how this is beneficial.

Tableau is a great visualization tool which helps you to understand your data, provide interactivity and assist in making business decisions. R integration is going to bring the capabilities of to your Tableau visualizations – such as statistical functions predictive analysis. The advantage of interactive visualizations in Tableau powered by the complex statistical analysis behind the scenes using R presents a strong case for data scientists to go for this integration.

I have added a high level representation below of this implementation. You can call R functions from Tableau and it passes the result back to Tableau which can be used to generate visualizations. You can utilize all packages (difficult to accomplish using Tableau scripting alone) that are running in R Server and generate visualizations using the resultant data (complex to accomplish using R alone).

 

R_Tableau

As first step, make sure that RServe is running as a service that you can connect to. The screenshots below shows how to install RServe from RStudio.

R_Tableau_Integration

Start the service

R_Tableau_Integration_1

Now RServe is ready for connections. Go to Tableau and choose the option Help–> Settings and Performance–>Manage External Service Connection

R_Tableau_Integration_2

In this case, my RServe is running on the same PC. So I selected localhost as server. Default RServe port is 6311. Leave that as is and test your connection as below

R_Tableau_Integration_3

Above message confirms that you have established the connectivity with R service.

Next, we will look at an example calling R scripts from Tableau.

On-Premises Data Gateway for Power BI

We have seen different Power BI implementation options in my article here. If you do not want their application or database infrastructure to cloud will be looking for options for generating visualizations from Power BI service(cloud) connecting to on-premises data source (database such as SQL Server, Oracle or even structured files such as spreadsheets or csv files.

Note : Complete on-premises solution is available with the release of Power BI Report Server which is part of Power BI Premium. This article assumes the Power BI service implementation as below and gateway component is highlighted.

Power BI Gateway 1

On-premises data gateway allows Power BI service to connect to your database to refresh dataset. Gateway allows both scheduled refresh and live/direct query.  Time for looking at the steps on how to install and configure the gateway.

Download the standalone data gateway and start the service in on-premises. To download, use the option below after logon to your accountPower BI Gateway Installation_1

After installation, you have to sign in using your Azure account and specify a name to your gateway so that Power BI service tie it to your Power BI account.

Power BI Gateway Installation_3

Power BI Gateway Installation_4

Make sure that On-Premises Data Gateway is running as a windows service in your PC. Ideally this will be one of your on-premises server with connectivity to your database.

Go to Power BI web login, select ‘Manage Gateway’ option as shown below

Power BI Gateway Installation_5

Note that your Data Gateway is already listed there. You need to create a data source and provide authentication details for database connectivity (Windows User details in case of a file data source such as excel spreadsheet)

Power BI Gateway Installation_6

Next, go to your workspace, select the data set and go to ‘settings’ as shown below

Power BI Gateway Installation_7

Under ‘Gateway connection’ section, select your data gateway and click ‘apply’

Power BI Gateway Installation_8

Under ‘Scheduled refresh’ section, turn on ‘Keep your data up to date’ and specify refresh frequency as below

Power BI Gateway Installation_9

That completes it. Your dataset is ready for scheduled refresh connecting to your on-premises data source!

For verifying, you can update data in SQL Server database (local server) and wait for the refresh to happen.

Power BI Gateway Installation_11

Once complete, launch your visualization which uses the dataset and see your updated data reflecting there!

Power BI Gateway Installation_10

Data Analytics – The Power BI way

In my previous articles related to BI, we have explored extensively on Tableau and SAP Business Objects. While I continue the series, it would be unfair if I don’t cover another ’emerging’ solution – Microsoft Power BI (in fact it is a market leading BI solution already, refer to my post for 2017 BI trends here Gartner Magic Quadrant for BI and Analytics)

Those who are familiar with Tableau will find themselves home with Power BI in many ways. However Power BI is unique in terms of features, the publish options, pricing and visualization options such as custom visualizations.

Power BI was completely cloud based solution until recently – i.e, until the release of Power BI Report Server in this month (June 2017).  In the cloud based solution, you have to publish your visualizations to Power BI Service on cloud. You have alternative to Power BI desktop for development in the form of Power BI Service Web Interface which provides similar options for creating and customizing visualizations.

I have sketched a high level cloud based (Power BI Service) implementation architecture below  connecting to on-premises database to understand the different components better.

Power BI Architecture

Implementation architecture using Power BI Service on cloud connecting to on-premises database

The implementation architecture below shows the on-premises solution using Power BI Report Server. In this implementation, development and publishing do not require cloud. Note that the development require installation of custom release version of Power BI Desktop (Power BI Desktop optimized for Power BI Report Server)

 

PowerBI Architecture_OnPremise

On-Premises implementation architecture using Power BI Report Server

In terms of connectivity, Power BI can connect to almost every data source. However the type and location of data should decide your design and type of connectivity.

On-Premises Database connectivity from Power BI Service : If you have an on-premises database, Power BI service allows the connectivity using gateways. The options are (a) data imported to Power BI service or (b) use direct query or live connections. We will explore these implementation models in detail in the upcoming articles.

PowerBI_DataSourcePowerBI_DataSource1

Custom Visuals : This is really great feature that Microsoft lets Power BI developers to create and publish their custom visualizations which can be added to your visualization tray and integrate with your reports/dashboards. Thanks for the support to creativity from developer community.

R Integration to Power BI :  Similar to Tableau R integration, we can connect to R for performing complex predictive analysis.

PowerBI_R_Integration

I will cover R integration details and many more in upcoming posts.

Data Analytics using Tableau – Trends & Trend Lines

In my previous blog, I had posted a bar chart showing declining trend in sales of a company over the years. If you notice, it had a trend line to aid visual analytics.

graph-1

Another example given below shows profit trends by increase in sales for each category.

Tableau_Trend_Lines

Trend lines are great visual tools for quick analysis. In the above diagram, its very easy to judge that the increase in sales for Supplies and Tables does not help increasing profit. The same judgement could not have been derived that easy from the clustered circles if the trend lines were not present. The steps to add Trend Lines are explained below.

I have created a scatter plot for Sales vs Profit using the Global Superstore database. Notice the “analytics” tab highlighted below.

Tableau_Trend_Lines1

Drag ‘Trend Line’ from the options to the visualization area and choose the line type. In this example, I have selected Linear model.

Tableau_Trend_Lines2

Tableau_Trend_Lines3

Additional options are available at (Right click -> Trend Lines-> Edit Trend Lines). For example, you can choose to show only one line (uncheck show trend line per color) and view for each category interactively for focused view/analysis as shown below

Tableau_Trend_Lines4

 

R Programming using RTVS – Part 4

This article assumes that you have RTVS extension added to your VS IDE.  You can refer my previous blogs on installation & configuration of RTVS.

  1. R Support in Visual Studio (RTVS)
  2. R Programming using RTVS – Part 2
  3. R Programming using RTVS – Part 3
Prerequisites:
1. Basic knowledge of R programming
2. RTVS Extension

Creating R projects using RTVS

Create a project in visual studio using R template.

RTVS_2

Open the Script.R file from solution explorer and type in your R program. In the example below, we create a simple line graph that compares sales and profit performance of a company over a period of 5 years.

#Define Data
sales <- c(2000, 3500, 3000, 4700, 6000)
profit <- c(1450, 2600, 750, 2000, 3750)
#range for axis
rnge <- range(0, sales, profit)
#Plot sales values
plot(sales, type = "o", col = "green", ylim = rnge, axes = FALSE, ann = FALSE)
#draw profit values as line
lines(profit, type = "o", pch = 22, lty = 2, col = "red")
box()
#draw X axis and y axis
axis(1, at = 1:5, lab = c("2013", "2014", "2015", "2016", "2017"))
axis(2, las = 1, at = 1000 * 0:rnge[2])
#Graph Title
title(main = "Sales by Year", col.main = "blue", font.main = 4)
#add axis labels
title(xlab = "Year", col.lab = rgb(0, 0, 1))
title(ylab = "Amount", col.lab = rgb(0, 0, 1))

Place your cursor in the first line of code and use Ctrl+Enter to step through each line to R Interactive and execute. You can see each line execution results to the R Plot window as shown below

RTVS_5_R_prioject

Here is the final plot view

RTVS_5_R_prioject_plot

Given below is an updated source which generate the output as PDF file

#Define Data
sales <- c(2000, 3500, 3000, 4700, 6000)
profit <- c(1450, 2600, 750, 2000, 3750)

#File name to save output
pdf(file = "sales_and_profit_byYr.pdf", height = 4, width = 5)

#range for axis
rnge <- range(0, sales, profit)

#Plot sales values
plot(sales, type = "o", col = "green", ylim = rnge, axes = FALSE, ann = FALSE)
#draw profit values as line
lines(profit, type = "o", pch = 22, lty = 2, col = "red")
box()

#draw X axis and y axis
axis(1, at = 1:5, lab = c("2013", "2014", "2015", "2016", "2017"))
axis(2, las = 1, at = 1000 * 0:rnge[2])

#Graph Title
title(main = "Sales by Year", col.main = "blue", font.main = 4)

#add axis labels
title(xlab = "Year", col.lab = rgb(0, 0, 1))
title(ylab = "Amount", col.lab = rgb(0, 0, 1))

#Flush the output to PDF file
dev.off()

R Programming using RTVS – Part 3

This article assumes that you have RTVS extension added to your VS IDE.  You can refer my previous blogs on installation & configuration of RTVS.

  1. R Support in Visual Studio (RTVS)
  2. R Programming using RTVS – Part 2
Prerequisites:
1. Basic knowledge of R programming
2. RTVS Extension

Key RTVS features that a developer should be aware of :

The following is just a list for you to explore rather than extensive walk through.

  1. Step through :  use Ctrl+Enter
  2. Execute a block of code in R interactive  : Select a block of code and Ctrl+ Enter (or right click and choose Execute in Interactive)
  3. Debug : Use Debug > Source startup file or F5
  4. Set Debug points : Native VS behavior (refer image below)rtvs_5_r_debug.png
  5. Reset current work space : To clear everything by using R Tools > Session > Reset
  6. Examine variables : Use ‘variable explorer’ (Ctrl + 8)
  7. IntelliSense : RTVS_5_R_IntelliSense
  8. Help : Use ? and ?? (Use ?? to search term in quotes if it include spaces)
  9. Code snippet manager :  View them at Tools > Code Snippets Manager.  RTVS_5_R_CodeSnipptType the abbreviated name and use tab to insert a snippet to your codeRTVS_5_R_CodeSnippt1
  10.  R interactive window or REPL (Read-Evaluate-Print-Loop) window : Use already illustrated in previous steps/posts
  11. Plot window : Ctrl + 6
  12. Plot history : Preserves all your plots in the session.  R Tools > Plots > Plot History Window (refer image below)RTVS_5_R_Plot_history

There are many more which an R programmer or VS developer can explore themselves. We will go through some samples using RTVS in the upcoming posts.