Creating a Web Data Connector / Part 1

One of the top features of Tableau is that they provide out-of-the-box connectivity to a variety of common data sources; the number of supported data sources is increasing with every release of Tableau. However, sometimes we may have a bespoke data source, a legacy data source, or perhaps an extremely bleeding edge data source, which may not be supported out of the box by any tool. In these edge cases, Tableau has provided the Tableau Web Data Connector (WDC) that allow developers to build our own data source connectivity.

In this series, we are going to build our own Tableau Web Data Connectors by introducing you to programming as well as the Tableau Web Data Connector JavaScript library.

Note: If you are a seasoned programmer, then the quickest way to get started is to go to the official Tableau documentation at http://tableau.github.io/webdataconnector.

Part 1: Getting Started

In the first part of this series, we will get started by installing and setting up the development environment (and relevant tools), getting the Tableau Web Data Connector (WDC) Library and integrating with GitHub for your storing and sharing your projects.

Installing GitHub

Git (/ɡɪt/) is a version control system for tracking changes in computer files and coordinating work on those files among multiple people. It is primarily used for source code management in software development, but it can be used to keep track of changes in any set of files. As a distributed revision control system, it is aimed at speed, data integrity, and support for distributed, non-linear workflows.

You can download Git for your desktop here: https://git-scm.com

GitHub Inc. is a web-based hosting service for version control using Git. It is mostly used for computer code. It offers all of the distributed version control and source code management (SCM) functionality of Git as well as adding its own features. It provides access control and several collaboration features such as bug tracking, feature requests, task management, and wikis for every project.

GitHub offers plans for both private repositories and free accounts which are commonly used to host open-source software projects. As of June 2018, GitHub reports having over 28 million users and 57 million repositories (including 28 million public repositories.), making it the largest host of source code in the world.

On June 4, 2018, Microsoft announced it had reached an agreement to acquire GitHub for US$7.5 billion. The purchase is expected to close by the end of the year.

Please go to the GitHub website and create yourself an account, we will save our work to here: https://github.com

Note: GitHub does come with a Desktop tool, however, for this tutorial we will be using a command line (cmd) to check in our new code into GitHub.

Installing Microsoft Visual Studio Code

Visual Studio Code is a source code editor developed by Microsoft for Windows, Linux and macOS. It includes support for debugging, embedded Git control, syntax highlighting, intelligent code completion, snippets, and code refactoring. It is also customizable, so users can change the editor’s theme, keyboard shortcuts, and preferences. It is free and open-source, although the official download is under a proprietary license.

Visual Studio Code is based on Electron, a framework which is used to deploy Node.js applications for the desktop running on the Blink layout engine. Although it uses the Electron framework, the software does not use Atom and instead employs the same editor component (codenamed “Monaco”) used in Visual Studio Team Services (formerly called Visual Studio Online).

In the Stack Overflow 2018 Developer Survey, Visual Studio Code was ranked the most popular development environment tool, with 34.9% of 75,398 respondents claiming to use it.

I use Microsoft Visual Studio Code for most of my coding and you can download it here: https://code.visualstudio.com

Installing Node.js

We will install the Yarn Package Manager to help manage our javascript packages and dependencies, don’t worry, we will go through this a little later, but for now a little bit about Node.js and Node Package Manager (NPM).

Node.js is an open-source, cross-platform JavaScript run-time environment that executes JavaScript code outside of a browser. Historically, JavaScript was used primarily for client-side scripting, in which scripts written in JavaScript are embedded in a webpage’s HTML and run client-side by a JavaScript engine in the user’s web browser. Node.js lets developers use JavaScript to write Command Line tools and for server-side scripting—running scripts server-side to produce dynamic web page content before the page is sent to the user’s web browser. Consequently, Node.js represents a “JavaScript everywhere” paradigm, unifying web application development around a single programming language, rather than different languages for server side and client side scripts.

npm is a package manager for the JavaScript programming language. It is the default package manager for the JavaScript runtime environment Node.js. It consists of a command line client, also called npm, and an online database of public and paid-for private packages called the npm registry. The registry is accessed via the client, and the available packages can be browsed and searched via the npm website. The package manager and the registry are managed by npm, Inc.

Download and install Node.js (this includes NPM) here: https://nodejs.org.

Installing YARN

Yarn is a package manager for your code. It allows you to use and share code with other developers from around the world. Yarn does this quickly, securely, and reliably so you don’t ever have to worry.

Yarn allows you to use other developers’ solutions to different problems, making it easier for you to develop your software. If you have problems, you can report issues or contribute back, and when the problem is fixed, you can use Yarn to keep it all up to date.

Code is shared through something called a package (sometimes referred to as a module). A package contains all the code being shared as well as a package.json file which describes the package.

We will use Yarn to get a hold of various Javascript packages. Download and install Yarn here: https://yarnpkg.com/en/docs/install.

Now that we have all the required software installed, let us go about putting this all together.

Putting it all together

Let us first start by creating a repository on GitHub to store your code and manage your changes.

• Log into github.com
• Click Create a repository
  • In Repository name enter tableau-wdc-tutorial-part-1
  • Leave the repository as Public
  • Tick Initialize this repository with a README
  • Click on Create repository

Congratulations, you have created a GitHub repository with a single file called README.md. This file is used to give browsers an idea of what is inside your repository. MD files have their own markup language that gives some basic formatting. You can read more about it, if you are curious, click here: https://github.com/adam-p/markdown-here/wiki/Markdown-Here-Cheatsheet 

Now that we have our GitHub repository setup, let us start up VS Code and import the directory.

• Open Visual Studio Code.
• Go to View and click on Terminal.
  • This is your command terminal which will send commands to your operating system.
• Let us test to see that we have all the software installed and available on your operating system.
  • Enter the following command in the Terminal and press enter: git --version
  • Enter the following command in the Terminal and press enter: node --version
  • Enter the following command in the Terminal and press enter: yarn --version
  • In all cases, you should receive a single line with the version number. 
• If all goes well, let us clone your newly created GitHub repository to a local folder, we will make changes, and then commit and push those changes back to your repository.
  • Enter the following command in the Terminal and press enter: https://github.com/<your username>/<your repository name>
  • For example, git clone https://github.com/tableaumagic/tableau-wdc-tutorial-part-1
  • If all is successful you will have copied the GitHub folder to the current folder set in your Terminal.

• Now go to File Menu and click Open Folder
  • Open the local copy of your directory that you have just created.
  • Once the folder is open you should see a single file called README.md.
• We are going to change this and push our changes back up to GitHub
  • Double click the README.md.
    • You should see the following: #tableau-wdc-tutorial-part-1.
  • Change this to # Tableau Web Data Connector Tutorial Part 1.
  • Add in a new line below: This is my first Web Data Connector.
  • Click Save.
• On the far left side menu, you should see a number 1 appearing in a blue next to the middle most button. This means you now have a file that you have modified by not committed.
  • Click on the third button on the left sidebar to see the Source Control tab.
  • In the Source Control: Git panel, click on the tick to commit your changes to your local repository.
    • You will be asked for a message, enter in My first commit, and click enter.
    • Your update has now been committed locally.
  • Now we will push the changes to GitHub by clicking on the … and select push.
    • A GitHub window may pop up, or you may see it in your tasks bar.
    • Enter in your credentials and click login.
  • Now go back to your GitHub directory in your web browser (or Desktop and see your changes).

Boom, so with that all said and done you have now installed the required tools, created a GitHub repository, started up Visual Studio Code, and connected to your GitHub Directory. We will now go about coding your first Tableau Web Data Connector (WDC) Project, but first…

Tableau Web Data Connector and Versions

The following Tableau displays which version of the Tableau WDC Library is compatible with which version of Tableau Desktop; always bare this in mind before you start developing.

Note: In this tutorial, I am using tableauwdc-2.3.latest.  

Your Project

To start the project:

• Open up your terminal by going to View and then clicking on Terminal.
  • Make sure your terminal is pointing to your project directory (see terminal commands).
  • Enter the following command: yarn init.
    • This will create your project.json where your project yarn details will be installed; this file will be used for dependency management.
    • Enter whatever information you fancy with regards to the project details.
  • Now in your Explorer window create the following folders.
    • js – for your javascript application code files.
    • css – for your component styling files.
    • libs – for your third-party libraries (reuse is awesome).
  • In the root directory of your project, create the following files (make sure both start if a period).
    • .yarnrc
      • Add the following line: --modules-folder libs.
      • This is where Yarn will save your third-party libraries.
    • .gitignore
      • Add the following line: /libs/.
      • This tells git to ignore the files within the libs directory. You should not be pushing your third-party libraries into your GitHub directory.

Now we are going to install two of the most common Javascript libraries, jQuery and Bootstrap, into our project; we will make use of these later in our project.

In your terminal, make sure you are in the root directory:

• Enter the following command and press enter: yarn add jquery.
• Enter the following command and press enter: yarn add bootstrap.
• Check your libs folder and see that you should have extra stuff added.
  • With those simple commands, you have added two of the most widely used web libraries to your project. As easy as that.

jQuery – jQuery is a lightweight, “write less, do more”, JavaScript library. The purpose of jQuery is to make it much easier to use JavaScript on your website. jQuery takes a lot of common tasks that require many lines of JavaScript code to accomplish and wraps them into methods that you can call with a single line of code.

Bootstrap – Bootstrap is a free and open-source front-end framework for designing websites and web applications. It contains HTML- and CSS-based design templates for typography, forms, buttons, navigation and other interface components, as well as optional JavaScript extensions. Unlike many earlier web frameworks, it concerns itself with front-end development only.

Now, let us get the Tableau Web Data Connector (WDC) library.

• Go to the following URL and save the latest file to your project js directory.
  • Open the following in your web browser: https://connectors.tableau.com/libs/tableauwdc-2.3.latest.js
  • Right click and save the following: tableauwdc-2.3.latest.js.

To this point, we have done a lot of work, and I assure you, you won’t have to do this all the time. We will now finally create the Tableau Web Data Connector.

Web Data Connector

Now that we have our project set up, we will now create our Tableau Web Data Connector (WDC) by creating the following three files:

index.html in the root of your project:

<html>
   <title>Tableau Web Data Connector Tutorial Part 1</title>
   <meta http-equiv="Cache-Control" content="no-store"/>
   <head>
      <link rel="stylesheet" type="text/css" href="libs/bootstrap/dist/css/bootstrap.min.css">
      <link rel="stylesheet" type="text/css" href="css/style.css">
   </head>
   <body>
      <div>
         <button id="submitButton" class="btn btn-success">Get Data</button>
      </div>
   </body>
   <script src="libs/jquery/dist/jquery.min.js"></script>
   <script src="js/tableauwdc-2.3.latest.js"></script>
   <script src="js/wdc.js"></script>
</html>

This is a very plain HTML page with a single button, but the HTML page imports all the required libraries.

style.css in the css folder:

.btn-success {
   margin: 20px;
}

This will be used to add margins around the button, however, this will be important to configure the look and feel of our page.

wdc.js in the js folder:

(function () {
   var myConnector = tableau.makeConnector();

   // This creates the Web Data Connector schema that
   // describes the information returned by hte WDC.
   myConnector.getSchema = function (schemaCallback) {
      var cols = [{
         id: "Country",
         dataType: tableau.dataTypeEnum.string
      }, {
         id: "Year",
         dataType: tableau.dataTypeEnum.int
      }, {
         id: "GDP",
         dataType: tableau.dataTypeEnum.float
      }];

      var tableSchema = {
         id: "WDC1",
         alias: "GDP by Country and Year",
         columns: cols
      };
      schemaCallback([tableSchema]);
   };

   // This function is called when data is required from the
   // Web Data Connector.
   myConnector.getData = function (table, doneCallback) {
      tableData = [];

      // We are manually adding data, but in future tutorials
      // we will connect to an external data source using the
      // ajax function.
      tableData.push(
         {"Country": "US", "Year": "2002", "GDP": "10.98"},
         {"Country": "China", "Year": "2002", "GDP": "1.45"},
         {"Country": "Japan", "Year": "2002", "GDP": "3.98"},
         {"Country": "Germany", "Year": "2002", "GDP": "2.01"},
         {"Country": "France", "Year": "2002", "GDP": "1.45"},
         {"Country": "United Kingdom", "Year": "2002", "GDP": "1.62"},
         {"Country": "Brazil", "Year": "2002", "GDP": "0.50"},
         {"Country": "Russian Federation", "Year": "2002", "GDP": "0.35"},
         {"Country": "Italy", "Year": "2002", "GDP": "1.23"},
         {"Country": "India", "Year": "2002", "GDP": "0.52"},
         {"Country": "US", "Year": "2012", "GDP": "16.24"},
         {"Country": "China", "Year": "2012", "GDP": "8.23"},
         {"Country": "Japan", "Year": "2012", "GDP": "5.94"},
         {"Country": "Germany", "Year": "2012", "GDP": "3.43"},
         {"Country": "France", "Year": "2012", "GDP": "2.61"},
         {"Country": "United Kingdom", "Year": "2012", "GDP": "2.46"},
         {"Country": "Brazil", "Year": "2012", "GDP": "2.25"},
         {"Country": "Russian Federation", "Year": "2012", "GDP": "2.02"},
         {"Country": "Italy", "Year": "2012", "GDP": "2.01"},
         {"Country": "India", "Year": "2012", "GDP": "1.86"}
      );

      table.appendRows(tableData);
      doneCallback();
   };

   // This is reqired to register the Web Data Connector.
   tableau.registerConnector(myConnector);

   // Once the document has loaded we will attached functionality
   // to the submitButton.
   $(document).ready(function () {
      $("#submitButton").click(function () {
         tableau.connectionName = "Web Data Connector Part 1";
         tableau.submit();
      });
   });
})();

This is where we implement our Tableau Web Data Connector (WDC) logic. We will revisit this file a lot in future articles, but for now, we just need to be aware of what we call:

• makeConnector() is our constructor that builds our connector.
• getScheme() which returns a schema object.
• getData() which returns our data.
• registerConnector() to validate the connector object before initialisation.
• click() to attach functionality to the click of the button.

In our next articles we will go into coding in far more detail, but for now, we have built our Tableau Web Data Connector, so let us give it a whirl.

Running the Web Data Connector

We will use npm to install an http-server by entering the following command in the Terminal: npm install http-server -g. Once installed, we will run the index.html file on a local web server by entering the following command in the terminal:http-server; this will launch a web server which hosts your web page at http://localhost:8080, so now you need to:

• Open Tableau Desktop.
• Under To Server, go to More and select Web Data Connector.
• Enter http://localhost:8080 in the address bar and click enter.
• Your Web Data Connector should load with a green button, click Get Data.

If all goes well, data should be returned and you should be able to interact with the data specified in the wdc.js file, and boom you are done. Commit your work to GitHub, mine is located here: https://github.com/simplifyinsights/tableau-wdc-tutorial-part-1

Summary

In this lengthy tutorial, we have covered a large amount of ground. We looked at:

• Setting up the development environment
• Set up and published your project to GitHub
• Downloaded various web libraries
  • Tableau Web Data Connector library
  • Bootstrap for the look and feel of the component
  • jQuery to manipulate and add remove the worksheets
• Built your first Tableau Web Data Connector (WDC).

In our next articles in the series, we will explore the Tableau Web Data Connector (WDC) library, and also connect to a bespoke data source using Asynchronous JavaScript (Ajax).

I hope you all enjoyed this article as much as I enjoyed writing it; writing this article made a three-hour train ride from Newcastle to London far more informative than I could have imagined. Do let me know if you experienced any issues recreating this Visualisation, and as always, please leave a comment below or reach out to me on Twitter @Tableau_Magic.

Read more about Tableau Web Data Connectors (WDC) on the official website: http://tableau.github.io/webdataconnector

If you like our work, do consider supporting us on Patreon, and for supporting us, we will give you early access to tutorials, exclusive videos, as well as access to current and future courses on Udemy:

• Patreon: https://www.patreon.com/tableaumagic

Also, do be sure to check out our various courses:

• Creating Bespoke Data Visualizations (Udemy)
• Introduction to Tableau (Online Instructor-Led)
• Advanced Calculations (Online Instructor-Led)
• Creating Bespoke Data Visualizations (Online Instructor-Led)