Wednesday, 21 October 2015

Web Data Connectors

Craig Bloodworth

There are many data sources that would help us understand the world around us that are still only exporting .csv so how do you automate these? Web Data Connector

There are resources out there. You need the Simulator and SDK. Don’t be afraid to google and use other people’s code.

The WDC allows you to use a connection that is hosted on a webpage (and been written by someone else) and works just like a Driver for a database.

HTML – the building blocks of the page
CSS – the styling of the page
Javascript – the engine that is running in the background.

In Javascript there are Variables (like a Parameter in Tableau), Objects (a Variable that has many properties) and Functions (the elements that do the work)

HTML DOM – native set of functions to Javascript
J Query – a bit quicker to write

Event Listeners – waiting for certain user actions like a click

Arrays – like a shopping list
AJAX – how to connect to an external web service
JSON – hopefully the data will be in JSON as it is made for Javascript
Loops & Logic – need to be able to loop through those

Recommended read: Javasccript & Jquery by Jon Duckett

Major components: 1. User interface, 2. API interactions & config, 3. Decalre column names and data types, 4. Build data table
Major Compnents in coded order: 1. Declare column  names and data types, 2. build data table, 3. API interactions & config, 4. User interface.

Build through adjusting an existing WDC to help you see the changes you are making to see the effects.

50 Shades of Data

Matt Francis

1666 Isaac Newton was looking at Optics when he discovered the Spectrum. Split out 7 colours, he chose 7 as there are 7 musical notes.

This led to the idea of the colour wheel. This developed in to colour theory.

Colour is a fantastic tool.

Las Vegas uses colour to draw your attention to suck the money out of your pockets.

Colour theory: 3 primary colours (fundamental). The colour wheel helps us pick complimentary colours. Complimentary colours are those that are on the opposite side of the colour wheel. Orange & Blue contrast nicely. Used in film posters a lot sun / sky.

When picking colours you have to be careful about the perception of colour. Colour should always enhance the visualisation.

People see colours differently. Should we use Red / Green? We understand Red is Bad and Green is Good. MF’s says yes you can use it. If you use it for yourself but if it goes public then you should avoid it. You can use high contrasting colour.

Use to check your visualisations for colour blind tests. Stepped colour makes it easier to use as tone can be distinguished.

Colour has associations and so can act as a short cut. Colour is one of the first things we see so those associations happen before we have read the content.

Colour highlighting has two types: 1 Biased highlighting (something is wrong) and 2 Impartial highlighting (interesting)

Colour can be used to bring emotions out. Downward bar chart to show gun deaths. Make it red and it adds the emotional element.

Colour Themes – Matt’s viz about fast food calorific content was perfected through colour choice. 

The colour matches the theme. Chart colours need to fit the theme and relate well to the theme of the overall dashboard. – upload an image and it lets you select a colour palette. Add these to your custom colour palettes in you preferences file (My Docs > My tableau Repository > Preference.tps)

Colour theory gets us 90% of the way there but we should play with colour too

Using the medium default colour palette is a nice tip to avoid overly contrasting colours.   

Data15 Keynote 2 – Daniel Pink

New style of work – free, independent from the Corporate fixed roles where you are passionate for what you do

Books include: “A whole new mind”, “Drive”

Today’s session we will look at “what motivates us” from a data driven perspective

Two types of knowledge: 1. Explicit knowledge (you know it and can show it) 2. Implicit knowledge (you know it but you don’t know you know it)

The laws of motivation are very evident. If you reward behaviour you get more of it. If you punish behaviour, you get less of it. You don’t need a hypothesis to test to understand this. If your unlying laws are a little off then you will misread situations. Punishing behaviour doesn’t always result in less of that behaviour.

4 economists did 9 tests in America and India. Everyone was treated the same way across a series of challenges except the reward they were given. Participants got 3 different levels of reward for good performance. India’s reward was a lot higher relatively. For mechanical tasks the highest reward group performed the best. “But once the task called for even rudimentary cognitive skill, a larger reward led to poorer performance”

Controlling contingent reward – if / then rewards – great for simple and short term work. Humans love rewards (it’s the definition of the word!). Rewards get our attention and focus. If / then rewards are not great for long term and complex tasks. Great for algorthymic tasks (ie follow a simple set of steps). If you are solving a creative task then you need an expansive view where you don’t have that laser-beam focus where you narrow your thinking. This contradicts our Implicit knowledge so this is why we don’t find this in every day society.

Animals are implicitly aware of fairness. If you have uneven pay levels, you will get rebellion. You have to pay people enough.

If you are getting people to do long term complex work, then you want them to stop thinking about the money

Autonomy, Mastery and Purpose – are the 3 key elements of work

1. Autonomy – let’s think about management – DP argues that management is a technology designed in the 1850s to produce improvements in task completion. You still need compliance but people don’t produce their great work when they are compliant, they do it when they are ‘engaged’. 2 in 10 people are actively disengaged in the workforce. You have to have sovereignty for your employees if you want engagement.

Zappos is the extremem version where there is no hierarchy or management.

Netflix is less extreme – their expense policy is “Act in Netflix’s best interest”

If you have autonomy on Time, Technique, Team and Task then this sovereignty gives you much higher engagement.

Atlassian – Australian Software company. Each week you have a ‘Ship It’ where devs work on what they want as long as they show it to the rest of company

Columbia Credit Union – one manager gives an hour each week to go and do something different than answer the phone. Called the ‘Genius Hour’

Manchester University – have ‘Friday Evening Experiments’ that “You’re allowed to do whatever you want as long as it is not boring” – no funding just try stuff. Led to Graphene discovery and a Nobel Prize.

So the message is carve out a few ‘Islands of autonomy’ – create space to try something different.

2. Mastery – “making progress in meaningful work” has been found as the key element. Feedback is vital to showing the progress is being made. Millennials have grown up where they have information at their fingertips the whole time. In organisations, the feedback disappears and is done every 6 months.

Two ideas – 1. weekly one-on-ones with a twist. Every monthly meeting ask ‘Love and Loathe’ rather than what you are working on. Career long term or Removing Barriers. 2. Progress Rituals – Humans create rituals to understand the world. Write down 3 good things that happened each day.

3. Purpose – How / Why – if you are struggling and find out the How it gets focus. Why gives the focus as it creates a purpose to deliver against. Have 2 fewer conversations about How and have 2 conversations about Why.

We have the chance to run organisations that work with the grain of how humans work.

Tuesday, 20 October 2015

Minority Report in Tableau

Allan Walker, Anya Ahern and Jeffrey Schaffer

Preface: There are not enough words that I can use to describe this session. The experimental work done by these amazing Tableau Zens is phenomenal. I have given hints at what the content is below but until you see some of the techniques the team have come up with, you can only use your imaginations. That is what this session is about. Using the fundamentals in Tableau and setting up experiences to allow you to interact with the tool in an entirely new way.


Where it started: Tableau Reader was all there used to be until Tableau Server arrived. But that wasn’t enough!

The Javascript API allows you so much access to try to do anything.

The CSS set-up allows you to do a whole lot of fun interaction.

The team have created the ability to take the visualisations from Tableau Server and create your own webpage.

Reveal.js – can create slide transitions but loads the visualisations up front

The team have used Leap Motion to interact with visualisations by just hand movements (and not with a mouse!)

Anya has taken inspiration from the James Bond film Skyfall to create real time crime, traffic, fire and weather data. All pulled together and controlled using voice and Leap Motion control.

I wish there was a video of this session as there is so much possible and it really is the future!
What the team wanted to do was to get the Minority Report all built in Tableau. And the team actually has. Wanted to build parallel processing, animated, movable and resizable and they nailed it.

Fighting Ebola with Tableau

Peter Gilks, Nelson Davis and John Mathis

The Tableau Foundation deployed Zen Masters and software to help fight Ebola

Tableau foundation began in 2012 and 2013 IPO of Tableau created funding for the Foundation. Aim 'to encourage the use of facts and analytical reasoning to solve the World's problems'. Does Mission Grants, Community Grants, Disaster Response and Employee Service & Giving.

The President of Guinea said “the Tableau Foundation work helped to transform the fight against Ebola”.

Volunteer network of Tableau experts eager to help non-profits to do more with data.

Background: 1st outbreak in Guinea in December 2013. There have been c.3800 cases and c. 2800 deaths. The GDP per capita is less than $500. There is a population of 10.5 million people
5 stage process: 1. Contact Identification, 2. Contact Tracing, 3. Diagnosis, 4. Treatment, 5. Safe burials

Electronic Data management and Contact Tracing was a big step forward. Used basic smartphones to capture the data. Data was sent straight back to the system so live updates were possible. It helps to increase transparency around the tracing.

CommCare created a simplistic app to capture data and create on the ground recommendations. Visualising the data was not the best so in stepped Tableau.

Data flow was a daily upload that goes in to SQL Server and the data was held on Tableau Server. 

Data was not always as clean as hoped. Training of colleagues was continual so this meant data quality was challenged here as well. Transactional data also posed challenges. It was also in French so that was added fun.

Cultural challenges was tough as trying to develop reporting for use of those that haven’t had high reporting exposure adds to the challenge.

Quick fire dashboards were the requirement rather than running extensive usability testing.

Level of Detail would have made the transactional data a lot easier to handle and measure but the team were using 8.2.

Scaffolding was used to get round date issues but adding a data filter to look at whether the data was less than today to make sure the records are reduced as much as possible.

Doctors used the dashboards to work out how to allocate resources and could make as many data led decisions as much as possible because if it wasn’t optimal more people would contract Ebola

You can volunteer at

Data15 - NFL Digital Media: 32 Teams, 1 Dashboard

Presented by Allison Brown (AB) Senior Analyst, National Football League

Analysing all the digital media channels the league has. Used by all the teams.

Challenges: Reports were in Excel, wanted simple reporting, siloed datasets and no transparency between the clubs

Solution: Surveyed clubs to see what they were using, 29 clubs agreed to share their data, wanted infographic feel, need to combine 100 data feeds in to one interactive dashboard

First formative step was to create basic visualisations to augment the initial Excel reports

Switched to Tableau Online to avoid server set-up

Custom iconography gives it more of an interactive feel

Education was a challenge: Held a webinar to walkthrough functionality, set-up a google form for feedback, utilised comments to increase communication

Dashboard first iteration promoted all 32 teams to share and not just the 29. 90% of clubs log in at least once a week. The decisions about content creations are now being data driven

A few of the teams are using Tableau independently just from using the initial dashboards

Data15 - Tableau Opening Keynote

It’s that time folks. Christian, Chris and the Devs are about to take the stage in the MGM arena where Tyson and the Stones have starred. Big expectations! I’ll blog summaries of all the sessions that I go to if you can’t make it (or can’t remember them due to your Vegas excesses). This is a stream of consciousness so apologies for the spelling mistakes and typos in advance.

Christian Chabot (CC)
8th annual customer conference
CC  highlights the customer examples where the impacts of visualisations done everyday is making the world a better place through improving medicine, monitoring manta ray movement or childcare in local councils (Leicestershire County Council example).
Data is still growing and to help people everyone needs to become a ‘data person’.
La Nacion – “people don’t believe our journalists by what they write. They believe it because they can analyse the data themselves” all using Tableau. Showing the example that data is a way to empower people. “It’s not the people in power, it the power in the people that will change South America”.

Francois Ajenstat (FA)
“Tableau helps me be creative, even though I‘m not a creative person” – Ben at a retailer
This is the kick for the devs to keep making the software better
People who know the data should ask the questions – everyday people.
Software should be designed for deeper thinking – simple design shouldn’t mean simplistic.
Analytics at can drive change – data led collaboration can only exist when the software fits the culture of the company it is used in
More to be invested in R&D in the last 2 years than in the last 10
Developers on Stage is back!!
Areas of investment:
  1.  Data 
    1. Formatted spreadsheets with multiple tables in the sheet can now split out these tables using the Data Interpretor automatically
    2. Date Wrangling - Tableau Public showed over 250 different were in use. More automatic data / time coversion as Tableau becomes more intelligent. Just change the data type to a ‘date’ or ‘date / time’ and Tableau converts funky formats
    3. Union is here!!! Hover over the second data item over the first in the data preparation screen and new rows are added. Wildcard file names allows you to union lots of files all in a couple of clicks.
    4. Cross Database Joins – Adding connections all in the data preparation screen, goodbye blending. Row level, live querying across multiple databases. Publishing the cross database joins can be published to the server
  2. Visualisation
    1. Tables – Highlight tables with totals are dominated by totals. Now able to unclick ‘show totals’ in the colour dialogue box. Totals can be moved from bottom to the top row, from the right side to the left side of the table.
    2. Interactivity – Data Highlighter – on the marks card, drop the data field on the marks card and select ‘highlight selected’ and as you search the view changed on each letter to update the scatterplot. Add a data item to the ‘Data Highlighter’ and you can hover through a list and the screen will show the update as you go
    3. Maps – More data added – postcodes added for 76 different countries (1/4 million geographic entites).
    4. Custom Territories – similar to using grouping but finds the outside of the mapping polygons as one shape.
    5. Spatial data – New data connection group – ‘Spatial File’. New field type – ‘Geometry’ that loads as custom polygons on your map.
    6. Mapbox integration – add Mapbox maps in just a few click. Can add custom layers to your maps.
    7. Inserting charts in to Tooltips – in just a couple of clicks.
  3. Analytics
    1. Outlier Detection – Select multiple fields, right click and Create ‘Outlier Set’. Simply use as you would normal sets
    2. Clustering – in analytics pane and drop it on the viz. Tableau works out the clustering and have even shown the cluster outliers. A group is created to capture the results of the cluster. Can drop additional metrics in to the clustering algorithm.
    3. Drag and drop analytics – drag the reference band to colour and drop it to change the colours of the marks only with the reference band. Can drag reference lines to filters to filter up to that point
  4. Self-service at Scale (to make all companies data driven)
    1. New home page for every single user on Tableau Server. Collects your favourites and most visited views
    2. Content Analytics – hover over the content you are interested in and it shows the consumption rates. Bar charts show content views etc
    3. Server search – Thumbnails in the search. Results are ordered by views
    4. Version Control in Tableau Server. Click on history and select the old version and it gets moved across. Tableau Server making it safe to change your mind
    5. Permissions at the Project level – can manage permission at the project, workbook or data source owners. Project permissions can locked by the owner
    6. Tableau Server web manager application – apply changes to the Configuration in this tool. Lot friendlier interface
  5. Dashboards
    1. Web authoring – Table calc dialogue box looks but more importantly you can create dashboards in the browser
    2. Global formatting – in web editor can update fonts in the entire workbook (also will be available in Desktop)
    3. Cross Database Filtering – filtering no longer related to just one data source at the highest level
    4. Device Preview – Device menu allows you to select the device size of the view and change between models of phones etc Allows you to set different views using the same dashboard
  6. Mobile
    1. 9.1 Mobile release (already available). Favourite snapshots available offline. Mobile editing is available
    2. Mobile now available for the phone – layout built for thumb navigation. Intelligent selection calculated as you click.
    3. Geolocation - Maps centred based on your location
  7. Project Elastic now called Vizable
    1. Opens data sets from your tablet as charts
    2. Swipe to filter
    3. Swipe categories to change them
    4. Pinch to zoom in to get more bar charts being available
    5. Pinch to next level of drilldown
    6. Share instantly via email
    7. Heatmap histogram is a new chart type in Vizable
    8. Can zoom in to the individual record and get the row level data back visualised as a nice data
    9. Only available on the iPad at the moment.
    10. Vizable is free and available now

Tuesday, 6 October 2015

Tableau Fringe Festival

If you didn't get a chance to catch the live session, here is the recording of my session from the inaugural Tableau Fringe Festival set up by Emily Kund and Jen Vaughan.

The overview - visualise what you love and you will find new depths (and friendships) all over the world. There's no need to be lonely!

Thursday, 1 October 2015

#WatchMeViz - The Data School does the 'Sorting Bar Chart'


In response to Jewel's crazy Viz dancing. The Data School (and Chief Data Jedi Choreographer) created the 'Sorting Bar Chart' in response. Enjoy!