Some additional details on the Tableau project I worked on at Southern California Edison (SCE):
During implementation, I was heavily involved with the Tableau consultants, stakeholders, and project managers to make sure that all requirements were met. I did testing myself and signed off on testing that others did. I provided my manager with constant feedback on our progress. We were able to get Tableau installed in one month based on a very aggressive deadline set by management. During this phase, I spent countless hours side by side with Tableau consultants to describe the performance issues we were facing at the time. Here are two of the main hurdles we faced during implementation: One of the main issues we faced was the use of Custom SQL to build our visualizations. Because the roles within my team were highly segmented, we preferred using Custom SQL vs. connecting live to the data. The reason for this is because I was tasked with building the visual. I was not tasked with building the query. The DBA I worked with would hand me the query and I would build the visual from there (creating the necessary calculated fields and formatting the data). Tableau advises its clients against the use of Custom SQL for performance reasons. In addition to performance, there are other drawbacks. We would notice that certain functions in the SQL statements themselves would not get translated correctly into Tableau’s Native SQL (there are workarounds for this). This issue was starkly apparent when connecting to Teradata. Towards the end of my role at SCE, I was working daily on improving the connection to Teradata. Before I left, I was able to take certain reports built in Business Objects and build them in Tableau (but this was only accomplished when connecting live to the data). Another issue was refreshing certain dashboards that were meant to run 24/7. Tableau has the capability to refresh the data behind the visual as frequently as every 5 minutes (I believe). But that refresh does not update the visual. In order for the refreshed query to update the visual, you are required to force the browser to update on an interval of your choosing. It is only with this combination that you can come close to seeing what is happening in real time. The workaround for this issue was to create an .html file that had the URL of the dashboard, which was published to Tableau server embedded in it. From here, you simply double click on the .html file and you would have your dashboard updating both the data (handled by Tableau Server) and the visual (handled by the browser).
Here is a list of the data sources that I would connect to when building visualizations:
1. MS SQL Server
2. Oracle
3. MS Excel
4. Teradata
5. MS Access
Data Blending – There were several instances where business requirements made it necessary to join data from two separate data sources (i.e. - MS SQL Server and Oracle or MS SQL Server and MS Excel). Tableau has an intuitive capability to recognize a data blend. This will be indicated by the use of the link icon next to the primary data source (it looks like an infinity sign). The ability to blend data together was very appealing in the conceptual sense. But there are limitations on the number of rows that you can blend together (I noticed a significant deterioration in performance after 100k rows). This seemed to always be our problem at the SmartConnect Operations Center. We would relentlessly push Tableau’s performance only to find out that our local machine didn’t have the RAM necessary to do what we wanted, or that Tableau was not intended for that specific application. Regardless, I learned a tremendous amount about how the application behaves in different scenarios.
Incremental Refresh – Sometimes it can be useful to use a Tableau generated extract. This is essentially a mini-Tableau database. My manager tasked me with building a report that would store historical data. The purpose of this report was to keep a running total of certain events in the network. The benefit of this report would be that it would automate a manual process that the end users were responsible for on a daily basis. Prior to the implementation of this new report, the end users were tracking the historical data using MS Excel. The challenge I faced was to find a way to store data from the daily query results. Because the original data source did not store historical data, it was necessary for me to take advantage of the “Incremental refresh” option available in Tableau. I do not advocate this method if the daily query results are on the scale of several hundred records or greater. This solution was feasible because the number of records were less than 100 per day. In addition, I worked with the end users and their team lead to create a backup method to get to the final row count in the instance that the Incremental Refresh option failed.
Experience with Teradata – SCE’s 5 million meter network produces around 15 million records per day in just one its tables. The data warehouse team built a set of views that could be accessed within the Teradata schema. After several attempts to connect to Teradata (unsuccessfully) using Custom SQL, I worked with both Tableau’s technical staff and SCE’s DW team to diagnose the problem. We spent multiple WebEx sessions documenting application behavior under various scenarios. Initially, my thoughts were that the problem was that the client application was trying to connect to a View versus a hard coded table. But Tableau assured me that the client application has no problem connecting to views. In this instance, the workaround was to connect live to the data. This worked on certain tables (with several joins in place) but not all. Finally, I was able to re-build a few reports taken from Business Objects. In order to do this, I extracted the SQL from the BO reports and then connected live to the same tables. The result was a visual that came very close to what was found in BO. The report reflected data that was aggregated in the form of a percentage. The percentage represented the performance of a particular district over a time interval (i.e. – 98.99% over a 3 day period).
Integration with ArcMap – The SmartConnect Operations Center’s business requirements made it necessary for me to find ways to enhance the mapping capability within Tableau. The default background map that is offered by Tableau falls short in the areas of zooming and satellite imagery rendering. Therefore, Tableau offers its users the ability to connect to a geospatial (WMS) server. To Tableau’s credit, the steps and information needed to connect to a WMS server are very simple. If you have the URL of the WMS, then you can connect to the server. In my experience, the biggest problem with using the layers from various WMS servers is in the rendering of the image. If your application requires a certain level of detail and/or the ability to zoom in to a certain extent, you may find that the static image stops rendering at a certain zoom level. In the SOC’s application, it was necessary to validate the latitude and longitude of meters in the field and their proximity to other meters (aka – the “mesh” network in wireless meter terminology) by visual inspection. When I left the SOC, I was in high-level discussions between Tableau’s technical team and SCE’s GIS team regarding the rendering of the different layers available through SCE’s internal WMS servers.
Comments
Please log in to join the conversation