Item: TIBCO Spotfire
Rating: 9
Author: Brent Meyers

Use Cases and Deployment Scope

As a field consultant I deploy TIBCO Spotfire, develop data visualization reports/dashboards/templates, and provide on-site training and mentoring. Most of my clients use Spotfire as an enterprise reporting solution. I have also helped the vast majority of those same organizations deploy ad hoc analysis environments. The ad hoc use cases are typically inter departmental analysis projects.

Almost all of my clients select TIBCO Spotfire for it's ability to easily visualize data, merge disparate data sets, scale across an expanding enterprise, and handle data loading in a variety of methods. Additionally, I work with a large number of clinical research and life sciences firms who have read about the growing number of Spotfire extensions (custom add-ins) that are geared specially to their industry. This level of customization generates a great level of interest in Spotfire.

Pros and Cons

Data transformation. Pivoting, un-pivoting, aggregation, and numerous other data transformations can be achieved through a simple "Information Link" GUI or through custom SQL. Additional transformations, joins, calculation expressions can be achieved within the client as well.
Standardized visualizations. Spotfire provides all of the typical types of visualizations (bar charts, scatter plots, tree maps, geo, dashboards, etc.) in a clean standard format that is easily recognized by users.
Strong SaaS capabilities. Strong web-player out of the box that can be embedded and manipulated in your own site.
Robust use of data structure. Spotfire provides a range of options from importing spreadsheets to connecting to existing databases. Data can be merged within Spotfire via the GUI, pushed to the database, or return select data sets based on user actions.
Simple web and client interface. Intuitive interface for users and visualization report developers. Visualization options are limited for the sake of standardization
Depth of reporting capabilities. Simple enough for a casual user to develop simple reports, but powerful enough data transformation and calculation expressions to build complex reports.
One the greatest improvements with the latest versions of Spotfire (v6 and v7) is its ability to load enormous data sets. Spotfire was already the industry leader as an in-memory data visualization tool (as well as a hybrid data on-demand method), but it now supports numerous "big-data" data connections. This allows Spotfire to push demanding data processing into the database and return only the necessary metadata to build a visualization.
Visual user interface customization has improved dramatically with the latest Spotfire versions. Version 6 allowed for much more custom HTML/CSS/Java Script to produce unique visual solutions. Version 7 took that further by allowing a GUI (point and click) to modify nearly the entire user interface. Essentially you can alter the UI to the point that Spotfire begins to look like any webpage you like.

Error handling. The data engine and expression builder have almost no organic error handling or troubleshooting capabilities. Program errors, while minimal, are vague and appear spontaneously.
Training for data transformation user interface. On-site and online training for visualization report development is good, but data engine training is minimal at best.
Flexibility of visualization design. Design options are very limited and are not as powerful as Tableau or MS Excel 2010+.
While version 7 has provided extensive flexibility in modifying the user interface, the amount of flexibility in adjusting data visualizations is still limited.
Auto-save or version control. There is no auto-save or automated version control. The client will crash sporadically and all work is lost since the last file save.
Progressiveness of visualization designs. Spotfire lags behind its competitors in the range of visualization types and the options available to manipulate the current visualizations.
Standardization of design controls. Report authoring options that you would expect to be ubiquitous are not and those that are available are sometimes found in different locations.
Resource hog! Spotfire requires a good deal of memory, processing power, and efficiently tuned data sources. Spotfire v7 is especially demanding on memory and processing power. All versions will demand efficiently tuned in database sources. Poorly tuned databases will become a bottleneck in performance.
Upgrading can be challenging if you have custom extensions or Spotfire reports using complex operations (eg. large in-memory data, large number of tabs, complex logic, multiple data sources merging, etc.)

Alternatives Considered

Spotfire is routinely selected for its data engine, its ability to merge data from various data sources, and then quickly visualize the data. Spotfire can be effectively used as an ad hoc analysis tool as well as an enterprise level reporting application.
Tableau, Qlik, and MicroStrategy seemed to have a shorter learning curve and a more appealing user interface, but none of them touched Spotfire's data capabilities.

Other Software Used

SQL Developer, VMware User Environment Manager, Amazon Web Services

Likelihood to Renew

-Easy to distribute information throughout the enterprise using the webplayer.
-Ad hoc analysis is possible throughout the enterprise using business author in the webplayer or the thick client.
-Low level of support needed by IT team. Access interfaces with LDAP and numerous other authentication methods.
-Possible to continually extend the platform with JavaScript, R scripts, HTML, and custom extensions.
-Ability to standardize data logic through pre-built queries in the Information Designer. Everyone in the enterprise is using the same logic
-Tagging and bookmarking data allows for quick sharing of insights.
-Integration with numerous data sources... flat files, data bases, big data, images, etc.
-Much improved mapping capability. Also includes the ability to apply data points over any image.

Breadth of Deployment

In previous roles as well as my current vendor consultation role, I have seen Spotfire start in one business unit and expand to the entire company. Operations, research, IT, accounting, finance, and executives all end up performing ad hoc analyses or reviewing reports through Spotfire. Typically their are analysts or developers in each business unit or group that build Spotfire analyses. Decision makers typically review analyses through the webplayer. The ability to allow everyone in an organization to review visualizations is very simple through the webplayer.

Data Integration

Spotfire's data engine is its number one asset. It allows the ability to load and merge data from flat files, data bases, big data sources, and images. For example, a user can build a data table in Spotfire that comprises of data from an excel file, sas7bdat file, a query from a MS SQL data base, and a query from a Oracle data base.
Spotfire also has multiple methods to load data into an analysis. Data can be loaded into memory, fetched from a database on demand based on a user's actions, or push all of data integration into an external data source.
Using Spotfire's Information Designer allows non-technical users to build their own queries against data bases. It also allows for central management of queries. In other words, the data and logic being used across all analyses can be centrally managed, thereby standardizing your results.

Collaboration and Sharing

Spotfire's webplayer is a superb tool to share analyses across an enterprise. Security and access can be seamlessly integrated with most of the common authentication tools. Bookmarks are a helpful functionality that allow users to capture a particular view or data set that they want to be able to revisit or share at the click of a button.
Additionally, the webplayer can be integrated into existing web portals. The webplayer can be further configured to interact with a web portal.
The business author license in the Spotfire webplayer grants limited ability for webplayer uses to edit analyses.

Return on Investment

External and internal revenue growth. SaaS program has been developed using Spotfire. Internally, any employee has the ability to quickly develop reports by tapping into enormous data sources.
Rapidness of reporting vast amounts of data. Reports are easy to construct, publish, share, and update.

Likelihood to Recommend

Fantastic data reporting, discovery, and analytics tool! Basic report development and sharing is simple and clean in design. Powerful and elegant visualization reports are possible, but require some time to learn the client and produce efficient data views within your databases.
The greatest strength of Spotfire is its ability to merge data from numerous data sources and easily visualize said data. Additionally, Spotfire allows several methods to load data to achieve optimal performance while also being able to analyze across data sets.
Don't like the stock visualizations in Spotfire? Spotfire now has the capability to display nearly any JavaScript visualization and connect it with your data. You can interact with the visualizations in the same manner as the organic Spotfire visualizations. On top of this, text areas within a Spotfire analysis can be used to curate your data and add further interactivity.

Users and Roles

994900 - Primarily clinical project management and executive oversight. Clinical research, IT, supply chain, finance, and labs are also consistent users. The majority of these users are consumers through Spotfire's webplayer. We have a consolidated, dedicated development team to produce enterprise dashboards and reports. Ad hoc development is broken down into each business function with a central support team run through the development group.

Support Headcount Required

359623 -

Administrative functions can be learned quickly with initial training from Tibco or existing experienced staff. Depending on your organization's development process you will need to closely coordinate the migration of development reports and dashboards with the development team.

Spotfire development can be accomplished by anyone who wants to use the tool. Initial use can become proficient in a few weeks. Advanced use will take at least six months. Typically your MS Excel power users will learn Spotfire quickly. The creative aspect of data visualization comes with experience. Recommend reading some Edward Tufte, Stephen Few, or any other the other data viz gurus. Spotfire v7 makes it easier to select appropriate data vizes with its Recommendations engine, but it won't build a properly flowing dashboard/report.

Future Planned Uses

Mobile reporting. The current Spotfire mobile solution is half-baked and the existing Spotfire data visualizations do not represent well on mobile devices. The data visualizations display satisfactorily on tablets.

Business Processes Supported

SaaS / enterprise product. Spotfire is used to visualize data and then published over the webplayer. This allows any user to see their clinical trials' progress in near real time.
Internal reporting. Any employee can tap into any formal or ah hoc data source within the company and quickly develop their own visualization reports. Spotfire has seen use in nearly all facets of the company.
Standardize reporting throughout the company. Much quicker than Excel and much easier to produce clean, standardized reports.

Innovative Uses

We built a SaaS program that allows internal and external clients to dynamically load their specific data and monitor the progress of their clinical trials.
Pre-filtering capability allows users to jump straight to their pertinent data set. This also helps manage load times when dealing with massive data sets.
Changes the way that status meetings are held. A proficient user can pull in live data and answer questions as they are asked. Updating a report to a client's question as they are asked builds trust and engagement.

Products Replaced

First dedicated data visualization program within the company. The company previously (and currently for other projects) used OBIEE for data reporting.
Previously I developed reports in MS Excel with VBA, while using Tableau and Minitab for academic/side-work purposes.

Implementation Rating

Not involved with the implementation.

Implementation Details / Implementation Partner

Vendor implemented
Implemented in-house
Professional services company

Tibco was on-site for the initial system implementation and for the first series of visualization report development. A third party consultant assisted in visualization design review.

Training Types Used

Online training
In-person training
Self-taught

In-Person Training

We've repeatedly subscribed to a two-day in-person training course that focuses on the visualization aspects of Spotfire. While the trainers are reluctant to deviate from the curriculum it has generally received good reviews in helping users venture into the basic aspects of Spotfire. This course does not touch Spotfire's data engine or expressions used with visualizations. We've also had a one-day "ad-hoc" session that focused on Spotfire's data engine. This format was limited to a small group and tailored to dedicated developers. While this was a great course, it is one of the only ways to understand Spotfire's data engine other than trial and error.

Online Training

We subscribed to the 8-week "Report Author Bootcamp." This is sufficient to familiarize one with the visualization side of Spotfire, but not the data engine side of Spotfire. At the time that I started using Spotfire there was not a "data engine" training course. We learned this side of Spotfire primarily through trail and error.

Ease of TIBCO Spotfire Training

The basic visualizations within Spotfire use a drag and drop GUI that can be mastered by anyone. Anyone with a moderate level of MS Excel and/or SQL experience will easily grasp the data expressions used in the visualizations. Mastering the data engine requires an understanding of basic data structure and basic SQL. Developing complex reports that are efficient requires creativity and a high level of persistence that borders on stubbornness. I recommend that a visualization developer take the "boot camp" online course and sit down with Tibco for one or two ad-hoc courses on the data engine.

Support Rating

Tibco is very responsive to each of our questions. They are quick to jump on webex and troubleshoot issues with techs and/or receive files to analyze on their own. On occasion, Tibco has pushed back on their willingness to recognize certain issues that we have located.

Usability

Out of the box, Spotfire is easily usable for simple report development and display in its web-player. Within 3-6 months complex reports can be built by dedicated developers. After 8-12 months, highly efficient, quick, and extraordinary reports can be developed by a developer. Significant work with your database team is required to produce quick reports that use a data connection with large data sets.

Reliability and Availability

The Spotfire client is very reliable. There are sporadic crashes that are made worse due to the lack of an auto-save. Most product upgrades are uneventful. The one major issue we experienced was quickly responded to by Tibco. Our system admins have indicated that the Spotfire servers and the ancillary web-player systems are reliable, but requires time to learn how to deal with the unique nature of the web player and resource loading.

Performance

The client on its own is impressively quick with rendering numerous visualizations, loading large amounts of data, and joining several data tables. Performance begins to degrade with the use certain data transformations (eg. pivoting data within the client), calculation expressions (eg. "Over" expressions), and large data aggregations within the Spotfire data engine. We achieved significant performance gains by having our data teams aggregate data at within our data views (before it reaches Spotfire), employing data-on-demand (user initiated data queries in a report), and by employing data limiting options within visualization reports.

Systems Integrated With

Integrated into a portal environment to support SaaS product

Not involved with the integration.

Upgrade Process

Yes - I've implemented or participated in 15+ upgrades, each one with zero down time. The most common challenges are compatibility issues with custom extensions, extensions or files built in significantly older versions of Spotfire (jumping 2 versions or .NET versions), and unique authentication configurations. Text areas and bookmarks can also pose problems when jumping into Spotfire v7.0+.
Highly recommend employing the Spotfire Qualification Tool and Application Profiling Tool to test visual/data differences between Spotfire files and performance.

Robust data engine & standardized visualizations

Overall Satisfaction with TIBCO Spotfire Platform