A Conversation with Five IT Executives: User Reviews as a New Strategic Intelligence Tool

Traditionally, industry analysts like Gartner and Forrester have played a central role in informing IT purchasing decisions. However, they tend to focus on larger, better known vendors and have very limited coverage of emerging or open source solutions. Moreover, the primary data sources for researchers are conversations with vendors, augmented by customer surveys and calls (the audience is often directed by vendors) and occasionally lab tests. Few analysts have hands-on experience with every product, nor do they work with the products on an ongoing basis to experience customer support, upgrades, test integrations, etc., so coverage can lack usage details highly relevant to buyers.

For this reason, IT executives find that conversations with peers are often vital to securing unfiltered, direct knowledge of a product’s capabilities, and any hidden issues like difficulty of implementation, integration with other products, or customer support experience. This kind of information can be invaluable to help formulate areas of concern to drill into during vendor calls, or as part of an RFP process.

The challenge with peer conversations is scalability. Most IT executives have limited personal networks so it is challenging to talk to enough people who have used/evaluated the same set of products recently, and who operate in the same business context, i.e. have the same use case. A more scalable methodology is needed to drive an efficient technology acquisition process. Judicious changes or additions to the corporate technology stack, from underlying infrastructure software to the business applications that run on it, can make a significant difference to how well the business performs and to its long-term survival.

Making sure that the right technology is in place to keep the business humming efficiently requires constant vigilance and significant research to understand which products should be retired, and which new technologies should be purchased.

This is where user review platforms like TrustRadius can make a significant difference. While the utility of customer reviews has been well understood in the B2C world for many years, it has started to gain traction in the B2B world, and many IT executives are also turning to user review platforms to harvest the quantity of unfiltered user insights required to make an informed purchase decision.

To better understand the utility of reviews to senior IT executives, I talked to five of them; Four of them have used reviews during the software acquisition process.

The executives interviewed for this article are Sean Warren, Director of IT at Domo (Business Intelligence software, 501-1,000 employees); Everett Plante, CIO Silicon Labs (Semiconductors, 1,001-5,000 employees); Rich Robinson, VP Digital Transformation, SunPower Corporation (Renewables and Environment, 5,001-10,000 employees); Meerah Rajavel, CIO Forcepoint (Security Software, 1,001-5,000 employees); and Kevin Monagle, Partner QBA Corporation (Management Consulting, 11-50 employees).

I asked each to explain the ways in which technology reviews have been helpful. Five common value propositions emerged.

1. Helps you build a shortlist.

All five executives said that they use TrustRadius to help build a shortlist of relevant products when they have a specific business problem to solve, or when they have decided to invest in a product in a specific software category. Rich Robinson, VP Digital Transformation, SunPower Corporation, puts it like this:

“When I don’t have experience in a certain area, I look to peers that I respect. For example, I’m a member of a couple of CIO groups, and there are lots of people I respect who I have worked with, but the range of personal contact is limited. TrustRadius is just a scaled way of providing that information and the data behind it…Making this available in a scalable way. I thought it was brilliant at inception and I still think it’s a brilliant service.”

Meerah Rajavel, CIO Forcepoint, also thinks of TrustRadius as a crucial tool for developing a shortlist:

“We look at TrustRadius to help build a shortlist to figure out who we want to do an RFI or RFP with. That is the preparation homework that I get my team to do well before we get to RFP. It’s very important to understand the market and who the key players are.”

Tip: Look at TrustRadius category pages and TrustMaps to see the key players in a given software category. For example, here is the business intelligence category page, and the marketing automation category page.

2. No Spin, Unfiltered Feedback

TrustRadius reviews represent the unfiltered voice of the vendor’s customers, rather than vendor claims for the product. User insights are unmediated content that is highly trusted by product evaluators. Kevin Monagle, Partner QBA Corporation, is primarily interested in understanding which product claims made by the vendor are accurate and which are just hype.

“The great thing about TrustRadius data is that it is unfiltered. I want plain user feedback which has not been vetted by the vendor to help me distinguish fact from fiction or opinion. Analyst data is generally not unfiltered, but can reflect vendor spin and analyst opinions.”

Sean Warren, Director of IT at Domo concurs:

Tip: Use customer reviews as a source of ammunition to ask vendors probing questions about specific shortcomings, or to manage internal expectations about things like resources needed or time for implementation.

3. Identifying emerging products/capabilities

Information about available technology in specific categories tends to be backward looking. Many technical executives feel that analyst 2X2 grids in any given software category, like the Gartner Magic Quadrant, tend to feature the usual suspects—products that have dominated a category in the past. Analysts tend to be less helpful though in revealing emerging products based, perhaps, on a different technology, or taking a different approach to solving a business problem.

For Meerah Rajavel, CIO Forcepoint, this is one significant advantage of TrustRadius over traditional analyst services:

“The thing I like with TrustRadius is that it provides an opportunity to learn about the newcomers rather than the traditional big guns. Because insight is provided by users you get a broad focus—you get to see who are the up-and-comers, who are the innovative new players solving problems in a new way that fits our business strategy…One of the main reasons I like looking at the site, is because I have not even heard of some of the vendors and products mentioned!”

This innovation lens is an appropriate one for a more senior executive:

“My team does a lot of the detail work, looking at reviews in detail. I’m not doing that. For me it’s all about watching for trends; who is showing up in a category, and how they are they showing up.”

Even when looking at better-known incumbent products, it’s not always clear what these products are capable of today, rather than two years ago. Innovation does not just come from emerging products and platforms, but from incumbents too. The speed of product development has never been so rapid, and incumbent vendors are frequently adding new capabilities, taking advantage of open-source components which can significantly change the perceived value of these products. Kevin Monagle is familiar with a lot of software, but:

“Even if you now the software inside and out, you don’t necessarily know what it’s capable of at this moment. TrustRadius helps us come up to speed with the core parts of the software today. We may know the basic underlying components, but not what has been added recently, which may make a huge difference.”

Tip: When researching a specific software category, do not focus exclusively on the established players with which you are already familiar, but consider products that are emerging and may take a disruptive approach.

4. Insights from many types of users

Contrary to what some vendors might believe, IT executives are not only interested in review insights from other IT executives. Vendor materials and even product demos can be useful in helping to understand capabilities, but, if the product is difficult to implement or has poor support, it may still be a poor choice. TrustRadius uncovers these important details. Meerah Rajavel, CIO Forcepoint, argues that:

“Sometimes vendor claims make a product sound very grandiose and promising, but it often comes down to execution and how well a product solved a business problem in practice. I want to see a 360-degree view of the product from many different perspectives—from a technical executive, to an engineer, to a middle manager, to a business person tasked with solving a specific problem.”

Rich Robinson, VP digital Transformation, SunPower Corporation, also feels that people in different roles see products from a specific perspective, and that a multiplicity of roles is needed to get a completely rounded view.

“I don’t want to exclusively hear just CIO views. I like to hear from experts, but I also like to know what hat they are wearing when they write their review. Reading a review by a head of security versus a head of operations gives a very different perspective and it’s very valuable to understand that.”

Rich feels that during product evaluations, usability is often missed by IT executives, although it is a critical factor in achieving success with a new technology. CIOs might be more concerned by things like technical capabilities and compatibility with the existing IT stack than how easy it is for staff to actually use the product:

“I’m a huge believer in usability as a main function, so getting reviews outside of IT is very important. CIOs tend to have group-think and it’s often more valuable to see what people outside of the IT function think.”

The great advantage of a platform like TrustRadius is that it provides product insight from a plethora of different perspectives, giving a much more rounded view of a product, and a much better basis for decision-making.

Tip: Use role-based filters to find and read reviews from a broad variety of user types. For example, executive-level reviews will often not provide much usage detail, but may provide a wealth of information about why a specific product was selected. End users who are in the product every day can provide crucial insights into usability and support.

5. Independent reference checks

Software vendor reference checking is challenging for buyers and vendors. Vendors often struggle to line up happy reference customers in various industries who are willing to go on the record and talk directly to evaluators or worry about fatiguing references. On the other side of the equation, evaluators do not ascribe much value to references supplied by the vendor. These managed conversations are sometimes non-representative, and don’t provide much real insight.

Rich Robinson, VP Digital Transformation, SunPower Corporation, is passionate on this topic:

“Will a vendor even provide you with a reference? If they do, well that tells you something, and it’s a way of checking the reference box. But what you really want, is to talk to a reference that the vendor did not give you. That is the most powerful reference because it’s spontaneous and authentic. TrustRadius makes this much easier because, normally I would have to network by contacting a CIO that I know personally and maneuvering to the right person, which can take some time. On TrustRadius I can just contact someone who has written a review that is relevant to my concerns, and talk to them.”

Sean Warren, Director of IT at Domo agrees:

“Talking to reviewers is extremely helpful, because I can choose reviewers who match my use case, and drill down to the issues important to me. And I know they are giving me an unvarnished view.”

Everett Plante, CIO Silicon Labs, typically talks to peers when evaluating a software platform, using a CIO discussion platform provided by Gartner. He has not, so far, talked to reviewers, but felt that doing so would be helpful.

Sometimes CIOs have questions which are more strategic than product-specific, but where TrustRadius reviewers can also be very helpful. To give one simple example, if someone writes a review of Hadoop, and says that it has become their repository for all corporate data, replacing corporate data warehouses, a conversation with the reviewer on the pros and cons of this approach can be invaluable.

Tip: Select some relevant reviewers with whom to make direct contact. A short discussion with a knowledgeable reviewer can yield much better dividends that a conversation with a vendor-supplied reference.