About TrustRadius Scoring — Frequently Asked Questions

This FAQ covers the methodologies and algorithms behind TrustRadius scores and visuals, all of which are designed to create the most accurate, data-based picture of products in the marketplace.

What is trScore?

trScore is an algorithm that calculates a product’s scores based on a weighted average of reviews and ratings, rather than a simple average.

The problem: Simple averages can be misleading

Most review sites use simple averages to determine a product’s score or overall star rating. At first, TrustRadius did as well, until we noticed a couple of problems with simple averages:

  1. Over time, simple averages don’t always tell the most up-to-date story. Products can change quickly, and for products that have a large review base with a significant number of older reviews, a score based on a simple average won’t reflect current customer sentiment.
  2. Simple averages allow vendors to artificially inflate their product’s score by inviting only known promoters to write a review. Happy customers are valid users and reviewers, but when they are over-represented in a product’s review base, a score based on a simple average will be biased.

The solution: A weighted average called trScore

Our trScore algorithm weights ratings and reviews differently in the following ways, to provide a more reliable score than a simple average:

  1. More recent reviews and ratings are weighted more heavily than older ones.
  2. Reviews from a random, representative sample of customers are weighted more heavily than biased samples.
  3. Ratings given as part of an in-depth review are weighted more heavily than single-click star ratings, since they are more considered and include more context.

All average scores on TrustRadius are calculated using the trScore algorithm, including the overall Likelihood to Recommend score, scores for attributes like usability and support, and scores for specific features like dashboarding and reporting.

The trScore algorithm allows us to present the most accurate and objective picture of customer sentiment to buyers, as well as a level playing field for vendors.

What are TrustMaps?

TrustMaps are two-dimensional charts that help you compare software products in a particular category based on authenticated, end-user data. You might liken TrustMaps to the Gartner Magic Quadrant or the Forrester Wave, with some key differences:

  • TrustMaps are based purely on user data rather than analyst investigation.
  • TrustMaps are dynamically updated as new data comes in.
  • TrustMaps can be filtered based on the parameters that are important to you (specifically, company size).
  • TrustMaps are backed by in-depth, quality review content, so you can drill down to understand the context behind the numbers.
  • A product’s inclusion on the map does not depend on company size, revenue, popularity or status as a TrustRadius customer. Any product that fits the category and has the required number of ratings and reviews (to ensure sufficient data) will appear on the map.
  • Unlike other review sites, once a product clears the threshold for inclusion, its position on the map is not tied to the number of reviews it has on TrustRadius.

What data goes into a TrustMap?

Products are plotted on the TrustMap based on two dimensions: the overall star rating (trScore) on a 10-point scale, and research frequency by prospective buyers measured by pageviews on TrustRadius.

The Y-axis indicates customer satisfaction, and is supported by the hundreds of thousands of authenticated, user ratings and reviews on TrustRadius.

The X-axis indicates market share and product momentum. Each month hundreds of thousands of B2B technology buyers research software products on TrustRadius to inform their decisions. Thus the amount of website traffic a product receives relative to its competitors is a good proxy for buyer interest.

There is a third dimension to the TrustMaps as well – the data can be filtered based on company size, to show how products compare in different market segments. This is important because a small business and an enterprise have very different needs; i.e., a software product might be highly rated by enterprise users but too complex or overkill for small businesses. You can filter TrustMaps so that they reflect the most relevant feedback for you.

How does a product get on the TrustMap?

A few factors go into this, all of which are geared toward ensuring TrustMaps are useful resources for buyers:

  • Category - A product must fit the particular category. In addition to external research, TrustRadius researchers will often speak with vendors and experts to arrive at a meaningful set of parameters or requirements for a category. The category overview page (below the TrustMaps) defines these parameters. In some cases, a software product fits multiple categories and can show up on multiple TrustMaps.
  • Data - A product must meet the data threshold to appear on the TrustMap. In many categories, the threshold is a least 10 ratings and reviews overall, and at least 5 in a particular company-size segment. In categories where TrustRadius has collected far more data, the threshold is higher.
  • Products per Category - A TrustMap with one or two products is not very meaningful, so not all have software categories have TrustMaps (yet!). There must be a sufficient number of products that meet the threshold for the map to be published.

What is Customer Verified?

Products with the Customer Verified badge have at least 10 reviews from the past 12 months. This standard was selected based on buyer feedback about how many recent reviews they usually need to get the information they need to make a more confident decision.

Customer Verified status can be earned or lost at any time. When a product is Customer Verified, that means you'll find recent, in-depth feedback from real users about the product.

What does it mean when a product is Top Rated?

The TrustRadius Top Rated awards are based entirely on customer feedback via reviews, and have never been influenced by analyst opinion or status as a TrustRadius customer.

TrustRadius issues Top Rated badges to products in the top tier of their categories. To earn the award in a category, a product must have at least 10 recent reviews, be present on the TrustMap where the TrustMap is present, earn traffic for that category, and have a trScore of at least 7.5. Vendors can use the Top Rated badge on their own sites, in a press release, on a blog, in email, or on social media. The award allows vendors to share social proof with their prospects based on the authentic voice of real users.

What determines the order in which products and reviews are listed?

As a website with tens of thousands of products and services, organized into hundreds of categories, with hundreds of thousands of ratings and reviews, how items are sorted is important.

Our guiding principles for sorting are:

  1. Our site should be as useful to our readers as possible
  2. Sorting logic should be objective and transparent
  3. Sorting can’t be artificially influenced by vendors

Products in a category

The default sort order of products in a category is by number of reviews in descending order -- Products with the most reviews are displayed first. Since the most widely used products are often the most reviewed, this tends to be a useful sort-order for buyers, and has the added benefit of being easy to understand.

You can also change the sort order based on your needs: date, highest overall rating, and lowest overall rating are other options.

Reviews on a product page

The default sort order of reviews on a product page is called “Relevance.” It’s a simple algorithm that prioritizes reviews based on recency, attribution (whether the reviewer chose to be public or anonymous), and level of detail or quality of the review. The “Relevance” algorithm is designed to put the most helpful reviews at the top of the page, by prioritizing the factors that our readers have told us matter most.

You can also change the sort order based on your needs: date, highest overall rating, and lowest overall rating are other options.

Alternatives and suggested comparisons

Two of the main reasons B2B technology buyers visit TrustRadius are to discover alternative products and to compare products. Every product page on TrustRadius offers a list of alternatives and suggested comparisons.

These products are determined based on site traffic patterns on TrustRadius and Google search patterns (e.g. searches for Product X vs. Product Y). Products that are most often compared are suggested as alternatives and comparisons on each other’s pages.