Was this helpful?

(12) (0)

Are Paid Reviews Biased? Not in B2B

May 18th, 2018 9 min read

Fake reviews have become a significant issue in the B2C world. To ensure reviews are authentic and unbiased, many consumer sites have banned the use of incentives, or in the case of Yelp, even prohibited businesses or third parties from asking customers to write reviews. These moves were driven by research that showed offering rewards or free products in exchange for reviews resulted in higher ratings.

This has led to a perception among B2B buyers that the same is true for reviews of business technology.

Yet in the B2B world, the opposite is actually true—invitations with small incentives actually lead to a larger, more representative sample, a more even distribution of ratings (not just promoters and detractors), and more detailed product reviews.

Why do paid reviews in B2C and B2B look so different?

B2B is on a smaller scale

First, consumer-facing businesses enjoy exponentially larger customer bases, which translates to many more potential reviewers. Relying on a fraction of a percentage of customers to self-motivate to contribute reviews for B2C shops might produce enough quantity to be statistically meaningful.

But many B2B vendors have just a couple hundred customers. At best, they could hope for one or two self-motivated reviews per product—far below the 10 or more most buyers say they need. So vendors need to motivate a much higher percentage of their user base to participate.

B2B requires more detail

Writing a review of a consumer product is generally easier, and less time consuming, and maybe even more fun. But writing a review of a software product used by your business requires a certain level of detail and insight to be useful. That means reviewers need to have sufficient experience with the product, as well as a certain level of professional experience to understand the broader business context.

And buyers like you need detailed feedback. A software purchase decision can commonly be in the range of $50k to even $100k or $1M per year. Given the significant investment of both time and money in implementation, you definitely don’t want to make the wrong decision. Reviews are one of the best ways to access in-depth pros and cons from people like you, and the more information they can provide on their specific use case, the more helpful it will be when compared it to your own situation.

Rewards reduce participation bias in B2B reviews

For B2B technology, actively inviting users and offering them thank-you incentives is often necessary to get enough reviews, and to get reviews with enough detail. And they actually help produce less positive or biased results. If you rely on self-motivation in B2B, you’ll end up with the two extreme ends of the sentiment spectrum: the happiest customers, who are passionate and motivated to share their positive experience, and the most dissatisfied customers, who have a serious bone to pick. This is participation bias.

But B2B buyers also want to hear from the middle-of-the-road users—those who can offer balanced feedback about the product, including in-depth pros and cons. Small thank-you rewards offered for honest reviews help reduce participation bias. Employer review sites like Glassdoor experience a similar outcome as well. Unlike consumer reviews, writing reviews about work can feel like more work, and it’s no surprise that people do their best work when they get paid.

Ratings Distribution on Amazon

Source: ReviewMeta

Ratings Distribution on TrustRadius

Why do incentives lead to more positive ratings in consumer reviews, but actually lead to less positive ratings in B2B reviews? There are a few possible reasons for these opposite patterns:

  • The way these incentives are offered
  • The review process itself
  • The business model of review sites

B2B reviewers are offered thank-you rewards, not free products

For many consumer reviews, the practice is to offer free merchandise in exchange for a promise to review the item. In this case, the reviewer isn’t already a customer with an opinion about the product—rather, it is truly a purchased review. In B2B, the practice is rather different. Incentives are generally offered to existing customers, who already purchased the product to solve a real business problem and have feedback to share. The reward is presented as a thank-you for the reviewer’s time and effort sharing honest feedback.

B2B reviewers are prompted to share balanced feedback

Many consumer review sites ask for a star rating and then allow for open-text feedback about the product, leaving it to the reviewer to figure out what they want to say. B2B review sites lead the reviewer through structured questions, prompting them to think about topics like pros, cons, ROI, and use case, sometimes before providing the actual rating. This ensures reviewers have put considered thought into the product before offering up their likelihood to recommend rating.

Most B2B review sites aren’t marketplaces

Unlike many consumer review sites (most notably Amazon), many B2B review sites aren’t selling the products listed on their site. Considering the length of B2B deal cycles and the size of purchases, transactional review sites in B2B aren’t very feasible.

Thus, B2B review sites are not intrinsically motivated to influence a sale or motivate more transactions. Instead, the goal of B2B review sites is to help buyers with their product research.

As with any information source, it’s important to vet the company’s business model, and make sure they’re set up to be an objective resource. Business models vary among B2B technology review sites, but most are not transactional marketplaces. Here is an overview of TrustRadius’ business model.

But who offers the reward matters

The above ratings distribution data from TrustRadius shows the resulting sentiment when an independent third-party—TrustRadius—offers the reviewer a thank-you reward for sharing their feedback and helping our community of buyers.

But B2B technology vendors sometimes offer rewards as well, to encourage their customers to write reviews. Often the rewards themselves are similar to what TrustRadius offers (gift cards or perhaps branded items), but the results could be a little different. If vendors aren’t explicit with their customers that they’re looking for honest, in-depth feedback to help them build a better product or provide a better experience, then reviewers could feel more motivated to emphasize the positive and advocate for the vendor.

Even when vendors do solicit honest feedback, receiving a reward from the vendor can have a short-term influence over the customer’s feelings about that company and its products. Customers may be wearing rose-colored glasses while they write the review. So it’s possible for incentives from vendors to bias results to some extent.

The biggest bias to watch out for is cherry-picking

Additionally, software vendors often end up inviting only their happiest of customers to write reviews on third-party sites. These are real customers, with real feedback to offer, but the end result is selection bias. Vendors cherry-pick customers who have already rated them a 9 or 10 in internal NPS surveys, or those who they know will advocate for them and show them in a positive light. If the review site doesn’t intervene, an entire slice of customers with middle-of-the-road sentiment might be missing or underrepresented.

TrustRadius has established various practices to combat vendor-driven selection bias, including:

  • Sourcing as many reviews as possible independently from vendors
  • Clearly marking the source of each review and whether an incentive was used
  • Encouraging vendors to invite all customers
  • Introducing a score algorithm that doesn’t allow biased samples to inflate scores

But it’s an important trend to understand as you engage with TrustRadius reviews, and especially other B2B review sites.

What do you think of incentives in B2B?

We’d love to hear your feedback on this topic to inform our research! Have you encountered incentive disclosures on TrustRadius? Or other review sites? What was your reaction? Send your thoughts to research@trustradius.com.

Was this helpful?

(12) (0)

TrustRadius Weekly