The Anatomy of a Fake Review

Where there’s a rule, there’s a loophole. Where there’s a security feature, there’s someone to hack it. Where there’s a system, there’s someone figuring out how to game it.

So, once online reviews revolutionized how we buy stuff—allowing us to use the real experiences of others to decide which vacuum cleaner to buy, which hotel to stay at, which doctor to consult, which cruise to take, which book to read—that system got gamed too.

Amazon is the quintessential example, where a slew of fake review schemes have led to a system where even legitimate sellers feel they need to buy fake reviews to stay competitive. And the latest news—that the Elon Musk-based AI nonprofit OpenAI has created an AI tool that can generate fake text—doesn’t bode well for the consumer online review industry (though it’s not releasing the code for fear of misuse).

What about business software reviews?

Our process ensures that they don’t get published, but we see (and reject!) our share of fake reviews submitted to TrustRadius too. (And I feel confident our review readers would spot an AI-generated review off the bat.) But the motivation is different. In the consumer world, it’s the sellers buying fake reviews to drive up their ratings. It’s extremely rare to see a software vendor try to get fake reviews on TrustRadius, however.

When it comes to business software, it’s not all about ratings. With a complex (and high stakes) purchase like business software, buyers are looking more at the qualitative feedback in a review than the headline rating. They want to know things like: “Does this CRM work for a company of my size? How long will it take to implement? What are the product limitations, so I’m not surprised once I sign the deal?”

You can’t answer those questions without reading reviews in detail. And you can’t answer those questions without detailed reviews!

Since business software reviews need to be detailed, and take more than a few minutes to write, software review sites often offer small thank-you gifts to increase response rates—something in the range of $10 to $25 gift cards. (Sidenote: this practice of incentivizing reviews actually leads to less biased results.)

Follow the money

And therein lies the motivation for fake software reviewers. It’s not sellers trying to drive up their scores. It’s individuals, sitting on their couches, creating a bunch of fake profiles to rake in the small dollar gift cards.

(I don’t really know where they’re sitting, but that’s how I imagine it.)

Five secrets to uncovering fake software reviews

If you’re choosing software for your company—or making a recommendation to your boss—you definitely don’t want to base your decision on fake reviews—from individuals who have never touched the product they’re reviewing, no less.

There’s an art and a science to finding fake reviews. TrustRadius employs a proprietary set of tools and a team of sleuths to snuff out fake reviews before they even show up on our site. Last year, 11% of reviews got rejected on suspicion of being inauthentic.

But if you’re reading software reviews on other sites, here are five signs you might be reading a fake review.

1. Ariana Grande reviews Google Drive

Stolen profile picture = probably a fake review.

My personal favorite at TrustRadius was when someone used David Boreanaz as their profile picture—perhaps forgetting how many Buffy fans there are out there? But we’ve seen profile pics stolen from modeling sites, wedding photography portfolios, famous investors, and even a teen from an anti-drug campaign.

2. Marketing Director for Pied Piper, Inc reviews Mailchimp

Sham company = probably a fake review.

If you’re not sure about the person, you can check out the company they work for. The review site should name the company where they used the product, or you can find their profile on LinkedIn. In the case of a fake reviewer, often the company they work for doesn’t really exist, or, upon visiting their website, seems like a sham. Visit the company’s website, or google the company name to see if there’s any news about it. (Although, to be fair, ‘middle-out compression’ is the kind of jargon you can find on a lot of real company websites even though our research shows that buyers hate it).

3. Surgeon reviews Data Center Virtualization software

Mismatched role = probably a fake review.

Look at the reviewer’s title. Does it make sense that someone in that role would use the product? At TrustRadius, we’ve seen everything from a chef reviewing identity management software, to wait staff reviewing Oracle databases.

Certainly some people wear multiple hats—but there’s a certain diversity of hat wearing that’s simply unbelievable. But if the role isn’t a clear nail in the coffin, the biggest clue to look at is the content in the review itself.

4. “Top level management decided to purchase this product to improve business agility and aid digital transformation”

No detail = probably a fake review.

Remember: fake software reviewers are trying to get their reviews published so they can earn gift cards. In that vein, they want to slip under the radar. Blend in. So their reviews tend to be middle of the road. Generally, the sentiment is positive, but there is a conspicuous lack of detail, since they don’t actually have anything real to share.

So instead of answering the questions with concrete detail (at the risk of being nonsensical), they write something generic. Great places to look are use case questions, cons and ROI—these are areas where authentic reviewers will generally provide an answer with details that are specific to their scenario.

Fake reviewers, when asked to describe how their organization uses such-and-such contact center software, might say something like: “ProductX is a unique, multi-channel customer engagement platform that is simple to use, flexible in adapting to your changing organization, and available in various deployment modes from on-premise to in the cloud.”

It *almost* reads like marketing speak! (In fact, sometimes fake reviewers will pull their text straight from the vendor’s own website.)

Another clue is if the review, like a horoscope, reads like it could be describing just about any product out there. For example:

Pros: Ease of use; Customer support; Fast implementation

Cons: Better documentation; Prettier interface; Too expensive

5. Reviews Amazon Relational Database… actually talking about Amazon Prime

Ok, you could argue that this isn’t really a *fake* review—but it’s definitely a sign that the feedback isn’t useful to you. We see genuine mistakes on TrustRadius too—for example, people reviewing Salesforce App Cloud, the PaaS solution, who are really talking about the CRM Salesforce.com. Our sleuths do their best to switch those reviews over to the right spot.

But there’s a certain level of ridiculousness if there’s a review that’s talking about something that’s not even a software product at all, like this review of Amazon Relational Database (which is, obviously, a relational database) on another review site:

What can the review site do?

In addition to vetting every single review before it gets published, TrustRadius has a few signs we watch for ourselves on the backend.

1. Patterns

It’s all about the patterns! We dig under the hood on everything from IP address, to geography, to products reviewed to surface patterns that belie potential strings of fake reviewers.

2. Unnatural passion for gift cards

It’s totally normal for a reviewer to ask about their promised gift card, if it slips through the cracks. If a reviewer misses their reward, we want them to reach out and ask about it, to ensure a good experience!

But a reviewer who resorts to actual death threats about gift cards? (This actually happens!) Not the profile of the typical professional who wants to share detailed feedback on the products they use—and the thank-you gift just nudges them along rather than being their primary focus.

We recently had one reviewer even post a fake Glassdoor review after we didn’t issue gift cards for his reviews that were clearly inauthentic. Glassdoor—which has its own practices to sniff out fake reviews!—took the review down.

3. Quality over quantity

When it comes to complex software purchases, and figuring out how products in a category are really different from each other, detailed, trustworthy insights are what you need. Yes, you want a variety of opinions, but when push comes to shove, a few high quality reviews are more useful than thousands of generic ones. So that’s been our focus from the beginning.

4. Bias is more common than fakes

What’s more common than fake software reviews, are biased ones. These reviews are from authentic users of the product, with real feedback to share—but with a sampling bias, generally due to cherry-picking by the vendor. You can read about this practice—and what TrustRadius does to minimize it—here.

Have you stumbled across fake software reviews? What do you look for? We’d love to hear from you!