About unrepresentative reviewers and biased feedback

This page is part of a global project to create a better online reviews system. If you want to know more or give your feedback, write at [email protected] and we’ll grab a beer ;)

There are two main types of reviews: organic reviews and invited reviews. Organic reviews are voluntarily left by customers who visit a public reviews platform, often through a search engine. Invited reviews, on the other hand, are solicited by the company, usually via email. The barrier to leaving an organic review is higher because the customer must first decide to leave a review, then connect to the platform and find the company. In contrast, invited reviews simply require clicking a provided link.

As discussed in “Why do we leave online reviews”, emotion is a strong motivator for people to leave a review, particularly when their experience deviates significantly from expectations. Typically, this deviation is negative. Therefore, customers driven by strong emotions are more likely to leave organic reviews, leading to a trend where organic reviews are often more negative than the overall customer experience. Invited reviews, having a lower barrier, tend to be more balanced.

Ever wondered why some businesses have a terrible score on Trustpilot? Often, this is because they haven’t set up their account on Trustpilot or have exceeded the limit of the free plan. As a result, most of their reviews are organic and skew negative. To compete effectively, these businesses are left with a stark choice: subscribe and pay, or lose potential customers to competitors.

Because the mix of organic and invited reviews is unknown, and the profile of reviewers is unclear, the set of reviewers is not a fair representation of the entire customer base. Many customers who likely had valuable experiences for potential readers did not leave reviews.

Given the numerous barriers to leaving a review, catalyzers significantly influence the decision to leave a review (see “Why do we leave online reviews”). Some companies attempt to sway this decision by offering incentives for positive reviews or by making it difficult for dissatisfied customers to leave reviews, a tactic known as "review gating" which will be explored in the corresponding section.

Thus, readers are often limited to the opinions of individuals with emotionally charged experiences, those who left positive reviews out of altruism, or frequent reviewers with specific personality traits. Do these reviewers share the same tastes as the reader? Probably not.

💡
Exploration
  • Share of Organic and Invited Reviews. Readers should be able to know the source of a specific review. Additionally, showing the proportion of organic to invited reviews would provide valuable context for evaluating the overall sentiment.
  • User profile as additional context. Many people wish they knew if reviewers shared similar profiles. Including the user’s profile in the review information would help readers determine if it matches their own situation.
  • A design proposition for Airbnb. We can standardize stay info, indicating if a guest traveled for work or leisure, as a couple or solo, etc. Readers can relate to their situation.
    A design proposition for Airbnb. We can standardize stay info, indicating if a guest traveled for work or leisure, as a couple or solo, etc. Readers can relate to their situation.
  • Disclose the Review Rate. The higher the review rate, the more representative it is of the overall set of reviewers.

➡️ Next up: Repartition of reviewers’ sentiment