Two recent academic studies have raised questions about customer reviews found on online sites. Can we trust the motivations of the reviewers and how can we tell fake from genuine reviews?

A team of researchers from Cornell has developed a classifier to detect what they call "deception opinion spam". Interviewed by the New York Times the main author of the paper, PhD student Myle Ott, commented

“The whole system [of consumer reviews] falls apart if made-up reviews are given the same weight as honest ones.”

Co-author, Linchi Kwok, an assistant professor at Syracuse University who is researching social media and the hospitality industry, explained that as Internet shopping has become more “social,” customer reviews have become an essential part of the sales pitch:  

“Everyone’s trying to do something to make themselves look better,” he said. “Some of them, if they cannot generate authentic reviews, may hire somebody to do it.”

The Cornell study looked specifically at fake hotel reviews but in his article for the New York Times David Streitfeld tackled wider topic of detecting fake product reviews and spoke to Russell Dicker, Amazon's director of community. Amazon, which pioneered online reviews and later moved from having its own panel of reviewers to soliciting customer reviews as well as expanding its product range from books to a much wider range of goods has a high proportion of 5-star reviews.

Amazon attributes the preponderance of positive reviews to a feedback loop: products with high-star ratings sell more, so they get more reviews than products with poor ratings. However Dicker is concerned about the integrity of reviews, pointing out:

"Any one review could be someone's best friend, and it's impossible to tell that in every case"

and says this problem is something Amazon is concerned about. 

Book reviews on Amazon.com have also come under academic scrutiny in a study Trevor Pinch and Filip Kesler of Cornell University that sheds some light on why there is a bias toward positive comments as well as providing information about who contributes reviews and what motivates them to do so.

These researchers contacted as many as possible of the top ranked Amazon reviewers and ended up with data from 166 reviewers, discovering that compared to the demographic of book buyers they were disproportionately male (70%). They also differed from typical Amazon customer by being older, the peak age group was 51-60, and better educated - over half had Masters or Doctorate and 92% had at least college degree. Another interesting finding is that 39% of the  reviewers are also authors.

As regards motivation, the research finds that 80% of their respondents are motivated to contribute reviews for "self expression" and "enjoyment", 85% them report being sent free books or other products to review by publishers, authors and the like.

The researchers comment:

This is an important finding because it shows that Amazon top reviewers do receive some sort of direct material reward, however small, for their endeavors.

Because of its timing, this survey attracted responses from those with high ranking on the Classic review system as well as on the new system in which reviewers' rankings are influenced according to how helpful other customers perceive their comments. Rankings on the two systems can be widely different - only 4 of the respondents had rankings that stayed the same, 84 received a lower rank, and 73 a higher one - but perhaps more important was the finding that having achieved a high ranking most reviewers were concerned about maintaining it.

This gives some clues about the tendency for reviews to be positive: 88% of the respondents reported that they gave either "all" or "most" positive reviews and many said they avoided writing negative reviews because such reviews are more likely to be found "unhelpful" - something that would adversely affect their ranking on the new system.

As book reviews editor for I-Programmer, co-ordinating our daily Book Review of the Day, I probably read even more customer reviews of computer-related books than most people and find that they range from the very conscientious, to the ill-informed or over-hyped.

While I don't often suspect the Amazon reviews I encounter as complete fakes, a proportion of the 5-star do seem to be little more than "cronyism".

You can get some idea of how likely a hyper-positive review is to be real by looking at the other reviews that are on offer by the same person.

• If there is only a single review of the book in question and it has five stars and lots of adjectives like "great", "wonderful", "best book ever written"... then suspect that it is a review from a friend of the author.

• If a reviewer has posted a few reviews but the range of items reviewed aren't related to the book in question, then the chances are that these are dummy reviews added to make the book review look more real. Be suspicious if you find  reviews of things that don't normally need a review, such as a pack of razor blades or a toothbrush. Another clue as to a review being suspect is when a single review of an advanced title is coupled with two or three reviews of some very trivial books. Ask yourself if the reviewer's reading habits seem coherent.

• The final sign that a reviewer is motivated by being offered freebies rather than providing a balanced opinion is when they have produced a large number of reviews but they are all five star. How can you trust a reviewer who thinks that everything is fine about everything?

So all you are left with to trust are the bad reviews? You can certainly rule out any attempt to hype a book if the recommendation is one star but you still can't take the review at face value. Some reviewers have motive to rubbish a book for gain in other directions - for example they are the author of, or associated with an author of a book, on the same topic.

You have to apply the same rule to very negative reviews as to very positive. You need to look to see what else the reviewer has reviewed. If they are all low scoring you have a very critical and angry person and perhaps you need to discount the review somewhat. If their reading or reviewing habits don't hang together then you again have grounds for suspicion.

A negative review is on average more trustworthy than one that is superlatively positive, but you still have to be careful.

So when you see a review that you suspect of being hype or setting out to rubbish a book rather than a considered evaluation of a title, just say "No" in response to the question "Was this review helpful to you?"

Finding Deceptive Opinion Spam by Any Stretch of the Imagination

New York Times Ferreting out Fake Reviews

How Aunt Ammy Gets Her Free Lunch: A Study of the Top-Thousand Customer Reviewers at Amazon.com

To access our store of over 700 book reviews in over 40 categories go to the Book Reviews menu.

If you would like to be informed about our daily Book Review of the Day follow us on Twitter or Facebook or  subscribe to our weekly newsletter for a regular digest.