An empirical study of the reported types of reputation manipulation scenarios that happen on Stack Overflow had led to two algorithms that could be used to detect, and hopefully deter, such fraud.

Reputation, Reputation, Reputation 

Oh, I have lost my reputation I have lost the immortal part of myself and what remains is bestial.

This quote come's from William Shakespeare's Othello, a play in which reputation and honor are key themes. The lament is made by Cassio a career soldier who has been stripped of his rank for drunken behavior.

Here's a more relevant quote - from Joel Spolsky co-founder of Stack Overflow and dating from November 2010 in an of-the-cuff remark in a thread about choice of programming language for work:

spend a few months earning a five digit Stack Overflow reputation, and you'll be getting job offers in the $100K+ range without an interview.

This idea has been hotly disputed many times over in the more than a decade since it was made and Stack Overflow produced evidence of a correlation between salary and number of reputation points in their 2015 Stack Overflow Developer Survey, as reported by Janet Swift at the time, see Developer Work and Pay. However it has to be remembered that such a correlation doesn't necessarily imply a causal relationship.

Suffice it to say for some programmers, their Stack Overflow reputation is both important and valuable and the idea that reputation points are gained and lost by underhanded methods is an uncomfortable one. Such suspicion does however exist and now there is third partly corroboration of how Stack Overflow reputation can be, and is manipulated. It comes in a paper entitled Reputation Gaming in Stack Overflow by researchers from Iran and Canada.

As most readers will be well aware, reputation on Stack Overflow is considered a rough measurement of how much the community trusts you and has confidence that you know what you’re talking about. The primary way to gain reputation is by posting good questions and useful answers. Votes on these posts cause you to gain (or sometimes lose) reputation. As the researchers point out  a Stack Overflow user can get upvotes for sharing useful contents and downvotes or no votes for sharing useless/harmful contents.

Acknowledging the importance of reputation scores as an incentive system whereby users with high reputation in Stack Overflow can significantly influence the overall knowledge sharing process and content quality, the researchers conducted two empirical studies to understand the possible types and prevalence of reputation manipulation in Stack Overflow.

Study 1. Understanding Reputation Fraud Scenarios

The researchers analyzed 1,697 posts from StackOverflow meta sites where developers discuss reputation manipulation of events. The analysis revealed six reputation fraud scenarios: 

• Voting Ring where fraud users form a ring and upvote each other repeatedly. This is the most prevalent 
• Bounty Gaming, when a bounty has been offered ti promote an unanswered question bounty.
• Revenge Voting where a victim’s posts are downvoted motivated by revenge.
• Badge Hunting - specifically the Reversal badge
• Edit Gaming - meaningless edits simply to gain +2 points 
• Closing Ring - to reduce the reputation scores of a victim

After this intial phase the rearchers looked for ways to automatically detect such reputation frauds and developed two algorithms. The first algorithm detects suspicious communities whose members show a high level of interactions among themselves by posting similar contents, responding to each other’s questions in a very short time, and accepting each other’s answers. The second algorithm detects suspicious users who show unusual jump in their reputation scores in a short time.

To evaluate the performance of their algorithms, the researchers examined the reputation history dashboard of Stack Overflow users and observed that around 60-80% of users considered to be suspicious by our algorithms had had their reputation scores removed by Stack Overflow. Not all the suspicious users identified by the research had been punished by Stack Overflow which led the researchers to suggest that Stack Overflow can utilize their proposed algorithms to improve the detection of fraudulent users.

The paper Reputation Gaming in Stack Overflow is a detailed account complete with the underlying statistics and discussion of threats to the validity of the study. 

Sad to say wherever there's community, there's potential for corruption. Perhaps these algorithms can deter fraudsters as well as detecting fraud.

More Information

Reputation Gaming in Stack Overflow

by Iren Mazloomzadeh, Gias Udin, Foutse Khomh, Ashkan Sami

Related Articles

Stack Overflow An Old Programmers Home

Stack Overflow Helps You Tell Your Story To Land Your Next Job

Developer Work and Pay

Stack Overflow Considered Harmful?

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info