As the result of extensive analysis of Stack Overflow questions and answers, researchers have come up with some dos and don'ts about framing questions that will result in useful answers. 

When we're stuck with a seemingly intractable programing problem, a Google search will often lead to Stack Overflow. Sometimes there will be the answer we're looking for. Other times what we'll discover is someone with the same, or very similar problem as an unanswered question.  

Fabio Calefato, Filippo Lanubile and Nicole Novielli from the University of Bari in Italy set out to investigate how information seekers can increase the chance of eliciting a successful answer to their questions on Stack Overflow.

They started from the premise that the success of Stack Overflow and similar community-based Q&A sites depends mainly on the will of their members to answer others' questions. They write:

When formulating requests on Q&A sites, we are not simply seeking for information. Instead, we are also asking for other people's help and feedback. Understanding the dynamics of the participation in Q&A communities is essential to improve the value of crowdsourced knowledge. 

The researchers proposed a conceptual framework of factors potentially influencing the success of questions in Stack Overflow. As shown in this diagram, they considered Time of asking the question, the Presentation Quality of the question and its Affect (positive versus negative in overall formulation) as "Actionable factors", i.e. those which the person asking the question could control. The other factor taken into account was Stack Overflow Reputation, which is something difficult to manipulate as it built over time and requires experience to answer questions, and was therefore classed "Not actionable"

(click in diagram to enlarge)

The actionable factors in this framework are influenced by existing guidelines for asking questions on Stack Overflow. Its policy of Be Nice and the advice to write questions using a neutral emotional style leads to including Affect.

As regards Presentation Quality, the main advice for asking questions that will elicit an answer is to use a title that summarizes the specific problem. Other hints on Writing the perfect question are those proposed in 2010 by Jon Skeet, who, the researchers point out, was the Stack Overflow contributor with the highest reputation. His advice can be summarized as:

• Use short, descriptive question titles
• Be concise
• Use capital letters where appropriate
• Provide sample code and data
• Provide context through tags

The researchers add to this the extra guideline:

• Provide context through URLs

With regard to the Time factor, the researchers guideline of:

• Be aware of low-efficiency hours

is derived from Bosu et al (2013) which reported that the most successful time slices correspond to the working time in the USA, where most of Stack Overflow experts resided at the time of their study. They observed that the highest proportion of questions with accepted answers to be in the time frame corresponding to 3:00-6:00 PM of West Coast US time. In addition, they found that questions posted during the weekend are more likely to be answered than questions posted during the week. 

The researchers made an analysis of over 87K questions extracted from an official Stack Overflow dump of 21 million answers to 13 million covering the period July 2008 to September 2014. The questions in the dataset were restricted to those in the final month prior to the dump. This was in order to use the reputation scores of those seeking technical help. The researchers argue:

Since Stack Overflow allows users to gain at most 200 reputation points per day, it is reasonable to assume that the reputation category of most users stays unvaried over a month. 

They removed from the dataset self-answered questions, those removed or closed by moderators and those edited after original posting. They also removed questions asked in the final 3 days prior to the creation of the dump to avoid introducing a bias into the results because the community had not had sufficient time to answer them.

From the set of 87,373 questions, 30,797 (35%) were Resolved - i.e. "successful" in terms of the analysis and overall those asked with a Positive Sentiment were twice as likely to succeed compared to Negative ones. 

The chance of having a question answered increased with Asker Reputation. Those from persons considered Trusted users according to Stack Overflow's system (i.e with a reputation score of 20K or more) were  rare in numerical terms - there were just 376 of them, but almost half 182 (48%) were Resolved. Trusted users were also able to get away more often with expressing negativity. For Established users (with reputation scores between 1000 and 20,000) the resolution rate was 44% and for Low Reputation users, with scores between 10 and 1000, which accounted for the majority of questions, it dropped to 40%.

The success rate for New users, with reputation score less than 10, was only 21% and their questions were the most penalized for expressing Negative sentiment  negativity. This could be interpreted as newbies being shunned by the community, however this isn't as extreme as it appears. To count as an answered question on Stack Overflow the person who posed the question has to mark it as being accepted. In fact many questions count as unanswered simply because new users are unaware of this requirement.

As part of their investigation the researchers looked at a sample of 100 questions that were deemed "unsuccessful" on Stack Overflow and found that 38 of them had useful answers and that 17 had answers that provided a solution. 

As a result of extensive analysis, backed up by a survey of Stack Overflow users, the researchers reported that Presentation Quality is the most relevant success factor concluding:

We found that regardless of user reputation, successful questions are short, contain code snippets, and do not abuse with uppercase characters. As regards affect, successful questions adopt a neutral emotional style. 

In relation to Time they observed an increased chance of success in GMT evening hours, i.e. corresponding to working hours in the US.

More Information

How to Ask for Technical Help? by Fabio Calefato, Filippo Lanubile, Nicole Noviellib on arxiv.org 

How do I ask a good question?

Related Articles

Stack Overflow: A Code Laundering Platform?

Stack Overflow Considered Harmful?

Stack Overflow An Old Programmers Home

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info