Does code quality matter in open source? New research suggests that we value other things.

One might imagine that contributions to open sources projects would be vetted for code quality, if for no other reason that the code is on display to the world. Do you really want the world laughing at your terrible code? The responsibility falls most heavily on the shoulders of the maintainers because while contributors move on maintainers tend to have a longer life span with a project and identify with it more. You would expect maintainers to be concerned about code quality of anything that they are considering accepting. New research suggests otherwise.

Valentina Lenarduzzi, Vili Nikkola, Nyyti Saarimaki and Davide Taibi  of Tampere University (Finland) decided to gather some data to see how important code quality was. They looked at 28 Java-based open source projects on  GitHub, including many Apache  projects and looked at code quality in 36,000 pull requests. They used the PMD code quality tool and measured: long methods, large class, duplicated code, high coupling, god class and style violations. These were summarized as a measure of technical debt, TD.  Previous work had confirmed that as code quality, as measured by PMD, goes down so the risk of faults increases - a good reason for maintainers to prefer good quality code.

The analysis failed to find any connection between technical debt and success of a pull request. What is more, even the more severe technical debt problems appeared to have no effect on pull request sucuss. In plain terms, code that was measured as very bad by PMD was as likely to be accepted. As the researchers say:

"Unexpectedly, code quality turned out not to affect the acceptance of a pull request at all. As suggested by other works, other factors such as the reputation of the maintainer and the importance of the feature delivered might be more important than code quality in terms of pull request acceptance."

So never mind the quality as long as it gets the job done, seems to be the conclusion. As already mentioned, this is surprising given the visibility of the code.

There are some possible explanations. The most obvious is that PMD is measuring a range of code problems that aren't really problems. In other words, the TD measure is false, but this seems unlikely as the factors that go into computing it have face validity and the TD measure is correlated with faults. More likely is that the average programmer doesn't really get measures like PMD and interprets code quality in more subjective and simpler ways - does it look good? Perhaps all that matters is getting the indents right, using good variable names and writing code that the maintainer can, or think they can, understand.

Open source may benifit from Linus's Law,  "given enough eyeballs, all bugs are shallow", but clearly even with enough eyeballs code can remain ugly.

More Information

Does Code Quality Affect Pull Request Acceptance? An empirical study

Related Articles

WiringPi - Deprecated

GitHub Launches Draft Pull Requests

Can a Language Make You a Better Programmer? Kotlin & Android

Does Strong Typing Help?

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info