//No Comment - TDD, C++ Lambdas & Agile Under Scrutiny
Written by Alex Armstrong
Wednesday, 09 November 2016
• Test-Driven Development
• An empirical study on the impact of C++ lambdas and programmer experience
• Software Quality - Traditional vs. Agile: an Empirical Investigation
Sometimes the news is reported well enough elsewhere and we have little to add other than to bring it to your attention.
No Comment is a format where we present original source information, lightly edited, so that you can decide if you want to follow it up.
Given how much in vogue test-driven development and tests are in general you would think that we had hard data on how much better the approach is. New research sheds some light on the matter. It seems that TDD doesn't make a huge amount of different but there is still room to argue:
Test-driven development (TDD) is an agile practice claimed to improve the quality of a software product, as well as the productivity of its developers. A previous study (i.e., baseline experiment) at the University of Oulu (Finland) compared TDD to a test-last development (TLD) approach through a randomized controlled trial. The results failed to support the claims.
We want to validate the original study results by replicating it at the University of Basilicata (Italy), using a different design. Method: We replicated the baseline experiment, using a crossover design, with 21 graduate students. We kept the settings and context as close as possible to the baseline experiment. In order to limit researchers bias, we involved two other sites (UPM, Spain, and Brunel, UK) to conduct blind analysis of the data.
The Kruskal-Wallis tests did not show any significant difference between TDD and TLD in terms of testing effort (p-value = .27 ), external code quality (pvalue = .82 ), and developers’ productivity (p-value = .83 ). Nevertheless, our data revealed a difference based on the order in which TDD and TLD were applied, though no carry over effect.
Conclusions: We verify the baseline study results, yet our results raises concerns regarding the selection of experimental objects, particularly with respect to their interaction with the order in which of treatments are applied.
Lambdas are also a hot topic and fast being introduced in just about every langauge that has enough syntactic flexibility, even those that don't even need lambdas. So is it worth the effort? New research suggests not. If you want to read the entire paper the bad news is that it is outragously behind a paywall.
Lambdas have seen increasing use in mainstream programming languages, notably in Java 8 and C++ 11. While the technical aspects of lambdas are known, we conducted the first randomized controlled trial on the human factors impact of C++ 11 lambdas compared to iterators. Because there has been recent debate on having students or professionals in experiments, we recruited undergraduates across the academic pipeline and professional programmers to evaluate these findings in a broader context.
Results afford some doubt that lambdas benefit developers and show evidence that students are negatively impacted in regard to how quickly they can write correct programs to a test specification and whether they can complete a task. Analysis from log data shows that participants spent more time with compiler errors, and have more errors, when using lambdas as compared to iterators, suggesting difficulty with the syntax chosen for C++.
Finally, experienced users were more likely to complete tasks, with or without lambdas, and could do so more quickly, with experience as a factor explaining 45.7% of the variance in our sample in regard to completion time.
Our final research report concerns the age old, well fairly old, argument about traditional v agle development.
It is well known that the software process impacts the quality of the resulting product. There are also anecdotal claims that agile processes result in higher level of quality than traditional methodologies. However, still solid evidence of this is missing.
This work reports in an empirical analysis of the correlation between software process and software quality with specific reference to agile and traditional processes.
More than 100 software developers and engineers from 21 countries have been surveyed with an online questionnaire. We have used the percentage of satisfied customers estimated by the software developers and engineers as the main dependent variable.
The results evidence some interesting patterns: architectural styles may not have a significant influence on quality, agile methodologies might result in happier customers, larger companies and shorter projects seem to produce better products.
Microsoft has announced TypeScript 5.7 in beta, with improvements including stronger checks for variables that have never been initialized before use, and path rewriting for relative paths.
At last ISO C23 has been published, but at $250 you probably aren't going to read it. Can we really tolerate this sort of profiteering on the work of others? This is worse than academic publishing!
