Size is the best predictor of code quality
After my last post, which hypothesized a relationship between the total size of a program and the bugs in it, I was led to this paper, via this blog post, via this comment. This, by the way, is my favorite thing about blogging.
Khaled El Emam, Saida Benlarbi, Nishith Goel, and Shesh N. Rai: “The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics“. IEEE Transasctions on Software Engineering, 27(7), July 2001.
It is a long and well-written paper, and contains a lot of background about code metrics.
If you want to understand all the nuances, I will not reproduce them here–go read the full paper.
However, here’s the tl;dr:
A long paper trail of software engineering studies has shown that many internal code metrics (such as methods per class, depth of inheritance tree, coupling among classes etc.) are correlated with external attributes, the most important of which is bugs. What the authors of this paper show is that when they introduce a second variable, namely, the total size of the program, into the statistical analysis and control for it, the correlation between all these code metrics and bugs disappears.
The old model was something like this: the structural properties of code, which can be measured with various metrics, affect its cognitive complexity, which in turn affects external attributes such as the number of bugs, maintainability etc.
However, this paper uncovers a strong association between most code metrics and the size of the code, both of which together lead to the externally observable attributes like bugs.
So this partially backs up my hypothesis that the number of bugs can primarily be predicted only by the total lines of code. Yegge got it right when he said that size was code’s worst enemy. However, I still haven’t found any studies which show what this relationship is like. Does the number of bugs grow linearly with code size? Sub-linearly? Super-linearly? My gut feeling still says “sub-linear”.