Assessing the Impact of Reuse on Software Quality and Productivity

Sep 27th, 2002 by Tony in * commentary, * papers

I’ve been trying to find more information on the “factor of 10” productivity differences, between either teams or individuals, frequently cited, but most of the primary articles don’t seem to be available on-line. A trip to the library is probably in order again early next week.

I did come across this study from 1995, however, which set out to measure the impact of reuse in OO systems. A graduate class was divided into teams, each of which was set the same programming task: to develop a system for a video rental store. Generic and domain specific libraries were made available for reuse, but they were free to choose whether or not to use these.

So far so good. Where the study seems to go bizarre, however, is in how productivity was actually measured: a team’s productivity was taken as “lines of code delivered” divided by “hours spent on analyzing, designing, implementing and repairing the system”. The authors point out that other measures than LOC “could have been used, but this one fulfilled our requirements and could be collected easily. More importantly, we are looking at the relative size of systems addressing similar requirements and, therefore, of similar functionality.”

This seems most bizarre. If the systems are all the same, why does the lines of code produced matter in the slightest? Surely all that matters is the time taken to produce the system – especially as this is meant to be testing re-use. If one team could reuse sufficient quantity of code to enable them to write the system in 10,000 lines, taking 100 hours (productivity = 100), but another team wrote an entire 250,000 LOC system from scratch taking 1250 hours (productivity = 200), is the second team really twice as productive?

But the paper seems even stranger than that. It counts the reused code within the total LOC for the team, thus distorting the productivity of a team who pull in a 10,000 line library that provides more functionality than they actually need (compare a team writing a 1,000 line subset of this in 40 hours, with another team who only need to write 10 extra lines to use this library, but who then have an extra 10,000 lines in their final total)

Using this methodology, the paper manages to show a productivity difference of 8.74 between the top and bottom teams, with a factor of 4.8 in LOC submitted. However, if you work back from their figures to calculate the actual lines of code written by the team (as opposed to being in their completed system), there only ends up being a factor of 1.77 in the LOC, and 2.6 in the time taken: still significant, but hardly as impressive:

Project LOC Delivered Reused Productivity Reuse rate LOC Written Time Spent
1 24698 16776 159.34 67.92% 7922 155
2 5105 113 18.23 2.21% 4992 280
3 11687 3061 32.01 26.19% 8626 365
4 10390 1545 34.3 14.87% 8845 303
5 8173 3273 51.4 40.05% 4900 159
6 8216 3099 31.12 37.72% 5117 264
7 9736 4206 69.54 43.20% 5530 140
8 5255 0 19.9 0.00% 5255 264

(italicised columns extrapolated from published results).

It’s also notable that the fastest/slowest teams in question are entirely different with each approach, and the “outlying” teams which deserve special explanation in the paper fare considerably differently.

Team 6 which seems to have a low productivity in the original paper, “considering its reuse rate”, is explained in terms of the team providing a particularly sophisticated “gold-plated” GUI. By solely measuring time taken to do the task, however, this team is one of the fastest.

Gotta go find those other papers…

No Comments