Studies of Methodologies

Nov 19th, 2002 by Tony in Questioning XP

Although opponents of Extreme Programming are always asking for studies about the effectiveness of XP, it is interesting to note that there are no real studies that support any methodology. There are no comparative studies that have ever taken place on reasonable-size projects. Indeed, it is hard to trust any of the studies in the software engineering field because they were either carried out a long time ago or because the numbers come from a small study that lasted a few weeks.

Studies that were carried out more than ten years ago are suspect because of the dramatic improvements in hardware and software tools over the years. Personally, I find that object-oriented programming at an interactive workstation is qualitatively different from the old days of batch compiles in low-level languages, when we were lucky to get three or four compilation runs a day.

Small short-duration studies are suspect because they are dominated by short-term learning effects, mostly because they are constrained to using novices. This is a problem because novices use different strategies than experts, and when experts switch to a new way of working, they experience a larger drop in performance than novices. These two factors make it dangerous to extrapolate long-term expert performance from that of novices.

— Pete McBreen, Questioning Extreme Programming, Chapter 1

No Comments