‘* commentary’ Category Archives

2
Sep

The Size/Complexity Dynamic

by Tony in * commentary, QSM1

Human brain capacity is more or less fixed, but software complexity grows at least as fast as the square of the size of the project

As we try to solve bigger and bigger problems, the Size/Complexity Dynamic drives us from a previously successful pattern to a new, untried pattern. Were it not for our insatiable ambition, we could rest comfortably with our present pattern until they carried us away in our rocking chairs. Many organizations have done just that, usually because their customers have no further ambitions for better software.

When there is ambition for more value in software, it quickly pushes against the barrier of the Size/Complexity Dynamic unless we can alter the capacity of our brain. Organizations sometimes do this by hiring smarter people, but there’s a definite limit to that tactic.

We can’t alter the capacity of our brain, but we can alter how much of that capacity we use, as well as what we use it for. That’s why software engineering was invented.

— Jerry Weinberg, Quality Software Management Vol 1, Chapter 9

In a nice twist of synchronicity, I was reading some old issues of JOOP this morning, and came across an interesting article from October 2000, by Nevin Pratt, entitled The Software Wall. In it he describes his two constants of software:

The First Constant of Software: Software is constantly becoming more complex.

The Second Constant of Software: A programmer’s ability to manage the complexity remains constant.

Pratt then explains that: these two constants, so long as they remain true, predict the existence of a developmental barrier or wall, beyond which we cannot progress. The reason that we cannot progress beyond “the wall” is because as complexity increases, productivity decreases. Productivity thus approaches zero as the complexity increases, until eventually productivity hits zero.

We can’t just add more people (cf. Mythical Man Month), so we need to constantly attempt to move the wall. Pratt shows how we’ve managed that by the move from machine language to assembly language, and then to 3GLs, and then to SQL, and then to OO (and stakes his claim that dynamically typed OO languages are the next in this series, thus the URL of the reprint of the article!)

However, Pratt claims that these are attacking the first constant – the complexity of the software, which I think gets it backwards. Each of these is really just another layer of abstraction – the software doesn’t become any less complex – the detail just gets hidden, meaning that the programmer doesn’t need to encounter as much of the complexity, thus minimising the impact of the second constant.

However, it’s an interesting partitioning of the problem, and I like the approach of asking, for any given suggestion, will this make the software less complex, or enable the programmer to handle more complexity?

27
Aug

Circular Reasoning

by Tony in * commentary, QSM1

The principle device for closing a culture to information is the circular argument.

Some years ago, I helped a client make the transition from Pattern 2 to Pattern 3. The transition was necessary because they were producing telephone equipment that had to have down time of no more than one hour in forty years! They were so successful in achieving their goal that their development anager published an account of the process.

The very first time I showed the article to another manager he asserted, “That would never work here.”.

“Why not?” I asked. “It should be straightforward. After all, their problems were much harder than yours. You don’t even have on-line systems, let alone systems that have to respond in real time.”

“Even so, their problems couldn’t have been as hard as ours because we could never get that kind of performance on our problems”

— Jerry Weinberg, Quality Software Management Vol 1, Chapter 3

I experienced this in action first hand a few months ago. We’d arranged for Damian Conway, one of the best Perl trainers in the world, to come to Belfast to do some training courses. We’d never put on training before, so we talked to a few of the local training companies, to see if they’d be interested in doing something with us. They all said the same thing: “There’s really not that much interest in Perl training in Belfast. We only really get about 5 or 6 people on our Perl courses, so we don’t think it would be worth it.”

Of course, these companies couldn’t see that the reason they didn’t get people coming on their Perl courses wasn’t that there was no interest, but that their courses weren’t very good. Whilst we were at BlackStar, we’d tried to find training courses for our developers to go on, and asked all the local training courses what Perl courses they had. We wouldn’t send our staff on any of them. (We talked of creating our own, in-house, training courses, but of course that never happened…)

So, despite everyone’s seeming lack of interest, we went ahead and arranged the courses ourselves, and had around 50 attendees over two days. We even lent Damian to Momentum, the Northern Ireland ICT Federation, for a morning to do a Breakfast Briefing on “10 Myths About OO” – one of their best attended breakfasts all year. (Of course Momentum then decided that they just wouldn’t bother paying Damian for this, but that’s NI for you.) The whole thing was so successful that Damian is coming back to Belfast in another few weeks to do even more courses.

How many more companies must fall into this “We did X (badly), and no-one wanted it, therefore no-one wants X” trap?

25
Aug

Increasing programmer productivity

by Tony in * commentary, QSM1

Studies of individuals have consistently shown variations of 20:1 or more in schedule, cost, and error performance among professional programmers, so it makes sense that this is the level of variation we see in Pattern 1.

— Jerry Weinberg, Quality Software Management Vol 1, Chapter 2

I’ve been thinking about this concept a lot again recently. This sort of figure crops up again and again in all sorts of places. It’s a reasonably well known concept, and reasonably self-evident to anyone who’s worked around enough programmers. So why is virtually no-one interested in the concept?

Yes, you get people writing about “software productivity”, but these tend to fall into two main camps. The first, who usually have something to sell, write about the newest, greatest Silver Bullet that will increase everyone‘s productivity by an order of magnitude.

The second, the more thoughtful consultant types, work hard with companies to smooth all their processes, and slowly move them up the CMM scale, or, in Weinberg’s case, his Six Degrees of Congruence.

But again, this is about raising the company’s level, as the totality of everyone. Although it’s rarely explicit, the implicit assumption is that it’s bad to rely on these “super-programmers”, and you need repeatable processes that work even if you lose your star performer(s). As Dijkstra put it:

Industry suffers from the managerial dogma that for the sake of stability and continuity, the company should be independent of the competence of individual employees. Hence industry rejects any methodological proposal that can be viewed as making intellectual demands on its workforce

Both of these approaches miss the point, in almost exactly the same way. They don’t concentrate on why some programmers are 20 times more productive than others, or whether it’s possible to create more of these star programmers.

Perhaps it’s just that people just don’t know how to cope with this entire concept. Bob Morris, in his keynote at the Emerging Technology conference earlier this year, talked about how in the last 100 years, $1000 worth of computational power has increased by 14 orders of magnitude – an unprecedented scale that people find difficult to conceptualise. But, in most fields of human endeavour, people can’t even conceptualise one order of magnitude. The world record time for running a mile is around 3 minutes 43 seconds. Even an average runner can run a mile in less than twice that length of time. This plays out again and again: the best is maybe only twice as good as the average. The difference between professionals is much less. In Formula 1, where cars go at speeds in excess of 200 miles per hour, the significant difference in lap speeds is often measured in hundredths of a second. (At those speeds we do get our factor of 20 over the human running that mile, but with huge costs and a lot of ‘artificial’ aids that can raise most people to the same levels)

In the workplace it’s not very different. You won’t find a call centre with an two employees, one of whom handles twenty times more calls than another (at the same, or even better, level of quality). Or two secretaries, one of whom can type twenty times more letters than the other.

We’re not used to dealing with these levels of differences between people, so we don’t know to deal with it when it happens.

And, with programmers, it’s hard to actually see it happening – not at the detail level. We know that Fred gets much more done than Barney, and can do most jobs in a fraction of the time, but we can never quite see how. When we look closely, they both seem to do the same thing. There’s no obvious flaw in Barney’s approach, or obvious methodological differences that could be applied. It just seems to happen, so we write it off as just an anomaly.

Managers in “lower” organisations on most of the “quality scales” rely on the fact that they can give the work to Fred, and have it done quickly, whilst always having that nagging fear of Fred moving somewhere else. Manager in slightly more “advanced” organisations know that they can’t rely on Fred, so they work out all their schedules and budgets based on Barney instead. Neither attempts to make Barney as productive as Fred. They may wish he was, but neither think it’s possible.

Why?

4
Jul

Understanding the Professional Programmer

by Tony in * commentary, * one-offs

This is quite an old book now by Jerry Weinberg (1982), and it has dated quite significantly – more so than Mythical Man Month which is a few years older.

Most of the programmers in this world were writing COBOL, PL/1 or raw assembler. And so many of the examples he gives don’t really translate well. (Although the extended example in “Say what you mean and mean what you say” could easily be translated in Java and show up in an Extreme Programming book). And cost analyses on the best way to teach computer programming when machine time was so expensive seem to come from an eerily distant past now.

But once we get into the middle of the book, much of the material tends more towards the “psychology” of programmers, which hasn’t really dated. And there are a few nice ideas that still haven’t really caught on. I particularly liked his idea that computer languages should be taught in pairs, with all the examples and exercises given in both languages. With natural languages the most difficult to learn is always the second. A third, fourth or even tenth language is much easier. The same seems to be true of computer languages also. So, teach people to be bilingual from the outset. Let them see how different languages would approach the same problem.

Although the book is on the surface aimed at programming managers, most of the articles really seem to be aimed at programmers who need to learn how to deal with programming managers. And, if anything, the problems he points out with the gap between programmers and managers seem to have widened rather than closed. As Weinberg writes: Code is to programming managers what dirty dishes are to a headwaiter. Once you have graduated from the garbage heap, you never touch the garbage again – even in jest.

The “Cost of Inadequate Software Testing Infrastructure” paper I linked to a few days ago had a chart that showed that the percentage of project time given to actually writing code had dropped significantly in the last few decades as more and more time went in to specifications, analysis, design etc. I doubt most managers read any more of the actual code that gets produced these days.

This is a book that treats coding seriously. Weinberg argues that contrary to popular opinion, now as much as then, coding is a valid profession, and a skill that takes years to master, not days or weeks. This idea has obviously had a resurgence recently with books such as Pete Breen’s Software Craftmanship, but it’s interesting to go back twenty years and see the articles that were being written then.

Probably not worth going out of your way to find, but worth a skim through if you find a copy.

9
May

Designing from Both Sides of the Screen

by Tony in * commentary, * one-offs

Most books I’ve read on User Interface Design support their theoretical approach with copious examples. This one takes the opposite approach. Most of the book is taken up with describing the evolution of an instant messaging client, and how its UI developed. This is made interesting because the software was developed simultaneously for Windows and for a PDA. However, I’d never heard of the product (Hubbub) before reading this book. This isn’t necessarily a problem, as there’s still a lot of useful ideas here (and as far as I’m concerned, the more books that reinforce the concept that most software is still really badly designed, the better), but I can’t help thinking that the book would have been more useful, or at least more interesting, if it was the story of the development of a product I actually used.

8
May

What Not How: The Business Rules Approach to Application Development

by Tony in * commentary, * one-offs

I spent an hour or so in MIT Coop reading this book, and taking copious notes. Date’s central thesis is that the trend in computing is always from Procedural to Declarative, from ‘How?’ to ‘What?’.

The biggest bottleneck in creating applications is turning the requirements into a functional system. His solution: eliminate the code, and merely state your business rules declaratively.

He spends quite a lot of time explaining how this is not the same as writing lots of event handlers in a database (they’re difficult to write, even more difficult to debug, and are usually database events (‘do something when this record is updated’), rather than business events (‘do something when an order is placed’).

I found his taxonomy of rules very handy. In the simplest form, all rules are either Constraints or Derivations.

There three forms of Constraints:

  1. State Constraints (Total Owed <= Credit Limit),
  2. Transition Constraints (You can move from Married to Divorced or Married to Widowed, but not from Widowed to Divorced, or Divorced to Widowed),
  3. Stimulus/Response (IF StockLevel – OrderedQuantity < ReorderLevel THEN reorder).

And two forms of Derivations:

  1. Computation (LineAmount = QuantityOrdered * OrderPrice)
  2. Inference (IF TotalOrders(c) > $100,000 THEN GoodCustomer(c))

If you can write a system that handles all these rules, then you can probably create 95% of business applications in a purely declarative manner.

I’ve been working for the last year on a framework that allows you to create database backed web sites mostly declaratively. You still need to do some Class::DBI set-up, although I’ve gradually made that less and less important, especially if you’re using a self-aware database, but after that you can create 90% of what you need for a reasonably complex site in a single config file. This book has given me lots of interesting ideas as to where I could take this next…

7
May

Love Is The Killer App

by Tony in * commentary, * one-offs

Tim Sanders is “Chief Solutions Expert” at Yahoo! and this books is, in part, his tale of how being a “lovecat” helped him get that job, and helped Yahoo! make lots of money. Whilst that section of the book, which takes about about the first third, is mildly interesting, I was much more impressed by the second section: “Knowledge”, which takes as its central thesis that you really need to read. Not magazine articles (the “between-meal snacks”), or electronic news (“candy and soda”), but real-life, dead-tree, books. Ideally in hardback (“A paperback is made to be read. A hardcover is made to be studied”). He spends a long time on how you know what to read, how to read what you’re reading, how to process what you’ve read, and how to apply what you’ve learnt. It’s all fairly basic stuff, but I’ve often been amazed at how little “professional” people actually read – particularly in the “computing” and “business” areas. If even encountered people who get offended if you suggest a book to them – as if you’re saying they’re not good at their job!

The rest of the book consists of fairly basic “Network” and “Compassion” sections (the book is classified as “Business – Motivational”, which is usually a bad sign IME), so I wouldn’t really advise buying it. But the “Knowledge” section is is worth a read, for people who don’t read. Getting them to read that of course is going to be the difficult bit.

 

6
May

Dream Merchants and Howboys

by Tony in * commentary, * one-offs

I discovered at the airport that Barry Gibbons had a new book. He’s the ex-CEO of Burger King, and writes exactly in the manner that you would not expect of such. His previous books (“This Indecision Is Final : 32 Management Secrets of Albert Einstein, Billie Holiday, and a Bunch of Other People Who Never Worked 9 to 5“, and “If You Want to Make God Really Laugh, Show Him Your Business Plan” were both excellent. One Amazon reviewer puts it well:  “This guy writes with the wit and wisdom of Tom Peters and the hysterical observations and honesty of Dave Barry.”

This new book, “Dream Merchants and Howboys: Mavericks, Nutters and the Road to Business Success“, is no exception. He spends a chapter each on some of his business heroes – each one definitely a maverick, and many indeed complete nutters –  Luciano Benneton, Richard Branson, James Dyson, Steve Jobs, Herb Kelleher etc. There’s not a huge amount of research into these – in many of the cases where I’ve read more detailed biographies I could spot either inaccuracies or important omissions – but it’s a great introduction to many of them, told not only with great wit but also with passion. Gibbons obviously admires (almost) all of these figures immensely, and it shows.

Probably the only business book that made me laugh aloud on this trip!