Testing & Software Blog

The problems with the American Healthcare.gov website is making the whole thing look like one of the highest profile software project management failures in history. As a contractor working for a UK software house supplying IT solutions for the public sector, it’s impossible not to take a professional interest in what went wrong, and what lessons should can be learned.

When it went live, Ben Simo was live-tweeting his experiences trying to set up an account, and highlighted some severe security issues with the website, and Bob Martin blogged about some of the problems. It’s clear that the system went live with wholly inadequate testing.

And then it occurred to me. The programmer did not write the questions! Some bureaucrat wrote the questions. The programmer never talked to that bureaucrat. The programmer never read the questions that the bureaucrat wrote. The programmer was simply told to display a set of questions from a database table, and to store the responses in the user’s account. The programmer had no idea that this particular question was asking for a date! So the programmer was not trying to match a date! The programmer was just accepting any string.

Well, not any string. After all, I had been typing strings for the last ten minutes. No, someone had told the programmer (or the programmer simply decided on his own) that certain characters would not be appropriate in the answers to the questions. One of those inappropriate characters was probably: "/". I think numbers must also have been considered to be inappropriate since I had tried: 17 July, 1973. Or perhaps it was the comma. Who knows? Who cares? (Apparently not the bureaucrat, the programmers, or the people who tested this system.)

I don’t think it’s fair to blame this solely on the testers failing to do their job properly. It sounds more like the time given to test the project thoroughly got severely truncated as the development overran, a scenario I’ve seen play out time and time again.

I’m sure we’ll be hearing inside stories detailing precisely what went wrong in the coming weeks and months. The whole thing sounds like a perfect storm of failure, in particular a fixed end date dictated by politics and legislation, with those same politics delaying the finalisation of requirements. Not to mention trying to save the failing project by throwing more warm bodies at the problem, despite the fact we’ve all known for thirty years that such an approach just doesn’t work. In other words, a classic software death march, a problem endemic in the IT industry.

This paragraph from the linked piece jumped out at me.

But when politics becomes the dominant “driving force” in a large, complex project, the project is likely to degenerate into a death march. Remember my definition of a death march project: It’s one where the schedule, budget, staff, or resources are 50–100 percent less than what they should be. Why are these constraints being placed on the project? There are many possible explanations, as we’ll see in the discussion below; but in many cases, the answer is simply, “Politics.” It may be a power struggle between two ambitious vice presidents in your organization, or the project may have been set up to fail as a form of revenge upon some manager who stepped on the wrong toes at the wrong time. The possibilities are endless.

I’ve worked on failed projects like that…