What was your all-time favourite bug?

Another of my occasional professional-related blog posts. I have considered spinning these off into a separate blog, but for the time being, it’s another category on my personal blog. As I’ve explained before, what I do in my day job isn’t completely unrelated from some of what I do outside of work.

So, the all-time favourite bug?

This is a question I’ve been asked recently. While it’s difficult to single out one bug from all my years of testing experience, this one does stand out, and comes from one of the most challenging projects from my testing career so far.

The application was a work scheduling system, part of a very comprehensive housing management product. A call centre would take calls from tenants reporting broken windows or blocked drains, then the system would schedule appointments for operatives to carry out repairs. The operatives themselves could update progress on these appointments in real time using a mobile app, allowing the system to maintain the daily work plan dynamically.

It was a complex application, involving two third-party products, one of them the mobile app, the other, which formed the focus of my own testing, being the scheduling engine itself. This was a standalone system which took XML messages containing the call information, and sent back XML containing either the appointment, or the reason why it couldn’t create one.

One thing that became apparent early on was the scope of my testing didn’t encompass just our own product and the XML interfaces. End-to-end testing soon revealed a lot of bugs in the scheduling engine itself. We had some complex business rules for dealing with things like multi-person jobs, multiple jobs which had to be performed in sequence, or jobs where the call centre wanted to assign a particular operatives rather than let the scheduling engine choose. With a near-infinite number of potential combinations of conditions I performed extensive exploratory testing on increasingly arcane permutations. A straightforward example might be “Remove wasps nest” followed by “Carry out repairs to guttering“, which required two different trades but had to be performed in the right order. Some of the less straightforward scenarios resulted in the scheduler returning gnomic utterances such as “The optimal schedule does not include this call“, or, if you were really unlucky, “Object reference is not set to an instance of this object“. Unfortunately I could never find a test case that would repeat that last one consistently.

The most significant bug occurred when the schedule filled up. The dynamic nature of the schedule meant that sometimes appointments did get missed. For example, if a previous appointment ended up taking far longer than anticipated, and the system couldn’t find another operative available in that slot because everyone else was busy, the appointment would fall out of the schedule and need to be re-scheduled. The same would happen if an operative went sick or crashed his van during the course of the day.

Unfortunately it did the same thing when I filled up the schedule and tried to schedule further appointments. Rather than reject them with “No operatives available with correct skill” as it should have done, it decided the plan would be more ‘efficient’ if it appointed the new jobs and displaced existing ones it had already committed to. Not a situation which would lead to happy tenants, who would not be impressed if they’d taken an afternoon off work for a plumber who never turned up.

Given the severity, that bug was reported to the third-party supplier and fixed before the system went live on any customer site. But that particular scenario ended up in the suite of regression tests run each time we received a new release of the third-party scheduling engine.