A post about assigning priorities to bugs.
My current client uses an internal defect logging system rather than a proprietary tool. It doesn’t have separate “severity” and “priority” fields, instead we have two fields called “Priority” and “Internal Priority”. In practice testers no longer use the former at all, since all priorities bar the default are now reserved for issues raised by customers via the help desk. So testers use the latter as “our” priority to identify those bugs that need fixing urgently. It’s a numeric field which allows priorities from 1 to 999, although we don’t use anything like the full range.
Without getting into jargon-laden moon-language or buzzword-parody words like “Seriosity”, I’ve used the following two factors in assigning values to this when logging defects found during testing.
First take the importance of the feature in question, on a scale of one to three.
1 – Critical feature on the end-to-end life-cycle of the object under test
2 – Important but non-critical feature
3 – Minor or little-used feature
Note that I’m using “Feature” rather than “Function” here, and the definition of a feature is a level of granularity below the module it’s logged against. For example, a search module blowing up when you press “Go” would be a 1, but an advanced search criteria on the third tab not returning the correct rows might be a 2 or a 3.
Second, take the impact of the bug on the feature in question:
1 – Feature unusable with no workaround
2 – Workaround exists but significant inconvenience to the user
3 – More irritation than inconvenience
Then multiply the two figures together, and you get a figure from 1 to 9 (The numerically-literate among you will notice that you’ll never get 5,7 or 8), and assign that as the Internal Priority. So a critical feature that just doesn’t work at all will be Internal Priority 1, an inconvenience with a workaround on an important function will be 4, and so on.
It’s a bit quick-and-dirty. I know. But it works well enough, and is a little less subjective than “Think of a number between one and six”, and the product owner always has the option of raising or lowering it later.
What rules of thumb to you use for assigning priorities, severities, seriosities or whatever? Are testers even responsible for assigning priorities at the time of logging, or is the deferred to the product owner or a bug triage meeting?
We use the word ‘Bug’ for all bugs, deviations, misunderstandings, fluctuation of reality… whatever, just bug.
We have a queue of bugs, 20 bugs with open status right now. We have 5 developers in our company, every week one of them has the support role, he deals with troubles from production and fixes bugs from the list, the support role iterates so all of them are aware of production pains and about existing bugs.
And that’s it.
As the tester over here, I do believe in what Antony Marcano (http://antonymarcano.com/blog/) once said about bug logs being a hidden technical debth backlog.
We could go faster, if every week this support dev was doing new features, but then we would not be doing better.
It is now a while since we were in full scale development mode, but our bugs were rated 1 to 4, where 1 was “it doesn’t work” and 4 was “this text is the wrong colour”. By the last six months we were at 1: “it’s going to be fixed” 2 to 4 “it’s not going to be fixed”, then we sent the priority 1 bugs to the prioritisation meeting. The prioritisation meeting ended up asking “does this bug impact on the validity of the main exported file?” and only bugs which invalidated that file were fixed. A valid file was eventually sent on the due date, but the due date was extended a few months as so many other teams didn’t have their files ready to exchange.
Of course that was last year. Now we have cleared all the 2 and 3 level faults and many of the level 4′s. The application is due to be replaced next year, so I doubt there will be many more releases.
A lot of the bugs that don’t get fixed but cause trouble further down the line are edge-cases that turn out not to be as obscure as first thought.
Spend a whole day earlier this week tracking down and isolating a bug that caused a wrong value in a report, then writing up a detailed bug report describing the problem.
We used to get a lot of bug reports which we would send back to the users and ask them to re-raise them a Change Requests.
To qualify as a bug the application must be performing at variance to its specification. If the business requirement changes then the fact that the code no longer meets the requirement is not a bug. The snag is that our department pays for time spent fixing bugs, but the user department has to fund Change Requests. Therefore it is in the users’ interest to raise everything as a bug.
This only matters at middle management levels, it’s all internal “funny money”, but when this application is outsourced next year the users will have to learn quickly as they will suddenly have to pay real money for their CRs and the IT team will bill everything they can because they need that real money.
The difference between a bug and a change request is highly subjective, and nothing good ever comes from a bureaucratic approach that places too much emphasis on that difference.
I once worked in an environment where developers’ performance appraisals (and hence bonuses) depended on the number of bugs found in their code. If there’s one thing more likely to politicise the entire fault logging process and get in the way of actually fixing issues…
One of our customers simply ignores the low, medium & high priorities and just logs EVERYTHING as critical. Even the one that said the web address on the printed statement must be underlined. Why? Because that’s what it looked like when they printed the design they had made in Excel ….
The users here used to put every bug down as priority 1. They were living in “the old days” when there were 13 people in the team. I simply explained that they now had one and a half people to work on the fixes and if they simply stuck everything in the top priority then we would choose which of those we would work on. Asking questions along the lines of “how much difference does this make to revenue?” helped, but really what they needed to know was that the priority setting was for their benefit rather than ours.
I recall an interesting approcah to priorities I saw once. An order could be given a “red” priority, which meant “drop everything else and do it now”. To stop it being abused, they ruled that the salesman who was responsible for the order had to stay with it all the time it was on the shop floor – on the basis that if the order was that important, then it deserved the salesman’s full attention. It worked well IIRC.
I like that rule to prevent abuse a *lot*!