In a thread on the Software Testing Club, one poster came up with this gem:
I repeat, if you have no requirements, you cannot test anything. I think it’s important to grasp that there is a fundamental difference between a test, and an experiment
To which I replied:
If you have no requirements, you have to make assumptions about how and what to test based on your knowledge of the domain and the system.
Being able to make those sorts of assumptions is part of the skill-set of a good tester. So is knowing when you lack sufficient domain knowledge to be able to make any meaningful assumptions.
It’s the difference between intelligent exploratory testing and mindless script-checking. I test using a mental model of the system. Detailed functional specifications are often a very important source for that model, and when they exist, a tester should certainly read them. But knowing who and what to ask when you have gaps in your mental model is just as important.
It’s worse than not being able to test anything!
If you don’t have the requirements there is no point trying to build the application in the first place. You cannot hope to deliver a high quality project if the users cannot tell you what they want.
You can start doing things, in fact I have frequently had to start work on projects without the specification, but this is a high risk option as you may well develop elements which are not useful in the end. It may be that the risk is worth taking if there are other constraints such as available developer time or even calendar time, but eventually this risk will hit you, and hit you hard.
Test driven development is not possible without a specification as you cannot define a test to be passed if you don’t know what a pass looks like.
You observation that it is important to know who to ask about gaps. However, this is making the assumption that there is someone to ask at all. This is not always the case!
It’s difficult for any software project to be a success if you don’t actually know what the user wants. But the poster I quoted was claiming you need a detailed specification to test against, and should not test until you have one. My counter-argument is that a tester’s job goes well beyond that sort of checking.
A significant number of product defects I deal with aren’t cases where what’s been coded doesn’t match the spec – it’s where requirements have been missed, or dealing with situations that the defined business rules didn’t cover.
A real-life example: A while ago I wanted to book a hotel room in York on a Friday night. The online booking system showed the hotel fully booked on Thursday and Saturday, but there was a vacant room on Friday. When I tried to make a booking, I got an error saying you can only book two-night stays at a weekend.
Do you need a specification to tell whether that’s a bug or not?
Well, it so happens that I do have some background knowledge of hotel booking practices and, absurd though it seems, I would not assume the situation you describe is a bug right out of hand. I would raise a query on it, but i could believe they did want the effect you describe.
Don’t know about hotels but I’ve worked on booking systems for caravan parks and forcing a minimum number of nights at different points in the week is perfectly normal. Allowing you to get an unbookable night is more likely to be a failure of the logic the business chooses to operate rather than a fault in the system that enforces that logic (i.e. PEBCAK – problem exists between chair and keyboard). Probably if you had phoned them, the human would have happily booked you in for one night because they don’t want to let the system have the discretion to do that.
How about this one. Steel stockholder. Lorry pulls up and driver hands the yard a piece of paper that says he is delivering 3 metre round bars. The yard says “put it there” and books in the receipt, signs the paper and the lorry dumps the load and leaves. The yard then go to put the stuff away and find that they’ve actually got a load of 4 metre flat strips. What do they do? Shrug and put it away and expect the system to automatically unpost the original receipt and book in the strips instead. Even though there is no purchase order for it. Or it could have been 10 tonnes of rubber chickens that got delivered for all they care ….
Anyway, back to the original question. I think there has to be some sort of requirement, however vague it may be, otherwise you could end up not going near the area that was changed at all. Especially in a big sprawling ERP system.
Business logic does lead to strange situations. I am assured by a telecoms engineer that he was once unable to get a part from the stores because it was the last one and the warehouseman was required to have one in stock at all times.
I’ve worked on more than one stock control system as an analyst/developer, and all of them have a function to return stock to the supplier because it doesn’t match what was ordered. It’s one of the standard functions I’d expect to see in such a system. For that “last one in stock” situation there might be a business rule that the last item can only be issued for high-priority jobs or require management authorisation.
Going back to the hotel reservations issue, that might just be a case of a situation that occurs rarely enough that a manual work-around (i.e. making the reservation by phone) is an acceptable solution.
Another hotel booking system (for a well known chain) does things differently – if I put in for the one night I wanted it told me “nothing available”. If I extended the stay to 2 nights, it suddenly found a room.
I dont think either system is right from a customer perspective, but I expect they both meet the agreed requirements / signed specification.
No problems with return to supplier. But what beggars belief is that someone can’t tell the difference between a round bar of steel and a flat bar of steel (or a bar of steel and a rubber chicken) before they even get the load off the lorry. But then again this outfit is a place where they bought 2 x £6000 printers for doing kevlar labels and the workforce refused to use them – so now the printers sit idle in the disabled toilet. They also have “holiday bars” – steel that disappears on one stocktake and then reappears on the next. Ho hum.