I received some interesting feedback on a previous post of mine (Defect Count is Constant), and it really prompted me to define what is a defect, and why I believe that the defect count is fairly constant. This comment was from a developer, but I'm sure applies to many more people and roles in software.
I definitely don't want to ship a non-perfect product. Especially because I just got out of school (a year ago) and every assignment I handed in (except one) was perfect.
Before I get into what is a defect, let's look at an official definition:
- Dictionary.com - "The lack of something necessary or desirable for completion or perfection."
So why do I say it is inevitable that all software will have defects? It's easy to say that my experience has proven this again and again, but that's a coward's answer. The truth is that there are a number of reasons. First, here is my definition of a defect:
The software does not do something that it is supposed to do.
I know my definition seems fairly vague, and that is by design. Every piece of software is supposed to do something. What that is, who determines it, what the customer wants, these are all negotiable and this is where some of the confusion sets in. If a customer wants a button that turns the screen green, and the product manager writes a specification to make the screen blue, the programmer could make the world's most perfect button, the code could be beautiful, and the testing could pass without a hitch; and it is still a defect. The customer is not getting what they want. Don't think that I feel the customer is always innocent, sometimes they say green and mean an off shade of lavender, but that is a different post for a different day. While this may seem like a very simple issue, and one that could be fixed quickly, start to image an exponentially more complex scenario. 250+ customers, 50+ high priority customers, all who want the same features to work in different ways, 10 people in product management writing specifications, 60 developers spread across 3 countries, 5 timezones, and 2 continents, and 60 testers, only 10 of which have any experience with the prior iterations of the product. Now you can start to see where things fall through the cracks.
Sources of defects:
- Poor Programming
- Poor Requirements
- Missing Requirements
- Missed Defects in Testing
- Misinterpreted Customer Requests
- Untested Usage Scenarios
- And Many More.....
The Point of this Post
Again, why does all software have defects? The truth is, there is software out there that literally has 0 defects. According to my previous post, that software's constant defect count could be 0. All versions of that software should ship with 0 defects. But as you increase complexity, dependencies, and requirements, the defect count will always increase. I won't begin to project percentages of the defect could that could be attributed to any particular source, and that is usually dependent on the team and its process. A team with a really strong customer interaction and strong product management probably has very few requirements defects. This also typically trends to a higher level of developer defects since the requirements contain additional detail, and that leaves additional details to be missed during the programming process.
The other hidden truth in a constant defect count is the severity of these defects. Sure, the number of defects remains virtually the same, but the average severity should continue to decrease throughout the testing phase of the product lifecycle. A critical defect, in my mind defined as a defect that you can't ship without fixing, will get fixed. While re-testing is, the tester may find a typo on the screen. Now, the total defect count hasn't changed, but I think everyone would agree it's better to ship a product with a typo then a product that doesn't install properly (example of a critical defect).
At the end of the day, everyone is human. The more complex the task, the higher the risk and occurrence of defects. This is to be expected, and it becomes the role of the leads and managers to work through the list of defects. So yes, it is possible for software to be perfect, but the likelihood of that is inversely proportional to the size of the project. The true skill is in being able to manage a product and ship a release that isn't perfect, but having a solid enough understanding to ship the important pieces well, and make sure that all of the known defects are manageable.
I'll be making another post soon on how to manage a product's defect list during the development cycle. Until next time .....
