The defect 4-step

What to do when you find a defect

The defect 4-step is not a new dance craze. It’s a way to accomplish organizational learning from the opportunity provided by a defect. So, here is what you should do when you find a defect:

  1. Fix it.
  2. Find and fix others “like it”. “Like it” could be along several dimensions and at multiple levels of abstraction (see below). Code query can really help here but sometimes a manual analysis can be effective.
  3. Prevent future occurrences “like it”…
    • from leaving the developers desk (preferable). This is often satisfied with some from of static analysis that runs during the build process or at the developers desk (think, upgrading the compiler). Frequently, the best you can do is share the bad news or conduct training so developers know what patterns lead to defects “like this”. Sometimes, you can change your technology or the programming paradigm to make the defect impossible. Switching from C to Java to avoid certain memory problems for example.
    • from getting into production (fallback). Tests that run overnight or manual tests that become part of the test plan are common approaches. Adding the meta-pattern(s) to a code review checklist can also help especially if it increases awareness and prevents defects “like it” from ever being written. Long-running overnight static analysis is sometimes in this category but it is preferable to run static analysis in the build process or on the desktop.

Why is this called a 4-step?

When I originally created this post, I had the two prevention “steps” broken out. In reality, you generally only do one or the other “prevent” steps. So, I guess I could rename this the defect 3-step.

What does “like it” mean?

The definition of “like it” for any given defect might be along several different dimensions and at multiple levels of abstraction. An example is the best way to illustrate this. We recently had a “stop-the-release” P1 defect found on the Friday before a Saturday release. An entire part of the application didn’t work in certain web browsers. There was an extra trailing comma added to the javascript that most browsers ignore but some complain about as a parsing error. Normal testing includes those browsers but didn’t catch it because testing was done on a version before the defect was injected. The definition of “like it” in this circumstance is along two different dimensions. One dimension of “like it” might try to address the need to make sure testing occurs on the latest version. A second dimension of “like it” might target the trailing comma pattern. The simple answer in this case was to implement JSLint, a static analysis tool for javascript. This is a great solution because it is at a higher level of abstraction from merely fixing a trailing comma defect. JSLint can be used to mitigate against ANY javascript parse error. It’s often more work to address the higher level of abstraction but it’s also usually more valuable. This trade-off decision should be made on a case-by-case basis but higher levels of abstraction should generally be favored. Testing is rarely able to climb the abstraction ladder so static analysis is favored.


This entry was posted in Software craftsmanship. Bookmark the permalink.

One Response to The defect 4-step

  1. Pingback: Organizational memory definition - Topic Research, Trends and Surveys

Leave a Reply

Your email address will not be published.

Spam Protection by WP-SpamFree