What?

When defective requirements contribute to significant system failures or expensive rework, consider the following preemptive testing strategy.

Assign the responsibility for significantly reducing requirements defects to software testers. Their job is not to develop requirements, but to identify the human causes of requirements defects and to identify mitigations for these defects. In many organizations, no one “owns” requirements defect prevention and detection, but identified ownership is much more effective than project team ownership.

This added responsibility will require a focus on requirements quality as well as code quality, a broader understanding of requirements and their development, and a variety of new collaborative skills. It will also significantly increase the organizational value of testing, leverage the inherent strengths of skilled testers, and reduce requirements volatility. We call this strategy preemptive testing (PT).

Why?

Requirements defects may account for most of the defect costs on a project because they are plentiful and expensive when detected late. IBM and Bell Labs studies show that 80 percent of all product defects are created during requirements definition. Many software problems begin with defective requirements that then breed defective designs, data, code, tests, and documentation.

The cost of a requirements defect discovered in production, is the total cost of:

  • Implementation failures, ranging from annoying to life-critical
  • Implementation defect isolation
  • Patches or corrections to designs, data, code, tests, documentation, and requirements
  • Change control and configuration management
  • Reverification and validation
  • Redistribution
  • Causal analysis
  • Hazard mitigation

But the cost of detecting the same defect during requirements development, is only the total cost of:

  • Early detection
  • Causal analysis
  • Hazard mitigation

Improved requirements is a natural side-effect of test strategies that entail early test design. Preemptive testing aims to make it a primary effect.

How?

Before implementation, baseline your reality by: (1) performing human causal analysis on each significant system failure that happens during development or production, and (2) monitoring the number, impact, and nature of requirements defects.

To use preemptive testing, testers must:

  1. Learn
    • requirements fundamentals, including types, forms, and specification alternatives
    • types and human causes of requirements defects
    • causal analysis of major system failures
    • techniques for demonstrating understanding
    • techniques for checking requirements
  2. Identify common (human) causes of your requirements defects based on:
    • requirements defects discovered by analysis of major failures
    • requirements misunderstood by developers
    • defects discovered by requirements checking
  3. Identify common causes of the human errors leading to requirements defects
  4. Identify and implement mitigations for common causes of major failures
  5. Monitor and report mitigation effectiveness
  6. Repeat steps 2 – 5