Blogs about Jazz

Blogs > Jazz Team Blog >

My engineers are telling me that we have to “qualify” DOORS Next Generation….!

But this is a Requirements Management tool – and its output is not part of my automotive/medical/avionics product… Are they pulling my leg?

They might be – engineers are prone to giving their managers heart attacks – but in this case they may simply be trying to ensure your company remains in regulatory compliance.

In our experience with providing tool qualification products at CertTech, LLC, we frequently hear this type of question. Before discussing a possible solution to your engineers’ concerns I think it would help if I first provide some background.  After all, tool qualification isn’t necessarily easy or inexpensive, every situation is unique, and it is worthwhile to explore their reasons for stating that DOORS NG requires tool qualification.

For safety and mission critical industries, all tools used in your development processes must be evaluated to determine how potential defects in each tool could negatively impact your product.  However, in many cases a full tool qualification process isn’t necessary because either:

  1. There are no potential tool defects that could lead to a product defect (or failure to detect the presence of a product defect); or
  2.  You have other cross-checking activities in your process stream that can clearly be shown to expose the product defects that would be associated with a defect in the tool itself.

For a Requirements Management tool, the first assumption is to allocate it to category 1.  For classic methods of managing requirements this is a perfectly justifiable decision. After all, early RM tools provided little to no “smarts” for operating on the requirements and any errors in the requirement content presented to the development and verification staff could be detected during a review.

DOORS NG, however, provides a rich set of interactions with the requirements beyond those available in the old “spreadsheet” approach for managing requirements. Instead, DOORS NG can perform operations on the data, thus opening the door for undetected tool defects to negatively impact the requirements data presented to the user.

Examples of where DOORS NG performs operations on the requirements data include:

  • Configuration Management (CM) operations to present requirements recorded in specific baselines or streams;
  • Presentation of, and reporting on, requirements based upon filters (and view) settings;
  • Comparison of different CM versions;
  • Maintenance of trace status (suspect links, link validity); and
  • Exporting and importing of requirements data.

Many of these intelligent features will be used by your engineering staff thus raising the possibility that a qualification is needed.  But how can you know for sure?  Well, some of the questions you might ask yourself that are applicable to any software tool could include:

  1.  “How much do you rely on the tool’s data operations to be working correctly?”
  2.  “What do you know about the known defects in your current version of the tool?”
  3.  “What kinds of defects have been found in earlier releases of the tool?”
  4.  “How confident are you that all of the tool defects have been found and fixed in the feature areas upon which you rely?”
  5.  “Could any of the known tool defects lead to a defect in your product that you would not have found using your current processes?”

These questions are not intended to diminish the value of the software tool, nor to imply that it has quality assurance concerns.  Instead, these questions are intended to illuminate the fact that people come to rely on tools and too easily forget that, just like with any product, tools are complex and can contain their own subtle errors.

If you can provide objective evidence that you have given due consideration to the possible consequences of defects in the Requirements Management tool, and you can clearly show how your processes would have detected these consequences, then (for you) performing a formal tool qualification of the tool can be avoided.

However, not all of us can be that confident. If you determine that you must qualify a software tool, what does that mean?

The basic steps for tool qualification include:

  • Identifying the key tool use cases where a tool defect could lead to shipping a product with undetected defects;
  • Decomposing those use cases to detailed tool operational requirements;
  • Developing and executing verification tests for each of the operational requirements;
  • Documenting test failures and describing options for working around those defects until they have been fixed;
  • Informing your tool users of the defects and any workaround steps;
  • Saving the tool qualification data in a configuration management system so that it can be accessed in case of future litigation; and
  • Updating all of the above listed qualification artifacts whenever you migrate to a new version of the tool.

A rigorous application of these steps can easily cost more than the tool itself. However, as I alluded at the beginning of the post, a cost-effective solution exists via the DOORS NG tool qualification kit provided through CertTech, LLC.

As you can see, the answer to the question “Do you need to qualify DOORS NG?” can vary from user to user.  For some of you the answer may be “no”, but even then you must document the reasons why.  For others it will be a clear “yes”. For the latter, and for those who are not sure, the DOORS NG tool qualification kit from CertTech can reduce the effort for meeting your tool qualification obligations.

(Dan Barbour is the Chief Technology Officer at CertTech.)