Summary
A discussion of how to write a requirement correctly for an FDA compliant medical device.
Conditions & Asserts
All requirements have the form:
If C then A
where C is a Condition and A is an Assert.
A Condition specifies the state of the system within which the Assert is allowed to be evaluated.
The Assert specifies the system's behavior when the Condition is in effect. An Assert has the word " shall" in it to indicate the Assert phrase must be met if the system is behaving correctly.
Fundamental Rules
A Condition C may or may not be true, and an Assert A may or may not be true. These rules describe what the possible combinations mean:
- If C is true and A is true then the requirement has been met. The system is behaving correctly.
- If C is true and A is false then the requirement is not met. The system has a defect.
- If C is false then the requirement does not apply and so it is indeterminate.
Indeterminacy
A requirement can be indeterminate. This can occur when, for example:
- the requirement applies elsewhere but not here
- the requirement applies at some other time, but not now
The Assert clause may still be true (or false), but we can't use the result. An example:
if on the ABC Screen, the Next button shall be displayed
is indeterminate if the current screen is the XYZ Screen i.e. it's not the ABC screen. There may, in fact, be a Next button displayed, but it doesn't matter. We are not on the ABC screen and so we have to throw out that result.
It may be indeterminate if, during the test execution, the system should have been on the ABC screen, but it is not. That is, when the tester was preparing to test, they incorrectly ended up on some other screen. Perhaps there is a defect in the system that prevents them from going to the correct screen. Or some system behavior can prevent them from testing the behavior at this point in time. In other words, they ran the preparatory test steps correctly, but the system has timing related behavior that prevents the correct screen from showing up now.
In any case, if a requirement is indeterminate, the automated test framework should exit the Test Case, or for manual testing, the test should be stopped. The rationale is that if a Condition is false:
- subsequent Conditions are likely to be false (whether rightly or wrongly)
- likely to lead to further indeterminate behavior
- may cause false positives
Implied Conditions
An Assert A is never implied. On the other hand, a Condition C may be implied.
Each requirement has a context. That context may be obvious or implied, e.g.
- The system is on
- The device under test is SystemABC
- The system has a color GUI
Specified in an indirect way. For example, the document section of the requirement may imply the Condition, e.g.
Section 1. ABC Screen Section
REQ-123 The Next button shall be displayed
In this case, Requirement REQ-123 only applies if you are testing the ABC Screen.
A Condition may be universal:
REQ-234 The background image shall be blue
Requirement REQ-234 applies no matter where in the system you are testing. Note, that there may still be implied Conditions even in this case, e.g. the system is on or the application has been started.
Compound Conditions
The Condition C may be compound, e.g.
if C1 and C2 then A
In this case both Conditions (C1 and C2) must be true for the Assert ***A''' to be evaluable. If either ''' C1''' or '''C2''' is false, the requirement is indeterminate.
Typically, the word "and" in a Condition is OK. However, the word "or" indicates that there are two (or more) requirements being specified, they just happen to have the same Assert. e.g.
SRS123 if C1 or C2 then A
It is usually not good practice to do this. Test Cases are tied to specific requirements. When the requirement is "C1 or C2", a question is raised as to which Condition is that Test Case is using, is it "C1" or is it " C2"? To remove this potential confusion, it is better to split the requirement into two:
SRS123 if C1 then A
SRS124 if C2 then A
Note this can be cumbersome at the very least and problematic as well. The Assert in both requirements need to change together, e.g. if the Assert in SRS123 is changed, then SRS124 must change as well.
Compound Asserts
The Assert '''A''' may be compound, e.g.
if C then
* A1
* A2
* A3
If Condition '''C''' is true then all of '''A1''', '''A2''' and '''A3''' must all evaluate to true for the requirement to be met. Conversely, if any of '''A1''', '''A2''' or '''A3''' is false, then the requirement is not met.
Using "and" in the Assert is OK, and it's OK for the "and" to be implied via list like the example above.
However, using the word "or" is problematic. For example:
If C then A1 or A2
Which option "A1" or "A2" should the test should confirm? The tester could check for both and confirm if either scenario occurs when the system is set up with Condition "C". But it does imply that the system is non-deterministic which is rarely ok. In that case, the system is randomly choosing which path to take. If that is the true expected behavior of the system, then having the "or" does make sense.
The more likely case is the Condition "C" is missing a clause that differentiates between the two scenarios. The requirement should be split into two or more requirements such that the "or" is not needed. For example:
If C and D then A1
If C and not D then A2
Note it is possible for the additional Condition to be embedded in language inside the Assert. For example:
REQ-500 On the Main Screen, if the User presses "Enter", the system shall navigate to the Therapy Screen if they've logged in already or to the Login Screen if they haven't.
The system is deterministic, and the requirement does capture the behaviors correctly. For clarity ( and tracing purposes), it should be split into two requirements:
REQ-500 On the Main Screen, if the User presses "Enter" and they've logged in already, the system shall navigate to the Therapy Screen
REQ-501 On the Main Screen, if the User presses "Enter" and they haven't logging in already, the system shall navigate to the Login Screen.
It is possible for the Assert to become very complex and may apply to multiple requirements. One way to do this is to use a document section to specify the Assert, for example:
Section 1 Main Screens
SRS-100 On the Main Screen, there shall be a navigation bar that complies with all requirements in the Navigation Bar Section
SRS-101 On the Therapy Screen, there shall be a navigation bar that complies with all requirements in the Navigation Bar Section
<snip>
Section 2 Navigation Bar
SRS-300.1 The navigation bar shall indicate the current screen's unique name
SRS-300.2 The navigation bar shall indicate which step in the current process the User is performing
SRS-300.3 The navigation bar shall be at the top of the screen
If this is done, there are complications in the testing. For example, the navigation bar has to be tested in each screen that it appears.
If a navigation bar requirement fails, then that "calling" requirement also has to fail, e.g. say SRS-300.2 failed for the Therapy Screen. Then the Test Report needs to indicate that SRS-300.2 failed and that SRS-101 failed as well. But note that SRS-300.2 passed on the Main Screen (SRS-100).
Multiple Asserts
Multiple Asserts are evaluated instantaneously. In other words the order that the Asserts are evaluated is irrelevant.
For example the following are all equivalent to each other:
1) If C1 then A1; A2; A3
2) If C1 then A3; A1; A2
3) If C1 then A2; A3; A1
etc.
This also spans requirements, e.g. If '''C1''' and '''C2''' are both true, then the following are equivalent:
1) If C1 then A1; If C2 then A2
2) If C2 then A2; If C1 then A1
3) If C1 & C2 then A1; A2
4) If C1 & C2 then A2; A1
etc.