Requirements verification matrix (RVTM): columns, methods and a worked example

What a requirements verification matrix is

Verification asks whether the system was built right: does it meet each requirement as written? The requirements verification matrix is the place where that question is answered requirement by requirement. Each row is one requirement. The columns say how it will be verified, at what level of assembly, by which event, and what came out of that event.

The same table goes by several names. RVTMRequirements Verification Traceability Matrix A table of each requirement with how it is verified, the tests that verify it and their latest result. (requirements verification traceability matrix) stresses the link from requirement to test. VCRMVerification Cross-Reference Matrix A table that maps requirements to verification methods (test, analysis, inspection, demonstration) and events. (verification cross-reference matrix) is common in defense and aerospace programs and often means the planning half: requirement, method and level. A verification matrix or compliance matrix may add a compliance statement per row for a customer. The content differs a little from one organization to the next, but the idea is the same: one row per requirement, nothing left without a planned way to show it is met.

ISO/IEC/IEEE 15288 describes verification as a process that plans the verification actions, sets up what they need, performs them and records the results, and it expects every requirement to be covered by at least one action. The matrix is the usual record of that coverage. The INCOSEInternational Council on Systems Engineering The professional society for systems engineering. It publishes the Systems Engineering Handbook. Systems Engineering Handbook treats it the same way: a working tool during development and a deliverable at reviews. The matrix is the verification end of requirements traceability: where traceability asks where a requirement came from and what satisfies it, the RVTM asks what proves it.

The worked example throughout is EO-1, a fictional 6U CubeSat that images farmland in four spectral bands from a 500 km orbit. Its specification has 32 requirements and 19 verification events in two test suites. It is the same project the MBSE example project walk-through follows from stakeholder needs to documents, so the rows here can be traced back to the functions, components and analyses that answer them.

Who uses it, and when

• Requirements reviews (for example a system requirements review). Reviewers check that every requirement has a method and a level. A requirement nobody can say how to verify is usually ambiguous or not measurable, and the review is the cheapest time to rewrite it.
• Design reviews. The matrix now names test cases, analyses and inspections. Reviewers look for requirements still without an event and for tests that need facilities or articles nobody has planned.
• Test readiness reviews. Before a campaign starts, the matrix shows which requirements the campaign will close and which procedures are ready.
• Qualification. Qualification shows that the design meets its requirements, often with margin, on a test article built for the purpose. The matrix lists each qualification result and its report.
• Acceptance. Acceptance shows that each delivered unit was built and works as designed. The customer signs off against the matrix, row by row, with evidence attached or referenced.
• Audits and certification. An auditor picks rows and follows them to the evidence. A matrix whose evidence column points to real, versioned files survives that; one that says "see test report" without a version does not.

The columns

A practical matrix has these columns. Some organizations split or rename them; few leave any out.

The first four columns are the plan. The last three are the results. Keeping the two halves apart matters, because they change at different times and for different reasons (see planning versus results).

Choosing a verification method

There are four standard methods. They are listed here from the least to the most expensive, which is also the usual order of preference when more than one would do.

Inspection

Examining the item or its documentation without operating it: measuring a dimension, reading a label, checking a certificate, a drawing or a configuration setting. Good for physical features, markings, materials and anything stated as "shall have" or "shall be made of". It also covers reviewing records produced elsewhere: EO-1 verifies 3.4 Bus Regulation (12 V within 2%) by inspecting the power distribution unit's qualification report and its load-step data instead of repeating the test.

Analysis

Showing the requirement is met by calculation, modeling or simulation, often using test data from lower levels. Good when testing the real thing is impossible or unsafe, or when the analysis covers many more cases than any test could. EO-1's 1.7 Mission Lifetime (730 days in orbit) can only be shown by a reliability analysis, and 1.6.1 Ground Track Repeat by propagating the orbit.

Demonstration

Operating the item to show that it does something, observed but not instrumented for precise measurement. Good for functional and operational requirements with a yes-or-no outcome: EO-1's 6.1 Contact Scheduling (contacts booked 48 hours ahead) is shown by running the operations team through 14 days of rehearsal.

Test

Operating the item under controlled conditions and measuring the result against a number. Good for performance requirements with a value and a tolerance, such as 2.1 Pointing Accuracy (within 0.1 degrees of the target). Test produces the strongest evidence and usually costs the most.

Some questions that settle the choice:

• Does the requirement contain a number? Then it usually needs test or analysis; demonstration cannot show "within 0.1 degrees".
• Can the condition be reproduced? Two years in orbit, or a 25 degree off-nadir pass over a real field, cannot be staged in a lab. Those go to analysis, backed by a test at a point the lab can reach.
• At what level can it be seen? Battery capacity can be tested on the battery pack (component level); pointing accuracy needs the attitude subsystem; the 72 hour revisit only exists for the whole mission. Verify each requirement at the lowest level where the property is fully present.
• What will the customer accept? Contracts often fix the method for key requirements. Agree on it early, not at acceptance.

Write the success criteria at the same time. "Meets the requirement" is not a criterion. "Error of at most 0.1 degrees on each of 20 commanded targets" is: anyone can run it and reach the same verdict.

Planning versus results

The planning half of the matrix is settled during requirements analysis and design. It answers "how will we know?" and should be complete long before any hardware exists. A requirement with no method at the requirements review is a defect in the requirement set, not a gap in the test program.

The results half is filled during integration and test. It changes every time an event runs, and it is only meaningful together with what was tested: the build or software version, the test article's serial number, the date and the person. A result without its configuration cannot be repeated or audited, and it cannot tell you which build fixed a failure.

Two habits keep the halves honest:

• One event per row of evidence. An analysis report, an inspection record and a demonstration are verification events just as a test case is. Give each its own entry with a result, so the matrix shows evidence for every method and not only for tests.
• Verify the leaves, roll up the parents. A parent requirement decomposed into children is verified when all its children are, or when it has its own event. Rolling up avoids planning duplicate tests for the parent and shows at once which branch is holding it back.

Status values and roll-up

Five status values cover the life of a row:

A verification event has its own, finer states: not run, in progress, blocked, failed or passed. For the matrix, an event that is in progress or blocked has no verdict yet, so its requirement counts as not run.

For a parent requirement, a failure anywhere below it makes it failed, and it passes only when every child passes (or its own event does). Otherwise it shows how far its least-covered branch has come. Counting rows by status gives the one number most reviews ask for first: how many requirements are verified out of how many.

Keep it generated from the model

Most matrices start life as a spreadsheet, and most spreadsheets drift. The requirement text is pasted in and goes stale when the requirement changes. Test status is typed by hand after each run, by whoever remembers. Two copies circulate before a review, and nobody is sure which one the customer saw. None of this is carelessness; a spreadsheet is a copy, and copies diverge.

The alternative is to keep the matrix's content where it already lives and generate the table from it:

• method, level and success criteria are attributes of the requirement;
• each verification event is an item linked to the requirements it verifies;
• each result, with its configuration and evidence, is recorded on the event when it runs;
• the matrix is a query over those links, rebuilt each time it is opened or exported.

The matrix then cannot disagree with the requirements or the test records, because it has no data of its own. An exported copy for a review is still a snapshot, but it is made in seconds from the current model rather than maintained by hand. This is one of the plainest payoffs of keeping requirements, design and verification in one model; the verification features page describes how Holarch does it.

How failed and changed results show

Two kinds of change make a row wrong after it looked finished.

A failed result

A failure is information, not an embarrassment, and the matrix should show it plainly. The failed event stays failed until it is re-run; it is not quietly reset. Each failure gets an issue (a problem report or discrepancy report) linked to the event and to the requirements it verifies, so the matrix row leads to the investigation. When the fix arrives, the event is re-run on the new configuration, and the record keeps both runs: the failure on the old build and the pass on the new one. Saving the results of each test campaign as a snapshot (often called a test cycle) lets you compare campaigns and see exactly which rows changed.

A changed requirement

The quieter problem is a requirement that changes after its test passed. If a power requirement rises from 28 W to 30 W, the old pass no longer proves anything about the new value, but a status column will still say "passed". A good matrix flags the link between the changed requirement and its verifying event as suspect until someone reviews it: either the event already covers the new value, or the procedure must change and the event must run again. The same applies when something upstream of the requirement changes. Suspect links in general are covered in the traceability guide's section on suspect links; for verification the rule is short: a pass recorded before the requirement last changed needs a review before it counts at a milestone.

Reviewing coverage before a milestone

A week before a review, walk the matrix with these questions:

1. Unverified rows. Every requirement should have at least a method. Rows without one are either missing a plan or are not verifiable as written.
2. Planned-only rows. For a design review, each should have an event or a reason it does not yet. For acceptance, none should remain.
3. Failed rows. Each needs a linked issue with an owner and a plan. A failure without an issue is the most common finding at test reviews.
4. Suspect rows. Each needs a decision: still valid, or re-run.
5. Level and method sanity. Look for system-level requirements verified only by a component test, for numeric requirements verified only by demonstration, and for events whose procedure does not measure what the requirement states.
6. Evidence. Spot-check a few passed rows: does the evidence exist, and does its configuration match what is being reviewed?
7. Orphan events. Test cases that verify no requirement cost money and prove nothing on the record. Link them or retire them.

Record the counts per status at each milestone. The trend (unverified falling, passed rising, failed closing) says more about the program's health than any single snapshot.

Worked example: the EO-1 CubeSat RVTM

The example is the CubeSat EO-1 demo project in Holarch, where the requirements live in SRS-001 EO-1 System Requirements Specification, verification events are Test Cases in two Test Suites, and each test case verifies its requirements. One change was added to the demo for this guide, the same one the MBSE example's change step makes: after the preliminary design review the orbit-average load grew to 26 W, so requirement 3.1 Orbit Average Power was raised from 28 W to 30 W.

Download the example: the EO-1 requirements verification matrix as an Excel workbook (.xlsx, 32 rows) or the same matrix as CSV. Both are Holarch's own RVTMRequirements Verification Traceability Matrix A table of each requirement with how it is verified, the tests that verify it and their latest result. export of the project shown below: requirement number and name, method, level, success criteria, verifying test cases with their status, own status and rolled-up status. Use them as a template for your own columns.

The plan

A selection of rows, one for each method and level, with the success criteria as written in the specification:

All four methods appear, at three levels. Battery capacity is tested on the pack alone, because the property is fully present there; pointing accuracy needs the attitude subsystem on an air-bearing table; the ground track repeat is a property of the orbit and goes to analysis. 6.5 is verified by a probabilistic analysis of the operations timeline, the kind of check the Monte Carlo guide walks through. 3.1's criterion is a prediction from a measurement: the test measures the arrays' peak power, and an equation in the power budget turns it into an orbit-average figure. 2.5 has no method because, as written, it cannot be verified.

The results and the test campaigns

The 19 test cases sit in two suites: TS-1 Spacecraft Qualification Tests (15 test cases, from optical bench tests to analyses) and TS-2 Communications and Ground Tests (4). Each recorded result carries the tester, the date, the configuration (flight software build, hardware serial numbers) and, where there is one, the evidence file.

TC-04 Pointing Accuracy Test failed: on its second run, with flight software 2.3.1, three of 20 targets were outside 0.1 degrees, the largest at 0.14 degrees, and wheel friction is suspected. The failure has an issue linked to the test case and, through it, to requirements 2.1 and 2.2. TC-09 Safe Mode Entry Test failed the same way (14 seconds against 10) and has its own issue. Two test cases, TC-11 UHFUltra High Frequency Radio frequencies from 300 MHz to 3 GHz, often used for spacecraft command links. Command Test and TC-17 Slew Time Analysis, are in progress; in the matrix they count as not run until they have a verdict.

The flatsat campaign was run twice, and each campaign's results were saved as a test cycle. Comparing the two shows what the second campaign changed:

• Flatsat Cycle 1: 5 of 9 passed. TC-03 and TC-06 were not run; TC-04 (0.16 degrees on 5 targets) and TC-09 (15 seconds) failed.
• Flatsat Cycle 2: 8 of 11 passed. TC-03 and TC-06 passed, the analysis TC-13 was added and passed, TC-14 was added but not run. TC-04 and TC-09 improved (0.14 degrees on 3 targets, 14 seconds) but still failed.

"Improved but still failed" is exactly what a status column hides and a cycle comparison keeps: the fixes between campaigns moved the numbers in the right direction, and neither is closed.

Reading the matrix

The matrix is generated from the requirements and test cases above; nothing in it was typed into a table. The cards count rows by rolled-up status, and the five counts add up to the 32 requirements: 11 passed, 3 failed, 12 not run, 3 planned-only, 3 unverified. Reading it the way a review board would:

• 11 of 32 verified (34%). Passes by test (1.2, 3.2), by analysis (1.6.1, 1.7) and by inspection (3.4). No demonstration has passed yet: all three (5.4, 6.1, 6.2) wait for the ground segment rehearsals in TS-2.
• 2.1 and 2.2 failed through one test. TC-04 verifies both, so one failure fails two rows. Look closer and 2.2 Pointing Stability has a second problem: its criterion is drift over each 40 s capture, but TC-04 measures only pointing error at each target. Even a passing TC-04 would not show 2.2 is met. That is a method-sanity finding: 2.2 needs its own measurement, or TC-04's procedure must record drift.
• 4.3 failed through TC-09. The issue and a task to raise the watchdog polling rate and rerun TC-09 are on record; the row stays failed until TC-09 passes on the new software.
• 1.6 Revisit Time rolls up. On its own it is only planned (analysis, no event), but its two children decide it: 1.6.1 passed, 1.6.2 Off-Nadir Imaging is not run, so 1.6 shows not run. No separate test of the parent is needed once both children pass.
• Planned-only: 1.5, 4.4 and 6.5. Each has a method and no event. 4.4 Downlink Encryption has neither level nor criteria either, so its plan is not finished.
• Unverified: 2.5, 3.5 and 6.4. No method, no event. These are also the three weakest requirements in the specification ("quickly … as appropriate", "warm enough (TBDTo Be Determined A requirement value or detail that is not known yet. Holarch tracks TBDs with an owner and a due date.)", "user friendly and/or easy to use"). Gaps in the matrix tend to point at requirements that need rewriting, not at missing tests.
• 3.1 is suspect. See the next section.

A changed requirement: 3.1 after PDR

TC-06 Solar Array Power Test passed: the expected result was at least 72 W peak, and the arrays delivered 75.1 W. Then 3.1 was raised from 28 W to 30 W. The status columns still read passed, because TC-06 did pass. The ⚠ after the requirement and after TC-06 says that the pass is older than the requirement. Clicking it lists every link from 3.1 that the change made suspect:

For the verification link the question is concrete: does 75.1 W of measured peak power still show 30 W orbit average? The power budget answers it. Its equation EQ-1.3 turns peak into orbit-average power with the sunlit fraction of the orbit (0.62) and the power path efficiency (0.6):

So the pass cannot be carried over: to show 30 W, the arrays would need about 80.6 W peak, and TC-06 measured 75.1 W on qualification panels at beginning of life, before any end-of-life degradation. The review also turns up an older problem the change exposed: TC-06's 72 W threshold corresponds to only 26.8 W orbit average, so even against 28 W the test's expected result was not derived from the requirement through the power budget. The right outcome is to keep the link suspect, raise an issue, revise TC-06's expected result from the equation (and decide with the power engineer whether the arrays or the margin change), then re-run it. The suspect mark stays until that decision is on the record, so nobody reads the old "Passed" as evidence for 30 W.

The exported matrix holds verification results only, not suspect flags: in the downloaded workbook 3.1 reads Passed. Review the suspect rows in the model before you export a copy for a milestone.

Planning the remaining rows before the review

The coverage list shows the 21 rows that are not yet verified (32 minus 11), with the controls to finish their plan. Before the critical design review, the work it lays out is:

1. Rewrite 2.5, 3.5 and 6.4 so they can be verified, then give each a method, level and criteria.
2. Give 4.4 a level and criteria, and create the inspection for it; create the analyses for 1.5 and 6.5.
3. Close the failures: TC-04 after the wheel investigation, TC-09 after the watchdog change, with 2.2's drift measured on its own.
4. Decide on 3.1: revise and re-run TC-06 against 30 W.
5. Schedule the not-run events, starting with TC-19 Array Deployment Test and the four TS-2 events, which between them verify eight requirements.

Recording the counts at each review (here 11 passed, 3 failed, 12 not run, 3 planned-only, 3 unverified) gives the trend the next review will compare against.

Common mistakes

• Restating the requirement as the success criterion. "The ADCSAttitude Determination and Control System The spacecraft subsystem that senses and controls which way the spacecraft points. points within 0.1 degrees" says nothing about how many targets, under what conditions or against what reference.
• Test thresholds not derived from the requirement. A test's expected result should come from the requirement through the analysis that links them, as TC-06's 72 W did not. Check the arithmetic when the test is written, not when the requirement changes.
• One test for several requirements without measuring each. Sharing a test is efficient, but only if the procedure records every quantity the linked requirements state.
• Verifying parents directly. Writing a test for a parent requirement that only repeats its children's tests doubles the work and the paperwork. Verify the leaves and let the parent roll up.
• Analysis without evidence. An analysis is a verification event and needs a report, a version and a verdict like any test.
• Linking a test to a requirement with a generic "related" link. The matrix counts only verification links; a test linked any other way leaves the requirement looking unverified.
• Resetting a failure to "not run" to tidy the matrix. Keep the failure and its issue; the history is what an auditor asks for.
• Ignoring suspect rows at acceptance. A pass that predates the requirement it claims to verify is not evidence for the current requirement.

Try it in Holarch

1. Create the EO-1 project: Manage Projects → Project Files ▾ → New Demo Project. Or start from your own requirements in a requirements document; see Requirements.
2. Open Verification → Test Center. In Verification Coverage, set each requirement's method with A, D, I and T, choose a level in Level…, and add success criteria with + Criteria.
3. Create test suites and test cases with + New ▾ and + Test Case, and link each test case to its requirements with + in the Verifies column. Analyses, inspections and demonstrations are test cases too.
4. Record results with the status buttons, or walk a campaign with Run Suite, which records the tester, configuration, test article and evidence file for each run. After a failure, click + Create Issue.
5. Save a campaign with Start New Cycle…, and compare campaigns in the suite's ⋯ → Test Cycles….
6. Open Verification → RVTMRequirements Verification Traceability Matrix A table of each requirement with how it is verified, the tests that verify it and their latest result.. Click a status card to filter the rows. To see a suspect mark, change requirement 3.1's text, then click the ⚠ next to it.
7. Click Export ▾ for Excel (.xlsx), CSV or Word (.docx). The downloadable example above is that export.

The full reference is in Verification.

Where to go next

• Verification in Holarch: test cases, runs with evidence, test cycles and the RVTMRequirements Verification Traceability Matrix A table of each requirement with how it is verified, the tests that verify it and their latest result. on one model.
• Verification docs: planning attributes, status rules, run mode, test cycles and the RVTM reference.
• Requirements docs: writing and approving requirements, and the Suspect Links page.