Verification vs validation in systems engineering

The short answer

Verification asks: did we build the system right? It compares the system with its requirements, statement by statement, and produces evidence that each one is met. Validation asks: did we build the right system? It compares the system with what its stakeholders need and how they will use it, and produces evidence that the system does the job it was bought for.

The two questions sound alike and are often run by the same team with the same equipment, but they can give different answers. A system can pass every requirement and still fail its users, because a requirement was missing, wrong or written for a different use than the real one. A system can also serve its users well while failing a requirement that turns out not to matter. Systems engineering keeps the two apart because each one catches mistakes the other cannot see.

The phrasing "building the product right" versus "building the right product" is usually credited to Barry Boehm's early work on software engineering, and it has stayed the shortest way to remember the difference.

Definitions in ISO/IEC/IEEE 15288

ISO/IEC/IEEE 15288, the international standard for system life cycle processes, treats verification and validation as two separate technical processes: the Verification process (clause 6.4.9) and the Validation process (clause 6.4.11). Integration sits before verification, and transition sits between the two: the system is verified as built, handed over to its operating environment, and then validated there.

• Verification is confirmation, through objective evidence, that specified requirements have been fulfilled. Its purpose in the standard is to show that the system or element meets its requirements and characteristics, and to report any anomaly found on the way.
• Validation is confirmation, through objective evidence, that the requirements for a specific intended use or application have been fulfilled. Its purpose is to show that the system, when used in its intended environment, fulfills the stakeholders' needs and achieves its business or mission objectives.

Both definitions rest on objective evidence: a measured result, an analysis report, an inspection record or an observed demonstration that someone else could check. Opinion, "it looked fine" and a test that nobody wrote down are not evidence. The difference between the two lies in the reference each one compares against.

The INCOSE Systems Engineering Handbook (5th edition, sections 2.3.5.9 and 2.3.5.11) follows the same split, and the NASA Systems Engineering Handbook (NASA/SP-2016-6105 Rev 2) gives them separate product realization processes: product verification (section 5.3) and product validation (section 5.4). NASA's handbook makes a useful point plainly: verification shows the product was built to the specification; validation shows the specification, and therefore the product, was the right one.

What each one checks against

The clearest way to tell them apart is to ask what the evidence is compared with.

• Verification checks against requirements. The reference is the "shall" statements of the system, subsystem and component specifications. They are written by the engineering team, agreed with the stakeholders, and phrased so that a test, analysis, inspection or demonstration can give a pass or fail. ISO/IEC/IEEE 29148 lists verifiable among the characteristics every requirement must have (clause 5.2.5) for exactly this reason: a requirement that cannot be verified cannot take part in verification at all.
• Validation checks against stakeholder needs and intended use. The reference is what users, operators, customers and other stakeholders expect, in their own terms, together with the conditions of real use: the operational scenarios, the environment, the people who will run the system and the measures of effectiveness that say what "good" means to them. Needs are often qualitative, so validation leans on scenarios, simulation, trials and stakeholder judgement as well as measurement.

The traceability guide's comparison of needs and requirements puts it in one line: needs are validated, requirements are verified. Everything else in this guide follows from that.

When each happens in the life cycle

The V

The Vee model draws the life cycle as a V. The left side goes down through decomposition and definition: concept of operations, system requirements, high-level design, detailed design. Implementation sits at the bottom. The right side goes up through integration and checking, and each step on the right answers to the step across from it:

The top pair is the only validation pair, which is why validation is often pictured as the last step. The V shows where the final evidence is produced, not when the work starts. Each verification and validation activity is planned on the left side, at the same time as the item it checks: the validation approach with the concept of operations, the verification method with each requirement.

Incremental and continuous V&V

Waiting for an integrated system before asking whether it is the right one is the most expensive way to learn that it is not. Practice spreads both activities across the life cycle:

• Early validation. Concepts, mock-ups, sample products and simulations go in front of stakeholders long before hardware exists. Each review lowers the risk that the requirements encode the wrong need.
• Verification at every level. Components are verified before integration, subsystems before the system. A defect found on the bench costs far less than one found in system test.
• Continuous V&V in iterative development. Teams that deliver in increments, such as software teams working with continuous integration, run automated verification on every build and validate each increment with users. The two questions stay the same; they are asked much more often.

The four verification methods

Most standards and handbooks use the same four methods. The RVTM guide explains how to choose among them; here each is shown with a row from the EO-1 CubeSat used throughout this guide, a fictional 6U spacecraft that images farmland in four spectral bands from a 500 km orbit, with 32 requirements in its specification.

Inspection

Examining the item or its records without operating it. EO-1's 3.4 Bus Regulation (12 V within 2%) is verified by TC-18 Bus Regulation Inspection, a review of the power distribution unit's design and its qualification load-step data. It passed: the worst case was 11.81 V, inside 11.76–12.24 V.

Analysis

Calculation, modeling or simulation, often fed by lower-level test data. 1.7 Mission Lifetime (730 days in orbit) cannot be tested on the ground, so TC-16 Lifetime Reliability Analysis combines part failure rates, radiation dose and battery fade. It passed with a probability of 0.86 of operating for 730 days, against at least 0.8.

Demonstration

Operating the item to show that it does something, observed but not precisely measured. 6.1 Contact Scheduling (contacts booked 48 hours ahead) is to be shown by TC-20 Operations Rehearsal: 14 days of contact scheduling with the partner ground network. It has not run yet.

Test

Operating the item under controlled conditions and measuring the result against a value. 3.2 Battery Capacity (at least 77 Wh at beginning of life) is verified by TC-07 Battery Capacity Test, a C/2 discharge at 20 °C. It passed at 78.3 Wh.

All four are verification. A demonstration is not "a softer test" and an analysis is not a validation activity just because it uses a model: what makes any of them verification is that the pass criterion comes from a requirement.

Validation activities

Validation uses many of the same techniques, aimed at a different reference. The common forms are:

• Operational scenarios. Walking the system, or a model of it, through the concept of operations: a nominal day, a degraded mode, a hand-over between operators. The question is whether the scenario achieves what the stakeholder wanted, not whether each step meets its number.
• Mission simulation. Running the operational concept in a simulation with realistic environments and variation, then reading the results against measures of effectiveness (MOEs), the stakeholder-facing measures of how well the mission does its job.
• Stakeholder reviews. Showing users concepts, prototypes, sample products or interfaces and recording their judgement. Early, cheap, and often the first time a missing need surfaces.
• Operational trials and pilots. Real users, real procedures, a representative environment. Often the decisive validation evidence, and in many programs a formal operational test and evaluation by an independent organization.

Validation for EO-1

The traceability guide added three stakeholder needs to the EO-1 model, each traced to the requirements that formalize it. For this guide, a third test suite, TS-3 Mission Validation, was added with one validation event per need; the needs, the suite and its three events are the only additions to the demo.

Each row shows why validation is not a repeat of verification. 1.6 asks for a revisit within 72 hours, a property of the orbit and the pointing limits, and TC-13 Ground Track Repeat Analysis has already shown a repeat of 3 days and 0.4 hours. But farmers need usable imagery twice a week, and clouds do not appear in 1.6. Only a simulation with cloud statistics (VAL-02) can say whether the requirement, met as written, delivers the need. If it does not, the fix is a new or changed requirement, not a better test.

N1 is similar. 6.2 and 6.5 put numbers on the delivery chain, and 6.5's Monte Carlo analysis (see the Monte Carlo guide) puts the probability of meeting 100 minutes at about 97.6%. Whether an agronomist actually learns of crop stress on the day of the overpass, in a form they can act on, is a different question, and VAL-01 asks it with real users.

VAL-03 ran first, before any flight hardware, because it needs only sample imagery and people. That is early validation: it confirmed that 4.5 m in four bands is detailed enough for field-level maps before the camera design was frozen. Two reviewers asked for field boundary overlays, a possible new need recorded with the result.

Model verification and model validation

In model-based systems engineering, the model itself is an engineering product, and the same two questions apply to it. Defense and aerospace practice has defined them for models and simulations for decades (DoD Instruction 5000.61 on verification, validation and accreditation; NASA-STD-7009 on the credibility of models and simulations):

• Model verification: is the model built correctly? Does the implementation match its own specification and the modeling rules: no missing durations in a simulated flow, no function performed by nobody, no requirement without a verifying event, equations that are dimensionally consistent, code that computes what its author intended.
• Model validation: is it the right model for its purpose? Does it represent the real system closely enough for the decisions it will support? That is answered by comparing the model with reality: test data, flight data, a trusted higher-fidelity model.
• Accreditation, in DoD usage, is the formal decision that a model is acceptable for a specific use, based on its verification and validation evidence.

EO-1 has a clear case. Its power budget converts measured peak array power into orbit-average power with equation EQ-1.3: orbit average = peak × 0.62 (sunlit fraction) × 0.6 (power path efficiency). Verifying that equation means checking the arithmetic, the units and its links. Validating it means asking whether 0.62 and 0.6 match the real orbit and the real power chain, for example by comparing the prediction with orbit-average power measured on the flatsat through a simulated orbit. The verdict on requirement 3.1 below depends entirely on that equation, so an unvalidated EQ-1.3 would make even a perfect test inconclusive.

The rule of thumb: when an analysis or simulation is used as verification evidence, its model needs validation evidence of its own, in proportion to how much the decision rests on it.

The traceability behind both

Verification and validation each need their own chain of links:

• Needs → validation. Each stakeholder need is linked to the validation activity that will show it is met in intended use, and to the requirements that formalize it.
• Requirements → verification. Each requirement is linked to the test, analysis, inspection or demonstration that verifies it, with a method, a level and success criteria. The requirements verification matrix is the report of this chain.

Together they answer the two coverage questions a review board asks: is every requirement verified, and is every need validated? The links between needs and requirements connect the two chains, so a gap on one side can be traced to the other: a need whose requirements all pass but whose validation fails points at a missing or wrong requirement.

In the matrix the two chains sit side by side: requirements are verified by TC test cases, needs are validated by VAL events. The matrix also shows a verification gap: 6.5 Planning Cycle Time has a method (analysis) but no verifying event in the model yet. The Monte Carlo guide runs that analysis; recording its result as a test case would close the row.

Verification evidence in practice: TC-06

EO-1's TC-06 Solar Array Power Test shows what makes verification evidence good, and how it stops counting. The record is complete: M. Chen ran it on the array qualification panels QM-1 and QM-2 with harness revision C, under the large-area solar simulator at AM0 (1361 W/m², uniformity 1.4%), and attached the I-V curve as tc06-array-iv-curve.csv. Peak power was 75.1 W against an expected result of at least 72 W. Passed.

Then requirement 3.1 Orbit Average Power was raised from 28 W to 30 W after the preliminary design review. The test result did not change, but what it proves did. Through EQ-1.3, 75.1 W peak is 27.9 W orbit average (75.1 × 0.62 × 0.6), below 30 W; showing 30 W would take about 80.6 W peak. The link from 3.1 to TC-06 is therefore flagged as suspect and stays suspect until TC-06's expected result is re-derived from the new requirement and the test is run again or the design changes. The RVTM guide walks through the review, and the change impact analysis guide follows the same change through the design.

Three lessons for verification evidence come out of it:

1. Evidence is tied to a configuration and a requirement version. A pass recorded against 28 W is not evidence for 30 W.
2. The pass criterion must come from the requirement. TC-06's 72 W limit corresponds to only 26.8 W orbit average, so it did not show even the old 28 W. A test can pass its own criterion and still fail to verify its requirement.
3. Analysis in the loop needs validation. The verdict depends on EQ-1.3's two factors, which is the model validation question from the previous section.

And none of this is validation. Even a clean 30 W result would show only that the arrays meet 3.1. Whether 30 W is enough power to image 120 scenes a day for farm customers is the kind of question the mission-level validation events answer.

Verification vs validation compared

Common confusions

• "Validation is the last test." System-level tests near the end are usually still verification. Validation is defined by its reference (needs and intended use), not by its timing, and it should start long before the end.
• "All requirements passed, so the system is validated." Passing requirements shows that the requirements were met. It says nothing about whether they were the right requirements. EO-1 could meet 1.6 exactly and still miss N2 under real cloud cover.
• Validating a requirement versus verifying the system. The words also apply to requirements themselves. The INCOSE Guide to Needs and Requirements separates requirement verification (is the statement well formed: necessary, unambiguous, verifiable?) and requirement validation (does it express the need it came from?) from system verification and system validation. Say which one you mean. The requirements writing guide covers making statements verifiable.
• "Demonstration is validation." A demonstration of a requirement is verification. A demonstration of an operational scenario with users, judged against their needs, is validation. The method does not decide it; the reference does.
• "A simulation is a validation." A simulation is a technique. Simulating 6.5's planning cycle against 100 minutes verifies 6.5. Simulating a growing season against "usable imagery twice a week" validates N2. And the simulation model itself needs its own validation.
• IV&V means "more V&V". Independent verification and validation is V&V performed by an organization separate from the developer in management, budget and technical judgement. It is about who does it, not a third activity.
• Counting links as results. A need linked to a validation event is planned for validation, not validated. A coverage count tells you what is planned; only results tell you what is shown.

How Holarch records each

In an LML project, verification and validation are kept in the same model as the needs and requirements, with the same kinds of items and links:

• Needs are Statements (not Requirements), linked to the requirements that formalize them with traced to.
• Verification planning lives on each Requirement: Verification Method (Analysis, Demonstration, Inspection, Test, in any combination), level and success criteria.
• Verification events are Test Cases in Test Suites, linked to requirements with verifies. Analyses, inspections and demonstrations are test cases too. Each run records the tester, configuration, test article and evidence file; failed tests become Issues, and test cycles save each campaign.
• Validation events can be Test Cases as well, in a suite of their own (such as TS-3 above), linked to the need with verifies. Create that link in the Traceability Matrix by choosing the relationship and clicking the cell. The RVTM and Verification Coverage count requirements only, so validation events do not change the "11 of 32 requirements verified" figure; the matrix shows them next to the requirements.
• Measures of effectiveness are Measures labeled MOE with threshold and objective values. A Monte Carlo study in the Simulator can check an output against a requirement or a Measure and record the result as a Test Case, which is how an analysis becomes verification evidence.
• Changes mark links suspect, including the link to a verifying test case, as with TC-06.
• Model verification is what Verification → Model Checks does: rules over the whole model, such as leaf requirements with no verifying test case and failed tests without an Issue. Action diagrams have their own checks before a simulation runs. Holarch has no separate record for model validation; keep the comparison with test or flight data as the evidence of the analysis test case that relies on the model.

The SE Process guide shows both sides of the V. With the Vee model process, the Verification links card pairs each right-side phase with the left-side phase it checks: system validation with the concept of operations ("needs validated"), system verification with the system requirements ("requirements verified"), and the subsystem and unit levels below.

Read the card as coverage, in the spirit of the confusion above. A need counts as validated once a test case is linked to it with verifies, or once a requirement traced from it has passed, so N1 and N2 count although VAL-01 and VAL-02 have not run. The card counts every traced Statement that is not a requirement as a need; in the demo that adds an assumption (1.1 Reference Orbit) and three operational rules (OR-1 to OR-3) to N1–N3, which is why it shows 7. The validation results themselves are in the Test Center.

Documents can carry the plans: the document templates include a Concept of Operations, a Verification and Validation Plan, a Test Plan, a Test Report and an IV&V Plan, with live VCRM and RVTM blocks that update from the model.

Try it in Holarch

1. Create the EO-1 project: Manage Projects → Project Files ▾ → New Demo Project.
2. Add the needs as Statements, for example with Add Item ▸ in a requirements document, and link each to its requirements with traced to.
3. In Verification → Test Center, click + New ▾ → Test Suite… for a validation suite, then + Test Case for each validation event, with its set-up and expected result in stakeholder terms.
4. Open Verification → Traceability Matrix with the needs in the rows and the validation test cases in the columns, choose the relationship and click each cell to link them.
5. Open Verification → RVTM to see requirement verification, and Plan & Deliver → SE Process with the Vee model to see both sides of the V.

The full references are Verification, Traceability, SE Process and Model Checks.

Frequently asked questions

Which comes first, verification or validation?

Formal system validation usually comes after system verification, because it needs a system that works as specified, in its intended environment. But validation activities start first: stakeholder reviews of the concept and the requirements happen before any verification is possible. Plan both at the start.

Can one activity do both?

Yes, if it is planned that way and judged against both references. An end-to-end operations rehearsal can verify 6.1 Contact Scheduling against its 48 hour criterion and, with real operators and customers taking part, gather validation evidence for N1. Record the two verdicts separately, because they can differ.

Who performs validation?

The developer usually plans and supports it, but the judgement belongs to the stakeholders: users, operators, the customer, sometimes an independent test organization. Validation without stakeholders is a guess about their needs.

What happens when validation fails but verification passed?

The requirements did not capture the need. Trace the failed need to its requirements, find the missing or wrong one, change it through change control, and follow the suspect links into design and verification. This is the most expensive failure in systems engineering, which is why early validation pays.

Where to go next

• Requirements verification matrix (RVTM): the practical next step, with columns, method choice, status roll-up and a milestone coverage review on the same CubeSat.
• How to write requirements: verifiable statements, the precondition for verification.
• Requirements traceability and change impact: needs, requirements, suspect links and the matrix.
• Verification in Holarch: test cases, runs with evidence, test cycles and the RVTM in one model.