Monte Carlo simulation in systems engineering: from point estimates to probabilities

Why point estimates mislead

Most engineering estimates are given as one number: the integration campaign takes 14 weeks, the payload draws 38 W, the ground station processes a pass in 45 minutes. Each number is usually the most likely value, and adding most-likely values gives a total that says nothing about how often it is exceeded, for two reasons.

• Skewed inputs. Real task durations are rarely symmetric. A task that most likely takes 6 minutes can finish in 4 but can also take 9 when something goes wrong. The average of such a task lies above its most likely value, and the sum of averages lies above the sum of most-likely values.
• Merge bias. Where parallel paths join, the work waits for the slowest one. The expected maximum of several uncertain paths is larger than the maximum of their expected values, because any path that runs late delays the whole.

The same applies to budgets. A mass or power budget built from nominal values hides the chance that several items come in high at once, and a timeline built from nominal durations hides the chance that the critical path runs long. A single number cannot say how likely it is to hold. A distribution can.

Monte Carlo simulation is the simplest way to get that distribution: describe each uncertain input as a probability distribution, draw one value for every input, compute the result, and repeat many times. The collection of results approximates the distribution of the output, and from it you read means, percentiles and the probability of meeting a requirement. In ISO/IEC/IEEE 15288 terms it is a tool of the system analysis process, and its result is often the evidence for verification by analysis, one of the four methods in a requirements verification matrix.

Uncertain inputs as distributions

A distribution states what you know about an input. Pick the simplest one that matches the information you have; three shapes cover most systems engineering work. The examples come from the satellite imaging pass worked through below.

• A uniform distribution is honest when you only know the bounds, but it gives the extremes as much weight as the middle.
• A triangular distribution is the usual choice for three-point estimates from engineers. Its mean is (min + most likely + max) / 3, so a long upper tail moves the mean above the most likely value: 4, 6 and 9 minutes give a mean of 6.33.
• A normal distribution has unbounded tails. For durations, masses and other quantities that cannot be negative, keep the standard deviation small relative to the mean, or use a distribution with a lower bound.
• Other shapes have specific uses: lognormal for quantities that are a product of factors, exponential for times between random events, Weibull for failure times, beta for fractions between 0 and 1.

Keep fixed what is not uncertain. A model in which every input varies is harder to review and hides the few inputs that drive the result.

Running many samples

One Monte Carlo run is one possible outcome: every uncertain input gets a random value from its distribution, every decision is drawn with its probability, and the model computes the result as if that were the real case. A study repeats this hundreds or thousands of times with independent draws. Two practical rules:

• Fix the random seed when the result becomes evidence. The same seed reproduces the same runs, so a reviewer can repeat the study and any single run can be replayed to see what happened in it.
• Keep the model logic in the model. Branches with probabilities, loops with counts and resource limits belong in the same flow the simulation runs, not in a separate spreadsheet that drifts from the design. The flow is usually the operational scenario from functional analysis, with durations added.

Reading the results

A study produces a list of outcomes, one per run. Read it in four ways.

• Histogram. The shape of the outcomes: where most runs fall, whether the distribution is skewed, and how long the tails are.
• Mean and standard deviation. Useful summaries, with a confidence interval on the mean that shrinks as runs are added. The mean alone says nothing about risk.
• Percentiles. P50 is the median: half the runs finish below it. P90 or P95 is a commitment value: the outcome you meet in 90% or 95% of cases. Schedules and budgets are often committed at P80 or P90 rather than at the mean.
• Probability of meeting a requirement. The share of runs that meet a threshold, for example "total time at most 100 minutes". Read it from the cumulative probability curve (the S-curve), which plots, for every value x, the share of runs with an outcome at most x. A requirement stated with a probability ("in at least 90% of cases") is checked directly against this number.

Margin can be read the same way: the difference between the threshold and the mean is the margin on average; the difference between the threshold and the P90 or P95 value is the margin you have at the confidence you commit to.

How many runs are enough

A Monte Carlo result is itself an estimate, and its precision grows with the square root of the number of runs. For a probability p estimated from N runs, the standard error is about √(p(1 − p) / N). Near 0% or 100% the simple ±2 standard errors interval is poor, so the table uses the Wilson interval, which stays inside 0 to 100%.

The practical test: the study has enough runs when the confidence interval of the number you decide on no longer crosses the decision line. In the example below the requirement asks for 90%; even 100 runs give an interval that stays above 90%, so the verdict is settled early. Had the estimate been 91% with an interval from 88% to 94%, more runs would be needed. Means settle quickly; percentiles in the tails (P95, P99) and small probabilities of failure need many more runs, because only a few runs land there.

Correlation and its effect

Drawing every input independently assumes that a long downlink says nothing about the processing that follows. Often that is false: a large data volume makes both slow, a cold spacecraft slows several warm-up steps, a heavy structure needs a heavier mechanism. Inputs that share a cause are correlated.

Positive correlation does not change the mean of a sum, but it widens the spread: high values come together and low values come together instead of cancelling. The tails move out, and the P90 or P95 value that a commitment rests on moves with them. Ignoring a real positive correlation therefore understates risk.

Ways to handle it:

• Model the common cause explicitly: draw the data volume once per run and derive both durations from it.
• Draw correlated inputs from a shared random number, so they move together run by run.
• At the least, run the study once independent and once fully correlated, and report both: the truth lies between them.

Common mistakes

Worked example: the EO-1 imaging pass

EO-1 is a 6U CubeSat that images farmland, the fictional mission of the MBSE example project. Its system requirements specification contains a timing requirement that is verified by analysis:

6.5 Planning Cycle Time. The mission shall complete each planning cycle, from target list uplink to product delivery, within 100 minutes.

The verification plan for 6.5 reads: a Monte Carlo simulation of the imaging pass meets 100 minutes with a probability of at least 90%. The Measure MOE-7 Planning Cycle Time specifies the requirement, with a threshold of 100 minutes and an objective of 80 minutes.

The operational timeline is the action OPS Run Nominal Imaging Pass. The ground station sends the imaging plan, the spacecraft decodes and loads it, then turns to the first target while the camera warms up. It images four targets, each a strip that is captured and then encoded. If a ground contact is in view (75% of passes), it transmits the data; otherwise it holds the data for the next pass. Finally, the battery recharges in sunlight while the ground receives the downlink and processes the image products.

The point estimates

• Most likely values, taking the more likely contact branch: 6 + 1 + 2 + max(3, 5) + 4 × (0.67 + 0.50) + 8 + max(35, 9 + 45) = 80.7 minutes. Against the 100-minute threshold this looks like almost 20 minutes of margin.
• Mean values, weighting the two branches by their probability (0.75 × 8 + 0.25 × 1 = 6.25): 6.33 + 1 + 2 + 5 + 4.67 + 6.25 + max(35, 54) = 79.25 minutes.

Both numbers say the pass fits comfortably. Neither says how often it does not, and the requirement asks exactly that.

The Monte Carlo result

A study of the same flow draws every duration and the contact decision in every run, and lets the parallel paths race. The table gives a one-million-run check computed separately in Python and the 2,000-run Holarch study shown in the screenshots below. They use different random number generators, so they differ by sampling error (for 2,000 runs about 0.25 minutes on the mean, 0.5 minutes on P95 and 0.35 percentage points on a probability).

What the numbers say:

• The requirement passes. About 97.6% of passes finish within 100 minutes, well above the required 90%, and the interval from 2,000 runs stays above 90% by a wide margin.
• The margin is smaller than it looks. The margin on the mean is about 20.6 minutes, but the P95 value is about 96.6 minutes, so at 95% confidence only about 3.4 minutes remain. About one pass in forty misses the threshold. In almost all of those runs OPS.12 Process Image Products took more than an hour (67 minutes on average, against its mean of 45), and in 95% of them the contact branch added the 8-minute downlink.
• The objective is a coin toss. The 80-minute objective of MOE-7 sits almost exactly at the median: it is met in only about half the passes. The 80.7-minute most-likely estimate hides this completely.
• Merge bias is small here. The mean of 79.35 minutes is only 0.1 minutes above the mean-value estimate, because the ground chain (OPS.11 and OPS.12, mean 54 minutes) is shorter than the 35-minute recharge in only about 3% of runs. In a flow whose parallel paths are closer in length, the gap grows.

The study also points at the lever. OPS.12 Process Image Products carries most of the spread (a standard deviation of 10 minutes, against about 3 minutes from the contact decision and 1 minute from everything else), so tightening the processing time moves the tail; speeding up the spacecraft steps does little.

The same example with correlation

The downlink and the processing both depend on how much image data the pass produced: more data means a longer transmission and a longer processing run. Drawing OPS.8 and OPS.12 from one shared random number per run (8 + 1.5z and 45 + 10z minutes for the same standard normal z), so a long downlink comes with long processing, keeps both distributions as they were and gives:

The mean is unchanged and the spread grows by about a tenth. The P95 value moves out by about two minutes, which cuts the margin at 95% confidence from about 3.4 minutes to about 1.5. The requirement still passes at 90%, but a stricter commitment, such as 100 minutes in 95% of passes, would now be marginal. For a commitment at P95, the correlated case is the safer one to plan with.

Download the example: the EO-1 imaging pass uncertainty study as an Excel workbook (inputs with their distributions, the independent and correlated results with percentiles and P(meet), and the run-count table), or the same data as CSV: imaging pass inputs and distributions and Monte Carlo results, percentiles and P(meet). Use it as a template for reporting your own study.

Try it in Holarch

In Holarch, Monte Carlo runs on the action flow of the model: the same Action Diagram that describes the operation is the model that is simulated. Each Action's Duration is a fixed number or a distribution with a unit, such as =triang.dist(4, 9, 6) minutes (minimum, maximum, peak), =norm.dist(40, 6) seconds or =uniform.dist(20, 40) seconds. Ten distributions are available, among them beta, lognormal, exponential and Weibull. See Simulation for the details.

1. Create the CubeSat project with Manage Projects → Project Files ▾ → New Demo Project. It already contains OPS Run Nominal Imaging Pass with the durations from the table, requirement 6.5 and Measure MOE-7.
2. Open the action's Action Diagram and choose ▶ Simulate ▾ → Monte Carlo. Set Runs to 2000, Random seed to 2026 and Vary fixed durations to None (the demo adds ±10% to fixed durations by default; the study here keeps them fixed), and press ▶ Run Monte Carlo.
3. Read Statistics, Duration distribution and Cumulative probability (S-curve) on the Distribution tab. Enter 100 in the S-curve's x field to read P(X ≤ 100 min).

4. Open the Drivers tab to see which actions take the time across all runs and which branch combinations occurred.
5. Click Check Against Requirement… and choose the Measure MOE-7 Planning Cycle Time. It fills the criterion (≤ 100 minutes, from its threshold and improvement direction) and the requirement 6.5; keep the required probability of 90%. The table gives P(meet) with its confidence interval, the margin and the verdict. Switch Use to Objective to check the 80-minute objective, which fails at about 52%.

6. Record Result… previews the changes: MOE-7's Projected Value becomes the mean, and a Test Case that verifies 6.5 gets the status Passed and the result text. Apply them as one undo step.

Holarch's Duration editor draws each distribution independently. The correlated comparison above used Durations driven by script hooks that read one shared value per run, which the Duration editor does not offer yet. To bracket correlation in your own model, compare the independent study with a fully correlated calculation made outside the tool. Battery energy over the pass is simulated too: the minimum and final amount of the Battery Charge resource are Monte Carlo outputs that the requirement check accepts. Mass and power budgets with contingency and margins are kept in Budgets and TPMs, and the same thinking applies to uncertain values in a trade study: a ranking that flips within the input ranges is not yet a decision.

Where to go next

• Simulation: Discrete Event and Monte Carlo runs, the panels and the requirement check.
• Budgets and TPMs: roll-ups with contingency, allocations, margins and performance measures.
• Action Diagrams: the flow, constructs, resources and scripts that a simulation reads.