Why we test: when one software assumption destroys a system.

Software defects become dangerous when software is connected to a physical system. A wrong conversion, an invalid state estimate or an unchecked assumption can move actuators, shut down equipment or feed nonsense into a controller. That is why serious testing is not about proving that code can execute. It is about discovering whether the complete system behaves correctly under the conditions it will actually experience.

1. The event: 37 seconds of normal flight

On 4 June 1996, the first Ariane 5 launch initially followed its planned trajectory. Roughly 37 seconds after main-engine ignition, both inertial reference systems stopped providing valid guidance information. The flight-control computer received diagnostic data where attitude data was expected, commanded extreme nozzle deflections, and the launcher rapidly departed its intended path. Structural loads increased until the vehicle broke up and its onboard destruction system terminated the flight.

2. The software defect was a numeric-range problem

A reused inertial-reference function converted a 64-bit floating-point horizontal-velocity-related value into a 16-bit signed integer. Under Ariane 4 flight dynamics, the value stayed inside the assumed range. Ariane 5 had a different early-flight trajectory and the value exceeded the representable range.

When the conversion overflowed, the software raised an exception. The exception was not handled in a way that allowed the inertial system to continue producing safe guidance data; instead the unit shut down. The backup inertial unit ran identical software and failed for the same reason almost immediately. Redundancy existed in hardware, but not in failure diversity.

// simplified illustration — not Ariane source code
int16_t convert(double value) {
    // hidden assumption: value is always representable
    return (int16_t)value;
}

// safer engineering intent
if (value < INT16_MIN || value > INT16_MAX) {
    raise_range_fault();
    enter_defined_safe_behavior();
}

3. Reuse transferred code — and transferred assumptions

The inertial-reference software came from Ariane 4. Reuse can reduce cost and risk when the environment is truly equivalent, but reuse is not evidence of validity. The old code contained a function related to alignment before launch. Ariane 5 continued executing that function after lift-off even though it was no longer needed during flight.

A component can be perfectly verified against yesterday’s assumptions and still be invalid in today’s system.

4. Why unit testing alone would not have been enough

A unit test can prove that a conversion throws an exception when fed an out-of-range value. But the real engineering questions are larger: could Ariane 5 produce that value? What happens to the inertial system after the exception? What does the flight-control computer do when both inertial units fail? Can diagnostic words be misinterpreted as flight data?

5. Representative test data is not optional

The investigation concluded that the development programme had extensive reviews and tests, but the inertial reference system and complete flight-control system were not tested with sufficiently representative Ariane 5 conditions. This distinction matters. High test volume does not compensate for a weak test basis.

For a numeric conversion, a useful test design would include nominal values, exact limits, one-step-outside limits, physically reachable extremes, transients and model-generated trajectories.

boundary_cases = [
    INT16_MIN,
    INT16_MIN - 1,
    -1,
    0,
    INT16_MAX,
    INT16_MAX + 1,
    worst_case_trajectory_value,
]

for x in boundary_cases:
    assert_defined_behavior(convert_or_reject(x))

6. Two identical backups can fail identically

The system had two inertial reference units. This protects well against some random hardware failures. It does not protect against a deterministic software defect present in both units. Common-cause failure is a systems problem: redundancy only helps when the duplicated channels do not share the same vulnerability.

For critical systems, engineers therefore analyse independence, diversity, fault containment and failure propagation—not merely component count.

7. Exception handling is part of the safety architecture

An exception handler is not just a programming detail. In embedded and safety-related systems it determines what the physical system does when assumptions fail. Options include rejecting the data, saturating a value, keeping the last trusted state, entering a degraded mode, switching channels or initiating a controlled shutdown. The correct strategy depends on hazards and system dynamics.

8. Verification should attack assumptions

A requirement says what the system shall do. An assumption says what engineers believe will always be true. Unwritten assumptions are dangerous because they rarely receive explicit verification. A strong review asks: what must remain true for this design to work, and what evidence proves that each condition is guaranteed?

• Numeric ranges and units are explicit.
• Every conversion has defined behavior at and beyond its limits.
• Unused legacy functions are removed or re-justified.
• Failure modes are exercised, not merely documented.
• Backup channels are analysed for common-cause failure.
• System simulations use representative mission trajectories.
• Interface consumers reject diagnostic or invalid data correctly.

9. Testing is evidence for a decision, not a ritual

The lesson from Ariane 5 is not “test more.” It is “test the right claims at the right level with the right environment.” A thousand passing unit tests cannot prove a system-level assumption that was never encoded in the test basis.

10. What this means for everyday engineering

Most of us are not launching rockets, but the pattern repeats in smaller systems: an API field overflows, an unsigned value wraps, a timeout is interpreted as zero, a reused module receives a new range, or two redundant services share the same configuration defect. The consequences may be less spectacular, but the engineering logic is identical.

Good testing turns assumptions into executable evidence. Good verification makes requirements traceable to that evidence. Good validation asks whether the complete system remains suitable in its real operating context. When software touches the physical world, those disciplines are not paperwork around engineering—they are part of engineering itself.