Testing and independent V&V: trust, but verify

Every program reaches a moment where the build team stands up and says the system is ready. They mean it. They have configured it, walked the demos, and watched the happy path run clean. They are right, as far as it goes: the system does what they built it to do.

"What they built" and "what the business needs" are two different statements, and only one of them shows up in the demo.

The same test, two ways

A demo is designed to succeed. Production is designed by nobody.

A test that looks like a demo proves the configuration is internally consistent. It says nothing about whether the business can run on it. Production-like data at production-like volume is not a nice-to-have, it is the only condition under which the result means anything.

1Why the build team cannot grade its own homework

Dishonesty has nothing to do with it. The people who configured the system are good at their jobs and want it to work. The problem is blind spots, and blind spots come from where you sit.

When you configure something you build a mental model of how it should behave, then you test the paths in that model. You confirm the cases you had in mind and miss the ones you never imagined, because you cannot test for a scenario you do not know exists.

The build team can tell you the system does what they built. Only independent testing tells you it does what the business needs. Those are not the same sentence, and the difference is where go-live failures come from.

2What independent verification actually adds

It is not a second QA team running the same scripts more carefully. It is a different vantage point, and it collapses back into the integrator's view of the world unless three things are set deliberately.

1A separate line of sightThe function reports to the program, not to the integrator. Its job is to tell you the truth about quality, not to protect a delivery date or a statement of work.Test: who writes their performance review?

2Risk-based coverageYou cannot test everything, so test what matters: processes that move money, touch employees, and carry regulatory or contractual weight.Test: can they name the top ten by exposure?

3Authority to say not readyAn assessment nobody is obligated to act on is theater. This only earns its keep when it can hold up a hand before go-live and be believed.Test: has it ever changed a date?

The clause that gets negotiated away

The third one is what disappears when schedules tighten. Resist it. The entire value of an independent voice is that it can deliver news the program would rather not hear, and a voice that can be overruled quietly is not independent.

3Test the ugly scenarios on purpose

If testing is going to mean something it has to look like the real world on its worst day. These are the cases configuration teams would rather not think about, which is exactly why they belong in the plan with a name against each one.

Scenario class	What it exposes
Retroactive change	A correction dated back two periods. Touches pay, accruals, ledger and reporting at once, and almost never behaves the way the demo implied
Mid-period movement	Transfers, promotions and cost centre changes landing between cycles, where proration and approval routing both have to be right
Termination and rehire	A rehire carrying a prior identifier, prior balances and prior tax history. The classic source of duplicate records
Leave and absence overlap	Concurrent leave types, statutory interaction, and the accrual that should not have accrued
Multi-jurisdiction	Employees working across states or entities, where tax, statutory reporting and entity allocation all disagree if configured in isolation
Volume at the boundary	Period close and payroll running against full production volume, not a sample. Throughput problems only appear at scale
Deliberately bad input	Missing fields, malformed dates, records that should be rejected. Silent acceptance of bad data is worse than a hard failure

4Parallel is the only test that argues back

Everything else compares the system against a specification. Parallel compares it against reality: the same period, the same population, run twice, and the two answers have to agree.

CyclesOne clean parallel is a data point. Two consecutive clean parallels is evidence. Programs that run one and declare success are reading noise.

To the centAgree the tolerance before you run, in writing. A variance explained after the fact is an argument. A variance against a stated tolerance is a decision.

By causeClassify every difference by root cause, not by size. Ten small variances from one bad rule matter more than one large keying error.

5Defects are only useful if severity means something

Severity has to be defined by business consequence and agreed before testing starts, or it becomes negotiable exactly when the pressure is highest. If severity can be argued down in the week before go-live, your gate has no floor.

Define before you test

What each severity level means in business terms
Who assigns severity, and who can change it
What the exit threshold is per level
Which defects block go-live outright

Watch during

Reopen rate, the strongest signal of a superficial fix
Age of the oldest open critical defect
Whether severity is drifting downward under pressure
Defects clustered in one process area

Gate on

Zero open critical defects at the gate, no exceptions
High-severity items closed or with an accepted, documented workaround
Two consecutive clean parallel cycles
The independent function on record with its own verdict

Trust, but verify. Not because the build team is careless, but because no team can see its own assumptions from the inside. That is the whole case, and it does not require anyone to be at fault.