| The artefact or action can be reproduced or inspected. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The artefact or action can be reproduced or inspected. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Important supplied context, assumptions, and inferences are visible. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Important supplied context, assumptions, and inferences are visible. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The intended use and boundaries are explicit. (publicly evidenced) | 44 of 75 (59%) | 39 of 75 (52%) | 41 of 75 (55%) | 45 of 75 (60%) |
|---|
| The intended use and boundaries are explicit. (claimed, no artefact linked) | 13 of 75 (17%) | 16 of 75 (21%) | 17 of 75 (23%) | 16 of 75 (21%) |
|---|
| Acceptance criteria are explicit and testable enough for the use. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Acceptance criteria are explicit and testable enough for the use. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Evidence shows the acceptance criteria were met. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Evidence shows the acceptance criteria were met. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Relevant failure modes and fallback behaviour were evaluated. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Relevant failure modes and fallback behaviour were evaluated. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Known limitations and out-of-scope conditions are stated. (publicly evidenced) | 36 of 75 (48%) | 25 of 75 (33%) | 47 of 75 (63%) | 54 of 75 (72%) |
|---|
| Known limitations and out-of-scope conditions are stated. (claimed, no artefact linked) | <10 | <10 | <10 | <10 |
|---|
| The Deliverable gate for the intended use has passed. (publicly evidenced) | <10 | <10 | <10 | <10 |
|---|
| The Deliverable gate for the intended use has passed. (claimed, no artefact linked) | <10 | <10 | <10 | <10 |
|---|
| Operational ownership is explicit. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Operational ownership is explicit. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The assurance mechanism is defined for the important failure modes. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The assurance mechanism is defined for the important failure modes. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Fallback or escalation is defined. (publicly evidenced) | <10 | <10 | <10 | 23 of 75 (31%) |
|---|
| Fallback or escalation is defined. (claimed, no artefact linked) | <10 | <10 | <10 | <10 |
|---|
| Recovery or rollback has been tested where failure consequence warrants it. (publicly evidenced) | <10 | <10 | <10 | <10 |
|---|
| Recovery or rollback has been tested where failure consequence warrants it. (claimed, no artefact linked) | <10 | <10 | <10 | <10 |
|---|
| Support, maintenance, debugging, and change responsibility are defined. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Support, maintenance, debugging, and change responsibility are defined. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The relevant operating-cost boundary is known well enough for the decision. (publicly evidenced) | 27 of 75 (36%) | 13 of 75 (17%) | 30 of 75 (40%) | 34 of 75 (45%) |
|---|
| The relevant operating-cost boundary is known well enough for the decision. (claimed, no artefact linked) | <10 | <10 | 11 of 75 (15%) | <10 |
|---|
| The outcome measure is defined. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The outcome measure is defined. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| A comparable baseline or counterfactual is defined. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| A comparable baseline or counterfactual is defined. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The outcome was measured after the intervention. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The outcome was measured after the intervention. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Before/after measurements use the same relevant definition. (publicly evidenced) | <10 | <10 | <10 | <10 |
|---|
| Before/after measurements use the same relevant definition. (claimed, no artefact linked) | <10 | <10 | <10 | <10 |
|---|
| Material confounds, uncertainty, or attribution limits are named. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Material confounds, uncertainty, or attribution limits are named. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Outcome evidence relevant to the decision is available. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Outcome evidence relevant to the decision is available. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The full relevant cost boundary is accounted for. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The full relevant cost boundary is accounted for. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Material risks and trade-offs are assessed. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Material risks and trade-offs are assessed. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Reasonable alternatives or the status quo are compared. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| Reasonable alternatives or the status quo are compared. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The rule for scale, renew, expand, or stop is explicit. (publicly evidenced) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|
| The rule for scale, renew, expand, or stop is explicit. (claimed, no artefact linked) | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule | withheld: withheld under the pre-registered shortfall rule |
|---|