# Annex U #### U.1 Test harness architecture and requirements U.1.1 **Purpose.** This Annex specifies the **NXOSI Conformance Harness**—the mandatory test architecture, vector structure, verifier behavior, and reporting outputs that make “NXOSI-compliant” **provable, repeatable, and comparable** across vendors and sovereign deployments. The harness is normative for conformance claims; implementations MAY differ internally but MUST pass the same vectors and produce the same verification outcomes within declared tolerances. U.1.2 **Core doctrine.** Conformance is **test-driven portability**. A conformance badge is merely a pointer to: * vector sets executed, * verifier outputs produced, * receipts/status resolution demonstrated, * and signed conformance reports issued. U.1.3 **Harness components (mandatory).** * **U.1.3.1 Vector Repository (VR).** Versioned collection of gold/negative/adversarial/drift/performance vectors with semantic versioning, overlays, and expected outcomes. * **U.1.3.2 Runner (RUN).** Executes vectors against a Target Under Test (TUT) in controlled modes (online/offline/degraded/federated). MUST support deterministic replay. * **U.1.3.3 Verifier Library (VER).** Canonical verification logic for PR signature checks, inclusion proof checks, status resolution, scope/time binding, and reliance tier enforcement. Implementers MAY replace internals but MUST match VER outputs. * **U.1.3.4 Oracle/Expected Outcome Set (ORC).** Signed expected outcomes for each vector, including allowable variability windows where probabilistic allowances are explicitly declared. * **U.1.3.5 Report Generator (REP).** Produces machine-readable report bundles + human summaries, both signed, with handling-aware redaction. * **U.1.3.6 Evidence Output Collector (EOC).** Captures CRRs/PRs/STOs/CPARs generated during tests and validates artefact linkage minima. * **U.1.3.7 Plugfest Mode (PLG).** Multi-vendor interop execution mode with regression lock rules and published interop matrices. U.1.4 **Target Under Test classes (mandatory).** The harness MUST support conformance testing across: * **artefact conformance** (schemas + signatures + handling markers), * **component conformance** (compiler, SAC runtime, check runtimes, proof/transparency, ontology fabric, control plane), * **interface conformance** (verify APIs, status APIs, registry APIs, docket APIs, ontology APIs), * **deployment conformance** (handling enforcement, sovereign zone constraints, offline/degraded behavior, records discipline). U.1.5 **Execution environments (mandatory).** * **U.1.5.1 Online mode.** Full connectivity: status queries and transparency operations online. * **U.1.5.2 Offline mode.** Verifier MUST succeed using receipt + cached snapshot + profile package + local status cache rules, subject to reliance tier constraints. * **U.1.5.3 Degraded mode.** Partial failures: transparency outage, delayed inclusion proofs, network partition, stale caches; harness validates mandated fallback behavior and declared reliance limits. * **U.1.5.4 Federated mode.** Cross-org verification: issuer and verifier in different administrative domains; enforcement of federation boundaries and authorization semantics. U.1.6 **Determinism and declared uncertainty (mandatory).** * For deterministic artefacts (PSA/SPP/OAR/XPL/PR/STO/CPAR): same inputs MUST yield identical canonical outputs. * For probabilistic systems (e.g., some ontology scoring, AI drift detection), variance MUST be declared in the vector metadata, bounded, and testable; failure to declare uncertainty is a conformance failure. U.1.7 **Test artefact envelope rules (mandatory).** All test artefacts MUST use the Common Envelope Schema (Annex B.16) with: * crypto suite ID, * time binding, * issuer authorization proof pointer (where required), * handling class marker, * deterministic canonicalization rules, * PQ fields when PQ-ready modes are claimed. U.1.8 **Security of the harness itself (mandatory).** The harness and vectors MUST be published as signed release bundles with SBOM and provenance; compromised harness invalidates results until re-run with verified harness versions. *** #### U.2 Mandatory gold vectors (core artefacts, receipts, overlays, ontology semantics) U.2.1 **Purpose.** Gold vectors prove correct behavior under normal conditions, forming the minimum backbone for claims of NXOSI compliance. U.2.2 **Gold vector structure (mandatory).** Each gold vector MUST include: * input artefacts (PSA/SPP/TRG and overlays), * initial ontology snapshot (NOF state) or canonical entity set, * expected OAR/XPL outputs, * expected CRR and PR patterns, * expected transparency inclusion proof behavior (online) and snapshot verification behavior (offline), * expected status semantics (STO none/active), * expected docket lifecycle actions (DKT open/update/close), * expected CPAR entries for operator actions where applicable, * permitted reliance tier inferences for generated receipts. U.2.3 **Mandatory gold vector categories (minimum set).** U.2.3.1 **Artefact chain baseline.** Compile PSA→SPP; evaluate TRG; issue OAR; generate XPL; produce CRR; issue PR; open DKT; resolve status. U.2.3.2 **Receipt verification baseline.** Verify PR signatures, time binding, profile hash binding, scope binding; validate inclusion proof pointer and status pointer behavior. U.2.3.3 **Overlay algebra baseline.** Apply global baseline + regional overlay + national overlay + operator policy; verify precedence and conflict resolution outputs; verify proof portability rules (receipt still verifiable under overlays). U.2.3.4 **Status semantics baseline.** Demonstrate supersession workflow with STO supersession object + migration notes; relying party resolution rules; permitted reliance stability. U.2.3.5 **Ontology semantics baseline.** Demonstrate entity/event bindings for triggers; graph-to-controls mapping correctness: event→trigger→OAR scope binding; receipts update graph state as declared. U.2.3.6 **Control plane baseline.** Governed authoring action produces CPAR; simulation produces evidence outputs; approval gates enforce separation-of-duties and handling controls. U.2.3.7 **Sovereign zone baseline (where claimed).** Evidence pack index points to in-zone store; receipt travels; controlled disclosure workflow exists; cross-border verifier succeeds with receipt-only reliance. U.2.4 **Gold invariants (mandatory).** * Every generated artefact MUST be linkable by immutable identifiers end-to-end. * No silent edits: any change affecting reliance MUST be represented via STO. * Handling markers MUST propagate correctly, and cross-class leakage MUST NOT occur. *** #### U.3 Mandatory negative tests (malformed inputs, revoked status, replay attempts) U.3.1 **Purpose.** Negative tests verify failure behavior is safe, deterministic, and auditable—rejecting malformed or unauthorized artefacts and preventing invalid reliance. U.3.2 **Mandatory negative test classes (minimum set).** U.3.2.1 **Schema invalidity.** Missing required fields; invalid enums; wrong handling markers; wrong version; bad canonicalization. Expected: reject with machine-readable error; emit audit event. U.3.2.2 **Signature invalidity.** Invalid signature; wrong signer; untrusted issuer; expired keys; wrong countersignature. Expected: reject; open/update DKT where policy requires. U.3.2.3 **Authorization invalidity.** Issuer not authorized for the namespace/profile; role key mismatch; SoD violation attempt (same actor approves and publishes). Expected: deny; CPAR logged for attempt if applicable. U.3.2.4 **Status revoked.** PR references status pointer that returns revocation; verifier MUST treat reliance as invalid for affected scope and tier; expected: fail verification. U.3.2.5 **Supersession beyond allowed compatibility.** Receipt references profile version that is superseded with breaking change where reliance tier requires upgrade; expected: fail or degrade per rules. U.3.2.6 **Replay attempts.** Same receipt reused outside time window or scope; expected: verification failure due to time binding/scope binding; emit security event. U.3.2.7 **Stale caches.** Offline verification uses stale snapshots beyond permitted window; expected: degraded reliance or failure depending on reliance tier. U.3.3 **Negative behavior invariants (mandatory).** * Fail closed for high reliance tiers; fail safely for lower tiers with explicit limitation statements. * Produce deterministic error outputs suitable for auditors/supervisors. * Never “auto-repair” invalid artefacts without correction objects and recorded actions. *** #### U.4 Mandatory adversarial tests (equivocation posture, poisoning attempts, override abuse) U.4.1 **Purpose.** Adversarial tests prove NXOSI remains trustworthy under active attack, coercion, capture attempts, and compromised dependencies. U.4.2 **Mandatory adversarial test categories (minimum set).** U.4.2.1 **Equivocation / split-view (transparency).** Simulate inconsistent views of the log (different inclusion proofs/checkpoints to different verifiers). Expected: detection or explicit equivocation risk posture outcomes; mandated fallback and reliance constraints. U.4.2.2 **Selective logging / delayed inclusion.** Receipt issued but transparency append delayed beyond publish time constraint. Expected: verification failure or degraded reliance per tier; docket opened per policy. U.4.2.3 **Ontology poisoning.** Inject conflicting or malicious assertions (automated feed or participatory). Expected: quarantine behavior, bounded influence enforcement, provenance checks, and correction pathways. U.4.2.4 **Privilege escalation / control plane misuse.** Attempt unauthorized authoring/deploy/override; attempt bypass of SoD approval gates. Expected: deny; CPAR attempt record; sanctions ladder hook where configured. U.4.2.5 **Override abuse / break-glass misuse.** Repeated emergency overrides beyond limits; override without post-incident review. Expected: failure; incident clock enforcement; status and correction workflow triggered. U.4.2.6 **Supply-chain compromise simulation.** Tampered compiler/runtime component; mismatched SBOM/provenance; dependency drift. Expected: conformance failure; emergency freeze or denial; requirement to re-run with verified releases. U.4.3 **Adversarial test invariants (mandatory).** * Attack outcomes MUST be observable and linkable (events→dockets→corrections). * Systems MUST prioritize containment and correctionability over continuity for high-impact tiers. * Any successful adversarial condition MUST produce a defined corrective action path (STO/COR + clocks). *** #### U.5 Mandatory drift tests (semantic drift, ontology drift, AI drift) U.5.1 **Purpose.** Drift tests validate that NXOSI can detect, record, and govern changes that would otherwise invalidate reliance silently. U.5.2 **Drift classes and required tests (minimum set).** U.5.2.1 **Semantic drift (standards/profile meaning drift).** Change in PSA/SPP semantics without explicit supersession is attempted. Expected: prevented or flagged; status object required; relying parties must see the change. U.5.2.2 **Ontology drift (meaning drift in entity/event semantics).** Ontology schema changes; event meaning changes; identity resolution changes. Expected: semantic regression tests fail unless migration notes + supersession issued; graph-to-controls mapping remains correct. U.5.2.3 **AI drift (behavior/data/policy/tool drift).** Simulated changes to model version, tool allowlist, retrieval corpus, prompt class, or safety posture. Expected: triggers fire; OARs updated or reissued; retest clocks start; receipts bind to new versions; reliance limits declared. U.5.3 **Drift test invariants (mandatory).** * Drift detection MUST produce explicit artefacts (events, OAR updates, dockets). * Retest obligations MUST be time-bound and enforceable via clocks. * Drift outcomes MUST be correctionable (COR + STO where reliance changes). *** #### U.6 Performance and resilience tests (offline/degraded; failover; clock compliance) U.6.1 **Purpose.** NXOSI must work under stress, not only in lab conditions. Performance/resilience tests validate operability under real-world constraints. U.6.2 **Mandatory resilience test categories (minimum set).** U.6.2.1 **Offline verification drill.** Verify receipts using cached transparency snapshots and profile packages; enforce freshness windows; declare reliance limits. U.6.2.2 **Degraded transparency service.** Transparency service down; delayed consistency; partial availability. Expected: mandated fallback behavior and reliance constraints by tier. U.6.2.3 **Network partition and federation disruption.** Cross-org status checks unavailable; store-and-forward event delivery; delayed sync. Expected: deterministic behavior with explicit constraints; no reliance over-claiming. U.6.2.4 **Failover / HA.** Receipt issuance service failover; status endpoint failover; control plane failover. Expected: SLO compliance within declared bounds; no artefact corruption; no equivocation introduced. U.6.2.5 **Clock compliance tests.** Verification/remediation/retest/dispute clocks are enforced: overdue items generate dockets/escalations; emergency mode expires; post-incident review required. U.6.3 **Performance budgets (mandatory to declare).** Any “NXOSI-compliant” claim MUST include declared performance budgets for: * receipt issuance latency, * verification latency, * status query latency, * maximum backlog under degraded operations, * recovery time objectives for core services,\ with measured evidence outputs from test runs. *** #### U.7 Signed conformance report requirements (machine-readable + human summary) U.7.1 **Purpose.** Conformance is only meaningful if it is portable, verifiable, and handling-aware. Reports themselves must be proof-carrying. U.7.2 **Report bundle contents (mandatory).** * **U.7.2.1 Machine-readable report (MRR).** Structured report with: * harness version, vector set IDs, execution environment mode(s), * TUT identification (implementation/deployment profile), * pass/fail per vector with error codes, * conformance claims (what is claimed; what is not), * declared uncertainty bounds where applicable, * hash pointers to generated artefacts (PR/STO/CRR/CPAR samples). * **U.7.2.2 Human-readable summary (HRS).** Public-safe or controlled summary stating: * what was tested, * what passed/failed, * permitted reliance tiers, * key limitations and required follow-ups. U.7.3 **Signing and authorization (mandatory).** Reports MUST be signed and MUST include: * issuer identity and authorization proof, * crypto suite IDs and time binding, * handling marker and disclosure tier, * status pointer for report supersession or revocation. U.7.4 **No implied endorsement (mandatory).** Reports MUST include a prohibition on implying regulator approval or universal security claims beyond the exact conformance scope. *** #### U.8 Plugfest playbook and regression lock discipline U.8.1 **Purpose.** Plugfests are the mechanism that turns specifications into ecosystem reality. They produce interoperability evidence and prevent fragmentation. U.8.2 **Plugfest minimum requirements (mandatory).** * shared vector sets (frozen for the event), * multi-vendor execution in federated mode, * cross-implementation verification (one vendor verifies another’s receipts/status), * publication of an interop matrix (public-safe + controlled detail as needed), * issue capture with defect taxonomy and correction workflows. U.8.3 **Regression lock discipline (mandatory).** * Each plugfest produces a **Regression Lock Release (RLR)**: a pinned vector set + expected outcomes that future releases MUST continue to pass for compatibility branches. * Breaking changes MUST be declared via semantic versioning and MUST ship with migration notes and supersession status objects. * No silent relaxations: tests may only be relaxed via explicit governance action and published correction objects. U.8.4 **Dispute and remedy during plugfests.** * Vendors MAY challenge vectors or expected outcomes via recorded objections. * Disputes MUST be resolved via correction objects and updated expected outcome sets; interim guidance MUST be published with reliance constraints. U.8.5 **Ecosystem neutrality safeguards.** * Plugfest governance MUST enforce competition-safe participation rules and COI controls. * Results MUST not be used as marketing beyond the permitted claim language; badge misuse controls apply. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.therisk.global/organization/standardization/nexus-osi/annexes/annex-u.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.