Skip to content
Agent OS
AOS-VAL-004 Normative foundation

Hardware Test and Quality Matrix

Hardware Test and Quality Matrix: scope, decisions, requirements, evidence, risks, and traceability for the Agent OS programme.

Edit on GitHub →

Hardware Test and Quality Matrix

This document specifies evidence strong enough to support implementation, portability, quality, security, privacy, and release claims.

Table of Contents

Purpose and Scope

Area: Validation and Assurance.

This document specifies evidence strong enough to support implementation, portability, quality, security, privacy, and release claims.

This document owns the semantics implied by Hardware Test and Quality Matrix. It does not assert that every described subsystem already exists. It defines the target model, constraints, evidence needed to trust an implementation, and the boundary with adjacent documents.

Normative Position

  1. Define claims before tests and specify evidence, target, configuration, seed, tools, expected failure, reviewer, and retention.
  2. Use unit, property, model, fuzz, fault-injection, integration, conformance, performance, security, privacy, hardware, and recovery tests.
  3. A negative result is a valid deliverable when bounded and reproducible.

Operating Model

The operating model is contract-first and evidence-driven. A component declares its authority, resources, lifecycle, error model, cancellation and timeout behavior, observability, version, and compatibility promise. Backends are replaceable only when the same conformance suite passes and no forbidden platform type leaks into portable layers.

Implementation proceeds through a reference model or mock, deterministic QEMU evidence where relevant, documentation-first physical hardware, and quality-hardware evidence. Pixel 9 adapters remain quarantined according to ADR-0004.

Requirements

  • R01. Define claims before tests and specify evidence, target, configuration, seed, tools, expected failure, reviewer, and retention.
  • R02. Use unit, property, model, fuzz, fault-injection, integration, conformance, performance, security, privacy, hardware, and recovery tests.
  • R03. A negative result is a valid deliverable when bounded and reproducible.
  • R04. Specify normal, partial, denied, timeout, cancellation, restart, upgrade, and permanent-failure behavior.
  • R05. Expose structured diagnostics without leaking secrets or vendor-specific implementation details.
  • R06. Link material unknowns to a claim and, when testable, an experiment with an owner and gate.
  • R07. Update affected documentation and task data when evidence changes the model.

Failure and Degradation

Degradation must be explicit rather than accidental. The system reports capability absence, reduced quality, unavailable provider, stale data, or unsafe condition through typed states. It must not silently fall back to broader authority, unrestricted legacy execution, unverified firmware, lossy data migration, or irreversible agent action.

Recovery defines what state is retained, reconstructed, re-enrolled, compensated, or intentionally discarded. Unsupported hardware or providers are rejected at binding time where possible.

Evidence and Acceptance

  • Reproducible evidence bundle.
  • Two independent backend runs.
  • Traceability from claim to experiment to task to decision.
  • Evidence records target identity, hardware revision, firmware, source commit, toolchain, configuration, seed, timestamps, artifacts, expected result, actual result, and reviewer.
  • Acceptance requires the referenced tasks to meet their own criteria; prose completion is not implementation completion.

Implementation Obligations

Task Obligation Priority Gate/Milestone Verification
AOS-CAM-091 Bring up a documented i.MX 8M Plus RAW and metadata bench P0 M5 Run the relevant document, schema, conformance, build, experiment, or hardware checks; independently review boundary compliance and acceptance criteria.

Risks and Open Questions

  • Passing mocks can conceal backend divergence.
  • Benchmarks without thermal/power/configuration metadata are not comparable.
  • Verification scope can be overstated.
  • Open-question rule: an unanswered high-impact question becomes a claim/experiment record and cannot be hidden in meeting notes.
  • Stop rule: work stops or changes track when legal rights, recovery, debug access, safety, or the required evidence path is unavailable.

Related Documents

Planning Reference Anchors

Matrix

AOS-CAM-091 — Bring up a documented i.MX 8M Plus RAW and metadata bench