Skip to content
APMX

What an audit verdict means

apmx audit asks one question about a delivery: is this receipt intact, from an approved factory, with passing checks, bound to these files, and signed by a signer your policy trusts? It answers in rows, then in one final line. Read the final line first.

Three verdicts

Section titled “Three verdicts”
Verdict Exit Means
VALID 0 Everything checked is consistent.
INVALID 1 Something was checked and is wrong.
INCOMPLETE 2 Something could not be checked. Never a pass.

Real final lines, one per kind:

Final lines of four real audits (v0.7.1)
[+] VALID   (content-bound; not authenticated)
[+] VALID   (authenticated: danielmeppiel/apmx/.github/workflows/run.yml)
[x] INVALID   Factory: definition sha256:0bec0549b9af.. is not on factories.allow
[!] INCOMPLETE   Identity: signature not checked (offline)
             Not a pass: the receipt could not be fully audited (exit 2). --offline skips ingredients explicitly.

The reason after INVALID or INCOMPLETE is the first row that failed or could not be checked.

The rows

Section titled “The rows”
  1. PolicyWhich policy applies?
    Can show: pass, warning or not checked, info or not evaluated.
  2. IntegrityIs the bundle intact and well-formed?includes schema checks since the next release (v0.7.1: separate Standards row)
    Can show: pass, fail, info or not evaluated.
  3. FactoryIs this the factory, and is it on factories.allow?
    Can show: pass, fail, warning or not checked, info or not evaluated.
  4. ChecksDid every declared check pass?
    Can show: pass, fail, info or not evaluated.
  5. DependenciesDo the skills and packages reinstall to the same bytes?v0.7.1: Ingredients
    Can show: pass, fail, warning or not checked, info or not evaluated.
  6. RevocationIs any selected skill or instruction revoked?
    Can show: pass, fail, warning or not checked, info or not evaluated.
  7. OutputsAre these exactly the delivered files?
    Can show: pass, fail, warning or not checked, info or not evaluated.
  8. IdentityDid a trusted signer your policy names sign a verification summary that binds it?
    Can show: pass, fail, warning or not checked, info or not evaluated.
  1. Any [x][x] INVALID exit 1
  2. Else, anything that could not be checked[!] INCOMPLETE exit 2
  3. Else[+] VALID exit 0content-bound; not authenticatedauthenticated: <workflow>

[!] is either a warning (the verdict stays VALID) or a check that could not run (INCOMPLETE). The final line tells you which. [i] also marks a row not evaluated after an earlier failure.

Audit rows, one verdict. Next release: eight; v0.7.1: nine. Any failure wins over anything incomplete.

Each row prints one symbol: [+] pass, [x] fail, [!] warning or not checked, [i] information or not evaluated.

Since the next release Eight rows, named as above. v0.7.1 prints nine: the schema checks are a separate Standards row after Integrity, and Dependencies is named Ingredients.

Two kinds of VALID

Section titled “Two kinds of VALID”
  • VALID (content-bound; not authenticated). The receipt is intact and bound to these files, but nobody your policy trusts signed it. It binds content, not identity: anyone who can write the files can write a self-consistent unsigned receipt, and it still passes. Only factories.allow and a trusted signer constrain that.
  • VALID (authenticated: <workflow>). A trusted signer your policy names in factories.signers signed a verification summary of exactly these receipt lines, covering these files.

Warnings that keep VALID

Section titled “Warnings that keep VALID”

These print [!] and leave the verdict VALID:

  • Dependencies not audited offline: not audited (offline): .... Your gate audits them online.
  • Unsigned, and the policy does not require a signature: unsigned: proves content binding, not who ran it, or not authenticated: policy names no factories.signers.
  • Signers configured but checked offline, without require_signed: signature not checked (offline).
  • factories.enforcement: warn: Factory, Revocation and Identity violations print [!] instead of [x], for a staged rollout.

A signer pinned only by ref (or not pinned) still passes Identity, with a note asking you to pin workflow_sha.

When INCOMPLETE is the right answer

Section titled “When INCOMPLETE is the right answer”

INCOMPLETE means APMX could not check something it was asked to. It fails closed:

  • The policy cannot be loaded: a typo, a bad digest, an unreachable extends. A policy that cannot be loaded is never treated as a pass.
  • Dependencies could not be checked: the bundled APM is missing, or the packages could not be reinstalled (network, authentication, package access).
  • A required signature could not be checked: with require_signed, --offline, a missing delivered file to look it up by, or no gh CLI.
  • Any pull request gate offline, or a gate whose policy names no factories.signers.

--offline is not a pass for signatures: with require_signed, and always in a pull request, the verdict is INCOMPLETE.

What VALID does not mean

Section titled “What VALID does not mean”
  • Not correctness. A receipt is evidence of how a change was made, not a guarantee that the code is correct. Checks are the bar: code that passes weak checks still passes.
  • Not a sandbox. The run was not confined.
  • Not permission to merge, unless your policy and your gate say so. See auto-merge factory PRs.

Next: every row’s exact messages in the apmx audit reference, and what to do about each one in its troubleshooting table.