Skip to content

Coding

Two scenes for an agent that writes and ships code: what it may do about a red build, and whether its change may land. The second is the change gate, covered in depth on its own page; this page adds CI triage and shows how the two compose.

CI failure triage

The scene

The agent's PR goes red. Left to a prompt, the agent reruns the job (sometimes twice), or "fixes" the test, or edits the deploy script that failed. Each of those is a judgment a model made, none of it is recorded as a decision, and the only artifact is a chat log. The disposition of a failed build should be policy: known-flaky tests get exactly one rerun, a flaky test that fails again is not flaky anymore, deploy failures roll back before anyone investigates.

The overlay

# Gallery overlay: CI failure triage for a coding agent. The agent sees a red
# build; these rules decide what it is allowed to do about it — and a rerun
# is a tool the runtime fires, never a judgment call the model makes.
# Illustrative policy.

@prefix :      <https://example.org/ci#> .
@prefix horis: <https://w3id.org/horismos#> .
@prefix rdf:   <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd:   <http://www.w3.org/2001/XMLSchema#> .

## Domain (doc 02)

:BuildFailure a rdfs:Class ;
    rdfs:label "Failed CI run" .

:Stage a rdfs:Class ;
    horis:oneOf ( :Lint :Test :Build :Deploy ) .

:failingStage a rdf:Property ;
    rdfs:domain :BuildFailure ; rdfs:range :Stage ;
    horis:cardinality "1" .

:knownFlaky a rdf:Property ;
    rdfs:domain :BuildFailure ; rdfs:range xsd:boolean ;
    rdfs:comment "The failing test is on the repo's tracked flaky list." ;
    horis:cardinality "0..1" .

:reproducesOnRerun a rdf:Property ;
    rdfs:domain :BuildFailure ; rdfs:range xsd:boolean ;
    horis:cardinality "0..1" .

:priorReruns a rdf:Property ;
    rdfs:domain :BuildFailure ; rdfs:range xsd:integer ;
    horis:min 0 ;
    horis:cardinality "0..1" .

:Disposition a rdfs:Class ;
    horis:oneOf ( :RerunOnce :FixForward :EscalateToOwner :RollBack ) .

:disposition a rdf:Property ;
    rdfs:domain :BuildFailure ; rdfs:range :Disposition ;
    horis:cardinality "0..1" ;
    horis:assignedBy horis:RulesOnly .

## Rules (doc 04)

:deployFailuresRollBack a horis:Rule ;
    horis:priority 100 ;
    horis:when [
        horis:subject :BuildFailure ; horis:property :failingStage ; horis:is :Deploy
    ] ;
    horis:then [ horis:property :disposition ; horis:value :RollBack ] ;
    horis:explain "A failed deploy stage rolls back first and investigates second" .

:rerunKnownFlakyOnce a horis:Rule ;
    horis:priority 50 ;
    horis:when [
        horis:all (
            [ horis:subject :BuildFailure ; horis:property :reproducesOnRerun ; horis:isNot true ]
            [ horis:subject :BuildFailure ; horis:property :failingStage ; horis:is :Test ]
            [ horis:subject :BuildFailure ; horis:property :knownFlaky ; horis:is true ]
            [ horis:subject :BuildFailure ; horis:property :priorReruns ; horis:below 1 ]
        )
    ] ;
    horis:then [ horis:property :disposition ; horis:value :RerunOnce ] ;
    horis:explain "Known-flaky test, not yet rerun — one rerun is allowed" .

:flakyStillFailingEscalates a horis:Rule ;
    horis:priority 60 ;
    horis:when [
        horis:all (
            [ horis:subject :BuildFailure ; horis:property :failingStage ; horis:isNot :Deploy ]
            [ horis:subject :BuildFailure ; horis:property :knownFlaky ; horis:is true ]
            [ horis:subject :BuildFailure ; horis:property :reproducesOnRerun ; horis:is true ]
        )
    ] ;
    horis:then [ horis:property :disposition ; horis:value :EscalateToOwner ] ;
    horis:explain "The flaky test failed again on rerun — it is no longer flaky, escalate to the owner" .

:deterministicFailuresFixForward a horis:Rule ;
    horis:priority 10 ;
    horis:when [
        horis:all (
            [ horis:subject :BuildFailure ; horis:property :failingStage ; horis:inList ( :Lint :Test :Build ) ]
            [ horis:subject :BuildFailure ; horis:property :knownFlaky ; horis:is false ]
        )
    ] ;
    horis:then [ horis:property :disposition ; horis:value :FixForward ] ;
    horis:explain "Deterministic lint/test/build failure — the agent may propose a fix through the change gate" .

# PRP-005 completeness-cover:start
# Two regions had no disposition until PRP-005's completeness check named them, and
# in both the agent is being asked to act on a failure nobody has classified. The
# overlay already has the right outcome for that: hand it to the owner. Letting
# :FixForward be the default would have the agent proposing patches against a red
# build whose cause is unknown.
:failureNotClassified a horis:Rule ;
    horis:priority 40 ;
    horis:when [ horis:all (
        [ horis:subject :BuildFailure ; horis:property :failingStage ; horis:isNot :Deploy ]
        [ horis:subject :BuildFailure ; horis:property :knownFlaky ; horis:isEmpty true ]
    ) ] ;
    horis:then [ horis:property :disposition ; horis:value :EscalateToOwner ] ;
    horis:explain "The failure was never checked against the flaky list — its cause is unclassified, so the owner decides" .

# A known-flaky failure that is out of reruns, or that is not a test at all:
# :rerunKnownFlakyOnce needs `:Test` with `priorReruns below 1`, so a flaky-listed
# lint or build failure, or a test already rerun, matched no rule.
:flakyRerunExhausted a horis:Rule ;
    horis:priority 40 ;
    horis:when [ horis:all (
        [ horis:subject :BuildFailure ; horis:property :failingStage ; horis:isNot :Deploy ]
        [ horis:subject :BuildFailure ; horis:property :knownFlaky ; horis:is true ]
        [ horis:subject :BuildFailure ; horis:property :reproducesOnRerun ; horis:isNot true ]
        [ horis:any (
            [ horis:subject :BuildFailure ; horis:property :failingStage ; horis:isNot :Test ]
            [ horis:none ( [ horis:subject :BuildFailure ; horis:property :priorReruns ; horis:below 1 ] ) ]
        ) ]
    ) ] ;
    horis:then [ horis:property :disposition ; horis:value :EscalateToOwner ] ;
    horis:explain "Flaky-listed failure with no rerun left to spend — the owner decides rather than the agent" .
# PRP-005 completeness-cover:end

The decision, with proof

from horismos import HorismosValidationError, Ontology, World

onto = Ontology.load("examples/gallery/ci-triage.ttl")
world = World(onto)

def triage(subject: str, **observed: object) -> None:
    ref = world.assert_(":BuildFailure", subject=subject, **observed)
    proof = world.explain(ref, ":disposition")
    print(proof.render() if proof else f"{subject}: no disposition — needs a person")

triage(":run-901", failing_stage=":Test", known_flaky=True, prior_reruns=0)
triage(":run-902", failing_stage=":Lint", known_flaky=False)
triage(":run-903", failing_stage=":Deploy", known_flaky=False)

:disposition = :RerunOnce   via rerunKnownFlakyOnce — "Known-flaky test, not yet rerun — one rerun is allowed"
  ├─ reproducesOnRerun isNot true   (closed world)
  ├─ failingStage :Test is :Test   (asserted by the api)
  ├─ knownFlaky true is true   (asserted by the api)
  └─ priorReruns 0 below 1   (asserted by the api)

:disposition = :FixForward   via deterministicFailuresFixForward — "Deterministic lint/test/build failure — the agent may propose a fix through the change gate"
  ├─ failingStage :Lint inList (:Lint :Test :Build)   (asserted by the api)
  └─ knownFlaky false is false   (asserted by the api)

:disposition = :RollBack   via deployFailuresRollBack — "A failed deploy stage rolls back first and investigates second"
  └─ failingStage :Deploy is :Deploy   (asserted by the api)

The rerun happens, the failure reproduces. The agent does not get to decide the test is "still flaky"; the facts change and the disposition follows:

triage(":run-901", prior_reruns=1, reproduces_on_rerun=True)
try:
    world.assert_(":BuildFailure", subject=":run-904",
                  failing_stage=":Test", disposition=":RerunOnce")
except HorismosValidationError as e:
    print(f"[{e.constraint}] {e}")

:disposition = :EscalateToOwner   via flakyStillFailingEscalates — "The flaky test failed again on rerun — it is no longer flaky, escalate to the owner"
  ├─ failingStage isNot :Deploy   (closed world)
  ├─ knownFlaky true is true   (asserted by the api)
  └─ reproducesOnRerun true is true   (asserted by the api)

[provenance_refused] disposition may only be assigned by rules (horis:assignedBy horis:RulesOnly); assertion came from the api

In a full workflow, :RerunOnce is an ActionStep whose tool the runtime fires — the rerun is never a model's call — and :FixForward hands the agent's patch to the change gate.

Why verification matters here

Two rules here overlapped on purpose: rerunKnownFlakyOnce (priority 50) requires priorReruns below 1, and flakyStillFailingEscalates (priority 60) requires reproducesOnRerun true, so both matched a known-flaky test failure recorded with priorReruns = 0 and reproducesOnRerun = true — a state the CI adapter should never produce, and priority resolved it in favour of escalation if it did. The conflict check named that pair and its region, along with a second overlap nobody had intended: deployFailuresRollBack (priority 100) against the same escalation rule, because flakyStillFailingEscalates carried no stage condition at all. That is the check earning its place — one of the two overlaps was a design decision, the other was an oversight, and they are indistinguishable by eye. Both rules now carry explicit guards (failingStage isNot :Deploy, reproducesOnRerun isNot true) that state the resolution priority was making silently. Completeness reported the regions with no disposition, and they shared a shape: a failure nobody had classified. A build whose knownFlaky status was never recorded matched nothing, because deterministicFailuresFixForward needs knownFlaky is false and the rerun rule needs true; so did a flaky-listed lint or build failure, and a test already out of reruns. :EscalateToOwner is the default in all of them — letting :FixForward cover an unclassified red build would have the agent proposing patches against a failure whose cause is unknown. Non-vacuity flags a rule whose conclusion never differs from what a higher-priority rule already assigns.

Status and honest edges

Runs today with structured input. Materializing the CI state as a versioned projection (run id, stage, flaky-list version) and committing "rerun" or "roll back" as a conditional effect is the coding-harness adapter in PRP-008.

Merge and release gating

The change gate is the merge decision: 28 terms, eleven rules, and a RulesOnly decision the bot cannot write. Release gating is the same shape with different facts — signed artifact, passing smoke suite, change-freeze window, on-call acknowledged — and the same guarantee: the release is approved by a rule you can read, not by whichever model was on duty.

What composes them is the fail-closed default. FixForward does not mean merge; it means the agent may propose a change, which then has to clear the gate on its own facts.