Enable correctly attributed leaf-package remediation proposals #208

New issue

Closed

opened 2026-09-09 05:14:31 +01:00 by nimmo · 1 comment

nimmo commented

2026-09-09 05:14:31 +01:00

Owner

Publication status

Published config MR #210 (stacked on #207) and updater MR nimmo/nixos-auto-update#2. Config head b5f17bdf601c7b8bc8107e44599f38f86ea9b188 pins published helper 1f80cf332b8f459d4813467ac0c352717d4df362. Full flake and remediation regression checks pass with this exact published pin. MR #207's review fixes are published at c907fda; its repeat automated review says Ready to merge, and CI tests pass.

Integration order: merge updater MR #2, change MR #210's locked helper ref to main while retaining the validated revision, merge #207, then retarget #210 to main. Live deployment and no-op evidence remain pending; no merge/deployment performed here. The earlier checkpoint notes below describe the pre-publication replay history.

Context

Follow-up to #205/#207 and historical Moonlight #172 replay. A real leaf compatibility failure never reached pin validation: all .drv mentions were treated as failed builders, and Moonlight had no existing file inside the leaf patch allowlist.

Scope

Recognize explicit failed-builder diagnostics in legacy and modern Nix logs. Exclude plans and dependent/cancelled builds; retain multiple independent root failures as ineligible.
Give Moonlight a dedicated existing leaf expression exposed consistently through host pkgs. Preserve its current derivation without enabling a pin or installing it on extra hosts.
Clarify that a leaf-scoped dependency compatibility override is distinct from changing a core library globally. Keep evaluation/global/core/multiple-root exclusions.
Test the extraction with real Nix failures and exercise the leaf patch validator. Assess a further historical replay without falsifying the original incident or loosening its historical base validation.

Completion criteria

Regression/lint/flake checks pass, unmodified Moonlight derivation identity is unchanged, and a Moonlight-only compatibility patch can pass structural policy while changes outside the leaf file remain rejected. Record what real replay proves and any remaining proposal/validation blockers. No automatic merge or deployment.

Implemented and validated (2026-09-09)

Local updater branch fix/failed-builder-attribution, commit f5f90f6, extracts direct builder failures (legacy, modern, fixed-output hash mismatch), excludes plans/dependent failures, and preserves multiple roots. Regression and shell lint checks pass; a real Nix leaf/dependent probe identifies only the leaf.
Local config branch fix/remediation-leaf-proposals, commits 0792dc6 and 5a0eecd, adds the existing Moonlight leaf target, bounded baseline/candidate package recipes and dependency context, and controller-computed proposal checksums. Full remediation regressions pass. Existing lifecycle integration/regressions and the combined flake check passed. Moonlight derivation and override behavior remain identical before/after the leaf extraction.
The standalone updater tests/auto-update-regressions.sh is a pre-existing stale harness referencing configuration-repository paths; it is not a current flake/CI gate. Relevant incident-protocol tests pass.

Real package replay scope

A disposable Nova flake exposes the actual historical Moonlight package as one test target; it is not a real NixOS host. Baseline nixpkgs 8e2eeb9477c9d40009a5bd51cd3eef2f5abb26f1 builds successfully. Candidate 044bfe75bfe4c7bbe043dc17b5e42ea823b84a09 reproduces the actual AVCodec.pix_fmts compile failure. The new incident identifies exactly one builder, moonlight-qt-6.1.0. Original production incidents were not modified.

Verified dependency comparison: baseline FFmpeg 8.1.2, candidate FFmpeg 9.0; candidate ffmpeg_8 is 8.1.2. Independently building pkgs.moonlight-qt.override { ffmpeg = pkgs.ffmpeg_8; } against the candidate succeeds via the official binary cache. This proves the narrow compatibility repair, not model selection, fleet validation, or publication.

Early real-model replays correctly reached eligible intake but declined a proposal because packaging/dependency information was missing; two responses also incorrectly retained affectedHosts in diagnosis-only output and were rejected. The added evidence and explicit output checklist address these observed gaps without lowering the confidence floor or permitting global changes.

A later real-model replay selected remediate, Moonlight as sole culprit, and FFmpeg 8 at confidence 0.95, but targeted the copied candidate recipe under .remediation-incident/. The controller correctly rejected that path. The next refinement explicitly lists existing allowed repository leaf files and labels source copies as reference text; integration tests preserve rejection of evidence and gaming-profile patches.

Real-model proposal accepted

The explicit-target worker run produced the correct one-file ffmpeg = pkgs.ffmpeg_8 override at confidence 0.90 in 191.285 seconds. It had a miscounted unified-diff hunk. Replayed those exact model outputs through the updated controller, without semantic edits or another model call. Git recounts hunk counts, all existing path/content/build gates run, and successful validation persists a canonical Git diff plus the original advisory digest. Non-executable comment edits remain subject to safety deletion deny patterns.

Real validation exposed and fixed two additional bugs hidden by earlier fake-Nix fixtures: stderr deprecation warnings corrupted JSON parsing, and a Nix projection containing outPath serialized as a path string instead of the requested record. JSON commands now separate bounded stderr, and the Nix projection uses storePath before normalizing the verified record. Tests emulate both behaviors.

Final replay status: validated-ready. Actual baseline build passed, the exact candidate compiler failure reproduced, the model-selected patched package built, flake checks passed, closure impact stayed within policy, and the exact Moonlight 6.1.0 output was proven in the closure. This is a reduced real-package fixture with one synthetic replay target, not full-host/fleet validation. The fixture exposes the real package as its toplevel.

Canonical patch SHA-256: f3e228de6b808608eac700389100bc6625e4d622ef3e0e049be38ab4f2531a24. The controller also generated pin lifecycle metadata and a checksum-bound removal patch in a disposable clone; reverse application restores the original leaf file exactly. No synthetic incident or remediation MR was published.

Current local config commits: 0792dc6, 5a0eecd, eb44253, 8138968; updater f5f90f6. Both trees are clean. Full remediation and pin-lifecycle regression gates pass. Combined flake validation and original Moonlight derivation/interface identity checks pass. Phase-two helper integration was tested with an explicit local input override; the committed lock still references the published phase-one helper, so publication needs a coordinated helper branch/pin update.

Limitations: model prose still invents unsupported historical FFmpeg API details; deterministic builds validate the proposed executable remedy, not every explanatory claim. Human MR review remains necessary. No changes have been merged or deployed, and this issue remains open.

Recovery check also passed: after applying the controller-generated removal patch, the unpinned Moonlight package builds successfully against the repository's newer nixpkgs revision 94a0f8d9ca5da101ec1cddbdfecfdc1ae1766d04. This is an actual package build/cache realization, not a mocked lifecycle build, but it does not replace full-host lifecycle validation.

## Publication status Published config MR #210 (stacked on #207) and updater MR https://git.nimmog.uk/nimmo/nixos-auto-update/pulls/2. Config head `b5f17bdf601c7b8bc8107e44599f38f86ea9b188` pins published helper `1f80cf332b8f459d4813467ac0c352717d4df362`. Full flake and remediation regression checks pass with this exact published pin. MR #207's review fixes are published at `c907fda`; its repeat automated review says **Ready to merge**, and CI tests pass. Integration order: merge updater MR #2, change MR #210's locked helper ref to main while retaining the validated revision, merge #207, then retarget #210 to main. Live deployment and no-op evidence remain pending; no merge/deployment performed here. The earlier checkpoint notes below describe the pre-publication replay history. ## Context Follow-up to #205/#207 and historical Moonlight #172 replay. A real leaf compatibility failure never reached pin validation: all .drv mentions were treated as failed builders, and Moonlight had no existing file inside the leaf patch allowlist. ## Scope - Recognize explicit failed-builder diagnostics in legacy and modern Nix logs. Exclude plans and dependent/cancelled builds; retain multiple independent root failures as ineligible. - Give Moonlight a dedicated existing leaf expression exposed consistently through host pkgs. Preserve its current derivation without enabling a pin or installing it on extra hosts. - Clarify that a leaf-scoped dependency compatibility override is distinct from changing a core library globally. Keep evaluation/global/core/multiple-root exclusions. - Test the extraction with real Nix failures and exercise the leaf patch validator. Assess a further historical replay without falsifying the original incident or loosening its historical base validation. ## Completion criteria Regression/lint/flake checks pass, unmodified Moonlight derivation identity is unchanged, and a Moonlight-only compatibility patch can pass structural policy while changes outside the leaf file remain rejected. Record what real replay proves and any remaining proposal/validation blockers. No automatic merge or deployment. ## Implemented and validated (2026-09-09) - Local updater branch `fix/failed-builder-attribution`, commit `f5f90f6`, extracts direct builder failures (legacy, modern, fixed-output hash mismatch), excludes plans/dependent failures, and preserves multiple roots. Regression and shell lint checks pass; a real Nix leaf/dependent probe identifies only the leaf. - Local config branch `fix/remediation-leaf-proposals`, commits `0792dc6` and `5a0eecd`, adds the existing Moonlight leaf target, bounded baseline/candidate package recipes and dependency context, and controller-computed proposal checksums. Full remediation regressions pass. Existing lifecycle integration/regressions and the combined flake check passed. Moonlight derivation and override behavior remain identical before/after the leaf extraction. - The standalone updater `tests/auto-update-regressions.sh` is a pre-existing stale harness referencing configuration-repository paths; it is not a current flake/CI gate. Relevant incident-protocol tests pass. ## Real package replay scope A disposable Nova flake exposes the actual historical Moonlight package as one test target; it is not a real NixOS host. Baseline nixpkgs `8e2eeb9477c9d40009a5bd51cd3eef2f5abb26f1` builds successfully. Candidate `044bfe75bfe4c7bbe043dc17b5e42ea823b84a09` reproduces the actual `AVCodec.pix_fmts` compile failure. The new incident identifies exactly one builder, `moonlight-qt-6.1.0`. Original production incidents were not modified. Verified dependency comparison: baseline FFmpeg 8.1.2, candidate FFmpeg 9.0; candidate `ffmpeg_8` is 8.1.2. Independently building `pkgs.moonlight-qt.override { ffmpeg = pkgs.ffmpeg_8; }` against the candidate succeeds via the official binary cache. This proves the narrow compatibility repair, not model selection, fleet validation, or publication. Early real-model replays correctly reached eligible intake but declined a proposal because packaging/dependency information was missing; two responses also incorrectly retained affectedHosts in diagnosis-only output and were rejected. The added evidence and explicit output checklist address these observed gaps without lowering the confidence floor or permitting global changes. A later real-model replay selected `remediate`, Moonlight as sole culprit, and FFmpeg 8 at confidence 0.95, but targeted the copied candidate recipe under `.remediation-incident/`. The controller correctly rejected that path. The next refinement explicitly lists existing allowed repository leaf files and labels source copies as reference text; integration tests preserve rejection of evidence and gaming-profile patches. ## Real-model proposal accepted The explicit-target worker run produced the correct one-file `ffmpeg = pkgs.ffmpeg_8` override at confidence 0.90 in 191.285 seconds. It had a miscounted unified-diff hunk. Replayed those exact model outputs through the updated controller, without semantic edits or another model call. Git recounts hunk counts, all existing path/content/build gates run, and successful validation persists a canonical Git diff plus the original advisory digest. Non-executable comment edits remain subject to safety deletion deny patterns. Real validation exposed and fixed two additional bugs hidden by earlier fake-Nix fixtures: stderr deprecation warnings corrupted JSON parsing, and a Nix projection containing `outPath` serialized as a path string instead of the requested record. JSON commands now separate bounded stderr, and the Nix projection uses `storePath` before normalizing the verified record. Tests emulate both behaviors. Final replay status: **validated-ready**. Actual baseline build passed, the exact candidate compiler failure reproduced, the model-selected patched package built, flake checks passed, closure impact stayed within policy, and the exact Moonlight 6.1.0 output was proven in the closure. This is a reduced real-package fixture with one synthetic `replay` target, not full-host/fleet validation. The fixture exposes the real package as its toplevel. Canonical patch SHA-256: `f3e228de6b808608eac700389100bc6625e4d622ef3e0e049be38ab4f2531a24`. The controller also generated pin lifecycle metadata and a checksum-bound removal patch in a disposable clone; reverse application restores the original leaf file exactly. No synthetic incident or remediation MR was published. Current local config commits: `0792dc6`, `5a0eecd`, `eb44253`, `8138968`; updater `f5f90f6`. Both trees are clean. Full remediation and pin-lifecycle regression gates pass. Combined flake validation and original Moonlight derivation/interface identity checks pass. Phase-two helper integration was tested with an explicit local input override; the committed lock still references the published phase-one helper, so publication needs a coordinated helper branch/pin update. Limitations: model prose still invents unsupported historical FFmpeg API details; deterministic builds validate the proposed executable remedy, not every explanatory claim. Human MR review remains necessary. No changes have been merged or deployed, and this issue remains open. Recovery check also passed: after applying the controller-generated removal patch, the unpinned Moonlight package builds successfully against the repository's newer nixpkgs revision `94a0f8d9ca5da101ec1cddbdfecfdc1ae1766d04`. This is an actual package build/cache realization, not a mocked lifecycle build, but it does not replace full-host lifecycle validation.

nimmo added this to the Automated Update Remediation milestone

2026-09-09 05:14:31 +01:00

nimmo added the

project:auto-update-remediation

service:auto-update

labels

2026-09-09 09:41:24 +01:00

nimmo referenced this issue from nimmo/nixos-auto-update

2026-09-09 18:22:17 +01:00

Identify directly failed builders in rejected updates #2