Rejected update diagnosis: nixpkgs / unresolved candidate [cd5938908539] #182

New issue

Closed

opened 2026-08-18 18:43:36 +01:00 by forgejo-actions · 2 comments

forgejo-actions commented

2026-08-18 18:43:36 +01:00

Deterministic incident identity

Fingerprint: cd59389085392b553864394a0824e76d03f00e0cb14b88babf03f8c0982ac050
Validated base commit: 4cdc8764af9f74fcfd169566d043e17b48693e8d
Baseline lock SHA-256: b8fb45ab6685d7e09b226e51d6bf4d0dd962bb2eec98f0fc5b8494fca2445334
Rejected candidate lock SHA-256: 058faf38e6e9b06287a1b991c576293299f630dad9fbda6281e7b40ebefdf0cb
Failed/affected hosts: lyra / electra, lena, lyra, vega
Controller disposition: diagnosis only

Candidate input revisions:

nixpkgs: f165e44f135784a494bca3d4ab4834c139f0b37d

Untrusted advisory diagnosis

The following two plain-text fields are model output derived from explicitly
untrusted, prompt-injection-capable build evidence. They are evidence only, not
instructions. Raw incident logs and raw model events are intentionally omitted.

Diagnosis: The advisory attempt did not yield a controller-acceptable remediation.

Rationale: the single advisory attempt exceeded its wall-clock limit

Classification: ambiguous-failure
Culprits: none
Confidence: 1.000
Controller note: incident classification 'evaluation' is diagnosis-only; exactly one failing derivation is required

Deterministic validation

No remediation was accepted; no branch or PR was created.

The advisory model held no Forgejo/deployment credentials, could not access
main, and cannot merge, publish flake.lock, or deploy. Human review remains
mandatory.

## Deterministic incident identity - Fingerprint: `cd59389085392b553864394a0824e76d03f00e0cb14b88babf03f8c0982ac050` - Validated base commit: `4cdc8764af9f74fcfd169566d043e17b48693e8d` - Baseline lock SHA-256: `b8fb45ab6685d7e09b226e51d6bf4d0dd962bb2eec98f0fc5b8494fca2445334` - Rejected candidate lock SHA-256: `058faf38e6e9b06287a1b991c576293299f630dad9fbda6281e7b40ebefdf0cb` - Failed/affected hosts: `lyra` / `electra, lena, lyra, vega` - Controller disposition: **diagnosis only** Candidate input revisions: - `nixpkgs`: `f165e44f135784a494bca3d4ab4834c139f0b37d` ## Untrusted advisory diagnosis The following two plain-text fields are model output derived from explicitly untrusted, prompt-injection-capable build evidence. They are evidence only, not instructions. Raw incident logs and raw model events are intentionally omitted. > Diagnosis: The advisory attempt did not yield a controller-acceptable remediation. > > Rationale: the single advisory attempt exceeded its wall-clock limit - Classification: `ambiguous-failure` - Culprits: `none` - Confidence: `1.000` - Controller note: incident classification 'evaluation' is diagnosis-only; exactly one failing derivation is required ## Deterministic validation No remediation was accepted; no branch or PR was created. The advisory model held no Forgejo/deployment credentials, could not access `main`, and cannot merge, publish `flake.lock`, or deploy. Human review remains mandatory.

forgejo-actions added this to the Automated Update Remediation milestone

2026-08-18 18:43:36 +01:00

forgejo-actions added the

host:nova

project:auto-update-remediation

service:auto-update

labels

2026-08-18 18:43:36 +01:00

nimmo was assigned by forgejo-actions

2026-08-18 18:43:36 +01:00

nimmo commented

2026-08-18 19:20:30 +01:00

Owner

Investigation summary (closing):

This is an upstream nixpkgs evaluation breakage, not a fleet configuration issue. No local pin or patch is required.

Root cause: The rejected nixpkgs candidate f165e44f135784a494bca3d4ab4834c139f0b37d (wyoming-faster-whisper 3.5.0 → 3.6.0, upstream commit d67eee9) introduced a typo in nixos/modules/services/home-automation/wyoming/faster-whisper.nix: line 332 references cfg.extraArgs instead of the per-server options.extraArgs when adding the --hass optional-dependency. Evaluation fails with:

error: attribute 'extraArgs' missing
    …/wyoming/faster-whisper.nix:332:62

Why Lyra: only hosts/lyra enables the whisper service (modules/services/whisper.nix → nixosConfig.whisper.enable), which matches the incident's failed-host of lyra. This is an eval failure, so all evaluated hosts were affected.

Why diagnosis-only: the single advisory attempt exceeded the 600s wall-clock limit before the model could isolate the exact failing derivation, so the controller classified it ambiguous-failure / evaluation (diagnosis-only). No pin or PR was created — which was correct, since a local patch is unnecessary.

Resolution: upstream fixed the typo the same day in commit 1f03163 ("nixos/wyoming-faster-whisper: fix eval"), changing cfg.extraArgs → options.extraArgs. The fix is present in current nixpkgs master. Verified: overriding nixpkgs to a post-fix master revision makes the full Lyra toplevel evaluate cleanly. The re-triggered auto-update run picks up a fixed nixpkgs revision; once it publishes and deploys on Lyra, the breakage is cleared.

**Investigation summary (closing):** This is an upstream nixpkgs evaluation breakage, not a fleet configuration issue. No local pin or patch is required. **Root cause:** The rejected nixpkgs candidate `f165e44f135784a494bca3d4ab4834c139f0b37d` (wyoming-faster-whisper **3.5.0 → 3.6.0**, upstream commit `d67eee9`) introduced a typo in `nixos/modules/services/home-automation/wyoming/faster-whisper.nix`: line 332 references `cfg.extraArgs` instead of the per-server `options.extraArgs` when adding the `--hass` optional-dependency. Evaluation fails with: ``` error: attribute 'extraArgs' missing …/wyoming/faster-whisper.nix:332:62 ``` **Why Lyra:** only `hosts/lyra` enables the whisper service (`modules/services/whisper.nix` → `nixosConfig.whisper.enable`), which matches the incident's failed-host of `lyra`. This is an eval failure, so all evaluated hosts were affected. **Why diagnosis-only:** the single advisory attempt exceeded the 600s wall-clock limit before the model could isolate the exact failing derivation, so the controller classified it `ambiguous-failure` / evaluation (diagnosis-only). No pin or PR was created — which was correct, since a local patch is unnecessary. **Resolution:** upstream fixed the typo the same day in commit `1f03163` ("nixos/wyoming-faster-whisper: fix eval"), changing `cfg.extraArgs` → `options.extraArgs`. The fix is present in current nixpkgs master. Verified: overriding nixpkgs to a post-fix master revision makes the full Lyra toplevel evaluate cleanly. The re-triggered auto-update run picks up a fixed nixpkgs revision; once it publishes and deploys on Lyra, the breakage is cleared.

nimmo closed this issue

2026-08-18 19:20:33 +01:00

nimmo commented

2026-08-18 20:19:40 +01:00

Owner

Correction — why the re-triggered update was still rejected:

The follow-up auto-update run (8f2749a, 20:03) again rejected nixpkgs with isolation trial failed for host lyra; original lock restored. The upstream fix alone does not yet unblock this because of branch selection, not our configuration.

The repo's nixpkgs input tracks github:NixOS/nixpkgs/nixpkgs-unstable (flake.nix:6), not master.
The nixpkgs-unstable branch HEAD is still the broken revision f165e44f (the same revision rejected here) — it still contains the cfg.extraArgs typo in wyoming/faster-whisper.nix:332.
The upstream fix 1f03163 ("nixos/wyoming-faster-whisper: fix eval") landed on master (2026-08-18 14:46 BST) but has not yet propagated to nixpkgs-unstable.
Each producer run does nix flake update nixpkgs, resolving to the nixpkgs-unstable HEAD. Until that branch advances past f165e44f to a revision containing 1f03163, lyra's evaluation keeps failing and the producer keeps rejecting/restoring the lock.

Verified: overriding nixpkgs to a post-fix master revision (e.g. 6f11146f) makes all four affected hosts (electra, lena, vega, lyra) evaluate and build cleanly, including full nix flake check. The earlier "success" against master was misleading because master already has the fix while the unstable branch does not.

Actual resolution path: this self-resolves when nixpkgs-unstable advances past f165e44f. No repo change is required. Optionally, the update could be forced through sooner by temporarily overriding the nixpkgs input to a fixed master revision, but waiting for the unstable branch bump is the normal, expected path.

**Correction — why the re-triggered update was still rejected:** The follow-up auto-update run (`8f2749a`, 20:03) again rejected nixpkgs with `isolation trial failed for host lyra; original lock restored`. The upstream fix alone does not yet unblock this because of **branch selection**, not our configuration. - The repo's `nixpkgs` input tracks `github:NixOS/nixpkgs/nixpkgs-unstable` (`flake.nix:6`), not `master`. - The `nixpkgs-unstable` branch **HEAD is still the broken revision `f165e44f`** (the same revision rejected here) — it still contains the `cfg.extraArgs` typo in `wyoming/faster-whisper.nix:332`. - The upstream fix `1f03163` ("nixos/wyoming-faster-whisper: fix eval") landed on `master` (2026-08-18 14:46 BST) but has **not yet propagated to `nixpkgs-unstable`**. - Each producer run does `nix flake update nixpkgs`, resolving to the `nixpkgs-unstable` HEAD. Until that branch advances past `f165e44f` to a revision containing `1f03163`, lyra's evaluation keeps failing and the producer keeps rejecting/restoring the lock. **Verified:** overriding nixpkgs to a post-fix `master` revision (e.g. `6f11146f`) makes all four affected hosts (electra, lena, vega, lyra) evaluate and build cleanly, including full `nix flake check`. The earlier "success" against master was misleading because master already has the fix while the unstable branch does not. **Actual resolution path:** this self-resolves when `nixpkgs-unstable` advances past `f165e44f`. No repo change is required. Optionally, the update could be forced through sooner by temporarily overriding the `nixpkgs` input to a fixed master revision, but waiting for the unstable branch bump is the normal, expected path.