Expose partial input-update rejections instead of reporting green success #53

New issue

Closed

opened 2026-07-13 07:57:07 +01:00 by nimmo · 1 comment

nimmo commented

2026-07-13 07:57:07 +01:00

Owner

Context

Both the Forgejo path and local full-update path run:

nixos_update_run_input_groups || true

If at least one group is accepted, later steps commit/push and send a success notification even when FAILED_UPDATES=true. This is compatible with independent channel acceptance, but it hides failed or stale channels behind an overall green workflow and generic success message.

Scope

Track each group as unchanged, accepted, rejected-update, or infrastructure-failed.
Distinguish package incompatibility from transient build/network/storage failures.
Preserve safe independent acceptance, but make partial completion explicit in Actions status, job summary, commit message, and ntfy.
Retain enough diagnostic output to identify the failed input and host without reading an entire six-hour log.
Decide whether partial rejection should produce a non-green conclusion, warning/check, or follow-up retry.

Completion criteria

Every input group has an outcome in the run summary.
Partial acceptance cannot emit an unqualified success message.
Rejected inputs and failing hosts are named.
Transient infrastructure failures are retried separately from incompatible updates.
A subsequent run can retry rejected groups without needlessly rebuilding accepted unchanged groups.

## Context Both the Forgejo path and local full-update path run: ```bash nixos_update_run_input_groups || true ``` If at least one group is accepted, later steps commit/push and send a success notification even when `FAILED_UPDATES=true`. This is compatible with independent channel acceptance, but it hides failed or stale channels behind an overall green workflow and generic success message. ## Scope - Track each group as unchanged, accepted, rejected-update, or infrastructure-failed. - Distinguish package incompatibility from transient build/network/storage failures. - Preserve safe independent acceptance, but make partial completion explicit in Actions status, job summary, commit message, and ntfy. - Retain enough diagnostic output to identify the failed input and host without reading an entire six-hour log. - Decide whether partial rejection should produce a non-green conclusion, warning/check, or follow-up retry. ## Completion criteria - [ ] Every input group has an outcome in the run summary. - [ ] Partial acceptance cannot emit an unqualified success message. - [ ] Rejected inputs and failing hosts are named. - [ ] Transient infrastructure failures are retried separately from incompatible updates. - [ ] A subsequent run can retry rejected groups without needlessly rebuilding accepted unchanged groups.

nimmo added this to the Automated Update Reliability Review milestone

2026-07-13 07:57:07 +01:00

nimmo added the

host:fleet

project:auto-update-reliability

service:auto-update

phase:validation

labels

2026-07-13 07:57:07 +01:00

nimmo commented

2026-07-16 20:21:55 +01:00

Author

Owner

Implemented in fb892dc (fix: expose partial flake update failures).

Validation:

Auto-Update Regression Tests run #318 passed.
Attic cache build run #317 passed.
Regression coverage verifies all four per-group outcomes, non-green partial results, named failing inputs/hosts, infrastructure-vs-validation classification, and retrying rejected groups without rebuilding accepted unchanged groups.

Partial accepted updates are now preserved and published with a qualified commit/notification, then the workflow deliberately exits non-green. Closing as complete.

Implemented in `fb892dc` (`fix: expose partial flake update failures`). Validation: - Auto-Update Regression Tests run [#318](https://git.nimmog.uk/nimmo/nixos-config/actions/runs/318) passed. - Attic cache build run [#317](https://git.nimmog.uk/nimmo/nixos-config/actions/runs/317) passed. - Regression coverage verifies all four per-group outcomes, non-green partial results, named failing inputs/hosts, infrastructure-vs-validation classification, and retrying rejected groups without rebuilding accepted unchanged groups. Partial accepted updates are now preserved and published with a qualified commit/notification, then the workflow deliberately exits non-green. Closing as complete.

nimmo closed this issue

2026-07-16 20:21:59 +01:00