nimmo/nixos-config
nimmo/nixos-config
Watch
1
0
Fork
You've already forked nixos-config
 0
Code Issues 15 Pull requests Wiki Activity Actions

Implement guarded Nova remediation worker and Forgejo proposals #130

New issue
Closed
opened 2026-07-31 22:00:21 +01:00 by nimmo · 3 comments
nimmo commented 2026-07-31 22:00:21 +01:00
Owner
Copy link

Context

When the producer rejects a genuine package regression, a local model on Nova should diagnose it, propose a narrow temporary pin where safe, validate the candidate, and create review material. The model must remain advisory; deterministic code owns credentials, validation and publication.

Scope

Implement the isolated OpenCode worker and controller around the model policy from #128 and incident bundle from #129.

Checklist

  • Create a fresh worktree from the exact validated base and rejected candidate lock.
  • Run under a dedicated least-privilege context with no deployment secrets or direct main access.
  • Supply logs as explicitly untrusted evidence and constrain readable or writable paths.
  • Permit only bounded investigation and one remediation attempt per incident.
  • Initially allow only leaf applications and isolated services; require diagnosis-only output for kernels, drivers, core libraries, language package sets, module-schema failures and ambiguous cases.
  • Require a structured diagnosis, proposed patch, rationale, confidence and machine-readable validation request.
  • Reproduce the original failure without the patch and require success with the patch.
  • Build every affected host and all required flake checks before accepting a proposal.
  • Inspect closure impact and prove that the selected package comes from the intended revision or source.
  • Let a deterministic controller create or update a fingerprint-deduplicated Forgejo issue and issue-linked PR.
  • Never merge, publish flake.lock or deploy from the model process.
  • Add time, token, disk, build and retry limits with cleanup for interrupted worktrees.

Completion criteria

An eligible fixture produces a validated issue-linked PR; unsafe and ineligible fixtures produce diagnosis-only issues; no model process holds Forgejo publication or deployment authority.

Depends on #128 and #129.

## Context When the producer rejects a genuine package regression, a local model on Nova should diagnose it, propose a narrow temporary pin where safe, validate the candidate, and create review material. The model must remain advisory; deterministic code owns credentials, validation and publication. ## Scope Implement the isolated OpenCode worker and controller around the model policy from #128 and incident bundle from #129. ## Checklist - [ ] Create a fresh worktree from the exact validated base and rejected candidate lock. - [ ] Run under a dedicated least-privilege context with no deployment secrets or direct main access. - [ ] Supply logs as explicitly untrusted evidence and constrain readable or writable paths. - [ ] Permit only bounded investigation and one remediation attempt per incident. - [ ] Initially allow only leaf applications and isolated services; require diagnosis-only output for kernels, drivers, core libraries, language package sets, module-schema failures and ambiguous cases. - [ ] Require a structured diagnosis, proposed patch, rationale, confidence and machine-readable validation request. - [ ] Reproduce the original failure without the patch and require success with the patch. - [ ] Build every affected host and all required flake checks before accepting a proposal. - [ ] Inspect closure impact and prove that the selected package comes from the intended revision or source. - [ ] Let a deterministic controller create or update a fingerprint-deduplicated Forgejo issue and issue-linked PR. - [ ] Never merge, publish flake.lock or deploy from the model process. - [ ] Add time, token, disk, build and retry limits with cleanup for interrupted worktrees. ## Completion criteria An eligible fixture produces a validated issue-linked PR; unsafe and ineligible fixtures produce diagnosis-only issues; no model process holds Forgejo publication or deployment authority. Depends on #128 and #129.
nimmo commented 2026-07-31 22:03:32 +01:00
Author
Owner
Copy link

Recommended Codex model

Use codex/gpt-5.6-sol with max reasoning.

This is the highest-risk and most architecturally coupled phase: it combines untrusted-input handling, least privilege, bounded autonomous tool use, Nix validation, Forgejo side effects, cleanup, and fleet publication invariants. OpenAI recommends reserving max effort for the hardest quality-first workloads; that tradeoff is justified for the controller and security boundary. Use Terra only for mechanical test expansion after the design is fixed.

Model guidance: https://developers.openai.com/api/docs/guides/latest-model

## Recommended Codex model Use `codex/gpt-5.6-sol` with `max` reasoning. This is the highest-risk and most architecturally coupled phase: it combines untrusted-input handling, least privilege, bounded autonomous tool use, Nix validation, Forgejo side effects, cleanup, and fleet publication invariants. OpenAI recommends reserving max effort for the hardest quality-first workloads; that tradeoff is justified for the controller and security boundary. Use Terra only for mechanical test expansion after the design is fixed. Model guidance: https://developers.openai.com/api/docs/guides/latest-model
nimmo commented 2026-08-01 07:01:54 +01:00
Author
Owner
Copy link

PR #134 implementation and deterministic CI are green, but merge is blocked by the required automated-review service. Three bounded review attempts failed before publishing any review comment or code finding because Qwen returned empty/malformed JSON (latest: Extra data). Automatic retries have stopped. PR #134 remains open and unmerged pending repair of the separate forgejo-auto-pr-review path or explicit direction to bring that external repository into scope. No check is being bypassed, and dependent issue #131 has not started.

PR #134 implementation and deterministic CI are green, but merge is blocked by the required automated-review service. Three bounded review attempts failed before publishing any review comment or code finding because Qwen returned empty/malformed JSON (latest: Extra data). Automatic retries have stopped. PR #134 remains open and unmerged pending repair of the separate forgejo-auto-pr-review path or explicit direction to bring that external repository into scope. No check is being bypassed, and dependent issue #131 has not started.
nimmo commented 2026-08-01 16:19:59 +01:00
Author
Owner
Copy link

Passive rollout and no-op evidence

PR #134 merged as revision 32bb4093c9cca08bcf717146bb690289a9f409ca. The normal fleet timers deployed it without any forced or manual run:

  • Lyra: scheduled pull began 2026-08-01 10:00:21 BST, fast-forwarded from a90a87a to 32bb409, verified deployed revision 32bb4093c9cca08bcf717146bb690289a9f409ca at 10:00:41, and completed at 10:00:47.
  • Vega: scheduled pull began 2026-08-01 10:06:22 BST, fast-forwarded from a90a87a to 32bb409, verified the same deployed revision at 10:06:48, and completed at 10:06:49.

The subsequent normal even-hour window supplied the required no-op follow-up:

  • Lyra: scheduled run began 12:03:20; at 12:03:21 it reported origin/main and HEAD both at 32bb4093c9cca08bcf717146bb690289a9f409ca, then Nothing changed in pull, system is current.
  • Vega: scheduled run began 12:15:06; at 12:15:07 it reported the same matching revision and successful no-op.

Evidence was read from nixos-auto-update.service and detached-switch journals on both hosts. No deployment or updater run was forced. With deterministic CI, independent automated review, clean merge, genuine deployment, and the required no-op follow-up all complete, #130 is complete.

## Passive rollout and no-op evidence PR #134 merged as revision `32bb4093c9cca08bcf717146bb690289a9f409ca`. The normal fleet timers deployed it without any forced or manual run: - Lyra: scheduled pull began 2026-08-01 10:00:21 BST, fast-forwarded from `a90a87a` to `32bb409`, verified deployed revision `32bb4093c9cca08bcf717146bb690289a9f409ca` at 10:00:41, and completed at 10:00:47. - Vega: scheduled pull began 2026-08-01 10:06:22 BST, fast-forwarded from `a90a87a` to `32bb409`, verified the same deployed revision at 10:06:48, and completed at 10:06:49. The subsequent normal even-hour window supplied the required no-op follow-up: - Lyra: scheduled run began 12:03:20; at 12:03:21 it reported origin/main and HEAD both at `32bb4093c9cca08bcf717146bb690289a9f409ca`, then `Nothing changed in pull, system is current`. - Vega: scheduled run began 12:15:06; at 12:15:07 it reported the same matching revision and successful no-op. Evidence was read from `nixos-auto-update.service` and detached-switch journals on both hosts. No deployment or updater run was forced. With deterministic CI, independent automated review, clean merge, genuine deployment, and the required no-op follow-up all complete, #130 is complete.
nimmo closed this issue 2026-08-01 16:20:02 +01:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
feat/issue-236-electra-nova-vega
feat/issue-233-nova-builder-credentials
docs/wiki-final-cutover
fix/remediation-evaluation-evidence
fix/remediation-evidence-review
fix/chaos-synthseek-queue
pre-electra-disko-migration
Labels
Clear labels   
area:authentication
Issues related to authentication area

  
area:flake-utilities
Issues related to flake utilities area

  
area:performance
Issues related to performance area

  
area:tbd
Area to be determined

  
host:chaos
Issues specific to the chaos host.

  
host:electra
Issues specific to the electra host.

  
host:fleet
Issues that span multiple hosts or the fleet as a whole.

  
host:lyra
Issues specific to the lyra host.

  
host:nova
Issues specific to the nova host.

  
host:vega
Issues specific to the vega host.

  
investigation
Issue is an investigation task

  
phase:cutover
The service cutover execution phase.

  
phase:deploy

   
phase:mcp

   
phase:module

   
phase:packaging

   
phase:prep
Preparation and preflight work before service cutover.

  
phase:validation
Post-cutover validation and rollback-window work.

  
priority:high
High priority issue

  
priority:medium
Medium priority issue

  
project:attic-postgres-lyra-rollout
Issues in the Attic PostgreSQL migration and Lyra rollout workstream.

  
project:auto-update-reliability
Work arising from the automated update reliability review.

  
project:auto-update-remediation
Guarded Nova-assisted diagnosis and remediation of failed automated updates.

  
project:declarative-purity-cleanup
Issues in the declarative purity cleanup workstream.

  
project:external-review
Review of external configurations for ideas and improvements

  
project:fleet-boundary-cleanup
Maintainability work that clarifies fleet configuration and automation boundaries.

  
project:host-facts-refactor
Issues in the host facts refactor workstream.

  
project:lyra-nixos-deploy
Issues in the Lyra NixOS deployment workstream.

  
project:lyra-service-stack-migration
Issues in the lyra service stack migration workstream.

  
project:nebula-mesh-network
Issues in the Nebula mesh network workstream.

  
project:nixos-build-deployment-pipeline
Issues in the NixOS Build & Deployment Pipeline workstream.

  
project:security-hardening
Work belonging to the Security Hardening milestone

  
project:service-stack-migration
Issues in the service stack migration workstream.

  
project:vega-sillytavern-cutover
Issues in the Vega SillyTavern cutover workstream.

  
project:wiki-rebuild
Re-found the repository wiki as the fleet's curated knowledge architecture.

  
repo:numtide/flake-utils
Issues related to numtide/flake-utils repository

  
repo:numtide/nix-auth
Issues related to numtide/nix-auth repository

  
repo:numtide/nixos-passthru-cache
Issues related to numtide/nixos-passthru-cache repository

  
repo:numtide/nix-relay
Issues related to numtide/nix-relay repository

  
service:auto-update
NixOS flake update, cache build, and consumer deployment automation.

  
service:mem0

   
service:nix
Issues concerning Nix build, store, cache, or deployment infrastructure.

  
service:sillytavern
Issues related to the SillyTavern service.

  
service:slskd
Soulseek daemon (slskd) work.

  
service:synthseek
Synthseek music discovery/library manager work.

No labels
area:authentication
area:flake-utilities
area:performance
area:tbd
host:chaos
host:electra
host:fleet
host:lyra
host:nova
host:vega
investigation
phase:cutover
phase:deploy
phase:mcp
phase:module
phase:packaging
phase:prep
phase:validation
priority:high
priority:medium
project:attic-postgres-lyra-rollout
project:auto-update-reliability
project:auto-update-remediation
project:declarative-purity-cleanup
project:external-review
project:fleet-boundary-cleanup
project:host-facts-refactor
project:lyra-nixos-deploy
project:lyra-service-stack-migration
project:nebula-mesh-network
project:nixos-build-deployment-pipeline
project:security-hardening
project:service-stack-migration
project:vega-sillytavern-cutover
project:wiki-rebuild
repo:numtide/flake-utils
repo:numtide/nix-auth
repo:numtide/nixos-passthru-cache
repo:numtide/nix-relay
service:auto-update
service:mem0
service:nix
service:sillytavern
service:slskd
service:synthseek
Milestone
Clear milestone
No items
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
nimmo/nixos-config#130
No description provided.