01 – Establish Nova as the Central Nix Build Host #232

New issue

Closed

opened 2026-09-19 20:20:39 +01:00 by nimmo · 2 comments

nimmo commented

2026-09-19 20:20:39 +01:00

Owner

Goal

Establish Nova, currently a Fedora 44 host with Nix installed, as the first central remote Nix builder. This implementation is intentionally independent of Nova's pending NixOS migration: Fedora-specific builder setup must be documented now and the later migration must preserve the proven builder contract.

Scope

Record Nova's starting state (Nix daemon mode, Nix version, resources, cache configuration, and current CI/Attic workload) before changing it.
Create a dedicated remote-builder SSH identity. It must be distinct from nimmo, the Forgejo runner identity, and Nix's internal nixbld workers.
Use key-only authentication; disable password authentication for this identity and avoid general-purpose interactive administration as far as the Nix remote-build protocol permits.
Configure the least Nix-daemon authority needed to serve remote builds. Do not grant Forgejo write, Attic push, deployment-target SSH, or other controller credentials.
Set explicit supported systems/features and conservative build concurrency, disk/GC, and queue headroom that protect Nova's existing CI/remediation work.
Define builder failure and replacement recovery, including how a later NixOS installation takes over without changing client configuration.

Security model

Remote Nix builds execute builder-controlled derivations and therefore can run arbitrary build workloads on Nova. Treat Nova as privileged build infrastructure; an SSH key for it is not a general administrative credential and must be separately revocable.

Validation

From a separate Nix-capable client, build a non-substituted derivation on Nova and verify the build executes there.
Verify the requester receives the resulting store path.
Demonstrate the builder identity cannot log in with a password, deploy to a fleet target, or access Forgejo/Attic credentials.
Record the Fedora setup/recovery procedure and the measured resource policy.

## Goal Establish Nova, currently a Fedora 44 host with Nix installed, as the first central remote Nix builder. This implementation is intentionally independent of Nova's pending NixOS migration: Fedora-specific builder setup must be documented now and the later migration must preserve the proven builder contract. ## Scope - Record Nova's starting state (Nix daemon mode, Nix version, resources, cache configuration, and current CI/Attic workload) before changing it. - Create a dedicated remote-builder SSH identity. It must be distinct from `nimmo`, the Forgejo runner identity, and Nix's internal `nixbld` workers. - Use key-only authentication; disable password authentication for this identity and avoid general-purpose interactive administration as far as the Nix remote-build protocol permits. - Configure the least Nix-daemon authority needed to serve remote builds. Do not grant Forgejo write, Attic push, deployment-target SSH, or other controller credentials. - Set explicit supported systems/features and conservative build concurrency, disk/GC, and queue headroom that protect Nova's existing CI/remediation work. - Define builder failure and replacement recovery, including how a later NixOS installation takes over without changing client configuration. ## Security model Remote Nix builds execute builder-controlled derivations and therefore can run arbitrary build workloads on Nova. Treat Nova as privileged build infrastructure; an SSH key for it is not a general administrative credential and must be separately revocable. ## Validation - From a separate Nix-capable client, build a non-substituted derivation on Nova and verify the build executes there. - Verify the requester receives the resulting store path. - Demonstrate the builder identity cannot log in with a password, deploy to a fleet target, or access Forgejo/Attic credentials. - Record the Fedora setup/recovery procedure and the measured resource policy.

nimmo added this to the NixOS Build & Deployment Pipeline milestone

2026-09-19 20:20:40 +01:00

nimmo added the

host:nova

phase:prep

project:nixos-build-deployment-pipeline

service:nix

labels

2026-09-19 21:00:54 +01:00

nimmo referenced this issue

2026-09-19 21:01:10 +01:00

02 - Secure and Automate Access to the Nova Build Machine #233

nimmo referenced this issue

2026-09-19 21:01:24 +01:00

03 – Enable NixOS Hosts to Offload Builds to Nova #234

~~nimmo referenced this issue 2026-09-19 21:02:12 +01:00~~

07 – Harden Build and Deployment Trust Boundaries #238

nimmo referenced this issue

2026-09-19 21:02:36 +01:00

09 – Document Build, Deployment and Recovery Workflows #240

nimmo commented

2026-09-19 21:33:49 +01:00

Author

Owner

Progress (2026-09-19): Nova Fedora bootstrap and a disposable-key proof are complete. A dedicated nix-remote-builder account is now trusted by the Nix daemon and its verification key was restricted to nix-store --serve --write; a minimal non-substituted derivation built on Nova and copied back to Lyra. The disposable key was removed from both hosts after verification. Commit e9e6dda records the Fedora procedure, baseline, proof, and recovery steps. Remaining in this issue: align/remove the stale Nova substituters (nix-cache.nimmog.uk warns and cuda-maintainers.cachix.org returns 401; preserve https://attic.nimmog.uk/nixos) and establish a reviewed shared Nix-daemon resource policy that protects the existing Nova CI workload.

Progress (2026-09-19): Nova Fedora bootstrap and a disposable-key proof are complete. A dedicated `nix-remote-builder` account is now trusted by the Nix daemon and its verification key was restricted to `nix-store --serve --write`; a minimal non-substituted derivation built on Nova and copied back to Lyra. The disposable key was removed from both hosts after verification. Commit e9e6dda records the Fedora procedure, baseline, proof, and recovery steps. Remaining in this issue: align/remove the stale Nova substituters (`nix-cache.nimmog.uk` warns and `cuda-maintainers.cachix.org` returns 401; preserve `https://attic.nimmog.uk/nixos`) and establish a reviewed shared Nix-daemon resource policy that protects the existing Nova CI workload.

nimmo commented

2026-09-19 21:40:44 +01:00

Author

Owner

Follow-up (2026-09-19): Nova system and nimmo user Nix cache settings now retain Attic and use cache.nixos-cuda.org plus cache.nixos.org; the stale nix-cache.nimmog.uk and unauthorised cuda-maintainers.cachix.org entries are gone. Nova’s shared Nix policy is now max-jobs = 3, cores = 9, min-free = 32 GiB, and max-free = 80 GiB, matching the established CI allocation. nix store info succeeded from Nova against Attic, cache.nixos-cuda.org, and cache.nixos.org. Documentation is captured in local commits e9e6dda and e11eee6; CI cache alignment is local commit ed24e95 and passed actionlint. The issue is ready for review/publishing; it remains open until those commits are reviewed and merged.

Follow-up (2026-09-19): Nova system and `nimmo` user Nix cache settings now retain Attic and use `cache.nixos-cuda.org` plus `cache.nixos.org`; the stale `nix-cache.nimmog.uk` and unauthorised `cuda-maintainers.cachix.org` entries are gone. Nova’s shared Nix policy is now `max-jobs = 3`, `cores = 9`, `min-free = 32 GiB`, and `max-free = 80 GiB`, matching the established CI allocation. `nix store info` succeeded from Nova against Attic, cache.nixos-cuda.org, and cache.nixos.org. Documentation is captured in local commits e9e6dda and e11eee6; CI cache alignment is local commit ed24e95 and passed actionlint. The issue is ready for review/publishing; it remains open until those commits are reviewed and merged.

nimmo referenced this issue from a pull request that will close it,

2026-09-19 21:42:48 +01:00

feat(nova): establish restricted remote Nix builder #242

nimmo closed this issue

2026-09-19 22:08:58 +01:00