[INVESTIGATION] Evaluate numtide/nixos-passthru-cache for potential NixOS configuration integration #149

New issue

Closed

opened 2026-08-06 23:24:10 +01:00 by nimmo · 1 comment

nimmo commented

2026-08-06 23:24:10 +01:00

Owner

Investigation: numtide/nixos-passthru-cache

Overview

Investigate the numtide/nixos-passthru-cache repository to determine if it provides valuable enhancements, simplifications, or performance improvements for the NixOS configuration at https://git.nimmog.uk/nimmo/nixos-config.

Repository Information

URL: https://github.com/numtide/nixos-passthru-cache
Description: Caching mechanism for NixOS passthru attributes to improve evaluation performance

Investigation Goals

Understand the core functionality and purpose of nixos-passthru-cache
Evaluate relevance to current NixOS configuration evaluation performance
Identify potential integration points for improving flake check and evaluation speed
Assess maintenance burden and compatibility with existing flake structure
Determine if proof-of-concept implementation is warranted

Large flake with many inputs in flake.nix
Extensive module system in modules/ directory
Multiple host configurations (electra, lena, vega, lyra, cosmos)
Regular use of nix flake check for validation
Auto-update system that performs frequent evaluations
treefmt for formatting checks
Various passthru attributes throughout the configuration

Specific Areas to Examine

How the caching mechanism works for passthru attributes
Impact on flake evaluation performance (especially nix flake check)
Integration requirements with existing flake structure
Cache storage location and management
Cache invalidation strategies when inputs change
Compatibility with existing passthru usage patterns

Suggested Evaluation Approach

Review documentation and README
Examine example usage patterns in NixOS context
Check compatibility with NixOS unstable channel
Benchmark current nix flake check performance
Test implementation in a copy of the flake to measure performance impact
Evaluate cache behavior when inputs are updated
Review maintenance history and community adoption

Decision Criteria

Significant improvement in nix flake check performance (>20% reduction)
Noticeable improvement in general flake evaluation speed
Simple integration with minimal disruption to existing structure
Reliable cache invalidation when needed
Active maintenance and community support
Low maintenance overhead

Notes for Investigating Agent

Feel free to adjust investigation focus based on initial findings
If a repository shows little promise, document reasons for closing investigation
If additional related repositories are discovered during investigation, feel free to create issues for those as well
Initial investigation should be time-boxed (e.g., 2-4 hours per repository max) unless significant promise is found

# Investigation: numtide/nixos-passthru-cache ## Overview Investigate the numtide/nixos-passthru-cache repository to determine if it provides valuable enhancements, simplifications, or performance improvements for the NixOS configuration at https://git.nimmog.uk/nimmo/nixos-config. ## Repository Information - **URL**: https://github.com/numtide/nixos-passthru-cache - **Description**: Caching mechanism for NixOS passthru attributes to improve evaluation performance ## Investigation Goals 1. Understand the core functionality and purpose of nixos-passthru-cache 2. Evaluate relevance to current NixOS configuration evaluation performance 3. Identify potential integration points for improving flake check and evaluation speed 4. Assess maintenance burden and compatibility with existing flake structure 5. Determine if proof-of-concept implementation is warranted ## Current Related Infrastructure - Large flake with many inputs in flake.nix - Extensive module system in modules/ directory - Multiple host configurations (electra, lena, vega, lyra, cosmos) - Regular use of nix flake check for validation - Auto-update system that performs frequent evaluations - treefmt for formatting checks - Various passthru attributes throughout the configuration ## Specific Areas to Examine 1. How the caching mechanism works for passthru attributes 2. Impact on flake evaluation performance (especially nix flake check) 3. Integration requirements with existing flake structure 4. Cache storage location and management 5. Cache invalidation strategies when inputs change 6. Compatibility with existing passthru usage patterns ## Suggested Evaluation Approach - Review documentation and README - Examine example usage patterns in NixOS context - Check compatibility with NixOS unstable channel - Benchmark current nix flake check performance - Test implementation in a copy of the flake to measure performance impact - Evaluate cache behavior when inputs are updated - Review maintenance history and community adoption ## Decision Criteria - Significant improvement in nix flake check performance (>20% reduction) - Noticeable improvement in general flake evaluation speed - Simple integration with minimal disruption to existing structure - Reliable cache invalidation when needed - Active maintenance and community support - Low maintenance overhead ## Notes for Investigating Agent - Feel free to adjust investigation focus based on initial findings - If a repository shows little promise, document reasons for closing investigation - If additional related repositories are discovered during investigation, feel free to create issues for those as well - Initial investigation should be time-boxed (e.g., 2-4 hours per repository max) unless significant promise is found

nimmo added this to the NumTide Repository Evaluation milestone

2026-08-06 23:24:10 +01:00

nimmo added the

repo:numtide/nixos-passthru-cache

area:performance

priority:high

investigation

labels

2026-08-06 23:24:10 +01:00

nimmo commented

2026-08-09 23:03:44 +01:00

Author

Owner

Evaluation conclusion: defer; current infrastructure already covers the dominant path

Reviewed upstream numtide/nixos-passthru-cache at commit 964a6cb.

Corrected scope

The issue premise is inaccurate: this is not a cache for NixOS passthru attributes or flake evaluation. It is a beta nginx pull-through binary cache, primarily intended for organizations with many Nix machines repeatedly downloading the same objects from cache.nixos.org.

The module provisions nginx caching, defaults to a 200 GiB cache, opens ports 80 and 443, and selects either ACME-backed HTTPS or an HTTP/mDNS LAN mode. It can proxy one configurable upstream.

Fit with this repository

The existing Forgejo workflow builds every active host closure on the self-hosted Nova runner and pushes the complete validated closure set to Attic twice daily.
All hosts already query https://attic.nimmog.uk/nixos before cache.nixos.org. The normal fleet-update path therefore consumes the exact prebuilt closures from the local Attic service rather than independently reconstructing them from the public cache.
A pull-through cache could still capture direct cache.nixos.org misses from ad-hoc builds, but no traffic or egress evidence currently shows those misses are material enough to justify a second cache service.
The upstream module is deliberately opinionated around nginx, public ACME, or trusted-LAN mDNS. This environment normally exposes services through Pangolin/Newt or Nebula and would need additional policy work rather than a drop-in enablement.
Upstream labels the project beta and currently has no release or tag. That is acceptable for a proof of concept, but there is no measured need that warrants one yet.

Validation performed

nix flake check --no-build passed against the reviewed upstream commit.
The NixOS module evaluated successfully against this repository's pinned nixpkgs (26.11 at the evaluated revision) using a container test configuration.

The issue's proposed “greater than 20% flake-check speedup” criterion does not apply: this service caches binary-cache HTTP responses and does not cache Nix evaluation.

Decision

Do not integrate or deploy nixos-passthru-cache now. Attic already optimizes the predictable, high-value fleet update path, while the remaining benefit is unmeasured.

Revisit only after collecting evidence that multiple hosts are repeatedly downloading substantial data directly from cache.nixos.org despite Attic. A future proof of concept should first measure public-cache bytes and duplicate-request rate, then compare those savings against storage, nginx, monitoring, and ingress complexity.

## Evaluation conclusion: defer; current infrastructure already covers the dominant path Reviewed upstream `numtide/nixos-passthru-cache` at commit [`964a6cb`](https://github.com/numtide/nixos-passthru-cache/tree/964a6cbb54059794eef666aeefc8cfd61153a0f1). ### Corrected scope The issue premise is inaccurate: this is not a cache for NixOS `passthru` attributes or flake evaluation. It is a beta [nginx pull-through binary cache](https://github.com/numtide/nixos-passthru-cache/blob/964a6cbb54059794eef666aeefc8cfd61153a0f1/README.md#what-it-is), primarily intended for organizations with many Nix machines repeatedly downloading the same objects from `cache.nixos.org`. The module provisions nginx caching, defaults to a 200 GiB cache, opens ports 80 and 443, and selects either ACME-backed HTTPS or an HTTP/mDNS LAN mode. It can proxy one configurable upstream. ### Fit with this repository - The existing Forgejo workflow builds every active host closure on the self-hosted Nova runner and pushes the complete validated closure set to Attic twice daily. - All hosts already query `https://attic.nimmog.uk/nixos` before `cache.nixos.org`. The normal fleet-update path therefore consumes the exact prebuilt closures from the local Attic service rather than independently reconstructing them from the public cache. - A pull-through cache could still capture direct `cache.nixos.org` misses from ad-hoc builds, but no traffic or egress evidence currently shows those misses are material enough to justify a second cache service. - The upstream module is deliberately opinionated around nginx, public ACME, or trusted-LAN mDNS. This environment normally exposes services through Pangolin/Newt or Nebula and would need additional policy work rather than a drop-in enablement. - Upstream labels the project beta and currently has no release or tag. That is acceptable for a proof of concept, but there is no measured need that warrants one yet. ### Validation performed - `nix flake check --no-build` passed against the reviewed upstream commit. - The NixOS module evaluated successfully against this repository's pinned nixpkgs (`26.11` at the evaluated revision) using a container test configuration. The issue's proposed “greater than 20% flake-check speedup” criterion does not apply: this service caches binary-cache HTTP responses and does not cache Nix evaluation. ### Decision Do not integrate or deploy `nixos-passthru-cache` now. Attic already optimizes the predictable, high-value fleet update path, while the remaining benefit is unmeasured. Revisit only after collecting evidence that multiple hosts are repeatedly downloading substantial data directly from `cache.nixos.org` despite Attic. A future proof of concept should first measure public-cache bytes and duplicate-request rate, then compare those savings against storage, nginx, monitoring, and ingress complexity.

nimmo closed this issue

2026-08-09 23:03:54 +01:00