Chapter 6: Secrets Management

This chapter explains how secrets (passwords, API keys, repository URLs) are handled in a NixOS configuration where the entire system definition is stored in a public git repository.

The Problem: The Nix Store Is World-Readable

Everything in /nix/store is readable by all users on the system. If you put a password directly in a .nix file:

# DON'T do this
environment.etc."restic-password".text = "hunter2";

That string ends up in /nix/store/abc123-etc-restic-password where any user (or process) can read it. Even if the file is not committed to git, the Nix store is not a safe place for secrets.

NixOS needs a mechanism to deliver secrets to services outside the Nix store, at activation time, into a location that only the intended service can read.

The Solution: sops-nix

sops-nix bridges the gap between encrypted-at-rest secrets and runtime access. It works in three stages:

1. At rest: Secrets live in scoped encrypted YAML files under secrets/, encrypted with age. These files are safe to commit to git.

2. At activation: When the system activates (during nixos-rebuild switch), sops-nix decrypts only the configured scoped files using a key available on the host and places each secret as a file under /run/secrets/.

3. At runtime: Services read their secrets from /run/secrets/ (a tmpfs -- never written to disk).

Why host SSH keys?

sops-nix needs a private key to decrypt secrets. The obvious choice is a user SSH key (~/.ssh/id_ed25519), but this creates a problem: /home may not be mounted during early system activation. If secrets are needed before /home is available, decryption fails.

This configuration uses the host SSH key (/etc/ssh/ssh_host_ed25519_key) instead. This key:

• Lives on the root filesystem, available from the earliest stages of activation
• Is generated automatically by sshd on first boot
• Is unique per machine, so each host can only decrypt secrets intended for it
• Is root-readable only (mode 0600), protecting it from unprivileged users

How It Is Configured

The sops-nix module (modules/common/sops-host.nix)

This module is imported by modules/common/base.nix, so it applies to every host:

{ ... }:

{
  sops.age.sshKeyPaths = [ "/etc/ssh/ssh_host_ed25519_key" ];

  systemd.tmpfiles.rules = [
    "z /etc/ssh/ssh_host_ed25519_key 0600 root root - -"
  ];
}

sops.age.sshKeyPaths tells sops-nix which private key to use for decryption. The tmpfiles rule enforces 0600 root root permissions on the host private key, which sshd requires.

The host SSH key

The host SSH key (/etc/ssh/ssh_host_ed25519_key) is generated automatically by NixOS on first boot. It persists across rebuilds and is available from the earliest stages of system activation, which is why it is used for sops-nix decryption rather than a user SSH key.

The .sops.yaml file

This file in the repository root tells sops which keys can decrypt which files:

keys:
  # electra: host SSH key
  - &electra age1qkndq2jf4ezw3gp2mrq76xhh8u5aczkrux23wxgw35y9pmejj99s90s46l
  # lena: host SSH key
  - &lena age1een3f9hpldw6dcfylznhu6zy0mcqtnzmx0hn7wscxmtd9wv6nv0s4d0sfs
  # vega: host SSH key
  - &vega age1206h9mlmk7anqs7vqpqdzs7xrln20c9wgj95wdw4nuqw2uprt3ds2dnjel
  # YubiKey Nano 5C (backup)
  - &yubikey-nano5c age1yubikey1qf8kmphcjvftsvmp8nrp7xkywhu5qa26f2c6s2ta6t2c85uen2fc2v72slp
  # YubiKey 5 NFC (backup)
  - &yubikey-5-nfc age1yubikey1q0srxtgqt6pryp0gra8a6vq8uyksqh5d60gye8hlqk498gpwqdj8yj67jwj
  # Provisioning age key (private key in password manager)
  - &provision-key age1vlc2eans4eykdev7zrmhuph7w8ytfn50z9uc8gm2jxaccu5dluvqufzxzt

creation_rules:
  - path_regex: secrets/.*\.yaml$
    key_groups:
      - age:
          - *electra
          - *lena
          - *vega
          - *yubikey-nano5c
          - *yubikey-5-nfc
          - *provision-key

Each host's age public key is derived from its SSH host public key using ssh-to-age. The specific creation_rules entries scope each service or host file to its consumers plus the YubiKeys and provisioning key. Nebula files have an admin-only catch-all for newly created external nodes; host-specific Nebula rules appear before it and take precedence. The broad secrets/.*\.yaml$ rule remains only as a fallback for files not yet given a specific rule. Electra is included only where it is an actual consumer or deliberate administrative operator; YubiKeys provide recovery access for the remaining scoped files.

Scoped secret files

There is no monolithic secrets/secrets.yaml file. Each service or host has its own encrypted file, for example:

backrest:
    rest_password: ENC[AES256_GCM,data:...,type:str]
    repo_password: ENC[AES256_GCM,data:...,type:str]
sops:
    age:
        - recipient: age1qkndq2jf4...
          enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            ...

The current scoped files and their consumers include:

Secret key	Module	Purpose
backrest/rest_password, backrest/repo_password in secrets/backrest.yaml	modules/services/backrest.nix	Backrest REST and repository credentials
beszel/<hostname>/agent_env in secrets/beszel/<hostname>.yaml	modules/services/beszel-agent.nix	Beszel agent authentication token (per-host)
beszel/mcp/{email,password} in secrets/ai-agents.yaml	modules/profiles/ai-agents.nix	Shared Beszel MCP credentials
forgejo/mcp/token in secrets/git.yaml	modules/services/forgejo-mcp.nix	Repository-scoped Forgejo MCP bot token
opencode-server-password, opencode-openrouter-api-key in secrets/ai-runtime.yaml	modules/profiles/ai-runtime.nix, modules/services/hermes.nix	OpenCode and shared OpenRouter credentials
nebula/<hostname>-key in secrets/nebula/<hostname>.yaml	modules/networking/nebula.nix or just nebula-bundle	Per-node Nebula private keys
wireguard/<hostname>-key in secrets/wireguard/<hostname>.yaml	modules/networking/wireguard-vpn.nix	Per-host WireGuard private keys

When opened with sops secrets/<scope>.yaml or just edit-secrets <scope>, sops decrypts that one file in your editor. You see the plaintext YAML, make changes, and when you save, sops re-encrypts automatically.

How Modules Consume Secrets

A module that needs a secret declares it in sops.secrets and then references the decrypted path. Here is how modules/services/backrest.nix does it:

Step 1: Declare the secrets

sops.secrets = {
  "backrest_rest_password" = {
    sopsFile = ../../secrets/backrest.yaml;
    key = "backrest/rest_password";   # maps to YAML path: backrest.rest_password
  };
  "backrest_repo_password" = {
    sopsFile = ../../secrets/backrest.yaml;
    key = "backrest/repo_password";   # maps to YAML path: backrest.repo_password
  };
};

The key parameter maps to a path in the YAML structure. backrest/rest_password means the rest_password key under the backrest key. The secret name ("backrest_rest_password") determines the filename under /run/secrets/.

Step 2: Reference the decrypted paths

serviceConfig = {
  LoadCredential = [
    "rest_password:${config.sops.secrets."backrest_rest_password".path}"
    "repo_password:${config.sops.secrets."backrest_repo_password".path}"
  ];
};

config.sops.secrets."backrest_rest_password".path evaluates to /run/secrets/backrest_rest_password at build time. LoadCredential is a systemd mechanism that copies the secret into a private $CREDENTIALS_DIRECTORY for the service, so the secret is never exposed in the process environment or /proc.

Step 3: Read the credentials at runtime

export BACKREST_REST_PASSWORD=$(cat "$CREDENTIALS_DIRECTORY/rest_password")
export BACKREST_REPO_PASSWORD=$(cat "$CREDENTIALS_DIRECTORY/repo_password")

The service script reads from $CREDENTIALS_DIRECTORY (set by systemd) rather than from /run/secrets/ directly. This is defence in depth: even if another process could read /run/secrets/, the credentials directory is private to the service.

The Decryption Flow

Here is the complete path from encrypted YAML to a running service:

1. secrets/<scope>.yaml               Encrypted at rest in git (age + sops)
         │
2. nixos-rebuild switch              Triggers system activation
         │
3. sops-nix activation script        Reads /etc/ssh/ssh_host_ed25519_key
         │                           Converts it to age format internally
         │                           Decrypts configured scoped files
         │
4. /run/secrets/backrest_rest_password  Plaintext on tmpfs (lost on reboot)
   /run/secrets/backrest_repo_password
         │
5. systemd LoadCredential            Copies into private $CREDENTIALS_DIRECTORY
         │
6. backrest service                  Reads credentials, runs backup

Every step after step 1 happens automatically during nixos-rebuild switch. No manual decryption is needed.

Managing Secrets

Editing existing secrets

just edit-secrets backrest

This opens secrets/backrest.yaml in your editor, decrypted. Edit the values, save, and sops re-encrypts on exit. Commit the result:

git add secrets/backrest.yaml
git commit -m "chore: Update restic credentials"

Adding a new secret

1. Choose or create the appropriately scoped file, for example secrets/my-service.yaml:

just edit-secrets my-service
# Add your new key/value, save and exit

2. Declare it in a module:

sops.secrets."my_new_secret" = {
  sopsFile = ../../secrets/my-service.yaml;
  key = "path/to/key";   # maps to YAML path: path.to.key
};

3. Reference the path in your module:

# In a service module:
serviceConfig.LoadCredential = [
  "my_new_secret:${config.sops.secrets."my_new_secret".path}"
];

# In a home-manager activation (for user-facing credentials):
home.activation.myConfig = hmLib.hm.dag.entryAfter [ "writeBoundary" ] ''
  MY_KEY=$(cat "${config.sops.secrets."my_new_secret".path}")
  # ... use the key
'';

Registering a new host

When you add a new machine to this configuration, it needs its own age key so it can decrypt secrets. After the first deployment (which generates the host SSH key):

just add-secret HOSTNAME

This derives the age public key from the host's SSH key and adds it to .sops.yaml. Update only the scoped creation rules and files that the new host should consume; do not grant a new host access to every secret by default.

Rotating recipients and retiring hosts

To rotate a recipient, update the relevant creation_rules entry in .sops.yaml, then run sops updatekeys on each affected file from a machine that can already decrypt it:

sops updatekeys secrets/backrest.yaml
sops updatekeys secrets/nebula/vega.yaml

For host retirement, remove the host from the applicable creation rules and update every affected file. For a Nebula node, revoke or blocklist its certificate before deleting its scoped private-key file. Keep YubiKey recipients on service files so recovery and editing remain possible without the retired host.

Security Properties

Property	How it is achieved
Secrets never in Nix store	sops-nix decrypts to /run/secrets/ (tmpfs), bypassing the store entirely
Secrets never on disk	/run/secrets/ is tmpfs -- contents exist only in memory, lost on reboot
Per-host isolation	Each host has its own age key; .sops.yaml controls which hosts can decrypt which files
Service isolation	systemd LoadCredential gives each service a private copy; secrets do not appear in process environment
Safe to commit	Files under secrets/ are age-encrypted; plaintext never enters git
Early availability	Host SSH keys live on the root filesystem, available before /home is mounted
No manual steps	Decryption is automatic during nixos-rebuild switch