nimmo/nixos-config
nimmo/nixos-config
Watch
1
0
Fork
You've already forked nixos-config
 0
Code Issues 15 Pull requests Wiki Activity Actions
Page: 37 Electra Boot Trust Recovery
Pages
00 Documentation Map and Scope 00 Documentation Standards 00 Start Here and Governance 10 System Atlas 11 Fleet Overview 12 Host Inventory 13 Service Catalogue 14 Network and Access Topology 15 Storage and Data Map 16 Automation and Control Plane 20 Architecture Decisions 21 ADR 001 Electra Hardware Specialisations 22 ADR 002 Workload Placement 23 ADR 003 Network Exposure 23 Project Tracking 24 ADR 004 Engram Durable Agent Memory 24 Security Baseline 25 ADR 005 Auto Update Control Plane 26 ADR 006 Scoped Secret Files 30 Operations and Runbooks 31 Routine Fleet Operations 32 Safe Testing and Deployment 33 Host Provisioning 34 Monitoring and Service Investigation 35 Backup Verification and Partial Restore 36 Full Host Recovery 37 Electra Boot Trust Recovery 38 Secret Editing and Recipient Rotation 39 Auto Update Incident Response 39a Service Migration and Cutover 39b Wrapper Repository Packaging 40 Engineering Handbook 41 Repository Structure and Import Chain 42 Configuration Ownership Boundaries 43 Adding Packages and Flake Inputs 44 Adding a Module or Profile 45 Adding a Host 46 Adding or Migrating a Service 47 Testing and CI Contracts 48 Secrets Implementation Conventions 49 Documentation Maintenance 50 NixOS Learning Path 51 Guided Repository Tour 52 Nix Language Essentials 53 Declarative Configuration Store and Generations 54 Flakes and Inputs 55 NixOS Modules and Option Merging 56 SpecialArgs and Home Manager Argument Flow 57 Electra Specialisations Worked Example 58 Nix Troubleshooting Primer 90 Historical Material 91 Electra Secure Boot Hardening Record Home
2 37 Electra Boot Trust Recovery
Nimmo edited this page 2026-09-13 13:14:49 +01:00
Table of contents

Electra Boot-Trust and TPM Recovery

  • Type: Runbook
  • Status: Current
  • Scope: Electra Secure Boot signing, Lanzaboote, managed PCR policy, LUKS TPM tokens, and independent recovery access
  • Canonical sources: hosts/electra/default.nix; hosts/electra/disko.nix; justfiles/maintenance.just; Electra hardening record; wiki Git history of retired disaster-recovery chapter
  • Last verified: Source commit 556df88494686003b1c4f20c8e0b99b5afc16a6e (2026-09-12); source and historical recovery evidence reviewed 2026-09-12. No new live firmware, TPM, or archive test was performed.
  • Review triggers: Firmware, Secure Boot keys, Lanzaboote, PCR policy, specialisation, LUKS keyslot/token, Disko, recovery archive, or hardware replacement changes

Purpose

Recover Electra's boot trust without sacrificing its independent LUKS passphrase route. Treat these as distinct failures: a broken signed boot artefact, lost firmware trust, a changed TPM/PCR measurement, a damaged LUKS header, and a failed physical disk require different actions. For full-host installation or data restore, use Full-Host Recovery.

The original hardening record retains the 2026-06-29 deployment history and validation evidence. This page owns current recovery decisions. The old disaster-recovery chapter, now in wiki Git history, contained an archive example; it is not a verified current backup. The post-change archive remains tracked by issue #42.

Current trust model

  • Root, data, and swap are separate LUKS volumes identified by electra-cryptroot, electra-cryptdata, and electra-cryptswap partlabels. Disko creates encrypted volumes with passphrases; it does not create TPM tokens or restore Secure Boot signing keys.
  • Lanzaboote signs boot artefacts with the PKI bundle in /var/lib/sbctl. The configured boot menu is not editable, and only two generations are retained because each has battery, IGPU, and DGPU variants beneath the pcrlock policy-alternative limit.
  • In battery/IGPU mode, the managed pcrlock policy covers PCR 4+7 and can release each LUKS volume's managed TPM token without a PIN. The physical NVIDIA expansion bay must be absent for IGPU mode.
  • With the expansion bay installed, DGPU mode deliberately does not regenerate that policy: its firmware adds an incompatible PCR 4 event. A separate PCR 7 token with TPM PIN is the intended DGPU unlock path.
  • The LUKS passphrase remains the independent recovery route for every volume. Neither a TPM token nor a Secure Boot key archive replaces it.

Prerequisites and safety boundary

Have console access, recovery media whose boot path is known to work, independently stored passphrases/PIN, firmware administrator access, and the exact host/disk identity before altering trust. Check the latest signed working generation and the current physical expansion-bay state. When the machine still boots, preserve an available recovery generation and take a fresh verified backup before changing keys or tokens.

Stop if any volume cannot be unlocked with its independent passphrase, or if the only copy of Secure Boot keys or LUKS headers is on the failing disk. Do not remove a passphrase slot to make TPM recovery appear successful. Never paste keys, headers, PINs, token secrets, decrypted archives, or raw recovery logs into Forgejo or Engram.

The old archive procedure records the intended inventory: encrypted /var/lib/sbctl, separate root/data/swap LUKS headers, pcrlock state, signed ESP artefacts, and status evidence. Its archive must be encrypted before leaving Electra, stored off-host, and test-decrypted/listed. The post-change archive is still tracked as incomplete by issue #42; confirm the actual available files and date before relying on them. Refresh a verified archive after a Secure Boot key, LUKS header/keyslot, or TPM-token change. Keep the archive password separately. LUKS-header copies are exceptionally sensitive and a header restore can replace current keyslots and tokens.

Triage without changing trust

From a bootable Electra, collect bounded local evidence:

cat /etc/nixos-specialisation
bootctl status
sudo sbctl status
sudo sbctl verify
systemctl status systemd-pcrlock-make-policy.service --no-pager

On each confirmed Electra partition, sudo systemd-cryptenroll /dev/disk/by-partlabel/electra-cryptroot (and likewise cryptdata and cryptswap) lists token enrolment. This is inspection only; never pass wipe or enrol options until the recovery case and passphrase route are clear. Inspect the managed policy locally at /var/lib/systemd/pcrlock.json; the expected protection mask is PCRs 4 and 7, not PCR 7 alone. Treat all outputs as potentially sensitive operational evidence.

If Electra cannot boot, first determine whether firmware rejects the signed artefact, LUKS rejects TPM auto-unlock but accepts its passphrase, or storage is unreadable. A working passphrase plus a signed previous generation usually permits a non-destructive repair. A failed TPM unlock by itself is not a reason to format, clear the TPM, or replace a LUKS header.

Recovery procedures

Signed generation or policy update failed, disks still unlock

Boot the last known-good signed generation with the correct hardware tier. Use each volume's LUKS passphrase if TPM unlock fails. Preserve the failed generation's logs, confirm disk and mount health, and compare the deployed revision with the intended configuration. Correct the configuration and rebuild through Safe Testing and Deployment, which preserves the current specialisation. Check sbctl verify before rebooting, then validate the real boot path and all three volumes. Do not change TPM tokens merely because one signed generation is bad.

TPM was cleared, replaced, or lost its managed policy

Boot with passphrases, then use battery/IGPU mode with the NVIDIA expansion bay absent to regenerate the managed PCR 4+7 policy. Do not generate it from DGPU: that mode intentionally disables automatic policy generation. Verify systemd-pcrlock-make-policy.service succeeded and that the policy file exists with the expected PCR mask before reenrolling anything.

After confirming the passphrase works for each volume and the matching partlabel is a block device, the repository helper can replace the managed TPM token one volume at a time:

just tpm2-enrol root
just tpm2-enrol data
just tpm2-enrol swap

Each helper prompts, then wipes existing TPM2 enrolment for that volume before adding a token bound to /var/lib/systemd/pcrlock.json. That interval is safe only if the passphrase slot is tested and retained. Inspect the token list after every volume; stop on the first failure rather than sweeping all three. If the PCR 7 + PIN fallback was also lost, add it only after the managed path works:

just tpm2-enrol-pin root
just tpm2-enrol-pin data
just tpm2-enrol-pin swap

These commands add a second token; they do not remove the managed token. Test IGPU unattended unlock, then test DGPU PIN unlock only with the NVIDIA bay installed. A PIN fallback failure must not be “fixed” by weakening the managed policy to PCR 7 alone.

Firmware Secure Boot trust or signing keys were lost

Use the independent passphrase and recovery media to gain access. First verify that the encrypted off-host /var/lib/sbctl archive matches this machine and is readable. If that material or firmware access is unavailable, stop and prepare a separate key-rotation/recovery plan; do not generate replacement keys in place while expecting old signed generations to boot.

For a verified matching archive, restore the PKI bundle with its expected ownership/permissions while firmware trust is disabled or in Setup Mode. Rebuild signed boot artefacts from the reviewed configuration, verify signatures with sbctl verify, and only then enrol the matching keys in firmware. The historical command sequence and Microsoft-key compatibility choice are in the hardening record, but archive contents and current firmware state must be checked before executing it. Confirm Secure Boot user mode and a signed working boot before repairing TPM policy or tokens; changing firmware trust can change PCR 7.

LUKS header is damaged

This is not a TPM-enrolment problem. Boot recovery media, identify the exact disk by model, serial, and partlabel, and compare the matching encrypted header backup against the intended volume. Header restoration overwrites current keyslots and tokens and can make newer credentials unusable. Prepare an incident-specific offline restore plan and preserve the current damaged header/image before using cryptsetup luksHeaderRestore. Do not apply a root header to data or swap, or an old backup without accepting its keyslot loss.

Disk or whole host is lost

Follow Full-Host Recovery and Host Provisioning. Electra's Disko create/format path is still untested from scratch; validate it on a scratch disk/VM before a real reprovision. Re-establish boot trust only after the base host and independent passphrase access work. Then restore data separately from verified backups.

Success checks

  • The correct physical hardware tier boots a signed current generation and a retained signed rollback generation; the boot menu is not editable.
  • sbctl verify and firmware Secure Boot status agree on trusted artefacts.
  • Root, data, and swap each retain a working independent passphrase.
  • IGPU with the NVIDIA bay absent uses managed PCR 4+7 auto-unlock; DGPU with the bay installed uses PCR 7 + PIN without policy downgrade.
  • The recovery archive actually available off-host has been decrypted and inspected, or its absence is recorded as an unresolved recovery risk.
  • After a real repair, application/data checks and a fresh backup pass.

Rollback and escalation

If a new generation fails, boot the retained signed generation and unlock with passphrases. If managed token enrolment fails, stop at the affected volume; the passphrase is the recovery path while the policy is investigated. If firmware trust or a LUKS header was changed, a Nix generation rollback alone is insufficient: restore only the verified matching key/header material under the incident-specific plan. Preserve recovery media and the independent passphrase throughout.

The final post-change encrypted archive and its restore test remain open in issue #42. This page does not assert that an absent archive exists or that a whole-host boot-trust recovery has been rehearsed.