cryodev/docs/getting-started/first-install.md

Initial Installation (x86_64 Server)

This guide describes the initial installation of a new x86_64 server (e.g. cryodev-main).

For Raspberry Pi: See Creating an SD Image.

Overview

During initial installation there is a chicken-and-egg problem:

• SOPS secrets are encrypted with the SSH host key
• The SSH host key is only generated during installation
• Therefore: Install without secrets first, then configure secrets

Process

1. Disable services (that require secrets)
2. Install NixOS
3. Extract SSH host key, configure SOPS, create immediately available secrets
4. Enable stage-1 services and deploy (Headscale, Forgejo, Mail, Nginx)
5. Generate remaining secrets (Tailscale, Headplane, Forgejo Runner)
6. Enable stage-2 services and perform final deployment

Step 1: Prepare Host Configuration

If the host already exists in hosts/ and flake.nix, skip 1.1-1.2.

1.1 Create Host from Template

nix run .#create -- -t generic-server -n <hostname>

The script:

• Copies the template to hosts/<hostname>/
• Sets the hostname in networking.nix
• Creates an empty secrets.yaml
• Adds the files to Git

1.2 Register in flake.nix

nixosConfigurations = {
  <hostname> = mkNixosConfiguration "x86_64-linux" [ ./hosts/<hostname> ];
};

Also adjust hardware.nix and disks.sh for the target hardware.

1.4 Temporarily Disable Services

All services that reference SOPS secrets must be disabled for the initial installation. Otherwise the installation will fail because the secrets cannot yet be decrypted.

In hosts/<hostname>/services/default.nix, comment out the corresponding imports:

{
  imports = [
    # Disabled until SOPS secrets are configured:
    # ./forgejo.nix     # requires: forgejo-runner/token, forgejo/mail-pw
    # ./headplane.nix   # requires: headplane/cookie_secret, headplane/agent_pre_authkey
    # ./mailserver.nix  # requires: mailserver/accounts/*
    # ./tailscale.nix   # requires: tailscale/auth-key

    # These services do not require secrets:
    ./headscale.nix
    ./netdata.nix
    ./nginx.nix
    ./openssh.nix
    ./sops.nix
  ];
}

Additionally, in hosts/<hostname>/services/sops.nix, comment out the secret definitions:

sops = {
  defaultSopsFile = ../secrets.yaml;
  # secrets = {
  #   "forgejo-runner/token" = { };
  #   "tailscale/auth-key" = { };
  # };
};

1.5 Test the Configuration

nix eval .#nixosConfigurations.<hostname>.config.system.build.toplevel.name

Step 2: Perform Installation

2.1 Boot NixOS ISO

Boot from the NixOS Minimal ISO (USB/CD).

2.2 Set Up Network and SSH

passwd                  # Set root password for SSH access
ip a                    # Determine IP address

Optionally connect via SSH (more convenient):

ssh -o StrictHostKeyChecking=no root@<IP>

2.3 Install

nix --experimental-features "nix-command flakes" run \
  git+<REPO_URL>#apps.x86_64-linux.install -- \
  -n <hostname> \
  -r <REPO_URL>

Alternatively, if the repository has already been cloned to /tmp/nixos:

nix --experimental-features "nix-command flakes" run /tmp/nixos#install -- -n <hostname>

Note: The disk ID in hosts/<hostname>/disks.sh must match the hardware. Verify with ls -la /dev/disk/by-id/.

The script:

1. Clones the repository (when using -r)
2. Partitions the disk (via disks.nix or disks.sh)
3. Generates hardware.nix (if not present)
4. Installs NixOS

2.4 Reboot

reboot

Step 3: Configure SOPS Secrets

After the first boot, log in (password: changeme, change immediately with passwd).

3.1 Convert SSH Host Key to Age Key

On the new server:

nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'

Note the output (e.g. age1abc123...).

Alternatively, remotely:

nix-shell -p ssh-to-age --run 'ssh-keyscan -p 2299 -t ed25519 <IP> | ssh-to-age'

3.2 Update .sops.yaml

On the development machine, add the new host key to .sops.yaml:

keys:
  - &steffen_key age1e8p...          # steffen (local)
  - &hostname_key age1abc...         # Key from step 3.1

creation_rules:
  - path_regex: hosts/<hostname>/secrets.yaml$
    key_groups:
      - age:
          - *steffen_key
          - *hostname_key

3.3 Create Secrets

Open the secrets file:

sops hosts/<hostname>/secrets.yaml

The following table shows all secrets for cryodev-main and how they are generated:

Immediately Available Secrets

These secrets have no dependencies and can be generated directly:

Secret	Command
headplane/cookie_secret	openssl rand -hex 16
mailserver/accounts/admin	mkpasswd -sm bcrypt (remember the password!)
mailserver/accounts/forgejo	mkpasswd -sm bcrypt (remember the password!)
forgejo/mail-pw	Plaintext password matching the bcrypt hash of mailserver/accounts/forgejo

Secrets That Require Running Services

These secrets can only be created after step 4. Do not add them yet -- they will be added later.

Secret	Command	Prerequisite
tailscale/auth-key	See steps 4.1-4.2	Headscale is running
headplane/agent_pre_authkey	See steps 4.1-4.2	Headscale is running
forgejo-runner/token	Forgejo Admin Panel > Actions > Runners > Create Runner	Forgejo is running

Example secrets.yaml (Plaintext Before Encryption)

headplane:
    cookie_secret: "a1b2c3d4e5f6..."
mailserver:
    accounts:
        admin: "$2b$05$..."
        forgejo: "$2b$05$..."
forgejo:
    mail-pw: "the-plaintext-password"

3.4 Gradually Re-enable Services -- Stage 1

Important: Services that require Headscale or Forgejo secrets (Tailscale, Headplane, Forgejo Runner) must not be enabled yet, as these secrets can only be generated once those services are running.

On the development machine, in hosts/<hostname>/services/default.nix, enable the services without external dependencies:

{
  imports = [
    # Stage 1: Services without external dependencies
    ./forgejo.nix
    ./headscale.nix
    ./mailserver.nix
    ./netdata.nix
    ./nginx.nix
    ./openssh.nix
    ./sops.nix

    # Stage 2: Enable only after step 4
    # ./forgejo-runner.nix  # requires: forgejo-runner/token (Forgejo)
    # ./headplane.nix       # requires: headplane/agent_pre_authkey (Headscale)
    # ./tailscale.nix       # requires: tailscale/auth-key (Headscale)
  ];
}

3.5 Deploy (Stage 1)

nix run .#deploy -- -n <hostname>

This uses the configuration from deploy.json. Alternatively, deploy manually:

NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \
  --target-host <user>@<IP> --sudo --ask-sudo-password

After this deployment, Headscale, Forgejo, Mailserver, and Nginx are running.

3.6 Create Forgejo Admin Account

On first start, Forgejo has no users. Create an admin account via CLI (on the server):

forgejo admin user create \
  --username <username> \
  --email <email>@<domain> \
  --password <password> \
  --admin

Note: The forgejo shell alias is provided by the module and automatically runs the command as the forgejo user with the correct config. If the alias is not available, start a new shell (bash or zsh).

Since DISABLE_REGISTRATION = true is set, new accounts can only be created via CLI.

Step 4: Generate Remaining Secrets and Enable All Services

After the server is running with Headscale and Forgejo:

1. Create Headscale users (on the server):

   sudo headscale users create default
   sudo headscale users create headplane-agent
   

2. Determine user IDs (needed for the preauth keys):

   sudo headscale users list
   

   The output shows the numeric IDs (e.g. 1 for default, 2 for headplane-agent).

3. Generate preauth keys (using the IDs from step 2):

   # For Tailscale (use the user ID of "default")
   sudo headscale preauthkeys create --expiration 99y --reusable --user <ID>
   
   # For Headplane Agent (use the user ID of "headplane-agent")
   sudo headscale preauthkeys create --expiration 99y --user <ID>
   

4. Create the Forgejo Runner token via the Forgejo Admin Panel: Administration > Actions > Runners > Create new Runner

5. Add the remaining secrets:

   sops hosts/<hostname>/secrets.yaml
   

   Add the missing secrets:

   tailscale:
       auth-key: "tskey-..."
   forgejo-runner:
       token: "..."
   headplane:
       agent_pre_authkey: "..."
   

6. Enable stage-2 services in hosts/<hostname>/services/default.nix:

   {
     imports = [
       ./forgejo.nix
       ./forgejo-runner.nix
       ./headplane.nix
       ./headscale.nix
       ./mailserver.nix
       ./netdata.nix
       ./nginx.nix
       ./openssh.nix
       ./sops.nix
       ./tailscale.nix
     ];
   }
   

7. Deploy again:

   nix run .#deploy -- -n <hostname>
   

Next Steps

• SOPS Reference -- Detailed documentation on secret management
• Creating an SD Image -- Install Raspberry Pi
• Set Up CD -- Automatic deployment