cryodev/INSTRUCTIONS.md

Server Setup Instructions / Server-Einrichtungsanleitung

---

🇬🇧 English Instructions

1. Prerequisites

Ensure you have the following tools installed on your local machine:

• nix (with flakes enabled)
• sops
• age
• ssh

2. DNS Configuration

Configure the following DNS records for your domain cryodev.xyz:

Hostname	Type	Value	Purpose
@	A	<SERVER_IP>	Main entry point
@	AAAA	<SERVER_IPV6>	Main entry point (IPv6)
git	CNAME	@	Forgejo
headscale	CNAME	@	Headscale
headplane	CNAME	@	Headplane
netdata	CNAME	@	Netdata Monitoring
mail	A	<SERVER_IP>	Mailserver
mail	AAAA	<SERVER_IPV6>	Mailserver (IPv6)
@	MX	10 mail.cryodev.xyz.	Mail delivery
@	TXT	"v=spf1 mx ~all"	SPF Record
_dmarc	TXT	"v=DMARC1; p=none"	DMARC Record

3. Secret Management (SOPS)

This repository uses sops-nix to manage secrets encrypted with age, utilizing the SSH host keys of the servers.

3.1 Get Server Public Keys

You need to convert the servers' SSH host public keys to age public keys.

For cryodev-main:

nix-shell -p ssh-to-age --run 'ssh-keyscan -t ed25519 <MAIN_IP> | ssh-to-age'

For cryodev-pi:

nix-shell -p ssh-to-age --run 'ssh-keyscan -t ed25519 <PI_IP> | ssh-to-age'

3.2 Configure .sops.yaml

Edit the .sops.yaml file in the root of this repository. Add the age public keys to the keys section and ensure creation rules exist for both hosts.

keys:
  - &admin_key age1e8p35795htf7twrejyugpzw0qja2v33awcw76y4gp6acnxnkzq0s935t4t # Admin Key (Steffen)
  - &main_key age1... # cryodev-main Key
  - &pi_key age1...   # cryodev-pi Key
creation_rules:
  - path_regex: hosts/cryodev-main/secrets.yaml$
    key_groups:
      - age:
        - *admin_key
        - *main_key
  - path_regex: hosts/cryodev-pi/secrets.yaml$
    key_groups:
      - age:
        - *admin_key
        - *pi_key

3.3 Generating Secret Values

Mailserver Passwords (for cryodev-main):

nix-shell -p mkpasswd --run 'mkpasswd -sm bcrypt'

Headplane Secrets (for cryodev-main):

nix-shell -p openssl --run "openssl rand -hex 16"
# Agent Pre-Authkey requires Headscale running:
sudo headscale users create headplane-agent
sudo headscale preauthkeys create --expiration 99y --reusable --user headplane-agent

Tailscale Auth Keys (for both hosts): Requires Headscale running on cryodev-main.

# For cryodev-main:
sudo headscale preauthkeys create --expiration 99y --reusable --user default
# For cryodev-pi:
sudo headscale preauthkeys create --expiration 99y --reusable --user default

Netdata Child UUID (for cryodev-pi):

uuidgen

Forgejo Runner Token: Get from Forgejo Admin Panel.

3.4 Creating Secrets Files

hosts/cryodev-main/secrets.yaml:

sops hosts/cryodev-main/secrets.yaml

mailserver:
    accounts:
        forgejo: "$2y$05$..."
        admin: "$2y$05$..."

forgejo-runner:
    token: "..."

headplane:
    cookie_secret: "..."
    agent_pre_authkey: "..."

tailscale:
    auth-key: "..."

hosts/cryodev-pi/secrets.yaml:

sops hosts/cryodev-pi/secrets.yaml

tailscale:
    auth-key: "..."

netdata:
    stream:
        child-uuid: "..." # Output from uuidgen

4. Initial Deployment (Bootstrap)

Before the continuous deployment can take over, you must perform an initial deployment manually using the provided install script.

4.1 Prepare Target Machine

1. Boot into the NixOS Installation ISO.
2. Set a root password (for SSH): passwd.
3. Ensure internet connectivity.

4.2 Run Install Script

From your local machine (where this repo is), copy the script to the target or run it directly if you can fetch it.

Method A: Copy Script via SSH

scp scripts/install.sh nixos@<TARGET_IP>:install.sh
ssh nixos@<TARGET_IP>
sudo -i
chmod +x /home/nixos/install.sh
./home/nixos/install.sh -r <GIT_REPO_URL> -n <HOSTNAME>

Method B: Run on Target (if repo is public or reachable)

# On the target machine (as root)
nix-shell -p git
git clone <GIT_REPO_URL> /tmp/nixos
cd /tmp/nixos
bash scripts/install.sh -n <HOSTNAME>

Note: The script handles disk partitioning (via disko/script), hardware config generation, and installation.

5. Continuous Deployment (CD)

5.1 cryodev-pi (Pull-based via Comin)

The cryodev-pi host is configured to pull updates automatically via comin.

1. Create Repository: Create a new repository named cryodev-server on your Forgejo instance (https://git.cryodev.xyz).
2. Push Configuration: Push this entire NixOS configuration to the main branch of that repository.
3. Comin URL: The configuration expects the repository at: https://git.cryodev.xyz/steffen/cryodev-server.git.

5.2 cryodev-main (Push-based via Forgejo Actions)

The main server is deployed via a Forgejo Action.

1. Generate SSH Key:

   ssh-keygen -t ed25519 -f deploy_key -C "forgejo-actions"
   

2. Add Public Key: Add the content of deploy_key.pub to /root/.ssh/authorized_keys on cryodev-main.
3. Add Secret: Add the content of deploy_key (private key) as a secret named DEPLOY_SSH_KEY in your Forgejo repository settings.

6. Creating New Hosts (Templates)

To quickly bootstrap a new host configuration, you can use the provided templates.

1. Copy Template:

   # For a Raspberry Pi:
   cp -r templates/raspberry-pi hosts/new-pi-name
   
   # For a generic x86 server:
   cp -r templates/generic-server hosts/new-server-name
   

2. Adjust Configuration:
   • Hostname: Edit hosts/new-name/networking.nix.
   • Flake: Register the new host in flake.nix under nixosConfigurations.
   • Constants: Add IP and ports to constants.nix.
   • Secrets: Add keys to .sops.yaml and create hosts/new-name/secrets.yaml.

---

🇩🇪 Deutsche Anleitung

1. Voraussetzungen

Stellen Sie sicher, dass folgende Tools lokal installiert sind:

• nix (mit Flakes)
• sops
• age
• ssh
• ssh-to-age
• uuidgen

2. DNS-Konfiguration

Richten Sie folgende DNS-Einträge für cryodev.xyz ein:

Hostname	Typ	Wert	Zweck
@	A	<SERVER_IP>	Hauptserver
@	AAAA	<SERVER_IPV6>	Hauptserver (IPv6)
git	CNAME	@	Forgejo
headscale	CNAME	@	Headscale
headplane	CNAME	@	Headplane
netdata	CNAME	@	Netdata Monitoring
mail	A	<SERVER_IP>	Mailserver
mail	AAAA	<SERVER_IPV6>	Mailserver (IPv6)
@	MX	10 mail.cryodev.xyz.	Mail-Empfang
@	TXT	"v=spf1 mx ~all"	SPF-Record
_dmarc	TXT	"v=DMARC1; p=none"	DMARC-Record

3. Verwaltung von Geheimnissen (SOPS)

Dieses Repository nutzt sops-nix mit den SSH-Host-Keys der Server.

3.1 Public Keys abrufen

Für cryodev-main:

nix-shell -p ssh-to-age --run 'ssh-keyscan -t ed25519 <MAIN_IP> | ssh-to-age'

Für cryodev-pi:

nix-shell -p ssh-to-age --run 'ssh-keyscan -t ed25519 <PI_IP> | ssh-to-age'

3.2 .sops.yaml konfigurieren

Bearbeiten Sie .sops.yaml und fügen Sie die Keys sowie Regeln für beide Hosts hinzu:

keys:
  - &admin_key age1e8p35795htf7twrejyugpzw0qja2v33awcw76y4gp6acnxnkzq0s935t4t # Admin Key (Steffen)
  - &main_key age1... # cryodev-main Key
  - &pi_key age1...   # cryodev-pi Key
creation_rules:
  - path_regex: hosts/cryodev-main/secrets.yaml$
    key_groups:
      - age:
        - *admin_key
        - *main_key
  - path_regex: hosts/cryodev-pi/secrets.yaml$
    key_groups:
      - age:
        - *admin_key
        - *pi_key

3.3 Werte generieren

Mailserver: mkpasswd -sm bcrypt Headplane: openssl rand -hex 16 Netdata UUID: uuidgen

Tailscale Auth Keys (auf cryodev-main):

sudo headscale preauthkeys create --expiration 99y --reusable --user default

3.4 Secrets-Dateien erstellen

hosts/cryodev-main/secrets.yaml:

sops hosts/cryodev-main/secrets.yaml

mailserver:
    accounts:
        forgejo: "$2y$05$..."
        admin: "$2y$05$..."

forgejo-runner:
    token: "..."

headplane:
    cookie_secret: "..."
    agent_pre_authkey: "..."

tailscale:
    auth-key: "..."

hosts/cryodev-pi/secrets.yaml:

sops hosts/cryodev-pi/secrets.yaml

tailscale:
    auth-key: "..."

netdata:
    stream:
        child-uuid: "..." # Output von uuidgen

4. Erstes Deployment (Bootstrap)

Bevor das automatische Deployment funktionieren kann, müssen Sie das System einmal manuell mit dem Installationsskript installieren.

4.1 Zielmaschine vorbereiten

1. Booten Sie das NixOS Installations-ISO.
2. Setzen Sie ein Root-Passwort: passwd.
3. Stellen Sie eine Internetverbindung her.

4.2 Install-Script ausführen

Kopieren Sie das Skript von Ihrem lokalen Rechner auf das Zielsystem.

Methode A: Per SCP

scp scripts/install.sh nixos@<TARGET_IP>:install.sh
ssh nixos@<TARGET_IP>
sudo -i
chmod +x /home/nixos/install.sh
./home/nixos/install.sh -r <GIT_REPO_URL> -n <HOSTNAME>

Methode B: Direkt auf dem Ziel (bei öffentlichem/erreichbarem Repo)

# Auf der Zielmaschine (als root)
nix-shell -p git
git clone <GIT_REPO_URL> /tmp/nixos
cd /tmp/nixos
bash scripts/install.sh -n <HOSTNAME>

Hinweis: Das Skript kümmert sich um Partitionierung, Hardware-Config und Installation.

5. Continuous Deployment (CD)

5.1 cryodev-pi (Pull-basiert via Comin)

Der Host cryodev-pi zieht Updates automatisch via comin.

1. Repository erstellen: Erstellen Sie ein Repository namens cryodev-server auf https://git.cryodev.xyz.
2. Konfiguration pushen: Pushen Sie diese Konfiguration in den main-Branch.
3. Comin URL: https://git.cryodev.xyz/steffen/cryodev-server.git.

5.2 cryodev-main (Push-basiert via Forgejo Actions)

Der Hauptserver wird über eine Forgejo Action deployt.

1. SSH Key generieren:

   ssh-keygen -t ed25519 -f deploy_key -C "forgejo-actions"
   

2. Public Key hinzufügen: Inhalt von deploy_key.pub in /root/.ssh/authorized_keys auf cryodev-main eintragen.
3. Secret hinzufügen: Inhalt von deploy_key (Private Key) als Secret DEPLOY_SSH_KEY im Forgejo-Repository hinterlegen.

6. Neue Hosts erstellen (Templates)

Um schnell eine neue Host-Konfiguration zu erstellen, können Sie die bereitgestellten Templates nutzen.

1. Template kopieren:

   # Für einen Raspberry Pi:
   cp -r templates/raspberry-pi hosts/neuer-pi-name
   
   # Für einen generischen x86 Server:
   cp -r templates/generic-server hosts/neuer-server-name
   

2. Konfiguration anpassen:
   • Hostname: Bearbeiten Sie hosts/neuer-name/networking.nix.
   • Flake: Registrieren Sie den neuen Host in flake.nix unter nixosConfigurations.
   • Constants: Fügen Sie IP und Ports in constants.nix hinzu.
   • Secrets: Fügen Sie Keys zu .sops.yaml hinzu und erstellen Sie hosts/neuer-name/secrets.yaml.