steffen/cryodev
1
0
Fork 0
Code Issues Pull requests Projects Releases 2 Packages Wiki Activity Actions

Compare commits

base: steffen:33bc9f5e33003ad7d0f770c78fd05a66f2654dd3
Branches Tags
steffen:main
steffen:develop
steffen:v2026-03-14-b0f78c3
steffen:v2026-03-14-33bc9f5
..
compare: steffen:badf9700419b27358e958b13fee6cfba30988147
Branches Tags
steffen:main
steffen:develop
steffen:v2026-03-14-b0f78c3
steffen:v2026-03-14-33bc9f5

No commits in common. "33bc9f5e33003ad7d0f770c78fd05a66f2654dd3" and "badf9700419b27358e958b13fee6cfba30988147" have entirely different histories.
33bc9f5e33 ... badf970041

11 changed files with 1438 additions and 1681 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.forgejo/workflows/deploy.yml
View file

@ -46,7 +46,7 @@ jobs:
--extra-platforms aarch64-linux \ --extra-platforms aarch64-linux \
--out-link result-${{ matrix.host }} --out-link result-${{ matrix.host }}
IMAGE_PATH=$(find -L result-${{ matrix.host }} -name "*.img.zst" | head -1) IMAGE_PATH=$(find result-${{ matrix.host }} -name "*.img.zst" -type f | head -1)
if [ -z "$IMAGE_PATH" ]; then if [ -z "$IMAGE_PATH" ]; then
echo "Error: No image found!" echo "Error: No image found!"
exit 1 exit 1

231
AGENTS.md
View file

@ -2,33 +2,37 @@
## Project Overview ## Project Overview
NixOS infrastructure managed with Nix Flakes. Two hosts, reusable modules, SOPS secrets, Comin auto-deployment. This repository contains a NixOS configuration managed with Nix Flakes. It defines:
- **Hosts**: `cryodev-main` (x86_64 server), `cryodev-pi` (aarch64 Raspberry Pi)
- **Hosts**: `cryodev-main` (x86_64 server), `cryodev-pi` (aarch64 Raspberry Pi 4)
- **Modules**: Reusable NixOS modules in `modules/nixos/` - **Modules**: Reusable NixOS modules in `modules/nixos/`
- **Apps**: `create`, `deploy`, `install`, `rebuild` in `apps/` - **Packages**: Custom packages in `pkgs/`
- **Templates**: `raspberry-pi`, `generic-server` for bootstrapping new hosts - **Templates**: `raspberry-pi`, `generic-server` for bootstrapping new hosts
## Build & Development Commands ## Build & Development Commands
### Prerequisites
- **Nix** with Flakes enabled
- **Git**
### Core Commands
```bash ```bash
# Format code (required before committing, runs nixfmt via pre-commit) # Build host configuration
nix fmt
# Run all checks (formatting, package builds, overlay builds)
nix flake check
# Quick evaluation test (faster than full build, use to validate changes)
nix eval .#nixosConfigurations.cryodev-main.config.system.build.toplevel.name
nix eval .#nixosConfigurations.cryodev-pi.config.system.build.toplevel.name
# Full build
nix build .#nixosConfigurations.cryodev-main.config.system.build.toplevel nix build .#nixosConfigurations.cryodev-main.config.system.build.toplevel
nix build .#nixosConfigurations.cryodev-pi.config.system.build.toplevel nix build .#nixosConfigurations.cryodev-pi.config.system.build.toplevel
# Build Raspberry Pi SD image (requires binfmt on x86_64) # Build Raspberry Pi SD image (requires binfmt on x86_64)
nix build .#nixosConfigurations.cryodev-pi.config.system.build.sdImage nix build .#nixosConfigurations.cryodev-pi.config.system.build.sdImage
# Format code (required before committing)
nix fmt
# Run all checks (lint, formatting)
nix flake check
# Quick evaluation test (faster than full build)
nix eval .#nixosConfigurations.cryodev-main.config.system.build.toplevel.name
# Update flake inputs # Update flake inputs
nix flake update nix flake update
@ -38,143 +42,176 @@ nix develop
### Deployment ### Deployment
Both hosts use **Comin** for automatic pull-based deployment (polls the git repo).
Manual deployment is only needed for initial setup or emergencies:
```bash ```bash
# Deploy via deploy app (uses deploy.json, SSH port 2299, asks sudo password) # Deploy all hosts via deploy app (uses deploy.json)
nix run .#deploy
# Deploy a specific host
nix run .#deploy -- -n cryodev-main nix run .#deploy -- -n cryodev-main
# Manual deployment via nixos-rebuild # Manual deployment via SSH
NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \ NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \
--target-host <user>@<ip> --sudo --ask-sudo-password --target-host <user>@<ip> --sudo --ask-sudo-password
``` ```
> **Note:** Both hosts use Comin for automatic pull-based deployment.
> Manual deployment is only needed for the initial setup or emergencies.
### Apps ### Apps
```bash ```bash
nix run .#create -- -t generic-server -n <hostname> # Scaffold new host # Create a new host from template
nix run .#install -- -n <hostname> -r <REPO_URL> # Install from NixOS ISO nix run .#create -- -t generic-server -n <hostname>
nix run .#deploy -- -n <hostname> # Deploy to host
nix run .#rebuild -- nixos # Rebuild locally # Install NixOS on a new machine (run from NixOS live ISO)
nix run .#install -- -n <hostname> -r <REPO_URL>
# Deploy to all configured hosts
nix run .#deploy
# Rebuild NixOS/Home Manager configuration
nix run .#rebuild -- nixos
``` ```
## Code Style & Conventions ## Code Style & Conventions
### Formatting ### Formatting
- **Tool**: `nixfmt` via `git-hooks.nix` (pre-commit) - **Tool**: `nixfmt` via pre-commit hooks
- **Run**: `nix fmt` before every commit - **Run**: `nix fmt` before every commit
- **Indentation**: 2 spaces (enforced by formatter) - **Indentation**: 2 spaces
- **Line length**: 80-100 characters (follow formatter)
### Naming Conventions ### Module Structure
| Type | Convention | Example |
|------|------------|---------|
| Files | kebab-case | `hardware-configuration.nix` |
| NixOS options | camelCase | `services.myService.enable` |
| Let bindings | camelCase | `let myValue = ...;` |
| Hosts | kebab-case | `cryodev-main`, `cryodev-pi` |
| Secret paths | kebab-case with `/` | `forgejo-runner/token` |
### Module Pattern
```nix ```nix
{ config, lib, ... }: # Standard module pattern
{ config, lib, pkgs, inputs, outputs, constants, ... }:
let let
cfg = config.services.myService; cfg = config.services.myService;
inherit (lib) mkDefault mkEnableOption mkIf mkOption types;
in in
{ {
options.services.myService = { options.services.myService = {
enable = mkEnableOption "My service"; enable = lib.mkEnableOption "My service";
port = mkOption { port = lib.mkOption {
type = types.port; type = lib.types.port;
default = 8080; default = 8080;
description = "Port to listen on"; description = "Port to listen on";
}; };
}; };
config = mkIf cfg.enable { config = lib.mkIf cfg.enable {
assertions = [ # Implementation here
{ assertion = cfg.port > 1024; message = "Port must be > 1024"; }
];
# Implementation
}; };
} }
``` ```
### Key Rules ### Naming Conventions
| Type | Convention | Example |
|------|------------|---------|
| Files | kebab-case | `hardware-configuration.nix` |
| Options | camelCase | `services.myService.enable` |
| Variables | camelCase | `let myValue = ...;` |
| Hosts | kebab-case | `cryodev-main`, `cryodev-pi` |
- **Use `lib.mkDefault`** for all module defaults (allows host-level overrides) ### Imports
- **Use `constants.nix`** for domains, IPs, ports -- never hardcode these
- **Use `lib.utils`** helpers: `mkReverseProxyOption`, `mkVirtualHost`, `mkUrl`
- **Secrets via SOPS** only, never plaintext. Reference: `config.sops.secrets."path".path`
- **Imports**: relative paths for local files, `outputs.nixosModules.*` for shared modules, `inputs.*` for external
- **Assertions** for invalid configurations, `warnings` for non-critical issues
- **`lib.inherit`** pattern: extract needed functions in `let` block
### Host Service Files
Each service gets its own file in `hosts/<host>/services/`:
```nix ```nix
# hosts/cryodev-main/services/myservice.nix # Local modules: relative paths
{ outputs, constants, ... }: imports = [ ./hardware.nix ./networking.nix ];
# Shared modules: via outputs
imports = [ outputs.nixosModules.common ];
# External inputs
imports = [ inputs.sops-nix.nixosModules.sops ];
```
### Constants
Use `constants.nix` for domains, IPs, and ports:
```nix
{ constants, ... }:
{ {
imports = [ outputs.nixosModules.myservice ]; services.nginx.virtualHosts."${constants.services.forgejo.fqdn}" = { ... };
services.myservice = {
enable = true;
port = constants.services.myservice.port;
};
services.nginx.virtualHosts."${constants.services.myservice.fqdn}" = {
forceSSL = true;
enableACME = true;
locations."/" = {
proxyPass = "http://127.0.0.1:${toString constants.services.myservice.port}";
};
};
} }
``` ```
### Special Args Available in Modules ### Error Handling
```nix
config = lib.mkIf cfg.enable {
assertions = [
{ assertion = cfg.port > 1024; message = "Port must be > 1024"; }
];
warnings = lib.optional (cfg.debug) "Debug mode enabled!";
};
```
- `inputs` -- flake inputs (nixpkgs, sops-nix, comin, headplane, etc.) ### Option Conflicts
- `outputs` -- this flake's outputs (nixosModules, packages) Use `lib.mkDefault` for default values that can be overridden:
- `constants` -- values from `constants.nix` (domain, hosts, services) ```nix
- `lib` -- nixpkgs.lib extended with `lib.utils` services.nginx.enable = lib.mkDefault true;
```
## Directory Structure ## Directory Structure
``` ```
. .
├── flake.nix # Entry point, inputs/outputs, mkNixosConfiguration ├── flake.nix # Entry point, inputs/outputs
├── constants.nix # Central config (domains, IPs, ports) ├── constants.nix # Central config (domains, IPs, ports)
├── hosts/ ├── hosts/
│ ├── cryodev-main/ # x86_64 server (services/, secrets.yaml, binfmt.nix) │ ├── cryodev-main/ # x86_64 server
│ └── cryodev-pi/ # aarch64 RPi (services/, secrets.yaml, sd-image.nix) │ │ ├── default.nix # Host entry point
├── modules/nixos/ # Reusable modules (common, forgejo, headscale, ...) │ │ ├── hardware.nix # Hardware configuration
├── users/ # User definitions (steffen, ralph, benjamin) │ │ ├── services/ # Service configurations
├── apps/ # Nix apps (create, deploy, install, rebuild) │ │ └── secrets.yaml # SOPS-encrypted secrets
├── lib/utils.nix # Helper functions (mkUrl, mkVirtualHost, ...) │ └── cryodev-pi/ # aarch64 Raspberry Pi
├── modules/nixos/ # Reusable modules
│ ├── common/ # Shared base configuration
│ ├── sops/ # Secret management
│ ├── forgejo/ # Git server
│ ├── headscale/ # VPN control server
│ └── ...
├── lib/utils.nix # Helper functions
├── apps/ # Nix apps (rebuild)
├── pkgs/ # Custom packages ├── pkgs/ # Custom packages
├── overlays/ # Nixpkgs overlays ├── overlays/ # Nixpkgs overlays
├── templates/ # Host templates (generic-server, raspberry-pi) ├── templates/ # Host templates
├── deploy.json # Deploy app config (hosts, SSH port) ├── scripts/ # Helper scripts (install.sh)
├── .sops.yaml # SOPS encryption rules (age keys per host) └── docs/ # Documentation
├── .forgejo/workflows/ # CI pipelines (ci.yml, deploy.yml)
└── docs/ # Documentation (English)
``` ```
## Key Patterns
### Adding a New Raspberry Pi Host
1. Copy template: `cp -r templates/raspberry-pi hosts/new-pi`
2. Update `hosts/new-pi/networking.nix` (hostname)
3. Add to `flake.nix`: `new-pi = mkNixosConfiguration "aarch64-linux" [ ./hosts/new-pi ];`
4. Add to `.forgejo/workflows/build-pi-image.yml` matrix
5. Push → SD image is built automatically
### SOPS Secrets
- Secrets encrypted with age using SSH host keys
- Config in `.sops.yaml`, secrets in `hosts/<host>/secrets.yaml`
- Reference: `config.sops.secrets."path/to/secret".path`
### Special Args Available in Modules
- `inputs`: Flake inputs (nixpkgs, sops-nix, etc.)
- `outputs`: This flake's outputs (nixosModules, packages)
- `constants`: Values from `constants.nix`
- `lib`: Extended nixpkgs.lib with `lib.utils`
## Deployment Workflows
| Host | Strategy | Trigger |
|------|----------|---------|
| `cryodev-main` | Pull via Comin | Automatic polling |
| `cryodev-pi` | Pull via Comin | Automatic polling |
| SD Images | Built in CI | Push to main (for Pi hosts) |
## Verification Checklist ## Verification Checklist
Before committing: Before committing:
- [ ] `nix fmt` passes - [ ] `nix fmt` passes
- [ ] `nix flake check` passes (or at least `nix eval` works for both hosts) - [ ] `nix flake check` passes (or at least `nix eval` works)
- [ ] New hosts added to `flake.nix` nixosConfigurations - [ ] New hosts added to `flake.nix`
- [ ] Constants in `constants.nix`, not hardcoded - [ ] Constants in `constants.nix`, not hardcoded
- [ ] Secrets use SOPS, not plaintext - [ ] Secrets use SOPS, not plaintext
- [ ] New services have their own file in `hosts/<host>/services/`
- [ ] New modules registered in `modules/nixos/default.nix`
- [ ] Documentation in English

2114
digest.txt
View file

File diff suppressed because it is too large Load diff

42
docs/deployment/dns.md
View file

@ -31,12 +31,12 @@ Required DNS records for the cryodev infrastructure.
| `@` | MX | `10 mail.cryodev.xyz.` | Mail delivery | | `@` | MX | `10 mail.cryodev.xyz.` | Mail delivery |
| `@` | TXT | `"v=spf1 mx ~all"` | SPF | | `@` | TXT | `"v=spf1 mx ~all"` | SPF |
| `_dmarc` | TXT | `"v=DMARC1; p=none"` | DMARC | | `_dmarc` | TXT | `"v=DMARC1; p=none"` | DMARC |
| `mail._domainkey` | TXT | *(see below)* | DKIM | | `mail._domainkey` | TXT | *(siehe unten)* | DKIM |
### Reverse DNS (PTR) ### Reverse DNS (PTR)
For reliable mail delivery, a **PTR record** must be configured at the hosting Fuer zuverlaessige Mail-Zustellung muss ein **PTR Record** beim Hosting-Provider
provider (not in the domain's DNS panel): konfiguriert werden (nicht im DNS-Panel der Domain):
| IP | PTR Value | | IP | PTR Value |
|----|-----------| |----|-----------|
@ -45,18 +45,18 @@ provider (not in the domain's DNS panel):
#### Hetzner Robot (Dedicated Server) #### Hetzner Robot (Dedicated Server)
1. [robot.hetzner.com](https://robot.hetzner.com) > **Server** > Select the server 1. [robot.hetzner.com](https://robot.hetzner.com) > **Server** > Server auswaehlen
2. **IPs** tab 2. **IPs** Tab
3. Click the **pencil icon** next to the IPv4 address 3. Bei der IPv4-Adresse auf das **Stift-Symbol** klicken
4. Enter `mail.cryodev.xyz` and save 4. `mail.cryodev.xyz` eintragen und speichern
5. For IPv6: Under **Subnets**, repeat the same for the primary IPv6 address 5. Fuer IPv6: Unter **Subnets** dasselbe fuer die primaere IPv6-Adresse
#### Hetzner Cloud #### Hetzner Cloud
1. [cloud.hetzner.com](https://cloud.hetzner.com) > Select the server 1. [cloud.hetzner.com](https://cloud.hetzner.com) > Server auswaehlen
2. **Networking** tab 2. **Networking** Tab
3. Under "Primary IP", click the IP > **Reverse DNS** 3. Bei "Primary IP" auf die IP klicken > **Reverse DNS**
4. Enter `mail.cryodev.xyz` (for both IPv4 and IPv6) 4. `mail.cryodev.xyz` eintragen (fuer IPv4 und IPv6)
## Getting the DKIM Key ## Getting the DKIM Key
@ -70,15 +70,15 @@ Add this as a TXT record for `mail._domainkey.cryodev.xyz`.
## Complete Checklist ## Complete Checklist
- [ ] A/AAAA for `@` (root domain) - [ ] A/AAAA fuer `@` (Root-Domain)
- [ ] A/AAAA for `www` - [ ] A/AAAA fuer `www`
- [ ] A/AAAA for `mail` - [ ] A/AAAA fuer `mail`
- [ ] CNAME for `git`, `headscale`, `headplane`, `netdata` - [ ] CNAME fuer `git`, `headscale`, `headplane`, `netdata`
- [ ] MX record - [ ] MX Record
- [ ] TXT for SPF (`v=spf1 mx ~all`) - [ ] TXT fuer SPF (`v=spf1 mx ~all`)
- [ ] TXT for DMARC (`v=DMARC1; p=none`) - [ ] TXT fuer DMARC (`v=DMARC1; p=none`)
- [ ] TXT for DKIM (`mail._domainkey` -- after first deployment) - [ ] TXT fuer DKIM (`mail._domainkey` -- nach erstem Deploy)
- [ ] PTR record at hosting provider (reverse DNS) - [ ] PTR Record beim Hosting-Provider (Reverse DNS)
## Verification ## Verification

244
docs/getting-started/first-install.md
View file

@ -1,44 +1,44 @@
# Initial Installation (x86_64 Server) # Erstinstallation (x86_64 Server)
This guide describes the **initial installation** of a new x86_64 server (e.g. cryodev-main). Diese Anleitung beschreibt die **Erstinstallation** eines neuen x86_64 Servers (z.B. cryodev-main).
> **For Raspberry Pi:** See [Creating an SD Image](sd-image.md). > **Fuer Raspberry Pi:** Siehe [SD-Image erstellen](sd-image.md).
## Overview ## Uebersicht
During initial installation there is a chicken-and-egg problem: Bei der Erstinstallation gibt es ein Henne-Ei-Problem:
- SOPS secrets are encrypted with the SSH host key - SOPS-Secrets werden mit dem SSH-Host-Key verschluesselt
- The SSH host key is only generated during installation - Der SSH-Host-Key wird erst bei der Installation generiert
- Therefore: **Install without secrets first, then configure secrets** - Daher: **Erst ohne Secrets installieren, dann Secrets konfigurieren**
### Process ### Ablauf
``` ```
1. Disable services (that require secrets) 1. Services deaktivieren (die Secrets brauchen)
2. Install NixOS 2. NixOS installieren
3. Extract SSH host key, configure SOPS, create immediately available secrets 3. SSH-Host-Key extrahieren, SOPS konfigurieren, sofort erstellbare Secrets anlegen
4. Enable stage-1 services and deploy (Headscale, Forgejo, Mail, Nginx) 4. Stufe-1-Services aktivieren und deployen (Headscale, Forgejo, Mail, Nginx)
5. Generate remaining secrets (Tailscale, Headplane, Forgejo Runner) 5. Restliche Secrets generieren (Tailscale, Headplane, Forgejo-Runner)
6. Enable stage-2 services and perform final deployment 6. Stufe-2-Services aktivieren und final deployen
``` ```
## Step 1: Prepare Host Configuration ## Schritt 1: Host-Konfiguration vorbereiten
> If the host already exists in `hosts/` and `flake.nix`, skip 1.1-1.2. > Falls der Host bereits in `hosts/` und `flake.nix` existiert, ueberspringe 1.1-1.2.
### 1.1 Create Host from Template ### 1.1 Host aus Template erstellen
```bash ```bash
nix run .#create -- -t generic-server -n <hostname> nix run .#create -- -t generic-server -n <hostname>
``` ```
The script: Das Script:
- Copies the template to `hosts/<hostname>/` - Kopiert das Template nach `hosts/<hostname>/`
- Sets the hostname in `networking.nix` - Setzt den Hostname in `networking.nix`
- Creates an empty `secrets.yaml` - Erstellt eine leere `secrets.yaml`
- Adds the files to Git - Fuegt die Dateien zu Git hinzu
### 1.2 Register in flake.nix ### 1.2 In flake.nix registrieren
```nix ```nix
nixosConfigurations = { nixosConfigurations = {
@ -46,24 +46,24 @@ nixosConfigurations = {
}; };
``` ```
Also adjust `hardware.nix` and `disks.sh` for the target hardware. Ausserdem `hardware.nix` und `disks.sh` fuer die Zielhardware anpassen.
### 1.4 Temporarily Disable Services ### 1.4 Services temporaer deaktivieren
All services that reference SOPS secrets must be disabled for the initial installation. Otherwise the installation will fail because the secrets cannot yet be decrypted. Alle Services, die SOPS-Secrets referenzieren, muessen fuer die Erstinstallation deaktiviert werden. Andernfalls schlaegt die Installation fehl, weil die Secrets noch nicht entschluesselt werden koennen.
In `hosts/<hostname>/services/default.nix`, comment out the corresponding imports: In `hosts/<hostname>/services/default.nix` die entsprechenden Imports auskommentieren:
```nix ```nix
{ {
imports = [ imports = [
# Disabled until SOPS secrets are configured: # Deaktiviert bis SOPS-Secrets konfiguriert sind:
# ./forgejo.nix # requires: forgejo-runner/token, forgejo/mail-pw # ./forgejo.nix # braucht: forgejo-runner/token, forgejo/mail-pw
# ./headplane.nix # requires: headplane/cookie_secret, headplane/agent_pre_authkey # ./headplane.nix # braucht: headplane/cookie_secret, headplane/agent_pre_authkey
# ./mailserver.nix # requires: mailserver/accounts/* # ./mailserver.nix # braucht: mailserver/accounts/*
# ./tailscale.nix # requires: tailscale/auth-key # ./tailscale.nix # braucht: tailscale/auth-key
# These services do not require secrets: # Diese Services brauchen keine Secrets:
./headscale.nix ./headscale.nix
./netdata.nix ./netdata.nix
./nginx.nix ./nginx.nix
@ -73,7 +73,7 @@ In `hosts/<hostname>/services/default.nix`, comment out the corresponding import
} }
``` ```
Additionally, in `hosts/<hostname>/services/sops.nix`, comment out the secret definitions: Zusaetzlich in `hosts/<hostname>/services/sops.nix` die Secrets-Definitionen auskommentieren:
```nix ```nix
sops = { sops = {
@ -85,32 +85,32 @@ sops = {
}; };
``` ```
### 1.5 Test the Configuration ### 1.5 Konfiguration testen
```bash ```bash
nix eval .#nixosConfigurations.<hostname>.config.system.build.toplevel.name nix eval .#nixosConfigurations.<hostname>.config.system.build.toplevel.name
``` ```
## Step 2: Perform Installation ## Schritt 2: Installation durchfuehren
### 2.1 Boot NixOS ISO ### 2.1 NixOS ISO booten
Boot from the [NixOS Minimal ISO](https://nixos.org/download/#nixos-iso) (USB/CD). Vom [NixOS Minimal ISO](https://nixos.org/download/#nixos-iso) booten (USB/CD).
### 2.2 Set Up Network and SSH ### 2.2 Netzwerk und SSH einrichten
```bash ```bash
passwd # Set root password for SSH access passwd # Root-Passwort setzen fuer SSH-Zugang
ip a # Determine IP address ip a # IP-Adresse ermitteln
``` ```
Optionally connect via SSH (more convenient): Optional per SSH verbinden (bequemer):
```bash ```bash
ssh -o StrictHostKeyChecking=no root@<IP> ssh -o StrictHostKeyChecking=no root@<IP>
``` ```
### 2.3 Install ### 2.3 Installieren
```bash ```bash
nix --experimental-features "nix-command flakes" run \ nix --experimental-features "nix-command flakes" run \
@ -119,20 +119,20 @@ nix --experimental-features "nix-command flakes" run \
-r <REPO_URL> -r <REPO_URL>
``` ```
Alternatively, if the repository has already been cloned to `/tmp/nixos`: Alternativ, falls das Repository bereits unter `/tmp/nixos` geklont wurde:
```bash ```bash
nix --experimental-features "nix-command flakes" run /tmp/nixos#install -- -n <hostname> nix --experimental-features "nix-command flakes" run /tmp/nixos#install -- -n <hostname>
``` ```
> **Note:** The disk ID in `hosts/<hostname>/disks.sh` must match the hardware. > **Hinweis:** Die Disk-ID in `hosts/<hostname>/disks.sh` muss zur Hardware passen.
> Verify with `ls -la /dev/disk/by-id/`. > Pruefen mit `ls -la /dev/disk/by-id/`.
The script: Das Script:
1. Clones the repository (when using `-r`) 1. Klont das Repository (bei `-r`)
2. Partitions the disk (via `disks.nix` or `disks.sh`) 2. Partitioniert die Disk (via `disks.nix` oder `disks.sh`)
3. Generates `hardware.nix` (if not present) 3. Generiert `hardware.nix` (falls nicht vorhanden)
4. Installs NixOS 4. Installiert NixOS
### 2.4 Reboot ### 2.4 Reboot
@ -140,34 +140,34 @@ The script:
reboot reboot
``` ```
## Step 3: Configure SOPS Secrets ## Schritt 3: SOPS-Secrets konfigurieren
After the first boot, log in (password: `changeme`, change immediately with `passwd`). Nach dem ersten Boot einloggen (Passwort: `changeme`, sofort aendern mit `passwd`).
### 3.1 Convert SSH Host Key to Age Key ### 3.1 SSH-Host-Key zu Age-Key konvertieren
On the **new server**: Auf dem **neuen Server**:
```bash ```bash
nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age' nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
``` ```
Note the output (e.g. `age1abc123...`). Ausgabe notieren (z.B. `age1abc123...`).
Alternatively, remotely: Alternativ remote:
```bash ```bash
nix-shell -p ssh-to-age --run 'ssh-keyscan -p 2299 -t ed25519 <IP> | ssh-to-age' nix-shell -p ssh-to-age --run 'ssh-keyscan -p 2299 -t ed25519 <IP> | ssh-to-age'
``` ```
### 3.2 Update .sops.yaml ### 3.2 .sops.yaml aktualisieren
On the **development machine**, add the new host key to `.sops.yaml`: Auf dem **Entwicklungsrechner** den neuen Host-Key in `.sops.yaml` eintragen:
```yaml ```yaml
keys: keys:
- &steffen_key age1e8p... # steffen (local) - &steffen_key age1e8p... # steffen (lokal)
- &hostname_key age1abc... # Key from step 3.1 - &hostname_key age1abc... # Key von Schritt 3.1
creation_rules: creation_rules:
- path_regex: hosts/<hostname>/secrets.yaml$ - path_regex: hosts/<hostname>/secrets.yaml$
@ -177,38 +177,38 @@ creation_rules:
- *hostname_key - *hostname_key
``` ```
### 3.3 Create Secrets ### 3.3 Secrets erstellen
Open the secrets file: Secrets-Datei oeffnen:
```bash ```bash
sops hosts/<hostname>/secrets.yaml sops hosts/<hostname>/secrets.yaml
``` ```
The following table shows all secrets for **cryodev-main** and how they are generated: Die folgende Tabelle zeigt alle Secrets fuer **cryodev-main** und wie sie generiert werden:
#### Immediately Available Secrets #### Sofort erstellbare Secrets
These secrets have no dependencies and can be generated directly: Diese Secrets haben keine Abhaengigkeiten und koennen direkt generiert werden:
| Secret | Command | | Secret | Befehl |
|--------|---------| |--------|--------|
| `headplane/cookie_secret` | `openssl rand -hex 16` | | `headplane/cookie_secret` | `openssl rand -hex 16` |
| `mailserver/accounts/admin` | `mkpasswd -sm bcrypt` (remember the password!) | | `mailserver/accounts/admin` | `mkpasswd -sm bcrypt` (Passwort merken!) |
| `mailserver/accounts/forgejo` | `mkpasswd -sm bcrypt` (remember the password!) | | `mailserver/accounts/forgejo` | `mkpasswd -sm bcrypt` (Passwort merken!) |
| `forgejo/mail-pw` | Plaintext password matching the bcrypt hash of `mailserver/accounts/forgejo` | | `forgejo/mail-pw` | Klartext-Passwort das zum bcrypt-Hash von `mailserver/accounts/forgejo` passt |
#### Secrets That Require Running Services #### Secrets die laufende Services brauchen
These secrets can only be created after step 4. **Do not add them yet** -- they will be added later. Diese Secrets koennen erst nach Schritt 4 erstellt werden. **Jetzt noch nicht eintragen** -- sie werden spaeter ergaenzt.
| Secret | Command | Prerequisite | | Secret | Befehl | Voraussetzung |
|--------|---------|--------------| |--------|--------|---------------|
| `tailscale/auth-key` | See steps 4.1-4.2 | Headscale is running | | `tailscale/auth-key` | Siehe Schritt 4.1-4.2 | Headscale laeuft |
| `headplane/agent_pre_authkey` | See steps 4.1-4.2 | Headscale is running | | `headplane/agent_pre_authkey` | Siehe Schritt 4.1-4.2 | Headscale laeuft |
| `forgejo-runner/token` | Forgejo Admin Panel > Actions > Runners > Create Runner | Forgejo is running | | `forgejo-runner/token` | Forgejo Admin Panel > Actions > Runners > Create Runner | Forgejo laeuft |
#### Example secrets.yaml (Plaintext Before Encryption) #### Beispiel secrets.yaml (Klartext vor Verschluesselung)
```yaml ```yaml
headplane: headplane:
@ -218,22 +218,22 @@ mailserver:
admin: "$2b$05$..." admin: "$2b$05$..."
forgejo: "$2b$05$..." forgejo: "$2b$05$..."
forgejo: forgejo:
mail-pw: "the-plaintext-password" mail-pw: "das-klartext-passwort"
``` ```
### 3.4 Gradually Re-enable Services -- Stage 1 ### 3.4 Services stufenweise reaktivieren -- Stufe 1
> **Important:** Services that require Headscale or Forgejo secrets (Tailscale, > **Wichtig:** Services die Headscale- oder Forgejo-Secrets brauchen (Tailscale,
> Headplane, Forgejo Runner) must **not** be enabled yet, as these > Headplane, Forgejo-Runner) duerfen noch **nicht** aktiviert werden, da diese
> secrets can only be generated once those services are running. > Secrets erst generiert werden koennen, wenn die Services laufen.
On the **development machine**, in `hosts/<hostname>/services/default.nix`, enable Auf dem **Entwicklungsrechner** in `hosts/<hostname>/services/default.nix` die
the services **without external dependencies**: Services **ohne externe Abhaengigkeiten** aktivieren:
```nix ```nix
{ {
imports = [ imports = [
# Stage 1: Services without external dependencies # Stufe 1: Services ohne externe Abhaengigkeiten
./forgejo.nix ./forgejo.nix
./headscale.nix ./headscale.nix
./mailserver.nix ./mailserver.nix
@ -242,88 +242,88 @@ the services **without external dependencies**:
./openssh.nix ./openssh.nix
./sops.nix ./sops.nix
# Stage 2: Enable only after step 4 # Stufe 2: Erst nach Schritt 4 aktivieren
# ./forgejo-runner.nix # requires: forgejo-runner/token (Forgejo) # ./forgejo-runner.nix # braucht: forgejo-runner/token (Forgejo)
# ./headplane.nix # requires: headplane/agent_pre_authkey (Headscale) # ./headplane.nix # braucht: headplane/agent_pre_authkey (Headscale)
# ./tailscale.nix # requires: tailscale/auth-key (Headscale) # ./tailscale.nix # braucht: tailscale/auth-key (Headscale)
]; ];
} }
``` ```
### 3.5 Deploy (Stage 1) ### 3.5 Deployen (Stufe 1)
```bash ```bash
nix run .#deploy -- -n <hostname> nix run .#deploy -- -n <hostname>
``` ```
This uses the configuration from `deploy.json`. Alternatively, deploy manually: Dies nutzt die Konfiguration aus `deploy.json`. Alternativ manuell:
```bash ```bash
NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \ NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \
--target-host <user>@<IP> --sudo --ask-sudo-password --target-host <user>@<IP> --sudo --ask-sudo-password
``` ```
After this deployment, Headscale, Forgejo, Mailserver, and Nginx are running. Nach diesem Deploy laufen Headscale, Forgejo, Mailserver und Nginx.
### 3.6 Create Forgejo Admin Account ### 3.6 Forgejo Admin-Account erstellen
On first start, Forgejo has no users. Create an admin account via CLI Beim ersten Start hat Forgejo noch keine Benutzer. Admin-Account per CLI anlegen
(on the **server**): (auf dem **Server**):
```bash ```bash
forgejo admin user create \ forgejo admin user create \
--username <username> \ --username <benutzername> \
--email <email>@<domain> \ --email <email>@<domain> \
--password <password> \ --password <passwort> \
--admin --admin
``` ```
> **Note:** The `forgejo` shell alias is provided by the module and automatically > **Hinweis:** Das `forgejo` Shell-Alias wird vom Modul bereitgestellt und fuehrt
> runs the command as the `forgejo` user with the correct config. > automatisch den Befehl als `forgejo`-User mit der richtigen Config aus.
> If the alias is not available, start a new shell (`bash` or `zsh`). > Falls der Alias nicht verfuegbar ist, neue Shell starten (`bash` oder `zsh`).
> >
> Since `DISABLE_REGISTRATION = true` is set, new accounts > Da `DISABLE_REGISTRATION = true` gesetzt ist, koennen neue Accounts
> can only be created via CLI. > nur per CLI erstellt werden.
## Step 4: Generate Remaining Secrets and Enable All Services ## Schritt 4: Restliche Secrets generieren und alle Services aktivieren
After the server is running with Headscale and Forgejo: Nachdem der Server mit Headscale und Forgejo laeuft:
1. **Create Headscale users** (on the server): 1. **Headscale-User anlegen** (auf dem Server):
```bash ```bash
sudo headscale users create default sudo headscale users create default
sudo headscale users create headplane-agent sudo headscale users create headplane-agent
``` ```
2. **Determine user IDs** (needed for the preauth keys): 2. **User-IDs ermitteln** (wird fuer die Preauth-Keys benoetigt):
```bash ```bash
sudo headscale users list sudo headscale users list
``` ```
The output shows the numeric IDs (e.g. `1` for default, `2` for headplane-agent). Die Ausgabe zeigt die numerischen IDs (z.B. `1` fuer default, `2` fuer headplane-agent).
3. **Generate preauth keys** (using the IDs from step 2): 3. **Preauth-Keys generieren** (mit den IDs aus Schritt 2):
```bash ```bash
# For Tailscale (use the user ID of "default") # Fuer Tailscale (User-ID von "default" einsetzen)
sudo headscale preauthkeys create --expiration 99y --reusable --user <ID> sudo headscale preauthkeys create --expiration 99y --reusable --user <ID>
# For Headplane Agent (use the user ID of "headplane-agent") # Fuer Headplane Agent (User-ID von "headplane-agent" einsetzen)
sudo headscale preauthkeys create --expiration 99y --user <ID> sudo headscale preauthkeys create --expiration 99y --user <ID>
``` ```
4. **Create the Forgejo Runner token** via the Forgejo Admin Panel: 4. **Forgejo-Runner-Token** ueber das Forgejo Admin Panel erstellen:
Administration > Actions > Runners > Create new Runner Administration > Actions > Runners > Create new Runner
5. **Add the remaining secrets**: 5. **Secrets ergaenzen**:
```bash ```bash
sops hosts/<hostname>/secrets.yaml sops hosts/<hostname>/secrets.yaml
``` ```
Add the missing secrets: Die fehlenden Secrets eintragen:
```yaml ```yaml
tailscale: tailscale:
@ -334,7 +334,7 @@ After the server is running with Headscale and Forgejo:
agent_pre_authkey: "..." agent_pre_authkey: "..."
``` ```
6. **Enable stage-2 services** in `hosts/<hostname>/services/default.nix`: 6. **Stufe-2-Services aktivieren** in `hosts/<hostname>/services/default.nix`:
```nix ```nix
{ {
@ -353,14 +353,14 @@ After the server is running with Headscale and Forgejo:
} }
``` ```
6. **Deploy again**: 6. **Erneut deployen**:
```bash ```bash
nix run .#deploy -- -n <hostname> nix run .#deploy -- -n <hostname>
``` ```
## Next Steps ## Naechste Schritte
- [SOPS Reference](../services/sops.md) -- Detailed documentation on secret management - [SOPS-Referenz](../services/sops.md) -- Detail-Dokumentation zur Secret-Verwaltung
- [Creating an SD Image](sd-image.md) -- Install Raspberry Pi - [SD-Image erstellen](sd-image.md) -- Raspberry Pi installieren
- [Set Up CD](../deployment/cd.md) -- Automatic deployment - [CD einrichten](../deployment/cd.md) -- Automatisches Deployment

170
docs/getting-started/new-client.md
View file

@ -1,60 +1,60 @@
# Adding a New Raspberry Pi Client # Neuen Raspberry Pi Client hinzufügen
This guide describes how to add a **new Raspberry Pi client** to the infrastructure. Diese Anleitung beschreibt das Hinzufügen eines **neuen Raspberry Pi Clients** zur Infrastruktur.
## Overview: The Process ## Übersicht: Der Ablauf
``` ```
1. Create configuration ──► Copy template, customize 1. Konfiguration erstellen ──► Template kopieren, anpassen
│ │
▼ ▼
2. Add to image pipeline ──► Extend workflow matrix 2. Zur Image-Pipeline hinzufügen ──► Workflow-Matrix erweitern
│ │
▼ ▼
3. Push to main ──► Forgejo automatically builds SD image 3. Push auf main ──► Forgejo baut automatisch SD-Image
│ │
▼ ▼
4. Flash image & boot ──► Write SD card, start Pi 4. Image flashen & booten ──► SD-Karte beschreiben, Pi starten
│ │
▼ ▼
5. Configure SOPS ──► Retrieve age key, create secrets 5. SOPS konfigurieren ──► Age-Key holen, Secrets erstellen
│ │
▼ ▼
6. Final deployment ──► Activate Tailscale etc. 6. Finales Deployment ──► Tailscale etc. aktivieren
``` ```
## Prerequisites ## Voraussetzungen
- SSH access to cryodev-main (for Tailscale auth key) - SSH-Zugang zu cryodev-main (für Tailscale Auth-Key)
- Development machine with repository access - Entwicklungsrechner mit Repository-Zugriff
- SD card (at least 8 GB) - SD-Karte (mindestens 8 GB)
--- ---
## Step 1: Generate Tailscale Auth Key ## Schritt 1: Tailscale Auth-Key generieren
**On cryodev-main** (via SSH): **Auf cryodev-main** (per SSH):
```bash ```bash
# Determine user ID # User-ID ermitteln
sudo headscale users list sudo headscale users list
# Create preauth key (use user ID of "default") # Preauth-Key erstellen (User-ID von "default" einsetzen)
sudo headscale preauthkeys create --expiration 99y --reusable --user <ID> sudo headscale preauthkeys create --expiration 99y --reusable --user <ID>
``` ```
**Take note of the output!** (e.g. `tskey-preauth-abc123...`) **Ausgabe notieren!** (z.B. `tskey-preauth-abc123...`)
--- ---
## Step 2: Create Host Configuration ## Schritt 2: Host-Konfiguration erstellen
### 2.1 Copy Template ### 2.1 Template kopieren
```bash ```bash
cp -r templates/raspberry-pi hosts/neuer-pi cp -r templates/raspberry-pi hosts/neuer-pi
``` ```
### 2.2 Set Hostname ### 2.2 Hostname setzen
`hosts/neuer-pi/networking.nix`: `hosts/neuer-pi/networking.nix`:
@ -64,58 +64,58 @@ cp -r templates/raspberry-pi hosts/neuer-pi
} }
``` ```
### 2.3 Register in flake.nix ### 2.3 In flake.nix registrieren
```nix ```nix
nixosConfigurations = { nixosConfigurations = {
# ... existing hosts ... # ... bestehende Hosts ...
neuer-pi = mkNixosConfiguration "aarch64-linux" [ ./hosts/neuer-pi ]; neuer-pi = mkNixosConfiguration "aarch64-linux" [ ./hosts/neuer-pi ];
}; };
``` ```
### 2.4 Add to constants.nix ### 2.4 In constants.nix eintragen
```nix ```nix
{ {
hosts = { hosts = {
# ... existing hosts ... # ... bestehende Hosts ...
neuer-pi = { neuer-pi = {
ip = "100.64.0.X"; # Assigned by Headscale ip = "100.64.0.X"; # Wird von Headscale vergeben
}; };
}; };
} }
``` ```
### 2.5 Create Placeholder secrets.yaml ### 2.5 Placeholder secrets.yaml erstellen
```bash ```bash
touch hosts/neuer-pi/secrets.yaml touch hosts/neuer-pi/secrets.yaml
``` ```
### 2.6 Temporarily Disable SOPS ### 2.6 SOPS temporär deaktivieren
In `hosts/neuer-pi/default.nix`, comment out the `sops.secrets.*` references so the image can be built without secrets. In `hosts/neuer-pi/default.nix` die `sops.secrets.*` Referenzen auskommentieren, damit das Image ohne Secrets gebaut werden kann.
--- ---
## Step 3: Add to Image Pipeline ## Schritt 3: Zur Image-Pipeline hinzufügen
Edit `.forgejo/workflows/build-pi-image.yml`: Bearbeite `.forgejo/workflows/build-pi-image.yml`:
```yaml ```yaml
jobs: jobs:
build-pi-images: build-pi-images:
strategy: strategy:
matrix: matrix:
# Add new host here: # Neuen Host hier hinzufügen:
host: [cryodev-pi, neuer-pi] host: [cryodev-pi, neuer-pi]
``` ```
--- ---
## Step 4: Push and Build Image ## Schritt 4: Push und Image bauen lassen
```bash ```bash
git add . git add .
@ -123,86 +123,86 @@ git commit -m "Add neuer-pi host configuration"
git push git push
``` ```
The Forgejo workflow will now automatically build an SD image for `neuer-pi`. Der Forgejo Workflow baut jetzt automatisch ein SD-Image für `neuer-pi`.
**Wait** until the workflow completes (30-60 minutes). Check the status at: **Warten** bis der Workflow fertig ist (30-60 Minuten). Status prüfen unter:
`https://git.cryodev.xyz/steffen/cryodev-server/actions` `https://git.cryodev.xyz/steffen/cryodev-server/actions`
--- ---
## Step 5: Flash Image ## Schritt 5: Image flashen
### 5.1 Download Image ### 5.1 Image herunterladen
After a successful build, find the image under **Releases**: Nach erfolgreichem Build unter **Releases**:
```bash ```bash
wget https://git.cryodev.xyz/steffen/cryodev-server/releases/latest/download/neuer-pi-sd-image.img.zst wget https://git.cryodev.xyz/steffen/cryodev-server/releases/latest/download/neuer-pi-sd-image.img.zst
``` ```
### 5.2 Decompress ### 5.2 Dekomprimieren
```bash ```bash
zstd -d neuer-pi-sd-image.img.zst -o neuer-pi.img zstd -d neuer-pi-sd-image.img.zst -o neuer-pi.img
``` ```
### 5.3 Write to SD Card ### 5.3 Auf SD-Karte schreiben
**Warning:** Replace `/dev/sdX` with the correct device! **Achtung:** `/dev/sdX` durch das richtige Gerät ersetzen!
```bash ```bash
lsblk # Identify the correct device lsblk # Richtiges Gerät finden
sudo dd if=neuer-pi.img of=/dev/sdX bs=4M conv=fsync status=progress sudo dd if=neuer-pi.img of=/dev/sdX bs=4M conv=fsync status=progress
``` ```
### 5.4 Boot ### 5.4 Booten
1. Insert the SD card into the Raspberry Pi 1. SD-Karte in den Raspberry Pi einlegen
2. Connect Ethernet 2. Ethernet anschließen
3. Connect power 3. Strom anschließen
4. Wait until booted (approximately 2 minutes) 4. Warten bis gebootet (ca. 2 Minuten)
--- ---
## Step 6: Configure SOPS ## Schritt 6: SOPS konfigurieren
### 6.1 Find IP Address ### 6.1 IP-Adresse finden
The Pi should receive an IP address via DHCP. Check your router or scan the network: Der Pi sollte per DHCP eine IP bekommen. Prüfe deinen Router oder scanne das Netzwerk:
```bash ```bash
nmap -sn 192.168.1.0/24 | grep -B2 "Raspberry" nmap -sn 192.168.1.0/24 | grep -B2 "Raspberry"
``` ```
### 6.2 Connect via SSH ### 6.2 SSH verbinden
```bash ```bash
ssh steffen@<IP> # or the configured user ssh steffen@<IP> # oder der konfigurierte User
``` ```
For the default password, see `hosts/neuer-pi/users.nix`. Standard-Passwort siehe `hosts/neuer-pi/users.nix`.
### 6.3 Determine Age Key ### 6.3 Age-Key ermitteln
On the Pi: Auf dem Pi:
```bash ```bash
nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age' nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
``` ```
**Take note of the output!** (e.g. `age1xyz...`) **Ausgabe notieren!** (z.B. `age1xyz...`)
### 6.4 Update .sops.yaml ### 6.4 .sops.yaml aktualisieren
On the development machine: Auf dem Entwicklungsrechner:
```yaml ```yaml
keys: keys:
- &steffen_key age1e8p35795htf7twrejyugpzw0qja2v33awcw76y4gp6acnxnkzq0s935t4t # steffen (local) - &steffen_key age1e8p35795htf7twrejyugpzw0qja2v33awcw76y4gp6acnxnkzq0s935t4t # steffen (local)
- &neuer_pi_key age1xyz... # The new key - &neuer_pi_key age1xyz... # Der neue Key
creation_rules: creation_rules:
# ... existing rules ... # ... bestehende Regeln ...
- path_regex: hosts/neuer-pi/secrets.yaml$ - path_regex: hosts/neuer-pi/secrets.yaml$
key_groups: key_groups:
@ -211,30 +211,30 @@ creation_rules:
- *neuer_pi_key - *neuer_pi_key
``` ```
### 6.5 Create Secrets ### 6.5 Secrets erstellen
```bash ```bash
sops hosts/neuer-pi/secrets.yaml sops hosts/neuer-pi/secrets.yaml
``` ```
Contents: Inhalt:
```yaml ```yaml
tailscale: tailscale:
auth-key: "tskey-preauth-abc123..." # Key from Step 1 auth-key: "tskey-preauth-abc123..." # Key aus Schritt 1
netdata: netdata:
stream: stream:
child-uuid: "..." # uuidgen child-uuid: "..." # uuidgen
``` ```
### 6.6 Activate SOPS References ### 6.6 SOPS-Referenzen aktivieren
Re-enable the `sops.secrets.*` references that were commented out in Step 2.6. Die in Schritt 2.6 auskommentierten `sops.secrets.*` Referenzen wieder aktivieren.
--- ---
## Step 7: Final Deployment ## Schritt 7: Finales Deployment
```bash ```bash
git add . git add .
@ -242,9 +242,9 @@ git commit -m "Configure SOPS secrets for neuer-pi"
git push git push
``` ```
Since Comin is running on the Pi, it will automatically pull the new configuration. Da Comin auf dem Pi läuft, wird er die neue Konfiguration automatisch pullen.
Alternatively, deploy manually: Alternativ manuell:
```bash ```bash
NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#neuer-pi \ NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#neuer-pi \
@ -253,34 +253,34 @@ NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#neuer-pi \
--- ---
## Step 8: Verify ## Schritt 8: Verifizieren
### Tailscale Connection ### Tailscale-Verbindung
```bash ```bash
# On the Pi # Auf dem Pi
tailscale status tailscale status
# On cryodev-main # Auf cryodev-main
sudo headscale nodes list sudo headscale nodes list
``` ```
### Netdata Streaming ### Netdata-Streaming
Check whether the new client appears in the Netdata dashboard: Prüfe ob der neue Client im Netdata-Dashboard erscheint:
`https://netdata.cryodev.xyz` `https://netdata.cryodev.xyz`
--- ---
## Checklist ## Checkliste
- [ ] Tailscale auth key generated on cryodev-main - [ ] Tailscale Auth-Key auf cryodev-main generiert
- [ ] Host configuration created (template, flake.nix, constants.nix) - [ ] Host-Konfiguration erstellt (Template, flake.nix, constants.nix)
- [ ] Host added to workflow matrix - [ ] Host zur Workflow-Matrix hinzugefügt
- [ ] Pushed and waited for image build - [ ] Gepusht und auf Image-Build gewartet
- [ ] SD card flashed and Pi booted - [ ] SD-Karte geflasht und Pi gebootet
- [ ] Age key determined and added to .sops.yaml - [ ] Age-Key ermittelt und in .sops.yaml eingetragen
- [ ] secrets.yaml created (Tailscale key, Netdata UUID) - [ ] secrets.yaml erstellt (Tailscale-Key, Netdata-UUID)
- [ ] SOPS references activated and deployed - [ ] SOPS-Referenzen aktiviert und deployed
- [ ] Tailscale connection working - [ ] Tailscale-Verbindung funktioniert
- [ ] Netdata streaming working - [ ] Netdata-Streaming funktioniert

134
docs/getting-started/reinstall.md
View file

@ -1,70 +1,70 @@
# Reinstallation # Neuinstallation (Reinstall)
This guide describes the **reinstallation** of an existing host, e.g. after a hardware change or in case of issues. Diese Anleitung beschreibt die **Neuinstallation** eines bestehenden Hosts, z.B. nach Hardwarewechsel oder bei Problemen.
## Difference from Initial Installation ## Unterschied zur Erstinstallation
| Aspect | Initial Installation | Reinstallation | | Aspekt | Erstinstallation | Neuinstallation |
|--------|----------------------|----------------| |--------|------------------|-----------------|
| SOPS Secrets | Not yet present | Already configured | | SOPS-Secrets | Noch nicht vorhanden | Bereits konfiguriert |
| SSH Host Key | Newly generated | **Must be restored!** | | SSH-Host-Key | Neu generiert | **Muss wiederhergestellt werden!** |
| Disk IDs | Newly determined | Often changed (new hardware) | | Disk-IDs | Neu ermitteln | Oft geändert (neue Hardware) |
| secrets.yaml | Will be created | Already exists | | secrets.yaml | Wird erstellt | Bereits vorhanden |
## Important: SSH Host Key Issue ## Wichtig: SSH-Host-Key Problem
During a reinstallation, a **new SSH host key** is generated. This key will no longer match the age key in `.sops.yaml`! Bei einer Neuinstallation wird ein **neuer SSH-Host-Key** generiert. Dieser stimmt nicht mehr mit dem Age-Key in `.sops.yaml` überein!
### Possible Solutions ### Lösungsmöglichkeiten
**Option A: Back up and restore the old host key** (recommended) **Option A: Alten Host-Key sichern und wiederherstellen** (empfohlen)
**Option B: Generate a new key and update SOPS** **Option B: Neuen Key generieren und SOPS aktualisieren**
## Prerequisites ## Voraussetzungen
- Backup of the old SSH host key (if using Option A) - Backup des alten SSH-Host-Keys (falls Option A)
- Access to `.sops.yaml` and the admin age keys - Zugriff auf `.sops.yaml` und die Admin-Age-Keys
- Bootable NixOS ISO - Bootbares NixOS ISO
## Step 1: Preparation (before the installation) ## Schritt 1: Vorbereitung (vor der Installation)
### 1.1 Back Up the Old SSH Host Key (Option A) ### 1.1 Alten SSH-Host-Key sichern (Option A)
If the old host is still running: Falls der alte Host noch läuft:
```bash ```bash
# On the old host # Auf dem alten Host
sudo cat /etc/ssh/ssh_host_ed25519_key > ~/ssh_host_ed25519_key.backup sudo cat /etc/ssh/ssh_host_ed25519_key > ~/ssh_host_ed25519_key.backup
sudo cat /etc/ssh/ssh_host_ed25519_key.pub > ~/ssh_host_ed25519_key.pub.backup sudo cat /etc/ssh/ssh_host_ed25519_key.pub > ~/ssh_host_ed25519_key.pub.backup
``` ```
Copy the files securely to the development machine. Dateien sicher auf den Entwicklungsrechner kopieren.
### 1.2 Determine Disk IDs ### 1.2 Disk-IDs ermitteln
**With new hardware**, the disk IDs will change! **Bei neuer Hardware** ändern sich die Disk-IDs!
```bash ```bash
# In the NixOS live system # Im NixOS Live-System
lsblk -o NAME,SIZE,MODEL,SERIAL lsblk -o NAME,SIZE,MODEL,SERIAL
ls -la /dev/disk/by-id/ ls -la /dev/disk/by-id/
``` ```
Enter the new disk ID in `hosts/<hostname>/disks.sh` or `disks.nix`: Die neue Disk-ID in `hosts/<hostname>/disks.sh` oder `disks.nix` eintragen:
```bash ```bash
# Example disks.sh # Beispiel disks.sh
DISK="/dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_XXXXX" DISK="/dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_XXXXX"
``` ```
## Step 2: Perform the Installation ## Schritt 2: Installation durchführen
### 2.1 Boot the NixOS ISO ### 2.1 NixOS ISO booten
Boot from USB/CD, set a root password, and connect via SSH. Von USB/CD booten, Root-Passwort setzen, per SSH verbinden.
### 2.2 Clone the Repository ### 2.2 Repository klonen
```bash ```bash
sudo -i sudo -i
@ -73,30 +73,30 @@ git clone <GIT_REPO_URL> /tmp/nixos
cd /tmp/nixos cd /tmp/nixos
``` ```
### 2.3 Verify the Disk Configuration ### 2.3 Disk-Konfiguration prüfen
```bash ```bash
# Display current disk IDs # Aktuelle Disk-IDs anzeigen
ls -la /dev/disk/by-id/ ls -la /dev/disk/by-id/
# Compare with the configuration # Mit Konfiguration vergleichen
cat hosts/<hostname>/disks.sh | grep DISK cat hosts/<hostname>/disks.sh | grep DISK
``` ```
**If necessary:** Update the disk ID in the configuration. **Falls nötig:** Disk-ID in der Konfiguration anpassen.
### 2.4 Run the Install Script ### 2.4 Install-Script ausführen
```bash ```bash
bash scripts/install.sh -n <hostname> bash scripts/install.sh -n <hostname>
``` ```
### 2.5 Restore the SSH Host Key (Option A) ### 2.5 SSH-Host-Key wiederherstellen (Option A)
**Before rebooting!** **Vor dem Reboot!**
```bash ```bash
# Restore the host key from backup # Host-Key vom Backup wiederherstellen
cp /path/to/ssh_host_ed25519_key.backup /mnt/etc/ssh/ssh_host_ed25519_key cp /path/to/ssh_host_ed25519_key.backup /mnt/etc/ssh/ssh_host_ed25519_key
cp /path/to/ssh_host_ed25519_key.pub.backup /mnt/etc/ssh/ssh_host_ed25519_key.pub cp /path/to/ssh_host_ed25519_key.pub.backup /mnt/etc/ssh/ssh_host_ed25519_key.pub
chmod 600 /mnt/etc/ssh/ssh_host_ed25519_key chmod 600 /mnt/etc/ssh/ssh_host_ed25519_key
@ -110,75 +110,75 @@ umount -Rl /mnt
reboot reboot
``` ```
## Step 3: After the Reboot ## Schritt 3: Nach dem Reboot
### Option A (Key Restored) ### Bei Option A (Key wiederhergestellt)
SOPS secrets should work automatically. Verify: SOPS-Secrets sollten automatisch funktionieren. Testen:
```bash ```bash
sudo cat /run/secrets/tailscale/auth-key sudo cat /run/secrets/tailscale/auth-key
``` ```
### Option B (New Key) ### Bei Option B (Neuer Key)
The host cannot decrypt the secrets. Configure the new key: Der Host kann die Secrets nicht entschlüsseln. Neuen Key konfigurieren:
```bash ```bash
# Determine the new age key # Neuen Age-Key ermitteln
nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age' nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
``` ```
On the development machine: Auf dem Entwicklungsrechner:
```bash ```bash
# Update .sops.yaml with the new key # .sops.yaml aktualisieren mit neuem Key
vim .sops.yaml vim .sops.yaml
# Re-encrypt secrets with the new key # Secrets mit neuem Key neu verschlüsseln
sops updatekeys hosts/<hostname>/secrets.yaml sops updatekeys hosts/<hostname>/secrets.yaml
``` ```
Then redeploy the configuration: Dann Konfiguration neu deployen:
```bash ```bash
NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \ NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \
--target-host <user>@<IP> --sudo --ask-sudo-password --target-host <user>@<IP> --sudo --ask-sudo-password
``` ```
## Common Issues ## Häufige Probleme
### "No secret key available" ### "No secret key available"
SOPS cannot decrypt the secrets. Cause: SOPS kann die Secrets nicht entschlüsseln. Ursache:
- SSH host key does not match the age key in `.sops.yaml` - SSH-Host-Key stimmt nicht mit Age-Key in `.sops.yaml` überein
Solution: Follow Option B (configure the new key). Lösung: Option B durchführen (neuen Key konfigurieren).
### "Device not found" during partitioning ### "Device not found" beim Partitionieren
The disk ID in `disks.sh`/`disks.nix` is incorrect. Disk-ID in `disks.sh`/`disks.nix` ist falsch.
```bash ```bash
# Find the correct ID # Richtige ID finden
ls -la /dev/disk/by-id/ ls -la /dev/disk/by-id/
``` ```
### Outdated Hardware Config ### Hardware-Config veraltet
With new hardware, `hardware.nix` must be regenerated: Bei neuer Hardware muss `hardware.nix` neu generiert werden:
```bash ```bash
# The install script regenerates automatically if the file is missing # Install-Script generiert automatisch neu, falls Datei fehlt
rm hosts/<hostname>/hardware.nix rm hosts/<hostname>/hardware.nix
bash scripts/install.sh -n <hostname> bash scripts/install.sh -n <hostname>
``` ```
## Checklist ## Checkliste
- [ ] Old SSH host key backed up (if possible) - [ ] Alten SSH-Host-Key gesichert (falls möglich)
- [ ] Disk IDs in configuration verified/updated - [ ] Disk-IDs in Konfiguration geprüft/aktualisiert
- [ ] Installation completed - [ ] Installation durchgeführt
- [ ] SSH host key restored OR new key configured in SOPS - [ ] SSH-Host-Key wiederhergestellt ODER neuen Key in SOPS konfiguriert
- [ ] Secrets are functional (`sudo cat /run/secrets/...`) - [ ] Secrets funktionieren (`sudo cat /run/secrets/...`)
- [ ] Tailscale connected (`tailscale status`) - [ ] Tailscale verbunden (`tailscale status`)

98
docs/getting-started/sd-image.md
View file

@ -1,95 +1,95 @@
# SD Card Images for Raspberry Pi # SD-Karten-Images für Raspberry Pi
The repository automatically builds SD card images for all configured Raspberry Pi hosts. Das Repository baut automatisch SD-Karten-Images für alle konfigurierten Raspberry Pi Hosts.
## Automatic Build ## Automatischer Build
When changes are pushed to `main`, images are automatically built for all Pi hosts and published as a release. Bei Änderungen an `main` werden automatisch Images für alle Pi-Hosts gebaut und als Release veröffentlicht.
**Download:** [Releases on Forgejo](https://git.cryodev.xyz/steffen/cryodev-server/releases) **Download:** [Releases auf Forgejo](https://git.cryodev.xyz/steffen/cryodev-server/releases)
## Available Images ## Verfügbare Images
| Host | Image Name | | Host | Image-Name |
|------|------------| |------|------------|
| `cryodev-pi` | `cryodev-pi-sd-image.img.zst` | | `cryodev-pi` | `cryodev-pi-sd-image.img.zst` |
New hosts are built automatically once they are added to the workflow matrix. Neue Hosts werden automatisch gebaut, wenn sie zur Workflow-Matrix hinzugefügt werden.
## Flashing the Image ## Image flashen
### 1. Download ### 1. Herunterladen
```bash ```bash
wget https://git.cryodev.xyz/.../releases/latest/download/<hostname>-sd-image.img.zst wget https://git.cryodev.xyz/.../releases/latest/download/<hostname>-sd-image.img.zst
wget https://git.cryodev.xyz/.../releases/latest/download/<hostname>-sd-image.img.zst.sha256 wget https://git.cryodev.xyz/.../releases/latest/download/<hostname>-sd-image.img.zst.sha256
# Verify checksum # Checksum prüfen
sha256sum -c <hostname>-sd-image.img.zst.sha256 sha256sum -c <hostname>-sd-image.img.zst.sha256
``` ```
### 2. Decompress ### 2. Dekomprimieren
```bash ```bash
zstd -d <hostname>-sd-image.img.zst -o <hostname>.img zstd -d <hostname>-sd-image.img.zst -o <hostname>.img
``` ```
### 3. Write to SD Card ### 3. Auf SD-Karte schreiben
```bash ```bash
# Identify the correct device # Richtiges Gerät finden
lsblk lsblk
# Write (WARNING: make sure to select the correct device!) # Schreiben (ACHTUNG: richtiges Gerät wählen!)
sudo dd if=<hostname>.img of=/dev/sdX bs=4M conv=fsync status=progress sudo dd if=<hostname>.img of=/dev/sdX bs=4M conv=fsync status=progress
``` ```
Alternatively, use `balenaEtcher` or `Raspberry Pi Imager`. Alternativ: `balenaEtcher` oder `Raspberry Pi Imager` verwenden.
## What Is Included in the Image? ## Was ist im Image?
- Complete NixOS installation for the specific host - Vollständige NixOS-Installation für den spezifischen Host
- All configured services (except secrets) - Alle konfigurierten Services (außer Secrets)
- SSH server enabled - SSH-Server aktiviert
- Automatic root partition expansion on first boot - Automatische Root-Partition-Erweiterung beim ersten Boot
- Comin for automatic updates - Comin für automatische Updates
## What Is Missing? ## Was fehlt?
**SOPS secrets** cannot be included in the image (chicken-and-egg problem with the SSH host key). **SOPS-Secrets** können nicht im Image enthalten sein (Henne-Ei-Problem mit SSH-Host-Key).
After the first boot: Nach dem ersten Boot:
1. Retrieve the age key from the Pi 1. Age-Key vom Pi holen
2. Update `.sops.yaml` 2. `.sops.yaml` aktualisieren
3. Create `secrets.yaml` 3. `secrets.yaml` erstellen
4. Deploy the configuration 4. Konfiguration deployen
See [Adding a New Client](new-client.md) for the complete guide. Siehe [Neuen Client hinzufügen](new-client.md) für die vollständige Anleitung.
## Adding a New Host to the Pipeline ## Neuen Host zur Pipeline hinzufügen
1. Create the host configuration in `hosts/<hostname>/` 1. Host-Konfiguration in `hosts/<hostname>/` erstellen
2. Add it to the matrix in `.forgejo/workflows/build-pi-image.yml`: 2. In `.forgejo/workflows/build-pi-image.yml` zur Matrix hinzufügen:
```yaml ```yaml
matrix: matrix:
host: [cryodev-pi, new-host] # <- add here host: [cryodev-pi, neuer-host] # <- hier hinzufügen
``` ```
3. Push to `main` -- the image will be built automatically 3. Push auf `main` → Image wird automatisch gebaut
## Building Manually ## Manuell bauen
```bash ```bash
# On aarch64 (e.g., another Pi) # Auf aarch64 (z.B. anderem Pi)
nix build .#nixosConfigurations.<hostname>.config.system.build.sdImage nix build .#nixosConfigurations.<hostname>.config.system.build.sdImage
# On x86_64 with QEMU emulation (slow) # Auf x86_64 mit QEMU-Emulation (langsam)
nix build .#nixosConfigurations.<hostname>.config.system.build.sdImage \ nix build .#nixosConfigurations.<hostname>.config.system.build.sdImage \
--extra-platforms aarch64-linux --extra-platforms aarch64-linux
``` ```
Prerequisite on x86_64: Voraussetzung auf x86_64:
```nix ```nix
{ {
@ -99,18 +99,18 @@ Prerequisite on x86_64:
## Troubleshooting ## Troubleshooting
### Workflow Fails ### Workflow schlägt fehl
- Check whether `sd-image.nix` is imported in the host configuration - Prüfe ob `sd-image.nix` in der Host-Konfiguration importiert wird
- Check whether binfmt is enabled on cryodev-main - Prüfe ob binfmt auf cryodev-main aktiviert ist
### Image Does Not Boot ### Image bootet nicht
- Was the SD card written correctly? - SD-Karte korrekt beschrieben?
- Try a different SD card - Andere SD-Karte versuchen
- Check the power supply (minimum 3A for Pi 4) - Stromversorgung prüfen (min. 3A für Pi 4)
### No Network ### Kein Netzwerk
- Check the Ethernet cable - Ethernet-Kabel prüfen
- Is there a DHCP server on the network? - DHCP-Server im Netzwerk?

76
docs/index.md
View file

@ -1,33 +1,33 @@
# Cryodev NixOS Configuration Documentation # Cryodev NixOS Configuration Documentation
Welcome to the documentation for the **cryodev** NixOS infrastructure. Willkommen zur Dokumentation der **cryodev** NixOS-Infrastruktur.
## Quick Links ## Quick Links
### Getting Started ### Getting Started
- [Prerequisites](getting-started/prerequisites.md) - Required tools - [Voraussetzungen](getting-started/prerequisites.md) - Benötigte Tools
- [Adding a New Raspberry Pi](getting-started/new-client.md) - Complete workflow for new clients - [Neuen Raspberry Pi hinzufügen](getting-started/new-client.md) - Kompletter Workflow für neue Clients
- [SD Image Reference](getting-started/sd-image.md) - Details on image building - [SD-Image Referenz](getting-started/sd-image.md) - Details zum Image-Build
- [First Installation (Server)](getting-started/first-install.md) - Bootstrap for x86_64 hosts - [Erstinstallation (Server)](getting-started/first-install.md) - Bootstrap für x86_64 Hosts
- [Reinstallation](getting-started/reinstall.md) - Reinstall with hardware changes - [Neuinstallation](getting-started/reinstall.md) - Reinstall mit Hardware-Änderungen
### Services ### Services
- [SOPS Secrets](services/sops.md) - Secret management with sops-nix - [SOPS Secrets](services/sops.md) - Geheimnisverwaltung mit sops-nix
- [Headscale](services/headscale.md) - Self-hosted Tailscale server - [Headscale](services/headscale.md) - Self-hosted Tailscale Server
- [Headplane](services/headplane.md) - Web UI for Headscale - [Headplane](services/headplane.md) - Web-UI für Headscale
- [Tailscale](services/tailscale.md) - Mesh VPN client - [Tailscale](services/tailscale.md) - Mesh-VPN Client
- [Mailserver](services/mailserver.md) - Email stack (Postfix/Dovecot) - [Mailserver](services/mailserver.md) - E-Mail Stack (Postfix/Dovecot)
- [Forgejo](services/forgejo.md) - Git hosting with CI/CD - [Forgejo](services/forgejo.md) - Git-Hosting mit CI/CD
- [Netdata](services/netdata.md) - Monitoring and alerting - [Netdata](services/netdata.md) - Monitoring und Alerting
### Deployment ### Deployment
- [Continuous Deployment](deployment/cd.md) - Push- and pull-based deployment - [Continuous Deployment](deployment/cd.md) - Push- und Pull-basiertes Deployment
- [DNS Configuration](deployment/dns.md) - Required DNS records - [DNS-Konfiguration](deployment/dns.md) - Benötigte DNS-Einträge
## Architecture ## Architektur
``` ```
Internet Internet
@ -57,38 +57,38 @@ Welcome to the documentation for the **cryodev** NixOS infrastructure.
+-------------------+ +-------------------+
``` ```
## Installation Scenarios ## Installations-Szenarien
| Scenario | Description | Guide | | Szenario | Beschreibung | Anleitung |
|----------|-------------|-------| |----------|--------------|-----------|
| **New Raspberry Pi** | Create config, build image, flash | [new-client.md](getting-started/new-client.md) | | **Neuer Raspberry Pi** | Config erstellen → Image bauen → Flashen | [new-client.md](getting-started/new-client.md) |
| **First Installation (Server)** | x86_64 host, manual installation | [first-install.md](getting-started/first-install.md) | | **Erstinstallation (Server)** | x86_64 Host, manuelle Installation | [first-install.md](getting-started/first-install.md) |
| **Reinstallation** | Existing host, new hardware | [reinstall.md](getting-started/reinstall.md) | | **Neuinstallation** | Bestehender Host, neue Hardware | [reinstall.md](getting-started/reinstall.md) |
For Raspberry Pi: [SD Image Reference](getting-started/sd-image.md) Für Raspberry Pi: [SD-Image Referenz](getting-started/sd-image.md)
## Directory Structure ## Verzeichnisstruktur
``` ```
. .
├── flake.nix # Entry point, inputs and outputs ├── flake.nix # Entry point, inputs and outputs
├── constants.nix # Central configuration (domains, IPs, ports) ├── constants.nix # Zentrale Config (Domains, IPs, Ports)
├── hosts/ # Host-specific configurations ├── hosts/ # Host-spezifische Konfigurationen
│ ├── cryodev-main/ │ ├── cryodev-main/
│ └── cryodev-pi/ │ └── cryodev-pi/
├── modules/ # Reusable NixOS modules ├── modules/ # Wiederverwendbare NixOS-Module
│ └── nixos/ │ └── nixos/
├── pkgs/ # Custom packages ├── pkgs/ # Eigene Pakete
├── overlays/ # Nixpkgs overlays ├── overlays/ # Nixpkgs Overlays
├── templates/ # Templates for new hosts ├── templates/ # Templates für neue Hosts
├── scripts/ # Helper scripts (install.sh) ├── scripts/ # Helper-Scripts (install.sh)
├── apps/ # Nix apps (rebuild) ├── apps/ # Nix Apps (rebuild)
└── lib/ # Helper functions (utils.nix) └── lib/ # Helper-Funktionen (utils.nix)
``` ```
## Deployment Strategies ## Deployment-Strategien
| Host | Strategy | Tool | Description | | Host | Strategie | Tool | Beschreibung |
|------|----------|------|-------------| |------|-----------|------|--------------|
| `cryodev-main` | Pull-based | Comin | Polls the repository for changes | | `cryodev-main` | Pull-basiert | Comin | Pollt Repository auf Aenderungen |
| `cryodev-pi` | Pull-based | Comin | Polls the repository for changes | | `cryodev-pi` | Pull-basiert | Comin | Pollt Repository auf Aenderungen |

4
docs/services/forgejo.md
View file

@ -89,9 +89,9 @@ See [CD documentation](../deployment/cd.md) for details.
```bash ```bash
forgejo admin user create \ forgejo admin user create \
--username <username> \ --username <benutzername> \
--email <email>@<domain> \ --email <email>@<domain> \
--password <password> \ --password <passwort> \
--admin --admin
``` ```

4
docs/services/tailscale.md
View file

@ -14,9 +14,9 @@ Tailscale clients connect to the self-hosted Headscale server to join the mesh V
On the Headscale server (cryodev-main): On the Headscale server (cryodev-main):
```bash ```bash
# Look up user ID # User-ID ermitteln
sudo headscale users list sudo headscale users list
# Create preauth key (use the user ID for "default") # Preauth-Key erstellen (User-ID von "default" einsetzen)
sudo headscale preauthkeys create --expiration 99y --reusable --user <ID> sudo headscale preauthkeys create --expiration 99y --reusable --user <ID>
``` ```