Compare commits

..

No commits in common. "c141e22febe8251837b1ae27d3a10d1784772c34" and "c81b43530aa62a41d467eb7b602a8ab53a2afcd6" have entirely different histories.

10 changed files with 1437 additions and 1680 deletions

231
AGENTS.md
View file

@ -2,33 +2,37 @@
## Project Overview
NixOS infrastructure managed with Nix Flakes. Two hosts, reusable modules, SOPS secrets, Comin auto-deployment.
- **Hosts**: `cryodev-main` (x86_64 server), `cryodev-pi` (aarch64 Raspberry Pi 4)
This repository contains a NixOS configuration managed with Nix Flakes. It defines:
- **Hosts**: `cryodev-main` (x86_64 server), `cryodev-pi` (aarch64 Raspberry Pi)
- **Modules**: Reusable NixOS modules in `modules/nixos/`
- **Apps**: `create`, `deploy`, `install`, `rebuild` in `apps/`
- **Packages**: Custom packages in `pkgs/`
- **Templates**: `raspberry-pi`, `generic-server` for bootstrapping new hosts
## Build & Development Commands
### Prerequisites
- **Nix** with Flakes enabled
- **Git**
### Core Commands
```bash
# Format code (required before committing, runs nixfmt via pre-commit)
nix fmt
# Run all checks (formatting, package builds, overlay builds)
nix flake check
# Quick evaluation test (faster than full build, use to validate changes)
nix eval .#nixosConfigurations.cryodev-main.config.system.build.toplevel.name
nix eval .#nixosConfigurations.cryodev-pi.config.system.build.toplevel.name
# Full build
# Build host configuration
nix build .#nixosConfigurations.cryodev-main.config.system.build.toplevel
nix build .#nixosConfigurations.cryodev-pi.config.system.build.toplevel
# Build Raspberry Pi SD image (requires binfmt on x86_64)
nix build .#nixosConfigurations.cryodev-pi.config.system.build.sdImage
# Format code (required before committing)
nix fmt
# Run all checks (lint, formatting)
nix flake check
# Quick evaluation test (faster than full build)
nix eval .#nixosConfigurations.cryodev-main.config.system.build.toplevel.name
# Update flake inputs
nix flake update
@ -38,143 +42,176 @@ nix develop
### Deployment
Both hosts use **Comin** for automatic pull-based deployment (polls the git repo).
Manual deployment is only needed for initial setup or emergencies:
```bash
# Deploy via deploy app (uses deploy.json, SSH port 2299, asks sudo password)
# Deploy all hosts via deploy app (uses deploy.json)
nix run .#deploy
# Deploy a specific host
nix run .#deploy -- -n cryodev-main
# Manual deployment via nixos-rebuild
# Manual deployment via SSH
NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \
--target-host <user>@<ip> --sudo --ask-sudo-password
```
> **Note:** Both hosts use Comin for automatic pull-based deployment.
> Manual deployment is only needed for the initial setup or emergencies.
### Apps
```bash
nix run .#create -- -t generic-server -n <hostname> # Scaffold new host
nix run .#install -- -n <hostname> -r <REPO_URL> # Install from NixOS ISO
nix run .#deploy -- -n <hostname> # Deploy to host
nix run .#rebuild -- nixos # Rebuild locally
# Create a new host from template
nix run .#create -- -t generic-server -n <hostname>
# Install NixOS on a new machine (run from NixOS live ISO)
nix run .#install -- -n <hostname> -r <REPO_URL>
# Deploy to all configured hosts
nix run .#deploy
# Rebuild NixOS/Home Manager configuration
nix run .#rebuild -- nixos
```
## Code Style & Conventions
### Formatting
- **Tool**: `nixfmt` via `git-hooks.nix` (pre-commit)
- **Tool**: `nixfmt` via pre-commit hooks
- **Run**: `nix fmt` before every commit
- **Indentation**: 2 spaces (enforced by formatter)
- **Indentation**: 2 spaces
- **Line length**: 80-100 characters (follow formatter)
### Naming Conventions
| Type | Convention | Example |
|------|------------|---------|
| Files | kebab-case | `hardware-configuration.nix` |
| NixOS options | camelCase | `services.myService.enable` |
| Let bindings | camelCase | `let myValue = ...;` |
| Hosts | kebab-case | `cryodev-main`, `cryodev-pi` |
| Secret paths | kebab-case with `/` | `forgejo-runner/token` |
### Module Pattern
### Module Structure
```nix
{ config, lib, ... }:
# Standard module pattern
{ config, lib, pkgs, inputs, outputs, constants, ... }:
let
cfg = config.services.myService;
inherit (lib) mkDefault mkEnableOption mkIf mkOption types;
in
{
options.services.myService = {
enable = mkEnableOption "My service";
port = mkOption {
type = types.port;
enable = lib.mkEnableOption "My service";
port = lib.mkOption {
type = lib.types.port;
default = 8080;
description = "Port to listen on";
};
};
config = mkIf cfg.enable {
assertions = [
{ assertion = cfg.port > 1024; message = "Port must be > 1024"; }
];
# Implementation
config = lib.mkIf cfg.enable {
# Implementation here
};
}
```
### Key Rules
### Naming Conventions
| Type | Convention | Example |
|------|------------|---------|
| Files | kebab-case | `hardware-configuration.nix` |
| Options | camelCase | `services.myService.enable` |
| Variables | camelCase | `let myValue = ...;` |
| Hosts | kebab-case | `cryodev-main`, `cryodev-pi` |
- **Use `lib.mkDefault`** for all module defaults (allows host-level overrides)
- **Use `constants.nix`** for domains, IPs, ports -- never hardcode these
- **Use `lib.utils`** helpers: `mkReverseProxyOption`, `mkVirtualHost`, `mkUrl`
- **Secrets via SOPS** only, never plaintext. Reference: `config.sops.secrets."path".path`
- **Imports**: relative paths for local files, `outputs.nixosModules.*` for shared modules, `inputs.*` for external
- **Assertions** for invalid configurations, `warnings` for non-critical issues
- **`lib.inherit`** pattern: extract needed functions in `let` block
### Host Service Files
Each service gets its own file in `hosts/<host>/services/`:
### Imports
```nix
# hosts/cryodev-main/services/myservice.nix
{ outputs, constants, ... }:
# Local modules: relative paths
imports = [ ./hardware.nix ./networking.nix ];
# Shared modules: via outputs
imports = [ outputs.nixosModules.common ];
# External inputs
imports = [ inputs.sops-nix.nixosModules.sops ];
```
### Constants
Use `constants.nix` for domains, IPs, and ports:
```nix
{ constants, ... }:
{
imports = [ outputs.nixosModules.myservice ];
services.myservice = {
enable = true;
port = constants.services.myservice.port;
};
services.nginx.virtualHosts."${constants.services.myservice.fqdn}" = {
forceSSL = true;
enableACME = true;
locations."/" = {
proxyPass = "http://127.0.0.1:${toString constants.services.myservice.port}";
};
};
services.nginx.virtualHosts."${constants.services.forgejo.fqdn}" = { ... };
}
```
### Special Args Available in Modules
### Error Handling
```nix
config = lib.mkIf cfg.enable {
assertions = [
{ assertion = cfg.port > 1024; message = "Port must be > 1024"; }
];
warnings = lib.optional (cfg.debug) "Debug mode enabled!";
};
```
- `inputs` -- flake inputs (nixpkgs, sops-nix, comin, headplane, etc.)
- `outputs` -- this flake's outputs (nixosModules, packages)
- `constants` -- values from `constants.nix` (domain, hosts, services)
- `lib` -- nixpkgs.lib extended with `lib.utils`
### Option Conflicts
Use `lib.mkDefault` for default values that can be overridden:
```nix
services.nginx.enable = lib.mkDefault true;
```
## Directory Structure
```
.
├── flake.nix # Entry point, inputs/outputs, mkNixosConfiguration
├── flake.nix # Entry point, inputs/outputs
├── constants.nix # Central config (domains, IPs, ports)
├── hosts/
│ ├── cryodev-main/ # x86_64 server (services/, secrets.yaml, binfmt.nix)
│ └── cryodev-pi/ # aarch64 RPi (services/, secrets.yaml, sd-image.nix)
├── modules/nixos/ # Reusable modules (common, forgejo, headscale, ...)
├── users/ # User definitions (steffen, ralph, benjamin)
├── apps/ # Nix apps (create, deploy, install, rebuild)
├── lib/utils.nix # Helper functions (mkUrl, mkVirtualHost, ...)
│ ├── cryodev-main/ # x86_64 server
│ │ ├── default.nix # Host entry point
│ │ ├── hardware.nix # Hardware configuration
│ │ ├── services/ # Service configurations
│ │ └── secrets.yaml # SOPS-encrypted secrets
│ └── cryodev-pi/ # aarch64 Raspberry Pi
├── modules/nixos/ # Reusable modules
│ ├── common/ # Shared base configuration
│ ├── sops/ # Secret management
│ ├── forgejo/ # Git server
│ ├── headscale/ # VPN control server
│ └── ...
├── lib/utils.nix # Helper functions
├── apps/ # Nix apps (rebuild)
├── pkgs/ # Custom packages
├── overlays/ # Nixpkgs overlays
├── templates/ # Host templates (generic-server, raspberry-pi)
├── deploy.json # Deploy app config (hosts, SSH port)
├── .sops.yaml # SOPS encryption rules (age keys per host)
├── .forgejo/workflows/ # CI pipelines (ci.yml, deploy.yml)
└── docs/ # Documentation (English)
├── templates/ # Host templates
├── scripts/ # Helper scripts (install.sh)
└── docs/ # Documentation
```
## Key Patterns
### Adding a New Raspberry Pi Host
1. Copy template: `cp -r templates/raspberry-pi hosts/new-pi`
2. Update `hosts/new-pi/networking.nix` (hostname)
3. Add to `flake.nix`: `new-pi = mkNixosConfiguration "aarch64-linux" [ ./hosts/new-pi ];`
4. Add to `.forgejo/workflows/build-pi-image.yml` matrix
5. Push → SD image is built automatically
### SOPS Secrets
- Secrets encrypted with age using SSH host keys
- Config in `.sops.yaml`, secrets in `hosts/<host>/secrets.yaml`
- Reference: `config.sops.secrets."path/to/secret".path`
### Special Args Available in Modules
- `inputs`: Flake inputs (nixpkgs, sops-nix, etc.)
- `outputs`: This flake's outputs (nixosModules, packages)
- `constants`: Values from `constants.nix`
- `lib`: Extended nixpkgs.lib with `lib.utils`
## Deployment Workflows
| Host | Strategy | Trigger |
|------|----------|---------|
| `cryodev-main` | Pull via Comin | Automatic polling |
| `cryodev-pi` | Pull via Comin | Automatic polling |
| SD Images | Built in CI | Push to main (for Pi hosts) |
## Verification Checklist
Before committing:
- [ ] `nix fmt` passes
- [ ] `nix flake check` passes (or at least `nix eval` works for both hosts)
- [ ] New hosts added to `flake.nix` nixosConfigurations
- [ ] `nix flake check` passes (or at least `nix eval` works)
- [ ] New hosts added to `flake.nix`
- [ ] Constants in `constants.nix`, not hardcoded
- [ ] Secrets use SOPS, not plaintext
- [ ] New services have their own file in `hosts/<host>/services/`
- [ ] New modules registered in `modules/nixos/default.nix`
- [ ] Documentation in English

2114
digest.txt

File diff suppressed because it is too large Load diff

View file

@ -31,12 +31,12 @@ Required DNS records for the cryodev infrastructure.
| `@` | MX | `10 mail.cryodev.xyz.` | Mail delivery |
| `@` | TXT | `"v=spf1 mx ~all"` | SPF |
| `_dmarc` | TXT | `"v=DMARC1; p=none"` | DMARC |
| `mail._domainkey` | TXT | *(see below)* | DKIM |
| `mail._domainkey` | TXT | *(siehe unten)* | DKIM |
### Reverse DNS (PTR)
For reliable mail delivery, a **PTR record** must be configured at the hosting
provider (not in the domain's DNS panel):
Fuer zuverlaessige Mail-Zustellung muss ein **PTR Record** beim Hosting-Provider
konfiguriert werden (nicht im DNS-Panel der Domain):
| IP | PTR Value |
|----|-----------|
@ -45,18 +45,18 @@ provider (not in the domain's DNS panel):
#### Hetzner Robot (Dedicated Server)
1. [robot.hetzner.com](https://robot.hetzner.com) > **Server** > Select the server
2. **IPs** tab
3. Click the **pencil icon** next to the IPv4 address
4. Enter `mail.cryodev.xyz` and save
5. For IPv6: Under **Subnets**, repeat the same for the primary IPv6 address
1. [robot.hetzner.com](https://robot.hetzner.com) > **Server** > Server auswaehlen
2. **IPs** Tab
3. Bei der IPv4-Adresse auf das **Stift-Symbol** klicken
4. `mail.cryodev.xyz` eintragen und speichern
5. Fuer IPv6: Unter **Subnets** dasselbe fuer die primaere IPv6-Adresse
#### Hetzner Cloud
1. [cloud.hetzner.com](https://cloud.hetzner.com) > Select the server
2. **Networking** tab
3. Under "Primary IP", click the IP > **Reverse DNS**
4. Enter `mail.cryodev.xyz` (for both IPv4 and IPv6)
1. [cloud.hetzner.com](https://cloud.hetzner.com) > Server auswaehlen
2. **Networking** Tab
3. Bei "Primary IP" auf die IP klicken > **Reverse DNS**
4. `mail.cryodev.xyz` eintragen (fuer IPv4 und IPv6)
## Getting the DKIM Key
@ -70,15 +70,15 @@ Add this as a TXT record for `mail._domainkey.cryodev.xyz`.
## Complete Checklist
- [ ] A/AAAA for `@` (root domain)
- [ ] A/AAAA for `www`
- [ ] A/AAAA for `mail`
- [ ] CNAME for `git`, `headscale`, `headplane`, `netdata`
- [ ] MX record
- [ ] TXT for SPF (`v=spf1 mx ~all`)
- [ ] TXT for DMARC (`v=DMARC1; p=none`)
- [ ] TXT for DKIM (`mail._domainkey` -- after first deployment)
- [ ] PTR record at hosting provider (reverse DNS)
- [ ] A/AAAA fuer `@` (Root-Domain)
- [ ] A/AAAA fuer `www`
- [ ] A/AAAA fuer `mail`
- [ ] CNAME fuer `git`, `headscale`, `headplane`, `netdata`
- [ ] MX Record
- [ ] TXT fuer SPF (`v=spf1 mx ~all`)
- [ ] TXT fuer DMARC (`v=DMARC1; p=none`)
- [ ] TXT fuer DKIM (`mail._domainkey` -- nach erstem Deploy)
- [ ] PTR Record beim Hosting-Provider (Reverse DNS)
## Verification

View file

@ -1,44 +1,44 @@
# Initial Installation (x86_64 Server)
# Erstinstallation (x86_64 Server)
This guide describes the **initial installation** of a new x86_64 server (e.g. cryodev-main).
Diese Anleitung beschreibt die **Erstinstallation** eines neuen x86_64 Servers (z.B. cryodev-main).
> **For Raspberry Pi:** See [Creating an SD Image](sd-image.md).
> **Fuer Raspberry Pi:** Siehe [SD-Image erstellen](sd-image.md).
## Overview
## Uebersicht
During initial installation there is a chicken-and-egg problem:
- SOPS secrets are encrypted with the SSH host key
- The SSH host key is only generated during installation
- Therefore: **Install without secrets first, then configure secrets**
Bei der Erstinstallation gibt es ein Henne-Ei-Problem:
- SOPS-Secrets werden mit dem SSH-Host-Key verschluesselt
- Der SSH-Host-Key wird erst bei der Installation generiert
- Daher: **Erst ohne Secrets installieren, dann Secrets konfigurieren**
### Process
### Ablauf
```
1. Disable services (that require secrets)
2. Install NixOS
3. Extract SSH host key, configure SOPS, create immediately available secrets
4. Enable stage-1 services and deploy (Headscale, Forgejo, Mail, Nginx)
5. Generate remaining secrets (Tailscale, Headplane, Forgejo Runner)
6. Enable stage-2 services and perform final deployment
1. Services deaktivieren (die Secrets brauchen)
2. NixOS installieren
3. SSH-Host-Key extrahieren, SOPS konfigurieren, sofort erstellbare Secrets anlegen
4. Stufe-1-Services aktivieren und deployen (Headscale, Forgejo, Mail, Nginx)
5. Restliche Secrets generieren (Tailscale, Headplane, Forgejo-Runner)
6. Stufe-2-Services aktivieren und final deployen
```
## Step 1: Prepare Host Configuration
## Schritt 1: Host-Konfiguration vorbereiten
> If the host already exists in `hosts/` and `flake.nix`, skip 1.1-1.2.
> Falls der Host bereits in `hosts/` und `flake.nix` existiert, ueberspringe 1.1-1.2.
### 1.1 Create Host from Template
### 1.1 Host aus Template erstellen
```bash
nix run .#create -- -t generic-server -n <hostname>
```
The script:
- Copies the template to `hosts/<hostname>/`
- Sets the hostname in `networking.nix`
- Creates an empty `secrets.yaml`
- Adds the files to Git
Das Script:
- Kopiert das Template nach `hosts/<hostname>/`
- Setzt den Hostname in `networking.nix`
- Erstellt eine leere `secrets.yaml`
- Fuegt die Dateien zu Git hinzu
### 1.2 Register in flake.nix
### 1.2 In flake.nix registrieren
```nix
nixosConfigurations = {
@ -46,24 +46,24 @@ nixosConfigurations = {
};
```
Also adjust `hardware.nix` and `disks.sh` for the target hardware.
Ausserdem `hardware.nix` und `disks.sh` fuer die Zielhardware anpassen.
### 1.4 Temporarily Disable Services
### 1.4 Services temporaer deaktivieren
All services that reference SOPS secrets must be disabled for the initial installation. Otherwise the installation will fail because the secrets cannot yet be decrypted.
Alle Services, die SOPS-Secrets referenzieren, muessen fuer die Erstinstallation deaktiviert werden. Andernfalls schlaegt die Installation fehl, weil die Secrets noch nicht entschluesselt werden koennen.
In `hosts/<hostname>/services/default.nix`, comment out the corresponding imports:
In `hosts/<hostname>/services/default.nix` die entsprechenden Imports auskommentieren:
```nix
{
imports = [
# Disabled until SOPS secrets are configured:
# ./forgejo.nix # requires: forgejo-runner/token, forgejo/mail-pw
# ./headplane.nix # requires: headplane/cookie_secret, headplane/agent_pre_authkey
# ./mailserver.nix # requires: mailserver/accounts/*
# ./tailscale.nix # requires: tailscale/auth-key
# Deaktiviert bis SOPS-Secrets konfiguriert sind:
# ./forgejo.nix # braucht: forgejo-runner/token, forgejo/mail-pw
# ./headplane.nix # braucht: headplane/cookie_secret, headplane/agent_pre_authkey
# ./mailserver.nix # braucht: mailserver/accounts/*
# ./tailscale.nix # braucht: tailscale/auth-key
# These services do not require secrets:
# Diese Services brauchen keine Secrets:
./headscale.nix
./netdata.nix
./nginx.nix
@ -73,7 +73,7 @@ In `hosts/<hostname>/services/default.nix`, comment out the corresponding import
}
```
Additionally, in `hosts/<hostname>/services/sops.nix`, comment out the secret definitions:
Zusaetzlich in `hosts/<hostname>/services/sops.nix` die Secrets-Definitionen auskommentieren:
```nix
sops = {
@ -85,32 +85,32 @@ sops = {
};
```
### 1.5 Test the Configuration
### 1.5 Konfiguration testen
```bash
nix eval .#nixosConfigurations.<hostname>.config.system.build.toplevel.name
```
## Step 2: Perform Installation
## Schritt 2: Installation durchfuehren
### 2.1 Boot NixOS ISO
### 2.1 NixOS ISO booten
Boot from the [NixOS Minimal ISO](https://nixos.org/download/#nixos-iso) (USB/CD).
Vom [NixOS Minimal ISO](https://nixos.org/download/#nixos-iso) booten (USB/CD).
### 2.2 Set Up Network and SSH
### 2.2 Netzwerk und SSH einrichten
```bash
passwd # Set root password for SSH access
ip a # Determine IP address
passwd # Root-Passwort setzen fuer SSH-Zugang
ip a # IP-Adresse ermitteln
```
Optionally connect via SSH (more convenient):
Optional per SSH verbinden (bequemer):
```bash
ssh -o StrictHostKeyChecking=no root@<IP>
```
### 2.3 Install
### 2.3 Installieren
```bash
nix --experimental-features "nix-command flakes" run \
@ -119,20 +119,20 @@ nix --experimental-features "nix-command flakes" run \
-r <REPO_URL>
```
Alternatively, if the repository has already been cloned to `/tmp/nixos`:
Alternativ, falls das Repository bereits unter `/tmp/nixos` geklont wurde:
```bash
nix --experimental-features "nix-command flakes" run /tmp/nixos#install -- -n <hostname>
```
> **Note:** The disk ID in `hosts/<hostname>/disks.sh` must match the hardware.
> Verify with `ls -la /dev/disk/by-id/`.
> **Hinweis:** Die Disk-ID in `hosts/<hostname>/disks.sh` muss zur Hardware passen.
> Pruefen mit `ls -la /dev/disk/by-id/`.
The script:
1. Clones the repository (when using `-r`)
2. Partitions the disk (via `disks.nix` or `disks.sh`)
3. Generates `hardware.nix` (if not present)
4. Installs NixOS
Das Script:
1. Klont das Repository (bei `-r`)
2. Partitioniert die Disk (via `disks.nix` oder `disks.sh`)
3. Generiert `hardware.nix` (falls nicht vorhanden)
4. Installiert NixOS
### 2.4 Reboot
@ -140,34 +140,34 @@ The script:
reboot
```
## Step 3: Configure SOPS Secrets
## Schritt 3: SOPS-Secrets konfigurieren
After the first boot, log in (password: `changeme`, change immediately with `passwd`).
Nach dem ersten Boot einloggen (Passwort: `changeme`, sofort aendern mit `passwd`).
### 3.1 Convert SSH Host Key to Age Key
### 3.1 SSH-Host-Key zu Age-Key konvertieren
On the **new server**:
Auf dem **neuen Server**:
```bash
nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
```
Note the output (e.g. `age1abc123...`).
Ausgabe notieren (z.B. `age1abc123...`).
Alternatively, remotely:
Alternativ remote:
```bash
nix-shell -p ssh-to-age --run 'ssh-keyscan -p 2299 -t ed25519 <IP> | ssh-to-age'
```
### 3.2 Update .sops.yaml
### 3.2 .sops.yaml aktualisieren
On the **development machine**, add the new host key to `.sops.yaml`:
Auf dem **Entwicklungsrechner** den neuen Host-Key in `.sops.yaml` eintragen:
```yaml
keys:
- &steffen_key age1e8p... # steffen (local)
- &hostname_key age1abc... # Key from step 3.1
- &steffen_key age1e8p... # steffen (lokal)
- &hostname_key age1abc... # Key von Schritt 3.1
creation_rules:
- path_regex: hosts/<hostname>/secrets.yaml$
@ -177,38 +177,38 @@ creation_rules:
- *hostname_key
```
### 3.3 Create Secrets
### 3.3 Secrets erstellen
Open the secrets file:
Secrets-Datei oeffnen:
```bash
sops hosts/<hostname>/secrets.yaml
```
The following table shows all secrets for **cryodev-main** and how they are generated:
Die folgende Tabelle zeigt alle Secrets fuer **cryodev-main** und wie sie generiert werden:
#### Immediately Available Secrets
#### Sofort erstellbare Secrets
These secrets have no dependencies and can be generated directly:
Diese Secrets haben keine Abhaengigkeiten und koennen direkt generiert werden:
| Secret | Command |
|--------|---------|
| Secret | Befehl |
|--------|--------|
| `headplane/cookie_secret` | `openssl rand -hex 16` |
| `mailserver/accounts/admin` | `mkpasswd -sm bcrypt` (remember the password!) |
| `mailserver/accounts/forgejo` | `mkpasswd -sm bcrypt` (remember the password!) |
| `forgejo/mail-pw` | Plaintext password matching the bcrypt hash of `mailserver/accounts/forgejo` |
| `mailserver/accounts/admin` | `mkpasswd -sm bcrypt` (Passwort merken!) |
| `mailserver/accounts/forgejo` | `mkpasswd -sm bcrypt` (Passwort merken!) |
| `forgejo/mail-pw` | Klartext-Passwort das zum bcrypt-Hash von `mailserver/accounts/forgejo` passt |
#### Secrets That Require Running Services
#### Secrets die laufende Services brauchen
These secrets can only be created after step 4. **Do not add them yet** -- they will be added later.
Diese Secrets koennen erst nach Schritt 4 erstellt werden. **Jetzt noch nicht eintragen** -- sie werden spaeter ergaenzt.
| Secret | Command | Prerequisite |
|--------|---------|--------------|
| `tailscale/auth-key` | See steps 4.1-4.2 | Headscale is running |
| `headplane/agent_pre_authkey` | See steps 4.1-4.2 | Headscale is running |
| `forgejo-runner/token` | Forgejo Admin Panel > Actions > Runners > Create Runner | Forgejo is running |
| Secret | Befehl | Voraussetzung |
|--------|--------|---------------|
| `tailscale/auth-key` | Siehe Schritt 4.1-4.2 | Headscale laeuft |
| `headplane/agent_pre_authkey` | Siehe Schritt 4.1-4.2 | Headscale laeuft |
| `forgejo-runner/token` | Forgejo Admin Panel > Actions > Runners > Create Runner | Forgejo laeuft |
#### Example secrets.yaml (Plaintext Before Encryption)
#### Beispiel secrets.yaml (Klartext vor Verschluesselung)
```yaml
headplane:
@ -218,22 +218,22 @@ mailserver:
admin: "$2b$05$..."
forgejo: "$2b$05$..."
forgejo:
mail-pw: "the-plaintext-password"
mail-pw: "das-klartext-passwort"
```
### 3.4 Gradually Re-enable Services -- Stage 1
### 3.4 Services stufenweise reaktivieren -- Stufe 1
> **Important:** Services that require Headscale or Forgejo secrets (Tailscale,
> Headplane, Forgejo Runner) must **not** be enabled yet, as these
> secrets can only be generated once those services are running.
> **Wichtig:** Services die Headscale- oder Forgejo-Secrets brauchen (Tailscale,
> Headplane, Forgejo-Runner) duerfen noch **nicht** aktiviert werden, da diese
> Secrets erst generiert werden koennen, wenn die Services laufen.
On the **development machine**, in `hosts/<hostname>/services/default.nix`, enable
the services **without external dependencies**:
Auf dem **Entwicklungsrechner** in `hosts/<hostname>/services/default.nix` die
Services **ohne externe Abhaengigkeiten** aktivieren:
```nix
{
imports = [
# Stage 1: Services without external dependencies
# Stufe 1: Services ohne externe Abhaengigkeiten
./forgejo.nix
./headscale.nix
./mailserver.nix
@ -242,88 +242,88 @@ the services **without external dependencies**:
./openssh.nix
./sops.nix
# Stage 2: Enable only after step 4
# ./forgejo-runner.nix # requires: forgejo-runner/token (Forgejo)
# ./headplane.nix # requires: headplane/agent_pre_authkey (Headscale)
# ./tailscale.nix # requires: tailscale/auth-key (Headscale)
# Stufe 2: Erst nach Schritt 4 aktivieren
# ./forgejo-runner.nix # braucht: forgejo-runner/token (Forgejo)
# ./headplane.nix # braucht: headplane/agent_pre_authkey (Headscale)
# ./tailscale.nix # braucht: tailscale/auth-key (Headscale)
];
}
```
### 3.5 Deploy (Stage 1)
### 3.5 Deployen (Stufe 1)
```bash
nix run .#deploy -- -n <hostname>
```
This uses the configuration from `deploy.json`. Alternatively, deploy manually:
Dies nutzt die Konfiguration aus `deploy.json`. Alternativ manuell:
```bash
NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \
--target-host <user>@<IP> --sudo --ask-sudo-password
```
After this deployment, Headscale, Forgejo, Mailserver, and Nginx are running.
Nach diesem Deploy laufen Headscale, Forgejo, Mailserver und Nginx.
### 3.6 Create Forgejo Admin Account
### 3.6 Forgejo Admin-Account erstellen
On first start, Forgejo has no users. Create an admin account via CLI
(on the **server**):
Beim ersten Start hat Forgejo noch keine Benutzer. Admin-Account per CLI anlegen
(auf dem **Server**):
```bash
forgejo admin user create \
--username <username> \
--username <benutzername> \
--email <email>@<domain> \
--password <password> \
--password <passwort> \
--admin
```
> **Note:** The `forgejo` shell alias is provided by the module and automatically
> runs the command as the `forgejo` user with the correct config.
> If the alias is not available, start a new shell (`bash` or `zsh`).
> **Hinweis:** Das `forgejo` Shell-Alias wird vom Modul bereitgestellt und fuehrt
> automatisch den Befehl als `forgejo`-User mit der richtigen Config aus.
> Falls der Alias nicht verfuegbar ist, neue Shell starten (`bash` oder `zsh`).
>
> Since `DISABLE_REGISTRATION = true` is set, new accounts
> can only be created via CLI.
> Da `DISABLE_REGISTRATION = true` gesetzt ist, koennen neue Accounts
> nur per CLI erstellt werden.
## Step 4: Generate Remaining Secrets and Enable All Services
## Schritt 4: Restliche Secrets generieren und alle Services aktivieren
After the server is running with Headscale and Forgejo:
Nachdem der Server mit Headscale und Forgejo laeuft:
1. **Create Headscale users** (on the server):
1. **Headscale-User anlegen** (auf dem Server):
```bash
sudo headscale users create default
sudo headscale users create headplane-agent
```
2. **Determine user IDs** (needed for the preauth keys):
2. **User-IDs ermitteln** (wird fuer die Preauth-Keys benoetigt):
```bash
sudo headscale users list
```
The output shows the numeric IDs (e.g. `1` for default, `2` for headplane-agent).
Die Ausgabe zeigt die numerischen IDs (z.B. `1` fuer default, `2` fuer headplane-agent).
3. **Generate preauth keys** (using the IDs from step 2):
3. **Preauth-Keys generieren** (mit den IDs aus Schritt 2):
```bash
# For Tailscale (use the user ID of "default")
# Fuer Tailscale (User-ID von "default" einsetzen)
sudo headscale preauthkeys create --expiration 99y --reusable --user <ID>
# For Headplane Agent (use the user ID of "headplane-agent")
# Fuer Headplane Agent (User-ID von "headplane-agent" einsetzen)
sudo headscale preauthkeys create --expiration 99y --user <ID>
```
4. **Create the Forgejo Runner token** via the Forgejo Admin Panel:
4. **Forgejo-Runner-Token** ueber das Forgejo Admin Panel erstellen:
Administration > Actions > Runners > Create new Runner
5. **Add the remaining secrets**:
5. **Secrets ergaenzen**:
```bash
sops hosts/<hostname>/secrets.yaml
```
Add the missing secrets:
Die fehlenden Secrets eintragen:
```yaml
tailscale:
@ -334,7 +334,7 @@ After the server is running with Headscale and Forgejo:
agent_pre_authkey: "..."
```
6. **Enable stage-2 services** in `hosts/<hostname>/services/default.nix`:
6. **Stufe-2-Services aktivieren** in `hosts/<hostname>/services/default.nix`:
```nix
{
@ -353,14 +353,14 @@ After the server is running with Headscale and Forgejo:
}
```
6. **Deploy again**:
6. **Erneut deployen**:
```bash
nix run .#deploy -- -n <hostname>
```
## Next Steps
## Naechste Schritte
- [SOPS Reference](../services/sops.md) -- Detailed documentation on secret management
- [Creating an SD Image](sd-image.md) -- Install Raspberry Pi
- [Set Up CD](../deployment/cd.md) -- Automatic deployment
- [SOPS-Referenz](../services/sops.md) -- Detail-Dokumentation zur Secret-Verwaltung
- [SD-Image erstellen](sd-image.md) -- Raspberry Pi installieren
- [CD einrichten](../deployment/cd.md) -- Automatisches Deployment

View file

@ -1,60 +1,60 @@
# Adding a New Raspberry Pi Client
# Neuen Raspberry Pi Client hinzufügen
This guide describes how to add a **new Raspberry Pi client** to the infrastructure.
Diese Anleitung beschreibt das Hinzufügen eines **neuen Raspberry Pi Clients** zur Infrastruktur.
## Overview: The Process
## Übersicht: Der Ablauf
```
1. Create configuration ──► Copy template, customize
1. Konfiguration erstellen ──► Template kopieren, anpassen
2. Add to image pipeline ──► Extend workflow matrix
2. Zur Image-Pipeline hinzufügen ──► Workflow-Matrix erweitern
3. Push to main ──► Forgejo automatically builds SD image
3. Push auf main ──► Forgejo baut automatisch SD-Image
4. Flash image & boot ──► Write SD card, start Pi
4. Image flashen & booten ──► SD-Karte beschreiben, Pi starten
5. Configure SOPS ──► Retrieve age key, create secrets
5. SOPS konfigurieren ──► Age-Key holen, Secrets erstellen
6. Final deployment ──► Activate Tailscale etc.
6. Finales Deployment ──► Tailscale etc. aktivieren
```
## Prerequisites
## Voraussetzungen
- SSH access to cryodev-main (for Tailscale auth key)
- Development machine with repository access
- SD card (at least 8 GB)
- SSH-Zugang zu cryodev-main (für Tailscale Auth-Key)
- Entwicklungsrechner mit Repository-Zugriff
- SD-Karte (mindestens 8 GB)
---
## Step 1: Generate Tailscale Auth Key
## Schritt 1: Tailscale Auth-Key generieren
**On cryodev-main** (via SSH):
**Auf cryodev-main** (per SSH):
```bash
# Determine user ID
# User-ID ermitteln
sudo headscale users list
# Create preauth key (use user ID of "default")
# Preauth-Key erstellen (User-ID von "default" einsetzen)
sudo headscale preauthkeys create --expiration 99y --reusable --user <ID>
```
**Take note of the output!** (e.g. `tskey-preauth-abc123...`)
**Ausgabe notieren!** (z.B. `tskey-preauth-abc123...`)
---
## Step 2: Create Host Configuration
## Schritt 2: Host-Konfiguration erstellen
### 2.1 Copy Template
### 2.1 Template kopieren
```bash
cp -r templates/raspberry-pi hosts/neuer-pi
```
### 2.2 Set Hostname
### 2.2 Hostname setzen
`hosts/neuer-pi/networking.nix`:
@ -64,58 +64,58 @@ cp -r templates/raspberry-pi hosts/neuer-pi
}
```
### 2.3 Register in flake.nix
### 2.3 In flake.nix registrieren
```nix
nixosConfigurations = {
# ... existing hosts ...
# ... bestehende Hosts ...
neuer-pi = mkNixosConfiguration "aarch64-linux" [ ./hosts/neuer-pi ];
};
```
### 2.4 Add to constants.nix
### 2.4 In constants.nix eintragen
```nix
{
hosts = {
# ... existing hosts ...
# ... bestehende Hosts ...
neuer-pi = {
ip = "100.64.0.X"; # Assigned by Headscale
ip = "100.64.0.X"; # Wird von Headscale vergeben
};
};
}
```
### 2.5 Create Placeholder secrets.yaml
### 2.5 Placeholder secrets.yaml erstellen
```bash
touch hosts/neuer-pi/secrets.yaml
```
### 2.6 Temporarily Disable SOPS
### 2.6 SOPS temporär deaktivieren
In `hosts/neuer-pi/default.nix`, comment out the `sops.secrets.*` references so the image can be built without secrets.
In `hosts/neuer-pi/default.nix` die `sops.secrets.*` Referenzen auskommentieren, damit das Image ohne Secrets gebaut werden kann.
---
## Step 3: Add to Image Pipeline
## Schritt 3: Zur Image-Pipeline hinzufügen
Edit `.forgejo/workflows/build-pi-image.yml`:
Bearbeite `.forgejo/workflows/build-pi-image.yml`:
```yaml
jobs:
build-pi-images:
strategy:
matrix:
# Add new host here:
# Neuen Host hier hinzufügen:
host: [cryodev-pi, neuer-pi]
```
---
## Step 4: Push and Build Image
## Schritt 4: Push und Image bauen lassen
```bash
git add .
@ -123,86 +123,86 @@ git commit -m "Add neuer-pi host configuration"
git push
```
The Forgejo workflow will now automatically build an SD image for `neuer-pi`.
Der Forgejo Workflow baut jetzt automatisch ein SD-Image für `neuer-pi`.
**Wait** until the workflow completes (30-60 minutes). Check the status at:
**Warten** bis der Workflow fertig ist (30-60 Minuten). Status prüfen unter:
`https://git.cryodev.xyz/steffen/cryodev-server/actions`
---
## Step 5: Flash Image
## Schritt 5: Image flashen
### 5.1 Download Image
### 5.1 Image herunterladen
After a successful build, find the image under **Releases**:
Nach erfolgreichem Build unter **Releases**:
```bash
wget https://git.cryodev.xyz/steffen/cryodev-server/releases/latest/download/neuer-pi-sd-image.img.zst
```
### 5.2 Decompress
### 5.2 Dekomprimieren
```bash
zstd -d neuer-pi-sd-image.img.zst -o neuer-pi.img
```
### 5.3 Write to SD Card
### 5.3 Auf SD-Karte schreiben
**Warning:** Replace `/dev/sdX` with the correct device!
**Achtung:** `/dev/sdX` durch das richtige Gerät ersetzen!
```bash
lsblk # Identify the correct device
lsblk # Richtiges Gerät finden
sudo dd if=neuer-pi.img of=/dev/sdX bs=4M conv=fsync status=progress
```
### 5.4 Boot
### 5.4 Booten
1. Insert the SD card into the Raspberry Pi
2. Connect Ethernet
3. Connect power
4. Wait until booted (approximately 2 minutes)
1. SD-Karte in den Raspberry Pi einlegen
2. Ethernet anschließen
3. Strom anschließen
4. Warten bis gebootet (ca. 2 Minuten)
---
## Step 6: Configure SOPS
## Schritt 6: SOPS konfigurieren
### 6.1 Find IP Address
### 6.1 IP-Adresse finden
The Pi should receive an IP address via DHCP. Check your router or scan the network:
Der Pi sollte per DHCP eine IP bekommen. Prüfe deinen Router oder scanne das Netzwerk:
```bash
nmap -sn 192.168.1.0/24 | grep -B2 "Raspberry"
```
### 6.2 Connect via SSH
### 6.2 SSH verbinden
```bash
ssh steffen@<IP> # or the configured user
ssh steffen@<IP> # oder der konfigurierte User
```
For the default password, see `hosts/neuer-pi/users.nix`.
Standard-Passwort siehe `hosts/neuer-pi/users.nix`.
### 6.3 Determine Age Key
### 6.3 Age-Key ermitteln
On the Pi:
Auf dem Pi:
```bash
nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
```
**Take note of the output!** (e.g. `age1xyz...`)
**Ausgabe notieren!** (z.B. `age1xyz...`)
### 6.4 Update .sops.yaml
### 6.4 .sops.yaml aktualisieren
On the development machine:
Auf dem Entwicklungsrechner:
```yaml
keys:
- &steffen_key age1e8p35795htf7twrejyugpzw0qja2v33awcw76y4gp6acnxnkzq0s935t4t # steffen (local)
- &neuer_pi_key age1xyz... # The new key
- &neuer_pi_key age1xyz... # Der neue Key
creation_rules:
# ... existing rules ...
# ... bestehende Regeln ...
- path_regex: hosts/neuer-pi/secrets.yaml$
key_groups:
@ -211,30 +211,30 @@ creation_rules:
- *neuer_pi_key
```
### 6.5 Create Secrets
### 6.5 Secrets erstellen
```bash
sops hosts/neuer-pi/secrets.yaml
```
Contents:
Inhalt:
```yaml
tailscale:
auth-key: "tskey-preauth-abc123..." # Key from Step 1
auth-key: "tskey-preauth-abc123..." # Key aus Schritt 1
netdata:
stream:
child-uuid: "..." # uuidgen
```
### 6.6 Activate SOPS References
### 6.6 SOPS-Referenzen aktivieren
Re-enable the `sops.secrets.*` references that were commented out in Step 2.6.
Die in Schritt 2.6 auskommentierten `sops.secrets.*` Referenzen wieder aktivieren.
---
## Step 7: Final Deployment
## Schritt 7: Finales Deployment
```bash
git add .
@ -242,9 +242,9 @@ git commit -m "Configure SOPS secrets for neuer-pi"
git push
```
Since Comin is running on the Pi, it will automatically pull the new configuration.
Da Comin auf dem Pi läuft, wird er die neue Konfiguration automatisch pullen.
Alternatively, deploy manually:
Alternativ manuell:
```bash
NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#neuer-pi \
@ -253,34 +253,34 @@ NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#neuer-pi \
---
## Step 8: Verify
## Schritt 8: Verifizieren
### Tailscale Connection
### Tailscale-Verbindung
```bash
# On the Pi
# Auf dem Pi
tailscale status
# On cryodev-main
# Auf cryodev-main
sudo headscale nodes list
```
### Netdata Streaming
### Netdata-Streaming
Check whether the new client appears in the Netdata dashboard:
Prüfe ob der neue Client im Netdata-Dashboard erscheint:
`https://netdata.cryodev.xyz`
---
## Checklist
## Checkliste
- [ ] Tailscale auth key generated on cryodev-main
- [ ] Host configuration created (template, flake.nix, constants.nix)
- [ ] Host added to workflow matrix
- [ ] Pushed and waited for image build
- [ ] SD card flashed and Pi booted
- [ ] Age key determined and added to .sops.yaml
- [ ] secrets.yaml created (Tailscale key, Netdata UUID)
- [ ] SOPS references activated and deployed
- [ ] Tailscale connection working
- [ ] Netdata streaming working
- [ ] Tailscale Auth-Key auf cryodev-main generiert
- [ ] Host-Konfiguration erstellt (Template, flake.nix, constants.nix)
- [ ] Host zur Workflow-Matrix hinzugefügt
- [ ] Gepusht und auf Image-Build gewartet
- [ ] SD-Karte geflasht und Pi gebootet
- [ ] Age-Key ermittelt und in .sops.yaml eingetragen
- [ ] secrets.yaml erstellt (Tailscale-Key, Netdata-UUID)
- [ ] SOPS-Referenzen aktiviert und deployed
- [ ] Tailscale-Verbindung funktioniert
- [ ] Netdata-Streaming funktioniert

View file

@ -1,70 +1,70 @@
# Reinstallation
# Neuinstallation (Reinstall)
This guide describes the **reinstallation** of an existing host, e.g. after a hardware change or in case of issues.
Diese Anleitung beschreibt die **Neuinstallation** eines bestehenden Hosts, z.B. nach Hardwarewechsel oder bei Problemen.
## Difference from Initial Installation
## Unterschied zur Erstinstallation
| Aspect | Initial Installation | Reinstallation |
|--------|----------------------|----------------|
| SOPS Secrets | Not yet present | Already configured |
| SSH Host Key | Newly generated | **Must be restored!** |
| Disk IDs | Newly determined | Often changed (new hardware) |
| secrets.yaml | Will be created | Already exists |
| Aspekt | Erstinstallation | Neuinstallation |
|--------|------------------|-----------------|
| SOPS-Secrets | Noch nicht vorhanden | Bereits konfiguriert |
| SSH-Host-Key | Neu generiert | **Muss wiederhergestellt werden!** |
| Disk-IDs | Neu ermitteln | Oft geändert (neue Hardware) |
| secrets.yaml | Wird erstellt | Bereits vorhanden |
## Important: SSH Host Key Issue
## Wichtig: SSH-Host-Key Problem
During a reinstallation, a **new SSH host key** is generated. This key will no longer match the age key in `.sops.yaml`!
Bei einer Neuinstallation wird ein **neuer SSH-Host-Key** generiert. Dieser stimmt nicht mehr mit dem Age-Key in `.sops.yaml` überein!
### Possible Solutions
### Lösungsmöglichkeiten
**Option A: Back up and restore the old host key** (recommended)
**Option A: Alten Host-Key sichern und wiederherstellen** (empfohlen)
**Option B: Generate a new key and update SOPS**
**Option B: Neuen Key generieren und SOPS aktualisieren**
## Prerequisites
## Voraussetzungen
- Backup of the old SSH host key (if using Option A)
- Access to `.sops.yaml` and the admin age keys
- Bootable NixOS ISO
- Backup des alten SSH-Host-Keys (falls Option A)
- Zugriff auf `.sops.yaml` und die Admin-Age-Keys
- Bootbares NixOS ISO
## Step 1: Preparation (before the installation)
## Schritt 1: Vorbereitung (vor der Installation)
### 1.1 Back Up the Old SSH Host Key (Option A)
### 1.1 Alten SSH-Host-Key sichern (Option A)
If the old host is still running:
Falls der alte Host noch läuft:
```bash
# On the old host
# Auf dem alten Host
sudo cat /etc/ssh/ssh_host_ed25519_key > ~/ssh_host_ed25519_key.backup
sudo cat /etc/ssh/ssh_host_ed25519_key.pub > ~/ssh_host_ed25519_key.pub.backup
```
Copy the files securely to the development machine.
Dateien sicher auf den Entwicklungsrechner kopieren.
### 1.2 Determine Disk IDs
### 1.2 Disk-IDs ermitteln
**With new hardware**, the disk IDs will change!
**Bei neuer Hardware** ändern sich die Disk-IDs!
```bash
# In the NixOS live system
# Im NixOS Live-System
lsblk -o NAME,SIZE,MODEL,SERIAL
ls -la /dev/disk/by-id/
```
Enter the new disk ID in `hosts/<hostname>/disks.sh` or `disks.nix`:
Die neue Disk-ID in `hosts/<hostname>/disks.sh` oder `disks.nix` eintragen:
```bash
# Example disks.sh
# Beispiel disks.sh
DISK="/dev/disk/by-id/nvme-Samsung_SSD_970_EVO_Plus_XXXXX"
```
## Step 2: Perform the Installation
## Schritt 2: Installation durchführen
### 2.1 Boot the NixOS ISO
### 2.1 NixOS ISO booten
Boot from USB/CD, set a root password, and connect via SSH.
Von USB/CD booten, Root-Passwort setzen, per SSH verbinden.
### 2.2 Clone the Repository
### 2.2 Repository klonen
```bash
sudo -i
@ -73,30 +73,30 @@ git clone <GIT_REPO_URL> /tmp/nixos
cd /tmp/nixos
```
### 2.3 Verify the Disk Configuration
### 2.3 Disk-Konfiguration prüfen
```bash
# Display current disk IDs
# Aktuelle Disk-IDs anzeigen
ls -la /dev/disk/by-id/
# Compare with the configuration
# Mit Konfiguration vergleichen
cat hosts/<hostname>/disks.sh | grep DISK
```
**If necessary:** Update the disk ID in the configuration.
**Falls nötig:** Disk-ID in der Konfiguration anpassen.
### 2.4 Run the Install Script
### 2.4 Install-Script ausführen
```bash
bash scripts/install.sh -n <hostname>
```
### 2.5 Restore the SSH Host Key (Option A)
### 2.5 SSH-Host-Key wiederherstellen (Option A)
**Before rebooting!**
**Vor dem Reboot!**
```bash
# Restore the host key from backup
# Host-Key vom Backup wiederherstellen
cp /path/to/ssh_host_ed25519_key.backup /mnt/etc/ssh/ssh_host_ed25519_key
cp /path/to/ssh_host_ed25519_key.pub.backup /mnt/etc/ssh/ssh_host_ed25519_key.pub
chmod 600 /mnt/etc/ssh/ssh_host_ed25519_key
@ -110,75 +110,75 @@ umount -Rl /mnt
reboot
```
## Step 3: After the Reboot
## Schritt 3: Nach dem Reboot
### Option A (Key Restored)
### Bei Option A (Key wiederhergestellt)
SOPS secrets should work automatically. Verify:
SOPS-Secrets sollten automatisch funktionieren. Testen:
```bash
sudo cat /run/secrets/tailscale/auth-key
```
### Option B (New Key)
### Bei Option B (Neuer Key)
The host cannot decrypt the secrets. Configure the new key:
Der Host kann die Secrets nicht entschlüsseln. Neuen Key konfigurieren:
```bash
# Determine the new age key
# Neuen Age-Key ermitteln
nix-shell -p ssh-to-age --run 'cat /etc/ssh/ssh_host_ed25519_key.pub | ssh-to-age'
```
On the development machine:
Auf dem Entwicklungsrechner:
```bash
# Update .sops.yaml with the new key
# .sops.yaml aktualisieren mit neuem Key
vim .sops.yaml
# Re-encrypt secrets with the new key
# Secrets mit neuem Key neu verschlüsseln
sops updatekeys hosts/<hostname>/secrets.yaml
```
Then redeploy the configuration:
Dann Konfiguration neu deployen:
```bash
NIX_SSHOPTS="-p 2299" nixos-rebuild switch --flake .#<hostname> \
--target-host <user>@<IP> --sudo --ask-sudo-password
```
## Common Issues
## Häufige Probleme
### "No secret key available"
SOPS cannot decrypt the secrets. Cause:
- SSH host key does not match the age key in `.sops.yaml`
SOPS kann die Secrets nicht entschlüsseln. Ursache:
- SSH-Host-Key stimmt nicht mit Age-Key in `.sops.yaml` überein
Solution: Follow Option B (configure the new key).
Lösung: Option B durchführen (neuen Key konfigurieren).
### "Device not found" during partitioning
### "Device not found" beim Partitionieren
The disk ID in `disks.sh`/`disks.nix` is incorrect.
Disk-ID in `disks.sh`/`disks.nix` ist falsch.
```bash
# Find the correct ID
# Richtige ID finden
ls -la /dev/disk/by-id/
```
### Outdated Hardware Config
### Hardware-Config veraltet
With new hardware, `hardware.nix` must be regenerated:
Bei neuer Hardware muss `hardware.nix` neu generiert werden:
```bash
# The install script regenerates automatically if the file is missing
# Install-Script generiert automatisch neu, falls Datei fehlt
rm hosts/<hostname>/hardware.nix
bash scripts/install.sh -n <hostname>
```
## Checklist
## Checkliste
- [ ] Old SSH host key backed up (if possible)
- [ ] Disk IDs in configuration verified/updated
- [ ] Installation completed
- [ ] SSH host key restored OR new key configured in SOPS
- [ ] Secrets are functional (`sudo cat /run/secrets/...`)
- [ ] Tailscale connected (`tailscale status`)
- [ ] Alten SSH-Host-Key gesichert (falls möglich)
- [ ] Disk-IDs in Konfiguration geprüft/aktualisiert
- [ ] Installation durchgeführt
- [ ] SSH-Host-Key wiederhergestellt ODER neuen Key in SOPS konfiguriert
- [ ] Secrets funktionieren (`sudo cat /run/secrets/...`)
- [ ] Tailscale verbunden (`tailscale status`)

View file

@ -1,95 +1,95 @@
# SD Card Images for Raspberry Pi
# SD-Karten-Images für Raspberry Pi
The repository automatically builds SD card images for all configured Raspberry Pi hosts.
Das Repository baut automatisch SD-Karten-Images für alle konfigurierten Raspberry Pi Hosts.
## Automatic Build
## Automatischer Build
When changes are pushed to `main`, images are automatically built for all Pi hosts and published as a release.
Bei Änderungen an `main` werden automatisch Images für alle Pi-Hosts gebaut und als Release veröffentlicht.
**Download:** [Releases on Forgejo](https://git.cryodev.xyz/steffen/cryodev-server/releases)
**Download:** [Releases auf Forgejo](https://git.cryodev.xyz/steffen/cryodev-server/releases)
## Available Images
## Verfügbare Images
| Host | Image Name |
| Host | Image-Name |
|------|------------|
| `cryodev-pi` | `cryodev-pi-sd-image.img.zst` |
New hosts are built automatically once they are added to the workflow matrix.
Neue Hosts werden automatisch gebaut, wenn sie zur Workflow-Matrix hinzugefügt werden.
## Flashing the Image
## Image flashen
### 1. Download
### 1. Herunterladen
```bash
wget https://git.cryodev.xyz/.../releases/latest/download/<hostname>-sd-image.img.zst
wget https://git.cryodev.xyz/.../releases/latest/download/<hostname>-sd-image.img.zst.sha256
# Verify checksum
# Checksum prüfen
sha256sum -c <hostname>-sd-image.img.zst.sha256
```
### 2. Decompress
### 2. Dekomprimieren
```bash
zstd -d <hostname>-sd-image.img.zst -o <hostname>.img
```
### 3. Write to SD Card
### 3. Auf SD-Karte schreiben
```bash
# Identify the correct device
# Richtiges Gerät finden
lsblk
# Write (WARNING: make sure to select the correct device!)
# Schreiben (ACHTUNG: richtiges Gerät wählen!)
sudo dd if=<hostname>.img of=/dev/sdX bs=4M conv=fsync status=progress
```
Alternatively, use `balenaEtcher` or `Raspberry Pi Imager`.
Alternativ: `balenaEtcher` oder `Raspberry Pi Imager` verwenden.
## What Is Included in the Image?
## Was ist im Image?
- Complete NixOS installation for the specific host
- All configured services (except secrets)
- SSH server enabled
- Automatic root partition expansion on first boot
- Comin for automatic updates
- Vollständige NixOS-Installation für den spezifischen Host
- Alle konfigurierten Services (außer Secrets)
- SSH-Server aktiviert
- Automatische Root-Partition-Erweiterung beim ersten Boot
- Comin für automatische Updates
## What Is Missing?
## Was fehlt?
**SOPS secrets** cannot be included in the image (chicken-and-egg problem with the SSH host key).
**SOPS-Secrets** können nicht im Image enthalten sein (Henne-Ei-Problem mit SSH-Host-Key).
After the first boot:
1. Retrieve the age key from the Pi
2. Update `.sops.yaml`
3. Create `secrets.yaml`
4. Deploy the configuration
Nach dem ersten Boot:
1. Age-Key vom Pi holen
2. `.sops.yaml` aktualisieren
3. `secrets.yaml` erstellen
4. Konfiguration deployen
See [Adding a New Client](new-client.md) for the complete guide.
Siehe [Neuen Client hinzufügen](new-client.md) für die vollständige Anleitung.
## Adding a New Host to the Pipeline
## Neuen Host zur Pipeline hinzufügen
1. Create the host configuration in `hosts/<hostname>/`
2. Add it to the matrix in `.forgejo/workflows/build-pi-image.yml`:
1. Host-Konfiguration in `hosts/<hostname>/` erstellen
2. In `.forgejo/workflows/build-pi-image.yml` zur Matrix hinzufügen:
```yaml
matrix:
host: [cryodev-pi, new-host] # <- add here
host: [cryodev-pi, neuer-host] # <- hier hinzufügen
```
3. Push to `main` -- the image will be built automatically
3. Push auf `main` → Image wird automatisch gebaut
## Building Manually
## Manuell bauen
```bash
# On aarch64 (e.g., another Pi)
# Auf aarch64 (z.B. anderem Pi)
nix build .#nixosConfigurations.<hostname>.config.system.build.sdImage
# On x86_64 with QEMU emulation (slow)
# Auf x86_64 mit QEMU-Emulation (langsam)
nix build .#nixosConfigurations.<hostname>.config.system.build.sdImage \
--extra-platforms aarch64-linux
```
Prerequisite on x86_64:
Voraussetzung auf x86_64:
```nix
{
@ -99,18 +99,18 @@ Prerequisite on x86_64:
## Troubleshooting
### Workflow Fails
### Workflow schlägt fehl
- Check whether `sd-image.nix` is imported in the host configuration
- Check whether binfmt is enabled on cryodev-main
- Prüfe ob `sd-image.nix` in der Host-Konfiguration importiert wird
- Prüfe ob binfmt auf cryodev-main aktiviert ist
### Image Does Not Boot
### Image bootet nicht
- Was the SD card written correctly?
- Try a different SD card
- Check the power supply (minimum 3A for Pi 4)
- SD-Karte korrekt beschrieben?
- Andere SD-Karte versuchen
- Stromversorgung prüfen (min. 3A für Pi 4)
### No Network
### Kein Netzwerk
- Check the Ethernet cable
- Is there a DHCP server on the network?
- Ethernet-Kabel prüfen
- DHCP-Server im Netzwerk?

View file

@ -1,33 +1,33 @@
# Cryodev NixOS Configuration Documentation
Welcome to the documentation for the **cryodev** NixOS infrastructure.
Willkommen zur Dokumentation der **cryodev** NixOS-Infrastruktur.
## Quick Links
### Getting Started
- [Prerequisites](getting-started/prerequisites.md) - Required tools
- [Adding a New Raspberry Pi](getting-started/new-client.md) - Complete workflow for new clients
- [SD Image Reference](getting-started/sd-image.md) - Details on image building
- [First Installation (Server)](getting-started/first-install.md) - Bootstrap for x86_64 hosts
- [Reinstallation](getting-started/reinstall.md) - Reinstall with hardware changes
- [Voraussetzungen](getting-started/prerequisites.md) - Benötigte Tools
- [Neuen Raspberry Pi hinzufügen](getting-started/new-client.md) - Kompletter Workflow für neue Clients
- [SD-Image Referenz](getting-started/sd-image.md) - Details zum Image-Build
- [Erstinstallation (Server)](getting-started/first-install.md) - Bootstrap für x86_64 Hosts
- [Neuinstallation](getting-started/reinstall.md) - Reinstall mit Hardware-Änderungen
### Services
- [SOPS Secrets](services/sops.md) - Secret management with sops-nix
- [Headscale](services/headscale.md) - Self-hosted Tailscale server
- [Headplane](services/headplane.md) - Web UI for Headscale
- [Tailscale](services/tailscale.md) - Mesh VPN client
- [Mailserver](services/mailserver.md) - Email stack (Postfix/Dovecot)
- [Forgejo](services/forgejo.md) - Git hosting with CI/CD
- [Netdata](services/netdata.md) - Monitoring and alerting
- [SOPS Secrets](services/sops.md) - Geheimnisverwaltung mit sops-nix
- [Headscale](services/headscale.md) - Self-hosted Tailscale Server
- [Headplane](services/headplane.md) - Web-UI für Headscale
- [Tailscale](services/tailscale.md) - Mesh-VPN Client
- [Mailserver](services/mailserver.md) - E-Mail Stack (Postfix/Dovecot)
- [Forgejo](services/forgejo.md) - Git-Hosting mit CI/CD
- [Netdata](services/netdata.md) - Monitoring und Alerting
### Deployment
- [Continuous Deployment](deployment/cd.md) - Push- and pull-based deployment
- [DNS Configuration](deployment/dns.md) - Required DNS records
- [Continuous Deployment](deployment/cd.md) - Push- und Pull-basiertes Deployment
- [DNS-Konfiguration](deployment/dns.md) - Benötigte DNS-Einträge
## Architecture
## Architektur
```
Internet
@ -57,38 +57,38 @@ Welcome to the documentation for the **cryodev** NixOS infrastructure.
+-------------------+
```
## Installation Scenarios
## Installations-Szenarien
| Scenario | Description | Guide |
|----------|-------------|-------|
| **New Raspberry Pi** | Create config, build image, flash | [new-client.md](getting-started/new-client.md) |
| **First Installation (Server)** | x86_64 host, manual installation | [first-install.md](getting-started/first-install.md) |
| **Reinstallation** | Existing host, new hardware | [reinstall.md](getting-started/reinstall.md) |
| Szenario | Beschreibung | Anleitung |
|----------|--------------|-----------|
| **Neuer Raspberry Pi** | Config erstellen → Image bauen → Flashen | [new-client.md](getting-started/new-client.md) |
| **Erstinstallation (Server)** | x86_64 Host, manuelle Installation | [first-install.md](getting-started/first-install.md) |
| **Neuinstallation** | Bestehender Host, neue Hardware | [reinstall.md](getting-started/reinstall.md) |
For Raspberry Pi: [SD Image Reference](getting-started/sd-image.md)
Für Raspberry Pi: [SD-Image Referenz](getting-started/sd-image.md)
## Directory Structure
## Verzeichnisstruktur
```
.
├── flake.nix # Entry point, inputs and outputs
├── constants.nix # Central configuration (domains, IPs, ports)
├── hosts/ # Host-specific configurations
├── constants.nix # Zentrale Config (Domains, IPs, Ports)
├── hosts/ # Host-spezifische Konfigurationen
│ ├── cryodev-main/
│ └── cryodev-pi/
├── modules/ # Reusable NixOS modules
├── modules/ # Wiederverwendbare NixOS-Module
│ └── nixos/
├── pkgs/ # Custom packages
├── overlays/ # Nixpkgs overlays
├── templates/ # Templates for new hosts
├── scripts/ # Helper scripts (install.sh)
├── apps/ # Nix apps (rebuild)
└── lib/ # Helper functions (utils.nix)
├── pkgs/ # Eigene Pakete
├── overlays/ # Nixpkgs Overlays
├── templates/ # Templates für neue Hosts
├── scripts/ # Helper-Scripts (install.sh)
├── apps/ # Nix Apps (rebuild)
└── lib/ # Helper-Funktionen (utils.nix)
```
## Deployment Strategies
## Deployment-Strategien
| Host | Strategy | Tool | Description |
|------|----------|------|-------------|
| `cryodev-main` | Pull-based | Comin | Polls the repository for changes |
| `cryodev-pi` | Pull-based | Comin | Polls the repository for changes |
| Host | Strategie | Tool | Beschreibung |
|------|-----------|------|--------------|
| `cryodev-main` | Pull-basiert | Comin | Pollt Repository auf Aenderungen |
| `cryodev-pi` | Pull-basiert | Comin | Pollt Repository auf Aenderungen |

View file

@ -89,9 +89,9 @@ See [CD documentation](../deployment/cd.md) for details.
```bash
forgejo admin user create \
--username <username> \
--username <benutzername> \
--email <email>@<domain> \
--password <password> \
--password <passwort> \
--admin
```

View file

@ -14,9 +14,9 @@ Tailscale clients connect to the self-hosted Headscale server to join the mesh V
On the Headscale server (cryodev-main):
```bash
# Look up user ID
# User-ID ermitteln
sudo headscale users list
# Create preauth key (use the user ID for "default")
# Preauth-Key erstellen (User-ID von "default" einsetzen)
sudo headscale preauthkeys create --expiration 99y --reusable --user <ID>
```