# Netdata Monitoring

Netdata provides real-time performance monitoring with parent/child streaming.

## Architecture

```
┌─────────────────┐     Stream over      ┌─────────────────┐
│  cryodev-pi     │  ───────────────────>│  cryodev-main   │
│  (Child Node)   │     Tailscale VPN    │  (Parent Node)  │
└─────────────────┘                      └─────────────────┘
                                                  │
                                                  v
                                         https://netdata.cryodev.xyz
```

## References

- [Netdata Documentation](https://learn.netdata.cloud/)
- [Streaming Configuration](https://learn.netdata.cloud/docs/streaming/streaming-configuration-reference)

## Parent Node (cryodev-main)

### DNS

Set a CNAME record for `netdata.cryodev.xyz` pointing to your main domain.

### Generate Stream API Key

```bash
uuidgen
```

### Configuration

```nix
# hosts/cryodev-main/services/netdata.nix
{ config, ... }:
{
  sops.secrets."netdata/stream-api-key" = { };
  
  sops.templates."netdata-stream.conf" = {
    content = ''
      [${config.sops.placeholder."netdata/stream-api-key"}]
      enabled = yes
      default history = 3600
      default memory mode = ram
      health enabled by default = auto
      allow from = *
    '';
    owner = "netdata";
  };
  
  services.netdata = {
    enable = true;
    configDir."stream.conf" = config.sops.templates."netdata-stream.conf".path;
  };
}
```

## Child Node (cryodev-pi)

### Generate Child UUID

```bash
uuidgen
```

### Add to Secrets

```bash
sops hosts/cryodev-pi/secrets.yaml
```

```yaml
netdata:
  stream:
    child-uuid: "your-generated-uuid"
```

Note: The stream API key must match the parent's key. You can either:
1. Share the same secret between hosts (complex with SOPS)
2. Hardcode a known API key in both configurations

### Configuration

```nix
# hosts/cryodev-pi/services/netdata.nix
{ config, constants, ... }:
{
  sops.secrets."netdata/stream/child-uuid" = { };
  
  sops.templates."netdata-stream.conf" = {
    content = ''
      [stream]
      enabled = yes
      destination = ${constants.hosts.cryodev-main.ip}:19999
      api key = YOUR_STREAM_API_KEY
      send charts matching = *
    '';
    owner = "netdata";
  };
  
  services.netdata = {
    enable = true;
    configDir."stream.conf" = config.sops.templates."netdata-stream.conf".path;
  };
}
```

## Email Alerts

Configure Netdata to send alerts via the mailserver:

```nix
{
  services.netdata.configDir."health_alarm_notify.conf" = pkgs.writeText "notify.conf" ''
    SEND_EMAIL="YES"
    EMAIL_SENDER="netdata@cryodev.xyz"
    DEFAULT_RECIPIENT_EMAIL="admin@cryodev.xyz"
  '';
}
```

## Usage

### Access Dashboard

Open `https://netdata.cryodev.xyz` in your browser.

### View Child Nodes

Child nodes appear in the left sidebar under "Nodes".

### Check Streaming Status

On parent:
```bash
curl -s http://localhost:19999/api/v1/info | jq '.hosts'
```

On child:
```bash
curl -s http://localhost:19999/api/v1/info | jq '.streaming'
```

## Troubleshooting

### Check Service Status

```bash
sudo systemctl status netdata
```

### View Logs

```bash
sudo journalctl -u netdata -f
```

### Child Not Streaming

1. Verify network connectivity:
   ```bash
   tailscale ping cryodev-main
   nc -zv <parent-ip> 19999
   ```

2. Check API key matches between parent and child

3. Verify firewall allows port 19999 on parent

### High Memory Usage

Adjust history settings in `netdata.conf`:

```ini
[global]
history = 1800  # seconds to retain
memory mode = ram
```