tyto/docs/multi-device.md

# Multi-Device Setup

This guide covers setting up Tyto to monitor multiple hosts from a central dashboard.

## Architecture Overview

```
                        ┌─────────────────┐
                        │    Browser      │
                        │  (Dashboard)    │
                        └────────┬────────┘
                                 │ HTTPS
                                 ▼
┌────────────────────────────────────────────────────────────────┐
│                      Central Server                             │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │
│  │  HTTP API    │  │  SSE Broker  │  │   gRPC Hub (mTLS)    │  │
│  │  :8080       │  │  (metrics)   │  │   :9849              │  │
│  └──────────────┘  └──────────────┘  └──────────┬───────────┘  │
│  ┌──────────────┐  ┌──────────────┐             │              │
│  │  Database    │  │   Registry   │             │              │
│  │  (SQLite/PG) │  │   (agents)   │             │              │
│  └──────────────┘  └──────────────┘             │              │
└─────────────────────────────────────────────────┼──────────────┘
                                                  │ gRPC/mTLS
              ┌───────────────────────────────────┼───────────────────────────┐
              │                                   │                           │
              ▼                                   ▼                           ▼
     ┌───────────────┐                   ┌───────────────┐           ┌───────────────┐
     │  Agent        │                   │  Agent        │           │  Agent        │
     │  (web-01)     │                   │  (db-01)      │           │  (app-01)     │
     └───────────────┘                   └───────────────┘           └───────────────┘
```

**Components:**
- **Central Server**: Aggregates metrics, serves dashboard, manages agents
- **Agents**: Lightweight collectors running on each monitored host
- **mTLS**: Mutual TLS for secure agent-server communication
- **Registry**: Tracks agent status and approval

## Prerequisites

- Central server with Docker or native Tyto installation
- Network connectivity between agents and server (port 9849)
- (Optional) Reverse proxy for HTTPS dashboard access

## Step 1: Set Up the Central Server

### Using Docker Compose

```bash
git clone https://somegit.dev/vikingowl/tyto.git
cd tyto

# Start in server mode
TYTO_MODE=server docker compose up -d
```

### Using Native Install

```bash
curl -fsSL https://somegit.dev/vikingowl/tyto/raw/branch/main/scripts/install.sh | \
  sudo TYTO_MODE=server bash
```

### Configure the Server

Edit `/etc/tyto/config.yaml`:

```yaml
mode: server

http:
  host: "0.0.0.0"
  port: 8080

database:
  type: sqlite
  path: /var/lib/tyto/tyto.db

server:
  grpc_port: 9849
  tls:
    enabled: true
    ca_cert: /etc/tyto/pki/ca.crt
    server_cert: /etc/tyto/certs/server.crt
    server_key: /etc/tyto/certs/server.key
  registration:
    auto_enabled: true
    require_approval: true
```

## Step 2: Initialize PKI

Generate certificates for mTLS authentication.

### Create Certificate Authority

```bash
tyto pki init-ca \
  --cn "Tyto CA" \
  --org "My Organization" \
  --validity 3650 \
  --out /etc/tyto/pki/
```

This creates:
- `/etc/tyto/pki/ca.crt` - CA certificate
- `/etc/tyto/pki/ca.key` - CA private key (keep secure!)
- `/etc/tyto/pki/store.json` - Certificate store

### Generate Server Certificate

```bash
tyto pki gen-server \
  --ca-dir /etc/tyto/pki/ \
  --dns tyto.example.com \
  --dns localhost \
  --ip 192.168.1.100 \
  --out /etc/tyto/certs/
```

This creates:
- `/etc/tyto/certs/server.crt`
- `/etc/tyto/certs/server.key`

### Generate Agent Certificates

For each agent:

```bash
# Web server
tyto pki gen-agent \
  --ca-dir /etc/tyto/pki/ \
  --agent-id web-server-01 \
  --out /tmp/web-server-01/

# Database server
tyto pki gen-agent \
  --ca-dir /etc/tyto/pki/ \
  --agent-id db-server-01 \
  --out /tmp/db-server-01/

# Application server
tyto pki gen-agent \
  --ca-dir /etc/tyto/pki/ \
  --agent-id app-server-01 \
  --out /tmp/app-server-01/
```

Each creates:
- `agent.crt` - Agent certificate
- `agent.key` - Agent private key

### Distribute Certificates

Copy to each agent host:
- CA certificate: `/etc/tyto/certs/ca.crt`
- Agent certificate: `/etc/tyto/certs/agent.crt`
- Agent key: `/etc/tyto/certs/agent.key`

```bash
# Example using scp
scp /etc/tyto/pki/ca.crt user@web-server:/etc/tyto/certs/
scp /tmp/web-server-01/agent.* user@web-server:/etc/tyto/certs/
```

## Step 3: Deploy Agents

### Install Agent

On each monitored host:

```bash
curl -fsSL https://somegit.dev/vikingowl/tyto/raw/branch/main/scripts/install.sh | \
  sudo TYTO_MODE=agent bash
```

### Configure Agent

Edit `/etc/tyto/config.yaml`:

```yaml
mode: agent

agent:
  id: web-server-01
  server_url: tyto.example.com:9849
  interval: 5s
  tls:
    ca_cert: /etc/tyto/certs/ca.crt
    agent_cert: /etc/tyto/certs/agent.crt
    agent_key: /etc/tyto/certs/agent.key
```

### Start Agent

```bash
sudo systemctl start tyto
sudo systemctl enable tyto
```

### Verify Connection

Check agent logs:
```bash
journalctl -u tyto -f
```

You should see:
```
Connected to server tyto.example.com:9849
Sending metrics every 5s
```

## Step 4: Approve Agents

New agents require approval before their metrics appear.

### Via Dashboard

1. Open the Tyto dashboard
2. Navigate to **Agents** → **Pending**
3. Review agent details (ID, hostname, IP)
4. Click **Approve** or **Reject**

### Via CLI

```bash
# List pending agents
tyto agents list --pending

# Approve an agent
tyto agents approve web-server-01

# Reject an agent
tyto agents reject suspicious-agent
```

### Via API

```bash
# List pending
curl http://localhost:8080/api/v1/agents/pending

# Approve
curl -X POST http://localhost:8080/api/v1/agents/pending/web-server-01/approve
```

## Agent Log Collection

Enable log forwarding on agents:

```yaml
agent:
  logs:
    enabled: true
    buffer_size: 1000
    flush_interval: 5s

    # Systemd journal
    journal:
      enabled: true
      units:
        - nginx.service
        - docker.service
      priority: 4  # warning and above

    # File tailing
    files:
      - path: /var/log/nginx/access.log
        format: nginx
      - path: /var/log/nginx/error.log
        format: nginx_error

    # Docker containers
    docker:
      enabled: true
      containers: []  # Empty = all
```

## PKI Management

### List Certificates

```bash
tyto pki list --ca-dir /etc/tyto/pki/
```

Output:
```
Serial      Subject                 Expires              Status
ABC123      CN=web-server-01       2025-12-28 10:30:00  valid
DEF456      CN=db-server-01        2025-12-28 10:35:00  valid
GHI789      CN=old-server          2024-06-15 08:00:00  revoked
```

### Revoke Certificate

If an agent is compromised or decommissioned:

```bash
tyto pki revoke \
  --serial ABC123 \
  --ca-dir /etc/tyto/pki/ \
  --reason "Server decommissioned"
```

### Renew Certificate

Before expiration:

```bash
# Generate new certificate
tyto pki gen-agent \
  --ca-dir /etc/tyto/pki/ \
  --agent-id web-server-01 \
  --out /tmp/web-server-01-new/

# Deploy to agent and restart
scp /tmp/web-server-01-new/* user@web-server:/etc/tyto/certs/
ssh user@web-server 'sudo systemctl restart tyto'
```

### View CA Info

```bash
tyto pki info --ca-dir /etc/tyto/pki/
```

## Firewall Configuration

### Server

Open port 9849 for agent connections:

```bash
# UFW
sudo ufw allow 9849/tcp

# firewalld
sudo firewall-cmd --permanent --add-port=9849/tcp
sudo firewall-cmd --reload

# iptables
sudo iptables -A INPUT -p tcp --dport 9849 -j ACCEPT
```

### Agents

Agents initiate outbound connections only. No inbound ports required.

## High Availability

### Multiple Servers

For redundancy, run multiple servers with shared PostgreSQL:

```yaml
# Server 1
database:
  type: postgres
  url: postgres://tyto:pass@db.example.com:5432/tyto

# Server 2 (same config)
database:
  type: postgres
  url: postgres://tyto:pass@db.example.com:5432/tyto
```

Configure agents with multiple server URLs:

```yaml
agent:
  server_url: tyto1.example.com:9849,tyto2.example.com:9849
```

### Load Balancer

Place a TCP load balancer in front of servers:

```
                    ┌─────────────┐
   Agents ─────────▶│ Load Balancer │
                    │   (TCP:9849)  │
                    └───────┬───────┘
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
         Server 1      Server 2      Server 3
```

## Troubleshooting

### Agent Not Connecting

1. **Check network**: `telnet tyto.example.com 9849`
2. **Check certificates**: `openssl verify -CAfile ca.crt agent.crt`
3. **Check logs**: `journalctl -u tyto -f`

### Certificate Issues

```bash
# Verify certificate chain
openssl verify -CAfile /etc/tyto/certs/ca.crt /etc/tyto/certs/agent.crt

# Check certificate details
openssl x509 -in /etc/tyto/certs/agent.crt -text -noout

# Check expiration
openssl x509 -in /etc/tyto/certs/agent.crt -enddate -noout
```

### Agent Not Approved

1. Check the Agents → Pending section in dashboard
2. Verify agent ID matches certificate CN
3. Check server logs for registration attempts

### Metrics Not Appearing

1. Verify agent status: `systemctl status tyto`
2. Check agent is approved in dashboard
3. Verify collector permissions (proc, sys access)