## CPU and resource limiting - Tokio worker threads - Decide thread policy: - Option A: set TOKIO_WORKER_THREADS in the environment for deployments. - Option B: build a custom runtime with tokio::runtime::Builder::new_multi_thread().worker_threads(n). - Document your default policy (e.g., 50% of physical cores). - Concurrency guard for CPU-heavy tasks - Create a global tokio::sync::Semaphore with N permits (N = allowed concurrent heavy tasks). - Acquire a permit before invoking heavy module operations; release automatically on drop. - Expose the semaphore in app state so handlers/jobs can share it. - HTTP backpressure and rate limiting (if using API) - Add tower::limit::ConcurrencyLimitLayer to cap in-flight requests. - Add tower::limit::RateLimitLayer or request-size/timeouts as needed. - Optionally add tower::timeout::TimeoutLayer to bound handler latency. - Stronger isolation (optional, later) - Evaluate running certain modules as separate processes for strict CPU caps. - Use cgroups v2 (Linux) or Job Objects (Windows) to bound CPU/memory per process. - Reuse the same JSON interface over IPC (e.g., stdio or a local socket). ## Build and run - Build all crates - Run: cargo build --workspace - Build each plugin as cdylib - Example: cd crates/modules/summarizer && cargo build --release - Stage plugin libraries for the host to find - Create a modules directory the daemon will read, e.g. target/modules - Copy the built artifact into that directory: - Linux: copy target/release/libsummarizer.so -> target/modules/libsummarizer.so - macOS: copy target/release/libsummarizer.dylib -> target/modules/libsummarizer.dylib - Windows: copy target/release/summarizer.dll -> target/modules/summarizer.dll - Alternatively set OWLY_MODULES_DIR to your chosen directory. - Run the daemon - cargo run -p owly-news - Optionally set: - OWLY_MODULES_DIR=/absolute/path/to/modules - TOKIO_WORKER_THREADS=N ## Wire into the API - Share ModuleHost in app state - Create a struct AppState { host: Arc, cpu_sem: Arc , ... }. - Add AppState to Axum with .with_state(state). - In a handler (example: POST /summarize) - Parse payload as JSON. - Acquire a permit from cpu_sem before heavy work. - host.get("summarizer").await? to lazily load the module. - Call module.invoke_json("summarize", payload_value)?. - Map success to 200 with JSON; map errors to appropriate status codes. - Error handling and observability - Use thiserror/anyhow to classify operational vs. client errors. - Add tracing spans around module loading and invocation; include module name and op. - Return structured error JSON when module reports an error. - Configuration - Decide env vars and defaults: OWLY_MODULES_DIR, TOKIO_WORKER_THREADS, concurrency permits, rate limits. - Optionally add a config file (toml) and load via figment or config crate. - Health and lifecycle - Add a /health route that checks: - Tokio is responsive. - Optional: preflight-check that required modules are present (or skip to keep lazy). - Graceful shutdown: listen for SIGINT/SIGTERM and drain in-flight requests before exit.