owly-news/backend-rust/TODO.md

CPU and resource limiting

• Tokio worker threads

  • Decide thread policy:

    • Option A: set TOKIO_WORKER_THREADS in the environment for deployments.
    • Option B: build a custom runtime with tokio::runtime::Builder::new_multi_thread().worker_threads(n).

  • Document your default policy (e.g., 50% of physical cores).

• Concurrency guard for CPU-heavy tasks

  • Create a global tokio::sync::Semaphore with N permits (N = allowed concurrent heavy tasks).
  • Acquire a permit before invoking heavy module operations; release automatically on drop.
  • Expose the semaphore in app state so handlers/jobs can share it.

• HTTP backpressure and rate limiting (if using API)

  • Add tower::limit::ConcurrencyLimitLayer to cap in-flight requests.
  • Add tower::limit::RateLimitLayer or request-size/timeouts as needed.
  • Optionally add tower::timeout::TimeoutLayer to bound handler latency.

• Stronger isolation (optional, later)

  • Evaluate running certain modules as separate processes for strict CPU caps.
  • Use cgroups v2 (Linux) or Job Objects (Windows) to bound CPU/memory per process.
  • Reuse the same JSON interface over IPC (e.g., stdio or a local socket).

Build and run

• Build all crates

  • Run: cargo build --workspace

• Build each plugin as cdylib

  • Example: cd crates/modules/summarizer && cargo build --release

• Stage plugin libraries for the host to find

  • Create a modules directory the daemon will read, e.g. target/modules

  • Copy the built artifact into that directory:

    • Linux: copy target/release/libsummarizer.so -> target/modules/libsummarizer.so
    • macOS: copy target/release/libsummarizer.dylib -> target/modules/libsummarizer.dylib
    • Windows: copy target/release/summarizer.dll -> target/modules/summarizer.dll

  • Alternatively set OWLY_MODULES_DIR to your chosen directory.

• Run the daemon

  • cargo run -p owly-news
  • Optionally set:
    • OWLY_MODULES_DIR=/absolute/path/to/modules
    • TOKIO_WORKER_THREADS=N

Wire into the API

• Share ModuleHost in app state

  • Create a struct AppState { host: Arc, cpu_sem: Arc , ... }.
  • Add AppState to Axum with .with_state(state).

• In a handler (example: POST /summarize)

  • Parse payload as JSON.
  • Acquire a permit from cpu_sem before heavy work.
  • host.get("summarizer").await? to lazily load the module.
  • Call module.invoke_json("summarize", payload_value)?.
  • Map success to 200 with JSON; map errors to appropriate status codes.

• Error handling and observability

  • Use thiserror/anyhow to classify operational vs. client errors.
  • Add tracing spans around module loading and invocation; include module name and op.
  • Return structured error JSON when module reports an error.

• Configuration

  • Decide env vars and defaults: OWLY_MODULES_DIR, TOKIO_WORKER_THREADS, concurrency permits, rate limits.
  • Optionally add a config file (toml) and load via figment or config crate.

• Health and lifecycle

  • Add a /health route that checks:

    • Tokio is responsive.
    • Optional: preflight-check that required modules are present (or skip to keep lazy).

  • Graceful shutdown: listen for SIGINT/SIGTERM and drain in-flight requests before exit.