73 lines
3.2 KiB
Markdown
73 lines
3.2 KiB
Markdown
## CPU and resource limiting
|
|
- Tokio worker threads
|
|
- Decide thread policy:
|
|
- Option A: set TOKIO_WORKER_THREADS in the environment for deployments.
|
|
- Option B: build a custom runtime with tokio::runtime::Builder::new_multi_thread().worker_threads(n).
|
|
|
|
- Document your default policy (e.g., 50% of physical cores).
|
|
|
|
- Concurrency guard for CPU-heavy tasks
|
|
- Create a global tokio::sync::Semaphore with N permits (N = allowed concurrent heavy tasks).
|
|
- Acquire a permit before invoking heavy module operations; release automatically on drop.
|
|
- Expose the semaphore in app state so handlers/jobs can share it.
|
|
|
|
- HTTP backpressure and rate limiting (if using API)
|
|
- Add tower::limit::ConcurrencyLimitLayer to cap in-flight requests.
|
|
- Add tower::limit::RateLimitLayer or request-size/timeouts as needed.
|
|
- Optionally add tower::timeout::TimeoutLayer to bound handler latency.
|
|
|
|
- Stronger isolation (optional, later)
|
|
- Evaluate running certain modules as separate processes for strict CPU caps.
|
|
- Use cgroups v2 (Linux) or Job Objects (Windows) to bound CPU/memory per process.
|
|
- Reuse the same JSON interface over IPC (e.g., stdio or a local socket).
|
|
|
|
## Build and run
|
|
- Build all crates
|
|
- Run: cargo build --workspace
|
|
|
|
- Build each plugin as cdylib
|
|
- Example: cd crates/modules/summarizer && cargo build --release
|
|
|
|
- Stage plugin libraries for the host to find
|
|
- Create a modules directory the daemon will read, e.g. target/modules
|
|
- Copy the built artifact into that directory:
|
|
- Linux: copy target/release/libsummarizer.so -> target/modules/libsummarizer.so
|
|
- macOS: copy target/release/libsummarizer.dylib -> target/modules/libsummarizer.dylib
|
|
- Windows: copy target/release/summarizer.dll -> target/modules/summarizer.dll
|
|
|
|
- Alternatively set OWLY_MODULES_DIR to your chosen directory.
|
|
|
|
- Run the daemon
|
|
- cargo run -p owly-news
|
|
- Optionally set:
|
|
- OWLY_MODULES_DIR=/absolute/path/to/modules
|
|
- TOKIO_WORKER_THREADS=N
|
|
|
|
## Wire into the API
|
|
- Share ModuleHost in app state
|
|
- Create a struct AppState { host: Arc, cpu_sem: Arc , ... }.
|
|
- Add AppState to Axum with .with_state(state).
|
|
|
|
- In a handler (example: POST /summarize)
|
|
- Parse payload as JSON.
|
|
- Acquire a permit from cpu_sem before heavy work.
|
|
- host.get("summarizer").await? to lazily load the module.
|
|
- Call module.invoke_json("summarize", payload_value)?.
|
|
- Map success to 200 with JSON; map errors to appropriate status codes.
|
|
|
|
- Error handling and observability
|
|
- Use thiserror/anyhow to classify operational vs. client errors.
|
|
- Add tracing spans around module loading and invocation; include module name and op.
|
|
- Return structured error JSON when module reports an error.
|
|
|
|
- Configuration
|
|
- Decide env vars and defaults: OWLY_MODULES_DIR, TOKIO_WORKER_THREADS, concurrency permits, rate limits.
|
|
- Optionally add a config file (toml) and load via figment or config crate.
|
|
|
|
- Health and lifecycle
|
|
- Add a /health route that checks:
|
|
- Tokio is responsive.
|
|
- Optional: preflight-check that required modules are present (or skip to keep lazy).
|
|
|
|
- Graceful shutdown: listen for SIGINT/SIGTERM and drain in-flight requests before exit.
|