docs: add provider onboarding guide and update documentation for ProviderManager, health worker, and multi‑provider architecture

CHANGELOG.md

@@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Comprehensive documentation suite including guides for architecture, configuration, testing, and more.
 - Rustdoc examples for core components like `Provider` and `SessionController`.
 - Module-level documentation for `owlen-tui`.
+- Provider integration tests (`crates/owlen-providers/tests`) covering registration, routing, and health status handling for the new `ProviderManager`.
+- TUI message and generation tests that exercise the non-blocking event loop, background worker, and message dispatch.
 - Ollama integration can now talk to Ollama Cloud when an API key is configured.
 - Ollama provider will also read `OLLAMA_API_KEY` / `OLLAMA_CLOUD_API_KEY` environment variables when no key is stored in the config.
 - `owlen config doctor`, `owlen config path`, and `owlen upgrade` CLI commands to automate migrations and surface manual update steps.
@@ -26,6 +28,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Input panel respects a new `ui.input_max_rows` setting so long prompts expand predictably before scrolling kicks in.
 - Command palette offers fuzzy `:model` filtering and `:provider` completions for fast switching.
 - Message rendering caches wrapped lines and throttles streaming redraws to keep the TUI responsive on long sessions.
+- Model picker badges now inspect provider capabilities so vision/audio/thinking models surface the correct icons even when descriptions are sparse.
 - Chat history honors `ui.scrollback_lines`, trimming older rows to keep the TUI responsive and surfacing a "↓ New messages" badge whenever updates land off-screen.
 
 ### Changed
@@ -38,6 +41,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `config.toml` now carries a schema version (`1.2.0`) and is migrated automatically; deprecated keys such as `agent.max_tool_calls` trigger warnings instead of hard failures.
 - Model selector navigation (Tab/Shift-Tab) now switches between local and cloud tabs while preserving selection state.
 - Header displays the active model together with its provider (e.g., `Model (Provider)`), improving clarity when swapping backends.
+- Documentation refreshed to cover the message handler architecture, the background health worker, multi-provider configuration, and the new provider onboarding checklist.
 
 ---
 

README.md

@@ -9,10 +9,11 @@
 
 ## What Is OWLEN?
 
-OWLEN is a Rust-powered, terminal-first interface for interacting with local large
+OWLEN is a Rust-powered, terminal-first interface for interacting with local and cloud
-language models. It provides a responsive chat workflow that runs against
+language models. It provides a responsive chat workflow that now routes through a
-[Ollama](https://ollama.com/) with a focus on developer productivity, vim-style navigation,
+multi-provider manager—handling local Ollama, Ollama Cloud, and future MCP-backed providers—
-and seamless session management—all without leaving your terminal.
+with a focus on developer productivity, vim-style navigation, and seamless session
+management—all without leaving your terminal.
 
 ## Alpha Status
 
@@ -32,8 +33,9 @@ The OWLEN interface features a clean, multi-panel layout with vim-inspired navig
 - **Session Management**: Save, load, and manage conversations.
 - **Code Side Panel**: Switch to code mode (`:mode code`) and open files inline with `:open <path>` for LLM-assisted coding.
 - **Theming System**: 10 built-in themes and support for custom themes.
-- **Modular Architecture**: Extensible provider system (Ollama today, additional providers on the roadmap).
+- **Modular Architecture**: Extensible provider system orchestrated by the new `ProviderManager`, ready for additional MCP-backed providers.
-- **Dual-Source Model Picker**: Merge local and cloud Ollama models with live availability indicators so you can see at a glance which catalogues are reachable.
+- **Dual-Source Model Picker**: Merge local and cloud catalogues with real-time availability badges powered by the background health worker.
+- **Non-Blocking UI Loop**: Asynchronous generation tasks and provider health checks run off-thread, keeping the TUI responsive even while streaming long replies.
 - **Guided Setup**: `owlen config doctor` upgrades legacy configs and verifies your environment in seconds.
 
 ## Security & Privacy
@@ -110,7 +112,8 @@ For more detailed information, please refer to the following documents:
 - **[CHANGELOG.md](CHANGELOG.md)**: A log of changes for each version.
 - **[docs/architecture.md](docs/architecture.md)**: An overview of the project's architecture.
 - **[docs/troubleshooting.md](docs/troubleshooting.md)**: Help with common issues.
-- **[docs/provider-implementation.md](docs/provider-implementation.md)**: A guide for adding new providers.
+- **[docs/provider-implementation.md](docs/provider-implementation.md)**: Trait-level details for implementing providers.
+- **[docs/adding-providers.md](docs/adding-providers.md)**: Step-by-step checklist for wiring a provider into the multi-provider architecture and test suite.
 - **[docs/platform-support.md](docs/platform-support.md)**: Current OS support matrix and cross-check instructions.
 
 ## Configuration

docs/adding-providers.md

@@ -0,0 +1,62 @@
+# Adding a Provider to Owlen
+
+This guide complements `docs/provider-implementation.md` with a practical checklist for wiring a new model backend into the Phase 10 architecture.
+
+## 1. Define the Provider Type
+
+Providers live in their own crate (for example `owlen-providers`). Create a module that implements the `owlen_core::provider::ModelProvider` trait:
+
+```rust
+pub struct MyProvider {
+    client: MyHttpClient,
+    metadata: ProviderMetadata,
+}
+
+#[async_trait]
+impl ModelProvider for MyProvider {
+    fn metadata(&self) -> &ProviderMetadata { &self.metadata }
+    async fn health_check(&self) -> Result<ProviderStatus> { ... }
+    async fn list_models(&self) -> Result<Vec<ModelInfo>> { ... }
+    async fn generate_stream(&self, request: GenerateRequest) -> Result<GenerateStream> { ... }
+}
+```
+
+Set `ProviderMetadata::provider_type` to `ProviderType::Local` or `ProviderType::Cloud` so the TUI can label it correctly.
+
+## 2. Register with `ProviderManager`
+
+`ProviderManager` owns provider instances and tracks their health. In your startup code (usually `owlen-cli` or an MCP server), construct the provider and register it:
+
+```rust
+let manager = ProviderManager::new(config);
+manager.register_provider(Arc::new(MyProvider::new(config)?.into())).await;
+```
+
+The manager caches `ProviderStatus` values so the TUI can surface availability in the picker and background worker events.
+
+## 3. Expose Through MCP (Optional)
+
+For providers that should run out-of-process, implement an MCP server (`owlen-mcp-llm-server` demonstrates the pattern). The TUI uses `RemoteMcpClient`, so exposing `generate_text` keeps the UI completely decoupled from provider details.
+
+## 4. Add Tests
+
+Commit 13 introduced integration tests in `crates/owlen-providers/tests`. Follow this pattern to exercise:
+
+- registration with `ProviderManager`
+- model aggregation across providers
+- routing of `generate` requests
+- provider status transitions when generation succeeds or fails
+
+In-memory mocks are enough; the goal is to protect the trait contract and the manager’s health cache.
+
+## 5. Document Configuration
+
+Update `docs/configuration.md` and the default `config.toml` snippet so users can enable the new provider. Include environment variables, auth requirements, or special flags.
+
+## 6. Update User-Facing Docs
+
+- Add a short entry to the feature list in `README.md`.
+- Mention the new provider in `CHANGELOG.md` under the “Added” section.
+- If the provider requires troubleshooting steps, append them to `docs/troubleshooting.md`.
+
+Following these steps keeps the provider lifecycle consistent with Owlen’s multi-provider architecture: providers register once, the manager handles orchestration, and the TUI reacts via message-driven updates.

docs/architecture.md

@@ -6,7 +6,8 @@ This document provides a high-level overview of the Owlen architecture. Its purp
 
 The architecture is designed to be modular and extensible, centered around a few key concepts:
 
--   **Providers**: Connect to various LLM APIs (Ollama, OpenAI, etc.).
+-   **Provider Manager**: Coordinates multiple `ModelProvider` implementations, aggregates model metadata, and caches health status for the UI.
+-   **Providers**: Concrete backends (Ollama Local, Ollama Cloud, future providers) accessed either directly or through MCP servers.
 -   **Session**: Manages the conversation history and state.
 -   **TUI**: The terminal user interface, built with `ratatui`.
 -   **Events**: A system for handling user input and other events.
@@ -16,18 +17,20 @@ The architecture is designed to be modular and extensible, centered around a few
 A simplified diagram of how components interact:
 
 ```
-[User Input] -> [Event Loop] -> [Session Controller] -> [Provider]
+[User Input] -> [Event Loop] -> [Message Handler] -> [Session Controller] -> [Provider Manager] -> [Provider]
       ^                                                                                |
       |                                                                                v
-[TUI Renderer] <------------------------------------ [API Response]
+[TUI Renderer] <- [AppMessage Stream] <- [Background Worker] <--------------- [Provider Health]
 ```
 
 1.  **User Input**: The user interacts with the TUI, generating events (e.g., key presses).
-2.  **Event Loop**: The main event loop in `owlen-tui` captures these events.
+2.  **Event Loop**: The non-blocking event loop in `owlen-tui` bundles raw input, async session events, and background health updates into `AppMessage` events.
-3.  **Session Controller**: The event is processed, and if it's a prompt, the session controller sends a request to the current provider.
+3.  **Message Handler**: `App::handle_message` centralises dispatch, updating runtime state (chat, model picker, provider indicators) before the UI redraw.
-4.  **Provider**: The provider formats the request for the specific LLM API and sends it.
+4.  **Session Controller**: Prompt events create `GenerateRequest`s that flow through `ProviderManager::generate` to the designated provider.
-5.  **API Response**: The LLM API returns a response.
+5.  **Provider**: The provider formats requests for its API and streams back `GenerateChunk`s.
-6.  **TUI Renderer**: The response is processed, the session state is updated, and the TUI is re-rendered to display the new information.
+6.  **Provider Manager**: Tracks health while streaming; errors mark a provider unavailable so background workers and the model picker reflect the state.
+7.  **Background Worker**: A periodic task runs health checks and emits status updates as `AppMessage::ProviderStatus` events.
+8.  **TUI Renderer**: The response is processed, the session state is updated, and the TUI is re-rendered to display the new information.
 
 ## Crate Breakdown
 
@@ -106,7 +109,7 @@ The session management system is responsible for tracking the state of a convers
 -   **`SessionController`**: This is the high-level controller that manages the active conversation. It handles:
     -   Storing and retrieving conversation history via the `ConversationManager`.
     -   Managing the context that is sent to the LLM provider.
-    -   Switching between different models.
+    -   Switching between different models by selecting a provider ID managed by `ProviderManager`.
     -   Sending requests to the provider and handling the responses (both streaming and complete).
 
 When a user sends a message, the `SessionController` adds the message to the current `Conversation`, sends the updated message list to the `Provider`, and then adds the provider's response to the `Conversation`.

docs/provider-implementation.md

@@ -2,24 +2,22 @@
 
 This guide explains how to implement a new provider for Owlen. Providers are the components that connect to different LLM APIs.
 
-## The `Provider` Trait
+## The `ModelProvider` Trait
 
-The core of the provider system is the `Provider` trait, located in `owlen-core`. Any new provider must implement this trait.
+The core of the provider system is the `ModelProvider` trait, located in `owlen-core::provider`. Any new provider must implement this async trait so it can be managed by `ProviderManager`.
 
 Here is a simplified version of the trait:
 
 ```rust
 use async_trait::async_trait;
-use owlen_core::model::Model;
+use owlen_core::provider::{GenerateChunk, GenerateRequest, GenerateStream, ModelInfo, ProviderMetadata, ProviderStatus};
-use owlen_core::session::Session;
 
 #[async_trait]
-pub trait Provider {
+pub trait ModelProvider: Send + Sync {
-    /// Returns the name of the provider.
+    fn metadata(&self) -> &ProviderMetadata;
-    fn name(&self) -> &str;
+    async fn health_check(&self) -> owlen_core::Result<ProviderStatus>;
-
+    async fn list_models(&self) -> owlen_core::Result<Vec<ModelInfo>>;
-    /// Sends the session to the provider and returns the response.
+    async fn generate_stream(&self, request: GenerateRequest) -> owlen_core::Result<GenerateStream>;
-    async fn chat(&self, session: &Session, model: &Model) -> Result<String, anyhow::Error>;
 }
 ```
 
@@ -35,41 +33,66 @@ In your new crate's `lib.rs`, you will define a struct for your provider and imp
 
 ```rust
 use async_trait::async_trait;
-use owlen_core::model::Model;
+use owlen_core::provider::{
-use owlen_core::Provider;
+    GenerateRequest, GenerateStream, ModelInfo, ModelProvider, ProviderMetadata,
-use owlen_core::session::Session;
+    ProviderStatus, ProviderType,
+};
 
-pub struct MyProvider;
+pub struct MyProvider {
-
+    metadata: ProviderMetadata,
-#[async_trait]
+    client: MyHttpClient,
-impl Provider for MyProvider {
-    fn name(&self) -> &str {
-        "my-provider"
 }
 
-    async fn chat(&self, session: &Session, model: &Model) -> Result<String, anyhow::Error> {
+impl MyProvider {
-        // 1. Get the conversation history from the session.
+    pub fn new(config: &MyConfig) -> owlen_core::Result<Self> {
-        let history = session.get_messages();
+        let metadata = ProviderMetadata::new(
+            "my_provider",
+            "My Provider",
+            ProviderType::Cloud,
+            true,
+        );
 
-        // 2. Format the request for your provider's API.
+        Ok(Self {
-        //    This might involve creating a JSON body with the messages.
+            metadata,
+            client: MyHttpClient::new(config)?,
+        })
+    }
+}
 
-        // 3. Send the request to the API using a client like reqwest.
+#[async_trait]
+impl ModelProvider for MyProvider {
+    fn metadata(&self) -> &ProviderMetadata {
+        &self.metadata
+    }
 
-        // 4. Parse the response from the API.
+    async fn health_check(&self) -> owlen_core::Result<ProviderStatus> {
+        self.client.ping().await.map(|_| ProviderStatus::Available)
+    }
 
-        // 5. Return the content of the response as a String.
+    async fn list_models(&self) -> owlen_core::Result<Vec<ModelInfo>> {
+        self.client.list_models().await
+    }
 
-        Ok("Hello from my provider!".to_string())
+    async fn generate_stream(&self, request: GenerateRequest) -> owlen_core::Result<GenerateStream> {
+        self.client.generate(request).await
     }
 }
 ```
 
 ## Integrating with Owlen
 
-Once your provider is implemented, you will need to integrate it into the main Owlen application.
+Once your provider is implemented, you will need to register it with the `ProviderManager` and surface it to users.
 
-1.  **Add your provider crate** as a dependency to `owlen-cli`.
+1.  **Add your provider crate** as a dependency to the component that will host it (an MCP server or `owlen-cli`).
-2.  **In `owlen-cli`, modify the provider registration** to include your new provider. This will likely involve adding it to a list of available providers that the user can select from in the configuration.
+2.  **Register the provider** with `ProviderManager` during startup:
 
-This guide provides a basic outline. For more detailed examples, you can look at the existing provider implementations, such as `owlen-ollama`.
+```rust
+let manager = ProviderManager::new(config);
+manager.register_provider(Arc::new(MyProvider::new(config)?)).await;
+```
+
+3.  **Update configuration docs/examples** so the provider has a `[providers.my_provider]` entry.
+4.  **Expose via MCP (optional)** if the provider should run out-of-process. Owlen’s TUI talks to providers exclusively via MCP after Phase 10.
+5.  **Add tests** similar to `crates/owlen-providers/tests/integration_test.rs` that exercise registration, model aggregation, generation routing, and health transitions.
+
+For concrete examples, see the Ollama providers in `crates/owlen-providers/` and the integration tests added in commit 13.

docs/troubleshooting.md

@@ -31,6 +31,7 @@ Owlen now queries both the local daemon and Ollama Cloud and shows them side-by-
 4. **Keep the base URL local.** The cloud setup command no longer overrides `providers.ollama.base_url` unless `--force-cloud-base-url` is passed. If you changed it manually, edit `config.toml` or run `owlen config doctor` to restore the default `http://localhost:11434` value.
 
 Once the daemon responds again, the picker will automatically merge the updated local list with the cloud catalogue.
+Owlen runs a background health worker every 30 seconds; once the daemon responds it will update the picker automatically without needing a restart.
 
 ## Terminal Compatibility Issues