docs: add provider onboarding guide and update documentation for ProviderManager, health worker, and multi‑provider architecture
This commit is contained in:
@@ -2,24 +2,22 @@
|
||||
|
||||
This guide explains how to implement a new provider for Owlen. Providers are the components that connect to different LLM APIs.
|
||||
|
||||
## The `Provider` Trait
|
||||
## The `ModelProvider` Trait
|
||||
|
||||
The core of the provider system is the `Provider` trait, located in `owlen-core`. Any new provider must implement this trait.
|
||||
The core of the provider system is the `ModelProvider` trait, located in `owlen-core::provider`. Any new provider must implement this async trait so it can be managed by `ProviderManager`.
|
||||
|
||||
Here is a simplified version of the trait:
|
||||
|
||||
```rust
|
||||
use async_trait::async_trait;
|
||||
use owlen_core::model::Model;
|
||||
use owlen_core::session::Session;
|
||||
use owlen_core::provider::{GenerateChunk, GenerateRequest, GenerateStream, ModelInfo, ProviderMetadata, ProviderStatus};
|
||||
|
||||
#[async_trait]
|
||||
pub trait Provider {
|
||||
/// Returns the name of the provider.
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// Sends the session to the provider and returns the response.
|
||||
async fn chat(&self, session: &Session, model: &Model) -> Result<String, anyhow::Error>;
|
||||
pub trait ModelProvider: Send + Sync {
|
||||
fn metadata(&self) -> &ProviderMetadata;
|
||||
async fn health_check(&self) -> owlen_core::Result<ProviderStatus>;
|
||||
async fn list_models(&self) -> owlen_core::Result<Vec<ModelInfo>>;
|
||||
async fn generate_stream(&self, request: GenerateRequest) -> owlen_core::Result<GenerateStream>;
|
||||
}
|
||||
```
|
||||
|
||||
@@ -35,41 +33,66 @@ In your new crate's `lib.rs`, you will define a struct for your provider and imp
|
||||
|
||||
```rust
|
||||
use async_trait::async_trait;
|
||||
use owlen_core::model::Model;
|
||||
use owlen_core::Provider;
|
||||
use owlen_core::session::Session;
|
||||
use owlen_core::provider::{
|
||||
GenerateRequest, GenerateStream, ModelInfo, ModelProvider, ProviderMetadata,
|
||||
ProviderStatus, ProviderType,
|
||||
};
|
||||
|
||||
pub struct MyProvider;
|
||||
pub struct MyProvider {
|
||||
metadata: ProviderMetadata,
|
||||
client: MyHttpClient,
|
||||
}
|
||||
|
||||
impl MyProvider {
|
||||
pub fn new(config: &MyConfig) -> owlen_core::Result<Self> {
|
||||
let metadata = ProviderMetadata::new(
|
||||
"my_provider",
|
||||
"My Provider",
|
||||
ProviderType::Cloud,
|
||||
true,
|
||||
);
|
||||
|
||||
Ok(Self {
|
||||
metadata,
|
||||
client: MyHttpClient::new(config)?,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Provider for MyProvider {
|
||||
fn name(&self) -> &str {
|
||||
"my-provider"
|
||||
impl ModelProvider for MyProvider {
|
||||
fn metadata(&self) -> &ProviderMetadata {
|
||||
&self.metadata
|
||||
}
|
||||
|
||||
async fn chat(&self, session: &Session, model: &Model) -> Result<String, anyhow::Error> {
|
||||
// 1. Get the conversation history from the session.
|
||||
let history = session.get_messages();
|
||||
async fn health_check(&self) -> owlen_core::Result<ProviderStatus> {
|
||||
self.client.ping().await.map(|_| ProviderStatus::Available)
|
||||
}
|
||||
|
||||
// 2. Format the request for your provider's API.
|
||||
// This might involve creating a JSON body with the messages.
|
||||
async fn list_models(&self) -> owlen_core::Result<Vec<ModelInfo>> {
|
||||
self.client.list_models().await
|
||||
}
|
||||
|
||||
// 3. Send the request to the API using a client like reqwest.
|
||||
|
||||
// 4. Parse the response from the API.
|
||||
|
||||
// 5. Return the content of the response as a String.
|
||||
|
||||
Ok("Hello from my provider!".to_string())
|
||||
async fn generate_stream(&self, request: GenerateRequest) -> owlen_core::Result<GenerateStream> {
|
||||
self.client.generate(request).await
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Integrating with Owlen
|
||||
|
||||
Once your provider is implemented, you will need to integrate it into the main Owlen application.
|
||||
Once your provider is implemented, you will need to register it with the `ProviderManager` and surface it to users.
|
||||
|
||||
1. **Add your provider crate** as a dependency to `owlen-cli`.
|
||||
2. **In `owlen-cli`, modify the provider registration** to include your new provider. This will likely involve adding it to a list of available providers that the user can select from in the configuration.
|
||||
1. **Add your provider crate** as a dependency to the component that will host it (an MCP server or `owlen-cli`).
|
||||
2. **Register the provider** with `ProviderManager` during startup:
|
||||
|
||||
This guide provides a basic outline. For more detailed examples, you can look at the existing provider implementations, such as `owlen-ollama`.
|
||||
```rust
|
||||
let manager = ProviderManager::new(config);
|
||||
manager.register_provider(Arc::new(MyProvider::new(config)?)).await;
|
||||
```
|
||||
|
||||
3. **Update configuration docs/examples** so the provider has a `[providers.my_provider]` entry.
|
||||
4. **Expose via MCP (optional)** if the provider should run out-of-process. Owlen’s TUI talks to providers exclusively via MCP after Phase 10.
|
||||
5. **Add tests** similar to `crates/owlen-providers/tests/integration_test.rs` that exercise registration, model aggregation, generation routing, and health transitions.
|
||||
|
||||
For concrete examples, see the Ollama providers in `crates/owlen-providers/` and the integration tests added in commit 13.
|
||||
|
||||
Reference in New Issue
Block a user