99 lines
3.5 KiB
Markdown
99 lines
3.5 KiB
Markdown
# Provider Implementation Guide
|
||
|
||
This guide explains how to implement a new provider for Owlen. Providers are the components that connect to different LLM APIs.
|
||
|
||
## The `ModelProvider` Trait
|
||
|
||
The core of the provider system is the `ModelProvider` trait, located in `owlen-core::provider`. Any new provider must implement this async trait so it can be managed by `ProviderManager`.
|
||
|
||
Here is a simplified version of the trait:
|
||
|
||
```rust
|
||
use async_trait::async_trait;
|
||
use owlen_core::provider::{GenerateChunk, GenerateRequest, GenerateStream, ModelInfo, ProviderMetadata, ProviderStatus};
|
||
|
||
#[async_trait]
|
||
pub trait ModelProvider: Send + Sync {
|
||
fn metadata(&self) -> &ProviderMetadata;
|
||
async fn health_check(&self) -> owlen_core::Result<ProviderStatus>;
|
||
async fn list_models(&self) -> owlen_core::Result<Vec<ModelInfo>>;
|
||
async fn generate_stream(&self, request: GenerateRequest) -> owlen_core::Result<GenerateStream>;
|
||
}
|
||
```
|
||
|
||
## Creating a New Crate
|
||
|
||
1. **Create a new crate** in the `crates/` directory. For example, `owlen-myprovider`.
|
||
2. **Add dependencies** to your new crate's `Cargo.toml`. You will need `owlen-core`, `async-trait`, `tokio`, and any crates required for interacting with the new API (e.g., `reqwest`).
|
||
3. **Add the new crate to the workspace** in the root `Cargo.toml`.
|
||
|
||
## Implementing the Trait
|
||
|
||
In your new crate's `lib.rs`, you will define a struct for your provider and implement the `Provider` trait for it.
|
||
|
||
```rust
|
||
use async_trait::async_trait;
|
||
use owlen_core::provider::{
|
||
GenerateRequest, GenerateStream, ModelInfo, ModelProvider, ProviderMetadata,
|
||
ProviderStatus, ProviderType,
|
||
};
|
||
|
||
pub struct MyProvider {
|
||
metadata: ProviderMetadata,
|
||
client: MyHttpClient,
|
||
}
|
||
|
||
impl MyProvider {
|
||
pub fn new(config: &MyConfig) -> owlen_core::Result<Self> {
|
||
let metadata = ProviderMetadata::new(
|
||
"my_provider",
|
||
"My Provider",
|
||
ProviderType::Cloud,
|
||
true,
|
||
);
|
||
|
||
Ok(Self {
|
||
metadata,
|
||
client: MyHttpClient::new(config)?,
|
||
})
|
||
}
|
||
}
|
||
|
||
#[async_trait]
|
||
impl ModelProvider for MyProvider {
|
||
fn metadata(&self) -> &ProviderMetadata {
|
||
&self.metadata
|
||
}
|
||
|
||
async fn health_check(&self) -> owlen_core::Result<ProviderStatus> {
|
||
self.client.ping().await.map(|_| ProviderStatus::Available)
|
||
}
|
||
|
||
async fn list_models(&self) -> owlen_core::Result<Vec<ModelInfo>> {
|
||
self.client.list_models().await
|
||
}
|
||
|
||
async fn generate_stream(&self, request: GenerateRequest) -> owlen_core::Result<GenerateStream> {
|
||
self.client.generate(request).await
|
||
}
|
||
}
|
||
```
|
||
|
||
## Integrating with Owlen
|
||
|
||
Once your provider is implemented, you will need to register it with the `ProviderManager` and surface it to users.
|
||
|
||
1. **Add your provider crate** as a dependency to the component that will host it (an MCP server or `owlen-cli`).
|
||
2. **Register the provider** with `ProviderManager` during startup:
|
||
|
||
```rust
|
||
let manager = ProviderManager::new(config);
|
||
manager.register_provider(Arc::new(MyProvider::new(config)?)).await;
|
||
```
|
||
|
||
3. **Update configuration docs/examples** so the provider has a `[providers.my_provider]` entry.
|
||
4. **Expose via MCP (optional)** if the provider should run out-of-process. Owlen’s TUI talks to providers exclusively via MCP after Phase 10.
|
||
5. **Add tests** similar to `crates/owlen-providers/tests/integration_test.rs` that exercise registration, model aggregation, generation routing, and health transitions.
|
||
|
||
For concrete examples, see the Ollama providers in `crates/owlen-providers/` and the integration tests added in commit 13.
|