vikingowl/vessel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 23 Wiki Activity

feat: add multi-backend LLM support #7

Merged
vikingowl merged 5 commits from feature/multi-backend-support into dev 2026-01-23 15:53:13 +01:00
Conversation 0 Commits 5 Files Changed 58 +6461 -249
vikingowl commented 2026-01-23 15:53:00 +01:00
Owner
Copy Link

Summary

Add support for multiple LLM backends with a unified interface:

  • Ollama (default) - Full model management, pull/delete/create custom models
  • llama.cpp - High-performance inference with GGUF models
  • LM Studio - Desktop app integration

Backend (Go)

  • New backends package with interface, registry, and adapters
  • Unified API routes under /api/v1/ai/*
  • SSE to NDJSON streaming conversion for OpenAI-compatible backends
  • Auto-discovery of backends on default ports

Frontend (Svelte 5)

  • New backendsState store for backend management
  • Unified LLM client routing through backend API
  • AI Providers settings tab with Backends and Models sub-tabs
  • Backend-aware chat streaming
  • Persist and restore last selected backend

Documentation

  • New LLM-Backends wiki page
  • Updated README, Configuration, Getting-Started docs
  • Updated CLAUDE.md with backend architecture

Test Plan

  • All 547 tests pass
  • TypeScript: 0 errors
  • Chat works with Ollama backend
  • Chat works with llama.cpp backend
  • Backend switching persists across sessions
  • Auto-detection finds available backends
## Summary Add support for multiple LLM backends with a unified interface: - **Ollama** (default) - Full model management, pull/delete/create custom models - **llama.cpp** - High-performance inference with GGUF models - **LM Studio** - Desktop app integration ### Backend (Go) - New `backends` package with interface, registry, and adapters - Unified API routes under `/api/v1/ai/*` - SSE to NDJSON streaming conversion for OpenAI-compatible backends - Auto-discovery of backends on default ports ### Frontend (Svelte 5) - New `backendsState` store for backend management - Unified LLM client routing through backend API - AI Providers settings tab with Backends and Models sub-tabs - Backend-aware chat streaming - Persist and restore last selected backend ### Documentation - New LLM-Backends wiki page - Updated README, Configuration, Getting-Started docs - Updated CLAUDE.md with backend architecture ## Test Plan - [x] All 547 tests pass - [x] TypeScript: 0 errors - [x] Chat works with Ollama backend - [x] Chat works with llama.cpp backend - [x] Backend switching persists across sessions - [x] Auto-detection finds available backends
vikingowl added 5 commits 2026-01-23 15:53:00 +01:00
Merge pull request 'Release v0.6.0' (!4) from dev into main
Some checks failed
Create Release / release (push) Has been cancelled
Details
6dcaf37c7f
Merge pull request 'Release v0.6.1' (!6) from dev into main
Some checks failed
Create Release / release (push) Has been cancelled
Details
d6994bff48
feat: add multi-backend LLM support (Ollama, llama.cpp, LM Studio) a80ddc0fe4 
Add unified backend abstraction layer supporting multiple LLM providers:

Backend (Go):
- New backends package with interface, registry, and adapters
- Ollama adapter wrapping existing functionality
- OpenAI-compatible adapter for llama.cpp and LM Studio
- Unified API routes under /api/v1/ai/*
- SSE to NDJSON streaming conversion for OpenAI backends
- Auto-discovery of backends on default ports

Frontend (Svelte 5):
- New backendsState store for backend management
- Unified LLM client routing through backend API
- AI Providers tab combining Backends and Models sub-tabs
- Backend-aware chat streaming (uses appropriate client)
- Model name display for non-Ollama backends in top nav
- Persist and restore last selected backend

Key features:
- Switch between backends without restart
- Conditional UI based on backend capabilities
- Models tab only visible when Ollama active
- llama.cpp/LM Studio show loaded model name
fix: resolve all TypeScript errors and accessibility warnings a564f7ec77 
TypeScript error fixes:
- Fix UUID mock type in chunker.test.ts
- Remove invalid timestamp property from Message types in tests
- Fix mockFetch type in client.test.ts
- Add missing parameters property to tool definition in test

Accessibility fixes (109 → 40 warnings, remaining are CSS @apply):
- Add aria-labels to all toggle switches and icon-only buttons
- Add tabindex="-1" to all dialog elements with role="dialog"
- Add onkeydown handlers to modal backdrops for keyboard accessibility
- Fix form labels: change decorative labels to spans, use fieldset/legend for groups
- Convert fileInput variables to $state() for proper reactivity
- Fix closure captures in ThinkingBlock and HtmlPreview with $derived()
- Add role="region" to drag-and-drop zones
- Restore keyboard navigation to BranchNavigator

All 547 tests pass.
docs: update README for multi-backend LLM support 291871c3b5 
- Update tagline to 'local LLMs' instead of 'Ollama'
- Add LLM Backends section with Ollama, llama.cpp, LM Studio
- Update Prerequisites to list all supported backends
- Add LLM Backends to documentation table
- Update Roadmap with multi-backend as completed
- Update Non-Goals to clarify cloud providers not supported
vikingowl merged commit a65417eabe into dev 2026-01-23 15:53:13 +01:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
vikingowl
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: vikingowl/vessel#7
Delete Branch "feature/multi-backend-support"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?