docs: align CLI docs to models subcommands; host: scan XDG plugin dir; ci: add GitHub Actions; chore: add CHANGELOG
This commit is contained in:
33
.github/workflows/ci.yml
vendored
Normal file
33
.github/workflows/ci.yml
vendored
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
name: CI
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [ dev, main ]
|
||||||
|
pull_request:
|
||||||
|
branches: [ dev, main ]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
build:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Checkout code
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Set up Rust
|
||||||
|
uses: dtolnay/rust-toolchain@stable
|
||||||
|
|
||||||
|
- name: Cache cargo registry and target
|
||||||
|
uses: Swatinem/rust-cache@v2
|
||||||
|
|
||||||
|
- name: Install components
|
||||||
|
run: rustup component add clippy rustfmt
|
||||||
|
|
||||||
|
- name: Cargo fmt
|
||||||
|
run: cargo fmt --all -- --check
|
||||||
|
|
||||||
|
- name: Clippy
|
||||||
|
run: cargo clippy --workspace --all-targets -- -D warnings
|
||||||
|
|
||||||
|
- name: Test
|
||||||
|
run: cargo test --workspace --all --locked
|
||||||
|
|
14
CHANGELOG.md
Normal file
14
CHANGELOG.md
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
|
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
|
||||||
|
|
||||||
|
## Unreleased
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- Docs: Replace `--download-models`/`--update-models` flags with `models download`/`models update` subcommands in `README.md`, `docs/usage.md`, and `docs/development.md`.
|
||||||
|
- Host: Plugin discovery now scans `$XDG_DATA_HOME/polyscribe/plugins` (platform equivalent via `directories`) in addition to `PATH`.
|
||||||
|
- CI: Add GitHub Actions workflow to run fmt, clippy (warnings as errors), and tests for pushes and PRs.
|
||||||
|
|
||||||
|
|
1
Cargo.lock
generated
1
Cargo.lock
generated
@@ -1193,6 +1193,7 @@ name = "polyscribe-host"
|
|||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"anyhow",
|
"anyhow",
|
||||||
|
"directories",
|
||||||
"serde",
|
"serde",
|
||||||
"serde_json",
|
"serde_json",
|
||||||
"tokio",
|
"tokio",
|
||||||
|
17
README.md
17
README.md
@@ -27,7 +27,7 @@ Installation
|
|||||||
|
|
||||||
Quickstart
|
Quickstart
|
||||||
1) Download a model (first run can prompt you):
|
1) Download a model (first run can prompt you):
|
||||||
- ./target/release/polyscribe --download-models
|
- ./target/release/polyscribe models download
|
||||||
- In the interactive picker, use Up/Down to navigate, Space to toggle selections, and Enter to confirm. Models are grouped by base (e.g., tiny, base, small).
|
- In the interactive picker, use Up/Down to navigate, Space to toggle selections, and Enter to confirm. Models are grouped by base (e.g., tiny, base, small).
|
||||||
|
|
||||||
2) Transcribe a file:
|
2) Transcribe a file:
|
||||||
@@ -45,13 +45,14 @@ Model locations
|
|||||||
- Override via env var: POLYSCRIBE_MODELS_DIR=/path/to/models.
|
- Override via env var: POLYSCRIBE_MODELS_DIR=/path/to/models.
|
||||||
- Force a specific model file via env var: WHISPER_MODEL=/path/to/model.bin.
|
- Force a specific model file via env var: WHISPER_MODEL=/path/to/model.bin.
|
||||||
|
|
||||||
Most-used CLI flags
|
Most-used CLI flags and subcommands
|
||||||
- -o, --output FILE_OR_DIR: Output path base (date prefix added). If omitted, JSON prints to stdout.
|
- -o, --output FILE_OR_DIR: Output path base (date prefix added). If omitted, JSON prints to stdout.
|
||||||
- -m, --merge: Merge all inputs into one output; otherwise one output per input.
|
- -m, --merge: Merge all inputs into one output; otherwise one output per input.
|
||||||
- --merge-and-separate: Write both merged output and separate per-input outputs (requires -o dir).
|
- --merge-and-separate: Write both merged output and separate per-input outputs (requires -o dir).
|
||||||
- --set-speaker-names: Prompt for a speaker label per input file.
|
- --set-speaker-names: Prompt for a speaker label per input file.
|
||||||
- --update-models: Verify/update local models by size/hash against the upstream manifest.
|
- Subcommands:
|
||||||
- --download-models: Interactive model list + multi-select download.
|
- models update: Verify/update local models by size/hash against the upstream manifest.
|
||||||
|
- models download: Interactive model list + multi-select download.
|
||||||
- --language LANG: Language code hint (e.g., en, de). English-only models reject non-en hints.
|
- --language LANG: Language code hint (e.g., en, de). English-only models reject non-en hints.
|
||||||
- --gpu-backend [auto|cpu|cuda|hip|vulkan]: Select backend (auto by default).
|
- --gpu-backend [auto|cpu|cuda|hip|vulkan]: Select backend (auto by default).
|
||||||
- --gpu-layers N: Offload N layers to GPU when supported.
|
- --gpu-layers N: Offload N layers to GPU when supported.
|
||||||
@@ -65,9 +66,9 @@ Minimal usage examples
|
|||||||
- Merge multiple transcripts into one:
|
- Merge multiple transcripts into one:
|
||||||
- ./target/release/polyscribe -m -o output merged input/a.json input/b.json
|
- ./target/release/polyscribe -m -o output merged input/a.json input/b.json
|
||||||
- Update local models non-interactively (good for CI):
|
- Update local models non-interactively (good for CI):
|
||||||
- ./target/release/polyscribe --update-models --no-interaction -q
|
- ./target/release/polyscribe models update --no-interaction -q
|
||||||
- Download models interactively:
|
- Download models interactively:
|
||||||
- ./target/release/polyscribe --download-models
|
- ./target/release/polyscribe models download
|
||||||
|
|
||||||
Troubleshooting & docs
|
Troubleshooting & docs
|
||||||
- docs/faq.md – common issues and solutions (missing ffmpeg, GPU selection, model paths)
|
- docs/faq.md – common issues and solutions (missing ffmpeg, GPU selection, model paths)
|
||||||
@@ -77,7 +78,7 @@ Troubleshooting & docs
|
|||||||
- docs/release-packaging.md – packaging notes for distributions
|
- docs/release-packaging.md – packaging notes for distributions
|
||||||
- CONTRIBUTING.md – PR checklist and CI workflow
|
- CONTRIBUTING.md – PR checklist and CI workflow
|
||||||
|
|
||||||
CI status: [CI badge placeholder]
|
CI status: 
|
||||||
|
|
||||||
License
|
License
|
||||||
-------
|
-------
|
||||||
@@ -86,7 +87,7 @@ This project is licensed under the MIT License — see the LICENSE file for deta
|
|||||||
---
|
---
|
||||||
|
|
||||||
Workspace layout
|
Workspace layout
|
||||||
- This repo is a Cargo workspace using resolver = "2".
|
- This repo is a Cargo workspace using resolver = "3".
|
||||||
- Members:
|
- Members:
|
||||||
- crates/polyscribe-core — types, errors, config service, core helpers.
|
- crates/polyscribe-core — types, errors, config service, core helpers.
|
||||||
- crates/polyscribe-protocol — PSP/1 serde types for NDJSON over stdio.
|
- crates/polyscribe-protocol — PSP/1 serde types for NDJSON over stdio.
|
||||||
|
@@ -9,3 +9,4 @@ serde = { version = "1.0.219", features = ["derive"] }
|
|||||||
serde_json = "1.0.142"
|
serde_json = "1.0.142"
|
||||||
tokio = { version = "1.47.1", features = ["rt-multi-thread", "process", "io-util"] }
|
tokio = { version = "1.47.1", features = ["rt-multi-thread", "process", "io-util"] }
|
||||||
which = "6.0.3"
|
which = "6.0.3"
|
||||||
|
directories = { workspace = true }
|
||||||
|
@@ -1,7 +1,7 @@
|
|||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use std::process::Stdio;
|
use std::process::Stdio;
|
||||||
use std::{env, fs, os::unix::fs::PermissionsExt, path::Path};
|
use std::{env, fs, os::unix::fs::PermissionsExt, path::{Path, PathBuf}};
|
||||||
use tokio::{
|
use tokio::{
|
||||||
io::{AsyncBufReadExt, BufReader},
|
io::{AsyncBufReadExt, BufReader},
|
||||||
process::{Child as TokioChild, Command},
|
process::{Child as TokioChild, Command},
|
||||||
@@ -20,28 +20,22 @@ impl PluginManager {
|
|||||||
pub fn list(&self) -> Result<Vec<PluginInfo>> {
|
pub fn list(&self) -> Result<Vec<PluginInfo>> {
|
||||||
let mut plugins = Vec::new();
|
let mut plugins = Vec::new();
|
||||||
|
|
||||||
// Scan PATH entries for executables starting with "polyscribe-plugin-"
|
// 1) Scan PATH entries for executables starting with "polyscribe-plugin-"
|
||||||
if let Ok(path) = env::var("PATH") {
|
if let Ok(path) = env::var("PATH") {
|
||||||
for dir in env::split_paths(&path) {
|
for dir in env::split_paths(&path) {
|
||||||
if let Ok(read_dir) = fs::read_dir(&dir) {
|
scan_dir_for_plugins(&dir, &mut plugins);
|
||||||
for entry in read_dir.flatten() {
|
|
||||||
let path = entry.path();
|
|
||||||
if let Some(fname) = path.file_name().and_then(|s| s.to_str())
|
|
||||||
&& fname.starts_with("polyscribe-plugin-")
|
|
||||||
&& is_executable(&path)
|
|
||||||
{
|
|
||||||
let name = fname.trim_start_matches("polyscribe-plugin-").to_string();
|
|
||||||
plugins.push(PluginInfo {
|
|
||||||
name,
|
|
||||||
path: path.to_string_lossy().to_string(),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TODO: also scan XDG data plugins dir for symlinks/binaries
|
// 2) Scan XDG data dir: $XDG_DATA_HOME/polyscribe/plugins or platform equiv
|
||||||
|
if let Some(dirs) = directories::ProjectDirs::from("dev", "polyscribe", "polyscribe") {
|
||||||
|
let plugin_dir = dirs.data_dir().join("plugins");
|
||||||
|
scan_dir_for_plugins(&plugin_dir, &mut plugins);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3) De-duplicate by binary path
|
||||||
|
plugins.sort_by(|a, b| a.path.cmp(&b.path));
|
||||||
|
plugins.dedup_by(|a, b| a.path == b.path);
|
||||||
Ok(plugins)
|
Ok(plugins)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -107,6 +101,24 @@ fn is_executable(path: &Path) -> bool {
|
|||||||
true
|
true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn scan_dir_for_plugins(dir: &Path, out: &mut Vec<PluginInfo>) {
|
||||||
|
if let Ok(read_dir) = fs::read_dir(dir) {
|
||||||
|
for entry in read_dir.flatten() {
|
||||||
|
let path = entry.path();
|
||||||
|
if let Some(fname) = path.file_name().and_then(|s| s.to_str())
|
||||||
|
&& fname.starts_with("polyscribe-plugin-")
|
||||||
|
&& is_executable(&path)
|
||||||
|
{
|
||||||
|
let name = fname.trim_start_matches("polyscribe-plugin-").to_string();
|
||||||
|
out.push(PluginInfo {
|
||||||
|
name,
|
||||||
|
path: path.to_string_lossy().to_string(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[allow(dead_code)]
|
#[allow(dead_code)]
|
||||||
#[derive(Debug, Deserialize)]
|
#[derive(Debug, Deserialize)]
|
||||||
struct Capability {
|
struct Capability {
|
||||||
|
@@ -32,18 +32,20 @@ Run locally
|
|||||||
|
|
||||||
Models during development
|
Models during development
|
||||||
- Interactive downloader:
|
- Interactive downloader:
|
||||||
- cargo run -- --download-models
|
- cargo run -- models download
|
||||||
- Non-interactive update (checks sizes/hashes, downloads if missing):
|
- Non-interactive update (checks sizes/hashes, downloads if missing):
|
||||||
- cargo run -- --update-models --no-interaction -q
|
- cargo run -- models update --no-interaction -q
|
||||||
|
|
||||||
Tests
|
Tests
|
||||||
- Run all tests:
|
- Run all tests:
|
||||||
- cargo test
|
- cargo test
|
||||||
- The test suite includes CLI-oriented integration tests and unit tests. Some tests simulate GPU detection using env vars (POLYSCRIBE_TEST_FORCE_*). Do not rely on these flags in production code.
|
- The test suite includes CLI-oriented integration tests and unit tests. Some tests simulate GPU detection using env vars (POLYSCRIBE_TEST_FORCE_*). Do not rely on these flags in production code.
|
||||||
|
|
||||||
Clippy
|
Clippy & formatting
|
||||||
- Run lint checks and treat warnings as errors:
|
- Run lint checks and treat warnings as errors:
|
||||||
- cargo clippy --all-targets -- -D warnings
|
- cargo clippy --all-targets -- -D warnings
|
||||||
|
- Check formatting:
|
||||||
|
- cargo fmt --all -- --check
|
||||||
- Common warnings can often be fixed by simplifying code, removing unused imports, and following idiomatic patterns.
|
- Common warnings can often be fixed by simplifying code, removing unused imports, and following idiomatic patterns.
|
||||||
|
|
||||||
Code layout
|
Code layout
|
||||||
@@ -61,10 +63,10 @@ Adding a feature
|
|||||||
|
|
||||||
Running the model downloader
|
Running the model downloader
|
||||||
- Interactive:
|
- Interactive:
|
||||||
- cargo run -- --download-models
|
- cargo run -- models download
|
||||||
- Non-interactive suggestions for CI:
|
- Non-interactive suggestions for CI:
|
||||||
- POLYSCRIBE_MODELS_DIR=$PWD/models \
|
- POLYSCRIBE_MODELS_DIR=$PWD/models \
|
||||||
cargo run -- --update-models --no-interaction -q
|
cargo run -- models update --no-interaction -q
|
||||||
|
|
||||||
Env var examples for local testing
|
Env var examples for local testing
|
||||||
- Use a local copy of models and a specific model file:
|
- Use a local copy of models and a specific model file:
|
||||||
|
@@ -30,10 +30,10 @@ CLI reference
|
|||||||
- Choose runtime backend. Default is auto (prefers CUDA → HIP → Vulkan → CPU), depending on detection.
|
- Choose runtime backend. Default is auto (prefers CUDA → HIP → Vulkan → CPU), depending on detection.
|
||||||
- --gpu-layers N
|
- --gpu-layers N
|
||||||
- Number of layers to offload to the GPU when supported.
|
- Number of layers to offload to the GPU when supported.
|
||||||
- --download-models
|
- models download
|
||||||
- Launch interactive model downloader (lists Hugging Face models; multi-select to download).
|
- Launch interactive model downloader (lists Hugging Face models; multi-select to download).
|
||||||
- Controls: Use Up/Down to navigate, Space to toggle selections, and Enter to confirm. Models are grouped by base (e.g., tiny, base, small).
|
- Controls: Use Up/Down to navigate, Space to toggle selections, and Enter to confirm. Models are grouped by base (e.g., tiny, base, small).
|
||||||
- --update-models
|
- models update
|
||||||
- Verify/update local models by comparing sizes and hashes with the upstream manifest.
|
- Verify/update local models by comparing sizes and hashes with the upstream manifest.
|
||||||
- -v, --verbose (repeatable)
|
- -v, --verbose (repeatable)
|
||||||
- Increase log verbosity; use -vv for very detailed logs.
|
- Increase log verbosity; use -vv for very detailed logs.
|
||||||
@@ -42,6 +42,9 @@ CLI reference
|
|||||||
- --no-interaction
|
- --no-interaction
|
||||||
- Disable all interactive prompts (for CI). Combine with env vars to control behavior.
|
- Disable all interactive prompts (for CI). Combine with env vars to control behavior.
|
||||||
- Subcommands:
|
- Subcommands:
|
||||||
|
- models download: Launch interactive model downloader.
|
||||||
|
- models update: Verify/update local models (non-interactive).
|
||||||
|
- plugins list|info|run: Discover and run plugins.
|
||||||
- completions <shell>: Write shell completion script to stdout.
|
- completions <shell>: Write shell completion script to stdout.
|
||||||
- man: Write a man page to stdout.
|
- man: Write a man page to stdout.
|
||||||
|
|
||||||
|
Reference in New Issue
Block a user