76 lines
2.7 KiB
Markdown
76 lines
2.7 KiB
Markdown
# Development
|
|
|
|
This document describes how to build, test, and run PolyScribe locally, and how to work with models during development.
|
|
|
|
Prerequisites
|
|
- Rust toolchain via rustup (recommended).
|
|
- ffmpeg installed and on PATH.
|
|
- For GPU builds: appropriate toolkits and libraries installed (CUDA/ROCm/Vulkan).
|
|
|
|
Rust toolchain
|
|
- Target stable Rust.
|
|
- Recommended:
|
|
- rustup install stable
|
|
- rustup default stable
|
|
|
|
Build
|
|
- CPU-only (default):
|
|
- cargo build
|
|
- cargo test
|
|
- Enable GPU features at build time:
|
|
- CUDA: cargo build --features gpu-cuda
|
|
- HIP: cargo build --features gpu-hip
|
|
- Vulkan: cargo build --features gpu-vulkan
|
|
|
|
Run locally
|
|
- Development model directory defaults to ./models when built in debug mode.
|
|
- You can override:
|
|
- POLYSCRIBE_MODELS_DIR=/path/to/models
|
|
- WHISPER_MODEL=/path/to/model.bin (forces a specific file)
|
|
- Example run:
|
|
- cargo run -- -v -o output samples/example.mp3
|
|
|
|
Models during development
|
|
- Interactive downloader:
|
|
- cargo run -- models download
|
|
- Non-interactive update (checks sizes/hashes, downloads if missing):
|
|
- cargo run -- models update --no-interaction -q
|
|
|
|
Tests
|
|
- Run all tests:
|
|
- cargo test
|
|
- The test suite includes CLI-oriented integration tests and unit tests. Some tests simulate GPU detection using env vars (POLYSCRIBE_TEST_FORCE_*). Do not rely on these flags in production code.
|
|
|
|
Clippy & formatting
|
|
- Run lint checks and treat warnings as errors:
|
|
- cargo clippy --all-targets -- -D warnings
|
|
- Check formatting:
|
|
- cargo fmt --all -- --check
|
|
- Common warnings can often be fixed by simplifying code, removing unused imports, and following idiomatic patterns.
|
|
|
|
Code layout
|
|
- src/lib.rs: core library surface and re-exports
|
|
- backend: runtime backend selection and transcription glue
|
|
- models: model discovery, manifest, download/update logic
|
|
- src/main.rs: CLI parsing, I/O orchestration, logging, and workflows
|
|
- tests/: integration tests
|
|
|
|
Adding a feature
|
|
- Find the closest existing module (backend/models/main) and add a small, focused unit test.
|
|
- Keep user-facing changes documented in README/docs and update CHANGELOG.md.
|
|
- Prefer small functions with clear responsibilities; avoid exposing unnecessary items publicly.
|
|
- Follow existing logging style (elog!/wlog!/ilog!/dlog!).
|
|
|
|
Running the model downloader
|
|
- Interactive:
|
|
- cargo run -- models download
|
|
- Non-interactive suggestions for CI:
|
|
- POLYSCRIBE_MODELS_DIR=$PWD/models \
|
|
cargo run -- models update --no-interaction -q
|
|
|
|
Env var examples for local testing
|
|
- Use a local copy of models and a specific model file:
|
|
- export POLYSCRIBE_MODELS_DIR=$PWD/models
|
|
- export WHISPER_MODEL=$PWD/models/large-v3-turbo-q8_0.bin
|
|
- Test manifests/offline copies are handled internally. For full offline runs, pre-populate models/ with the desired .bin files.
|