docs: refresh agent + user docs
This commit is contained in:
104
README.md
104
README.md
@@ -1,20 +1,96 @@
|
||||
# studip-sync
|
||||
|
||||
Command-line tool written in Rust (edition 2024) to sync files from the Stud.IP JSON:API to a local filesystem tree.
|
||||
The repository contains the cargo project (with CLI/config/state scaffolding) plus an offline copy of the JSON:API documentation (in `jsonapi/`).
|
||||
`studip-sync` is a Rust CLI that performs a one-way sync of Stud.IP course materials (via the Uni Trier JSON:API) into a local directory tree. The tool persists config/state in TOML, talks to the API with HTTP Basic auth, and keeps the local filesystem organized as `<download_root>/<semester>/<course>/<folder>/<files>`.
|
||||
|
||||
## Current status
|
||||
## Key Features
|
||||
|
||||
- `cargo` binary crate scaffolded with name `studip-sync`, pinned to Rust edition 2024.
|
||||
- CLI implemented with `auth`, `sync`, and `list-courses` subcommands plus logging/verbosity flags.
|
||||
- Config/state loaders wired up with XDG path resolution, multi-profile support, and JSON/quiet/debug logging modes.
|
||||
- `studip-sync auth` prompts for credentials (or reads `--username/--password` / `STUDIP_SYNC_USERNAME|PASSWORD`) and stores the base64 Basic auth token in the active profile.
|
||||
- `studip-sync list-courses` now talks to the Stud.IP JSON:API, caches user/semester/course metadata, and prints a table of enrolled courses (with pagination + semester-key inference).
|
||||
- `studip-sync sync` walks courses → folders → file refs via the JSON:API, downloads missing or changed files (streamed to disk), and supports `--dry-run` / `--prune` cleanup.
|
||||
- Ready for further implementation of Stud.IP HTTP client, sync logic, and actual command behaviors.
|
||||
- `auth` subcommand stores Base64-encoded credentials per profile (passwords are never logged).
|
||||
- `list-courses` fetches `/users/me`, paginates enrolled courses, infers semester keys, caches the metadata, and prints a concise table.
|
||||
- `sync` traverses every course folder/file tree, normalizes names, streams downloads to disk, tracks checksums/remote timestamps, and supports `--dry-run` plus `--prune` to delete orphaned files.
|
||||
- XDG-compliant config (`~/.config/studip-sync/config.toml`) and state (`~/.local/share/studip-sync/state.toml`) stores everything in TOML.
|
||||
- Extensive logging controls: `--quiet`, `--verbose/-v`, `--debug`, and `--json`.
|
||||
|
||||
## Next steps
|
||||
## Directory Layout & Data Files
|
||||
|
||||
1. Add configurable download concurrency plus richer progress/logging (per-course summaries, ETA) while keeping memory usage low.
|
||||
2. Implement smarter state usage (incremental `filter[since]` queries, resume checkpoints) and expand pruning to detect/cleanup orphaned state entries.
|
||||
3. Add tests and ensure `cargo fmt` + `cargo clippy --all-targets --all-features -- -D warnings` + `cargo test` pass (and wire into CI if applicable).
|
||||
- Config lives under `${XDG_CONFIG_HOME:-~/.config}/studip-sync/config.toml`. A `default` profile is created automatically and stores the `basic_auth_b64`, base URL, JSON:API path, download root, etc.
|
||||
- State is cached in `${XDG_DATA_HOME:-~/.local/share}/studip-sync/state.toml` with per-profile sections for user/semester/course/file metadata.
|
||||
- Downloads default to `${XDG_DATA_HOME:-~/.local/share}/studip-sync/downloads`, but you can override `download_root` in the config to point anywhere else. Each path segment is sanitized to keep names human-readable yet filesystem-safe.
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. **Prerequisites** – Install a recent Rust toolchain (Rust 1.75+ recommended) and ensure you can reach `https://studip.uni-trier.de`.
|
||||
2. **Build & validate** – From the repo root run:
|
||||
```bash
|
||||
cargo fmt --all -- --check
|
||||
cargo clippy --all-targets --all-features -- -D warnings
|
||||
cargo test
|
||||
```
|
||||
3. **First run**:
|
||||
```bash
|
||||
# Store credentials (prompts for username/password by default)
|
||||
cargo run -- auth
|
||||
|
||||
# Inspect courses and cache semester data
|
||||
cargo run -- list-courses --refresh
|
||||
|
||||
# Perform a dry-run sync to see planned actions
|
||||
cargo run -- sync --dry-run
|
||||
|
||||
# Run the real sync (omit --dry-run); add --prune to delete stray files
|
||||
cargo run -- sync --prune
|
||||
```
|
||||
Use `--profile`, `--config-dir`, or `--data-dir` when working with multiple identities or non-standard paths.
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
Example `config.toml`:
|
||||
|
||||
```toml
|
||||
default_profile = "default"
|
||||
|
||||
[profiles.default]
|
||||
base_url = "https://studip.uni-trier.de"
|
||||
jsonapi_path = "/jsonapi.php/v1"
|
||||
basic_auth_b64 = "base64(username:password)"
|
||||
download_root = "/home/alex/StudIP"
|
||||
max_concurrent_downloads = 3 # placeholder for future concurrency control
|
||||
```
|
||||
|
||||
- The file is written with `0600` permissions. Never commit credentials—`auth` manages them interactively or through `--username/--password` / `STUDIP_SYNC_USERNAME|PASSWORD`.
|
||||
- Multiple profiles can be added under `[profiles.<name>]`; pass `--profile <name>` when invoking the CLI to switch.
|
||||
|
||||
## CLI Reference
|
||||
|
||||
| Subcommand | Description | Helpful flags |
|
||||
| --- | --- | --- |
|
||||
| `auth` | Collect username/password, encode them, and save them to the active profile. | `--non-interactive`, `--username`, `--password` |
|
||||
| `list-courses` | List cached or freshly fetched courses with semester keys and IDs. | `--refresh` |
|
||||
| `sync` | Download files for every enrolled course into the local tree. | `--dry-run`, `--prune`, `--since` *(reserved for future API filters)* |
|
||||
|
||||
Global flags: `--quiet`, `--debug`, `--json`, `-v/--verbose` (stackable), `--config-dir`, `--data-dir`, `--profile`.
|
||||
|
||||
## Sync Behavior
|
||||
|
||||
1. Resolve user ID (cached in `state.toml`) and fetch current courses.
|
||||
2. Cache missing semesters via `/semesters/{id}` and infer keys like `ws2425` / `ss25`.
|
||||
3. For each course:
|
||||
- Walk folders using the JSON:API pagination helpers; fetch nested folders via `/folders/{id}/folders`.
|
||||
- List file refs via `/folders/{id}/file-refs`, normalize filenames, and ensure unique siblings through a `NameRegistry`.
|
||||
- Skip downloads when the local file exists and matches the stored checksum / size / remote `chdate`.
|
||||
- Stream downloads to `*.part`, hash contents on the fly, then rename atomically to the final path.
|
||||
4. Maintain a set of remote files so `--prune` can remove local files that no longer exist remotely (and optionally delete now-empty directories).
|
||||
5. `--dry-run` prints planned work but never writes to disk.
|
||||
|
||||
## Development Notes
|
||||
|
||||
- The HTTP client limits itself to GETs with Basic auth; non-success responses are surfaced verbatim via `anyhow`.
|
||||
- All downloads currently run sequentially; `ConfigProfile::max_concurrent_downloads` is in place for a future bounded task executor.
|
||||
- Offline JSON:API documentation lives under `docs/studip/` to keep this repo usable without network access.
|
||||
|
||||
## Roadmap / Known Gaps
|
||||
|
||||
1. Implement real concurrent downloads that honor `max_concurrent_downloads`.
|
||||
2. Wire `--since` into Stud.IP filters (if available) or local heuristics to reduce API load.
|
||||
3. Add unit/integration tests (`semesters::infer_key`, naming helpers, pruning) and consider fixtures for Stud.IP responses.
|
||||
4. Improve auth failure UX by detecting 401/403 and prompting the user to re-run `studip-sync auth`.
|
||||
5. Evaluate whether the crate should target Rust 2021 (per the original requirement) or explicitly document Rust 2024 as the minimum supported version.
|
||||
|
||||
Reference in New Issue
Block a user