Wire since filter and refresh semester metadata

This commit is contained in:
2025-11-15 01:25:53 +01:00
parent c12310efca
commit d8de882cdd
7 changed files with 181 additions and 33 deletions

View File

@@ -7,7 +7,7 @@
- `init-config` writes a ready-to-edit config template (respecting `--download-root` and `--force` to overwrite).
- `auth` subcommand stores Base64-encoded credentials per profile (passwords are never logged).
- `list-courses` fetches `/users/me`, paginates enrolled courses, infers semester keys, caches the metadata, and prints a concise table.
- `sync` traverses every course folder/file tree, normalizes names, streams downloads to disk, tracks checksums/remote timestamps, and supports `--dry-run` plus `--prune` to delete orphaned files.
- `sync` traverses every course folder/file tree, normalizes names, streams downloads to disk, tracks checksums/remote timestamps, and supports `--dry-run`, `--prune`, and `--since <semester|date>` filters (e.g., `--since ws2526` or `--since 01032024`).
- XDG-compliant config (`~/.config/studip-sync/config.toml`) and state (`~/.local/share/studip-sync/state.toml`) stores everything in TOML.
- Extensive logging controls: `--quiet`, `--verbose/-v`, `--debug`, and `--json`.
@@ -15,7 +15,7 @@
- Config lives under `${XDG_CONFIG_HOME:-~/.config}/studip-sync/config.toml`. Override this with `--config-dir` if you want the config somewhere else.
- State is cached in `${XDG_DATA_HOME:-~/.local/share}/studip-sync/state.toml`; `--data-dir` only changes this location (and anything else the tool stores under data, such as the default downloads folder). Use this when you want the state cache on a different disk but keep the config where it is.
- `download_root` determines where files land. If omitted, it falls back to `<data-dir>/downloads`, so moving the data dir automatically relocates the default downloads. Setting `download_root` explicitly decouples it from the data dir. Each path segment is sanitized to keep names human-readable yet filesystem-safe.
- `download_root` determines where files land. If omitted, it falls back to `<data-dir>/downloads`, so moving the data dir automatically relocates the default downloads. Setting `download_root` explicitly decouples it from the data dir. Each path segment is sanitized to keep names human-readable yet filesystem-safe. Semester entries cached in `state.toml` now include start/end timestamps so CLI filters such as `--since ws2526` know when a term begins (`list-courses --refresh` also re-fetches any cached semester still missing those timestamps).
## Getting Started
@@ -70,20 +70,20 @@ max_concurrent_downloads = 3 # placeholder for future concurrency control
| `init-config` | Write a default config template (fails if config exists unless forced). | `--force`, `--download-root` |
| `auth` | Collect username/password, encode them, and save them to the active profile. | `--non-interactive`, `--username`, `--password` |
| `list-courses` | List cached or freshly fetched courses with semester keys and IDs. | `--refresh` |
| `sync` | Download files for every enrolled course into the local tree. | `--dry-run`, `--prune`, `--since` *(currently just records the user-provided timestamp; the API filtering hook is planned but not implemented yet)* |
| `sync` | Download files for every enrolled course into the local tree. | `--dry-run`, `--prune`, `--since <semester key | DDMMYY | DDMMYYYY | RFC3339>` |
Global flags: `--quiet`, `--debug`, `--json`, `-v/--verbose` (stackable), `--config-dir`, `--data-dir` (state + default downloads), `--profile`.
## Sync Behavior
1. Resolve user ID (cached in `state.toml`) and fetch current courses.
2. Cache missing semesters via `/semesters/{id}` and infer keys like `ws2425` / `ss25`.
2. Cache missing semesters via `/semesters/{id}` and infer keys like `ws2425` / `ss25`. When `--refresh` is passed, already-known semesters that never recorded a `start` timestamp are re-fetched so `--since` filters have the data they need.
3. For each course:
- Walk folders using the JSON:API pagination helpers; fetch nested folders via `/folders/{id}/folders`.
- List file refs via `/folders/{id}/file-refs`, normalize filenames, and ensure unique siblings through a `NameRegistry`.
- Skip downloads when the local file exists and matches the stored checksum / size / remote `chdate`.
- Stream downloads to `*.part`, hash contents on the fly, then rename atomically to the final path.
4. Maintain a set of remote files so `--prune` can remove local files that no longer exist remotely (and optionally delete now-empty directories). `--since` is accepted for future incremental sync work and currently acts as an annotation only—no API filters are applied yet.
4. Maintain a set of remote files so `--prune` can remove local files that no longer exist remotely (and optionally delete now-empty directories). When `--since` is provided, files whose remote `chdate` precedes the resolved timestamp (semester start or explicit date) are skipped; newer files continue through the regular checksum/size logic.
5. `--dry-run` prints planned work but never writes to disk.
## Development Notes