Wire since filter and refresh semester metadata
This commit is contained in:
10
README.md
10
README.md
@@ -7,7 +7,7 @@
|
||||
- `init-config` writes a ready-to-edit config template (respecting `--download-root` and `--force` to overwrite).
|
||||
- `auth` subcommand stores Base64-encoded credentials per profile (passwords are never logged).
|
||||
- `list-courses` fetches `/users/me`, paginates enrolled courses, infers semester keys, caches the metadata, and prints a concise table.
|
||||
- `sync` traverses every course folder/file tree, normalizes names, streams downloads to disk, tracks checksums/remote timestamps, and supports `--dry-run` plus `--prune` to delete orphaned files.
|
||||
- `sync` traverses every course folder/file tree, normalizes names, streams downloads to disk, tracks checksums/remote timestamps, and supports `--dry-run`, `--prune`, and `--since <semester|date>` filters (e.g., `--since ws2526` or `--since 01032024`).
|
||||
- XDG-compliant config (`~/.config/studip-sync/config.toml`) and state (`~/.local/share/studip-sync/state.toml`) stores everything in TOML.
|
||||
- Extensive logging controls: `--quiet`, `--verbose/-v`, `--debug`, and `--json`.
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
|
||||
- Config lives under `${XDG_CONFIG_HOME:-~/.config}/studip-sync/config.toml`. Override this with `--config-dir` if you want the config somewhere else.
|
||||
- State is cached in `${XDG_DATA_HOME:-~/.local/share}/studip-sync/state.toml`; `--data-dir` only changes this location (and anything else the tool stores under data, such as the default downloads folder). Use this when you want the state cache on a different disk but keep the config where it is.
|
||||
- `download_root` determines where files land. If omitted, it falls back to `<data-dir>/downloads`, so moving the data dir automatically relocates the default downloads. Setting `download_root` explicitly decouples it from the data dir. Each path segment is sanitized to keep names human-readable yet filesystem-safe.
|
||||
- `download_root` determines where files land. If omitted, it falls back to `<data-dir>/downloads`, so moving the data dir automatically relocates the default downloads. Setting `download_root` explicitly decouples it from the data dir. Each path segment is sanitized to keep names human-readable yet filesystem-safe. Semester entries cached in `state.toml` now include start/end timestamps so CLI filters such as `--since ws2526` know when a term begins (`list-courses --refresh` also re-fetches any cached semester still missing those timestamps).
|
||||
|
||||
## Getting Started
|
||||
|
||||
@@ -70,20 +70,20 @@ max_concurrent_downloads = 3 # placeholder for future concurrency control
|
||||
| `init-config` | Write a default config template (fails if config exists unless forced). | `--force`, `--download-root` |
|
||||
| `auth` | Collect username/password, encode them, and save them to the active profile. | `--non-interactive`, `--username`, `--password` |
|
||||
| `list-courses` | List cached or freshly fetched courses with semester keys and IDs. | `--refresh` |
|
||||
| `sync` | Download files for every enrolled course into the local tree. | `--dry-run`, `--prune`, `--since` *(currently just records the user-provided timestamp; the API filtering hook is planned but not implemented yet)* |
|
||||
| `sync` | Download files for every enrolled course into the local tree. | `--dry-run`, `--prune`, `--since <semester key | DDMMYY | DDMMYYYY | RFC3339>` |
|
||||
|
||||
Global flags: `--quiet`, `--debug`, `--json`, `-v/--verbose` (stackable), `--config-dir`, `--data-dir` (state + default downloads), `--profile`.
|
||||
|
||||
## Sync Behavior
|
||||
|
||||
1. Resolve user ID (cached in `state.toml`) and fetch current courses.
|
||||
2. Cache missing semesters via `/semesters/{id}` and infer keys like `ws2425` / `ss25`.
|
||||
2. Cache missing semesters via `/semesters/{id}` and infer keys like `ws2425` / `ss25`. When `--refresh` is passed, already-known semesters that never recorded a `start` timestamp are re-fetched so `--since` filters have the data they need.
|
||||
3. For each course:
|
||||
- Walk folders using the JSON:API pagination helpers; fetch nested folders via `/folders/{id}/folders`.
|
||||
- List file refs via `/folders/{id}/file-refs`, normalize filenames, and ensure unique siblings through a `NameRegistry`.
|
||||
- Skip downloads when the local file exists and matches the stored checksum / size / remote `chdate`.
|
||||
- Stream downloads to `*.part`, hash contents on the fly, then rename atomically to the final path.
|
||||
4. Maintain a set of remote files so `--prune` can remove local files that no longer exist remotely (and optionally delete now-empty directories). `--since` is accepted for future incremental sync work and currently acts as an annotation only—no API filters are applied yet.
|
||||
4. Maintain a set of remote files so `--prune` can remove local files that no longer exist remotely (and optionally delete now-empty directories). When `--since` is provided, files whose remote `chdate` precedes the resolved timestamp (semester start or explicit date) are skipped; newer files continue through the regular checksum/size logic.
|
||||
5. `--dry-run` prints planned work but never writes to disk.
|
||||
|
||||
## Development Notes
|
||||
|
||||
Reference in New Issue
Block a user