56 lines
6.4 KiB
Markdown
56 lines
6.4 KiB
Markdown
# Repository Guidelines
|
||
|
||
## Project Structure & Module Organization
|
||
- `geodata_to_unity.py` is the main CLI; library code lives in `geodata_pipeline/` (`heightmaps.py`, `orthophotos.py`, `config.py`, `setup_helpers.py`). Legacy wrapper scripts have been removed; use `geodata_to_unity.py` directly.
|
||
- Working inputs (ignored): `raw/dgm1/`, `raw/dop20/jp2/`, `raw/citygml/lod1/`, `raw/citygml/lod2/`.
|
||
- Archives (ignored): `archive/dgm1/`, `archive/dop20/`, `archive/citygml/lod1/`, `archive/citygml/lod2/` (zip storage + dop20 filelist).
|
||
- Config: `geodata_config.json` (generated) or `geodata_config.example.json` for defaults.
|
||
- `export_unity/` is generated output (heightmaps, orthophotos, manifest). `work/` holds intermediates and is disposable.
|
||
|
||
## Build, Test, and Development Commands
|
||
- Create a venv: `uv venv && source .venv/bin/activate`. Install deps: `uv sync` (generates `uv.lock`).
|
||
- If wheels fail, install system GDAL first (e.g., `brew install gdal` or `apt-get install gdal-bin libgdal-dev`), then rerun `uv sync`.
|
||
- Prepare the directory tree and config: `uv run python geodata_to_unity.py --setup` (or `bash scripts/setup_dirs.sh` for directories only).
|
||
- Heightmap export: `uv run python geodata_to_unity.py --export heightmap`.
|
||
- Orthophoto export: `uv run python geodata_to_unity.py --export textures` (requires JP2s under `raw/dop20/jp2/`; use `bash scripts/dlscript_dop20.sh` to fetch JP2/J2W/XML listed in `archive/dop20/filelist.txt`).
|
||
- Refresh VRT manually if needed: `gdalbuildvrt work/dgm.vrt raw/dgm1/*.tif`.
|
||
- Inspect a result: `gdalinfo export_unity/height_png16/<tile>.png | head` to sanity-check bounds and scaling.
|
||
- Populate raw data from archives: `uv run python geodata_to_unity.py --build-from-archive --export all` (unzips `archive/*`; dop20 filelist stays in archive for the downloader).
|
||
- Rebuild VRTs after moving data: add `--force-vrt`.
|
||
- Expected warning: `Computed -srcwin ... falls partially outside source raster extent` means the DOP coverage is slightly smaller than the tile footprint; edge pixels will be filled with NoData/zeros. Add adjacent JP2s or shrink the requested window if you need to silence it.
|
||
- Scripts accept CLI overrides (e.g., `--config`, `--raw-dgm1-path`, `--raw-dop20-path`, `--export`, `--build-from-archive`); run `uv run python geodata_to_unity.py -h` to see options.
|
||
- DOP20 downloader assumes Linux/OpenSSL with system CA at `/etc/ssl/certs/ca-certificates.crt` to build a trust chain from the geobasis site. macOS/Windows users should either set `CURL_CA_BUNDLE` to a combined CA or download manually and place files in `raw/dop20/`.
|
||
- Orthophotos depend on a prebuilt manifest: run the heightmap export first (or `--export all`) so `export_unity/tile_index.csv` exists.
|
||
- VRTs are built from whatever is present in the raw directories; empty directories will fail fast. Use `--force-vrt` after moving data or deleting `work/`.
|
||
|
||
## Coding Style & Naming Conventions
|
||
- Python scripts use 4-space indentation, early-exit error handling, and `SystemExit` for fatal issues; follow PEP 8 where practical.
|
||
- Keep tile IDs stable (base filename without extension); avoid renaming inputs to reduce churn downstream.
|
||
- Prefer explicit config fields for tunables (`heightmap.out_res`, `heightmap.resample`, `ortho.out_res`, `ortho.jpeg_quality`) and log with clear context.
|
||
|
||
## Testing Guidelines
|
||
- No automated tests yet; rely on manual validation: run exports on a small tile subset, open outputs in `gdalinfo` or GIS viewer, and confirm `tile_index.csv` aligns with Unity expectations.
|
||
- When changing scaling or resolution, compare before/after stats (min/max) and spot-check terrain in Unity.
|
||
|
||
## Current Pipelines (v1 WIP)
|
||
- **Heightmaps/Orthos**: unchanged; see README.
|
||
- **Buildings (new)**: `--export buildings` converts LoD2 CityGML → CityJSON (citygml-tools), triangulates (cjio), rebases to tile-local XY, merges per tile into one GLB (1 mesh, roof/wall primitives), decimates to budget, planar-UV roofs with embedded DOP20 tile texture (unlit), walls colored from ortho fallback, axes glTF-friendly (x=east, y=height, z=-north). Open: better wall coloring (BDOM/LPO), stronger simplification, footprint-aware ground snap (currently clamp to DTM).
|
||
- **Trees (new)**: `--export trees` uses DOM1–DGM1 CHM + CityGML building mask (buffered) to find trees, roughness/confidence heuristic with optional LPO/BDOM boost, caps by count. Outputs per-tile CSV and chunked GLBs (4×4 by default) built from 16 procedural proxies; instancing toggle in config; shared proxy library emitted.
|
||
- **Tools**: expects `citygml-tools-2.4.0/citygml-tools` and `cjio` on PATH (override `CJIO` env). Orthos must exist for roofs/walls.
|
||
|
||
## Commit & Pull Request Guidelines
|
||
- Commit messages are short, imperative summaries (e.g., "Ignore generated orthophotos"). Group related changes per commit; commit `uv.lock` when dependency versions change.
|
||
- Before opening a PR: describe the change, list commands run, note data locations (not committed), and include any screenshots from Unity/GIS if visuals changed.
|
||
- Ensure raw datasets and large intermediates stay out of git; verify `.gitignore` still covers generated files after changes.
|
||
|
||
## Security & Data Handling
|
||
- Keep raw geodata local; avoid publishing source tiles or credentials. Document download sources/scripts instead of committing data.
|
||
- Outputs may be large; prefer syncing `export_unity/` artifacts via project-specific channels rather than embedding in the repo.
|
||
|
||
## Pipeline Behavior (for maintainers)
|
||
- Heightmaps: a VRT of all DGM1 tiles is warped per tile footprint, scaled once using the global min/max from the VRT to `[0, 65535]`, and written with worldfiles. Manifest rows include bounds, global min/max, and `out_res`.
|
||
- Orthophotos: `export_orthophotos` reuses the manifest for target windows and will abort if it is missing; JPEGs are resampled to `ortho.out_res` with worldfiles and default JPEG quality 90.
|
||
- Temporary files are written to `work/*_tmp.tif` and cleaned with broad `*.aux.xml` patterns in `work/` and the raw DGM1 directory—avoid placing non-GDAL aux files there.
|
||
- `materialize_archives` unzips every `*.zip` under `archive/*` into the matching raw folders and copies `archive/dop20/filelist.txt` next to `raw/dop20/` for the downloader.
|
||
- `geodata_config.example.json` includes `archives.dop20_filelist` for human reference; the dataclass ignores it, so keep the example in sync with actual CLI options rather than adding new unused keys.
|