GeoData/AGENTS.md

Repository Guidelines

Project Structure & Module Organization

• geodata_to_unity.py is the main CLI; library code lives in geodata_pipeline/ (heightmaps.py, orthophotos.py, config.py, setup_helpers.py). Legacy wrapper scripts have been removed; use geodata_to_unity.py directly.
• Working inputs (ignored): raw/dgm1/, raw/dop20/jp2/, raw/citygml/lod1/, raw/citygml/lod2/.
• Archives (ignored): archive/dgm1/, archive/dop20/, archive/citygml/lod1/, archive/citygml/lod2/ (zip storage + dop20 filelist).
• Config: geodata_config.json (generated) or geodata_config.example.json for defaults.
• export_unity/ is generated output (heightmaps, orthophotos, manifest). work/ holds intermediates and is disposable.

Build, Test, and Development Commands

• Create a venv: uv venv && source .venv/bin/activate. Install deps: uv sync (generates uv.lock).
• If wheels fail, install system GDAL first (e.g., brew install gdal or apt-get install gdal-bin libgdal-dev), then rerun uv sync.
• Prepare the directory tree and config: uv run python geodata_to_unity.py --setup (or bash scripts/setup_dirs.sh for directories only).
• Heightmap export: uv run python geodata_to_unity.py --export heightmap.
• Orthophoto export: uv run python geodata_to_unity.py --export textures (requires JP2s under raw/dop20/jp2/; use bash scripts/dlscript_dop20.sh to fetch JP2/J2W/XML listed in archive/dop20/filelist.txt).
• Refresh VRT manually if needed: gdalbuildvrt work/dgm.vrt raw/dgm1/*.tif.
• Inspect a result: gdalinfo export_unity/height_png16/<tile>.png | head to sanity-check bounds and scaling.
• Populate raw data from archives: uv run python geodata_to_unity.py --build-from-archive --export all (unzips archive/*; dop20 filelist stays in archive for the downloader).
• Rebuild VRTs after moving data: add --force-vrt.
• Expected warning: Computed -srcwin ... falls partially outside source raster extent means the DOP coverage is slightly smaller than the tile footprint; edge pixels will be filled with NoData/zeros. Add adjacent JP2s or shrink the requested window if you need to silence it.
• Scripts accept CLI overrides (e.g., --config, --raw-dgm1-path, --raw-dop20-path, --export, --build-from-archive); run uv run python geodata_to_unity.py -h to see options.
• DOP20 downloader assumes Linux/OpenSSL with system CA at /etc/ssl/certs/ca-certificates.crt to build a trust chain from the geobasis site. macOS/Windows users should either set CURL_CA_BUNDLE to a combined CA or download manually and place files in raw/dop20/.
• Orthophotos depend on a prebuilt manifest: run the heightmap export first (or --export all) so export_unity/tile_index.csv exists.
• VRTs are built from whatever is present in the raw directories; empty directories will fail fast. Use --force-vrt after moving data or deleting work/.

Coding Style & Naming Conventions

• Python scripts use 4-space indentation, early-exit error handling, and SystemExit for fatal issues; follow PEP 8 where practical.
• Keep tile IDs stable (base filename without extension); avoid renaming inputs to reduce churn downstream.
• Prefer explicit config fields for tunables (heightmap.out_res, heightmap.resample, ortho.out_res, ortho.jpeg_quality) and log with clear context.

Testing Guidelines

• No automated tests yet; rely on manual validation: run exports on a small tile subset, open outputs in gdalinfo or GIS viewer, and confirm tile_index.csv aligns with Unity expectations.
• When changing scaling or resolution, compare before/after stats (min/max) and spot-check terrain in Unity.

Commit & Pull Request Guidelines

• Commit messages are short, imperative summaries (e.g., "Ignore generated orthophotos"). Group related changes per commit; commit uv.lock when dependency versions change.
• Before opening a PR: describe the change, list commands run, note data locations (not committed), and include any screenshots from Unity/GIS if visuals changed.
• Ensure raw datasets and large intermediates stay out of git; verify .gitignore still covers generated files after changes.

Security & Data Handling

• Keep raw geodata local; avoid publishing source tiles or credentials. Document download sources/scripts instead of committing data.
• Outputs may be large; prefer syncing export_unity/ artifacts via project-specific channels rather than embedding in the repo.

Pipeline Behavior (for maintainers)

• Heightmaps: a VRT of all DGM1 tiles is warped per tile footprint, scaled once using the global min/max from the VRT to [0, 65535], and written with worldfiles. Manifest rows include bounds, global min/max, and out_res.
• Orthophotos: export_orthophotos reuses the manifest for target windows and will abort if it is missing; JPEGs are resampled to ortho.out_res with worldfiles and default JPEG quality 90.
• Temporary files are written to work/*_tmp.tif and cleaned with broad *.aux.xml patterns in work/ and the raw DGM1 directory—avoid placing non-GDAL aux files there.
• materialize_archives unzips every *.zip under archive/* into the matching raw folders and copies archive/dop20/filelist.txt next to raw/dop20/ for the downloader.
• geodata_config.example.json includes archives.dop20_filelist for human reference; the dataclass ignores it, so keep the example in sync with actual CLI options rather than adding new unused keys.