GeoData/README.md

GeoData Heightmap Export

This repository converts DGM1 elevation tiles into Unity-ready 16-bit PNG heightmaps and a placement manifest. It relies on GDAL for mosaicking, resampling, and scaling to UInt16 ranges Unity expects.

Prerequisites

• GDAL installed with Python bindings (osgeo importable).
• Python 3.9+ available on PATH.
• DGM1 source tiles placed under raw/dgm1/ as dgm1_<utm_zone>_<easting>_<northing>.tif with matching .tfw files.
• Raw inputs (raw/dop20/, raw/citygml/lod1/, raw/citygml/lod2/) are kept out of git; keep them locally or document how to fetch/regenerate.

Environment setup (uv)

• Create a project venv: uv venv && source .venv/bin/activate.
• Install dependencies from pyproject.toml: uv sync (generates uv.lock).
• If wheels fail to resolve, ensure system GDAL is present (e.g., brew install gdal or apt-get install gdal-bin libgdal-dev), then rerun uv sync.
• Create the default directory tree and config: uv run python geodata_to_unity.py --setup (or bash scripts/setup_dirs.sh for directories only).

Repository Layout

• raw/ — working inputs (not versioned): raw/dgm1/, raw/dop20/jp2/, raw/citygml/lod1/, raw/citygml/lod2/.
• archive/ — offline storage for untouched downloads (e.g., zipped DOP/CityGML tiles, dop20 filelist).
• work/ — intermediates such as dgm.vrt and _tmp.tif files; safe to delete/regenerate.
• export_unity/height_png16/ — final 16-bit PNG heightmaps for Unity import.
• export_unity/tile_index.csv — manifest mapping tile IDs to world bounds and global min/max used for scaling.
• export_unity/ortho_jpg/ — cropped orthophoto tiles aligned to the terrain grid (JPEG + worldfiles).
• geodata_to_unity.py — main CLI (uses geodata_pipeline/ library modules).
• scripts/ — helpers to create the directory tree and fetch DOP20 inputs.
• geodata_config.json — generated config (see geodata_config.example.json for defaults).
• AGENTS.md — contributor guide.

Quick Start

1. Activate the uv venv (source .venv/bin/activate) or prefix commands with uv run.
2. Initialize config + directories: uv run python geodata_to_unity.py --setup.
3. Export assets (builds VRTs automatically if missing):

   uv run python geodata_to_unity.py --export all
   # heightmaps only: uv run python geodata_to_unity.py --export heightmap
   # textures only:   uv run python geodata_to_unity.py --export textures
   

4. Import the PNGs into Unity Terrains using tile_index.csv for placement and consistent height scaling (0–65535).

Key Commands

• Refresh VRT: gdalbuildvrt work/dgm.vrt raw/dgm1/*.tif
• Run export pipeline: uv run python geodata_to_unity.py --export all
• Inspect an output tile: gdalinfo export_unity/height_png16/<tile>.png | head
• Override config paths: use --config <path>, --raw-dgm1-path <dir>, --raw-dop20-path <dir>.
• Build raws from archives: uv run python geodata_to_unity.py --build-from-archive --export all (unzips archive/*; dop20 filelist stays in archive/dop20/ for the downloader).
• Rebuild VRTs after moving data: add --force-vrt.

Workflow Notes

• The pipeline computes a global min/max from the VRT to scale all tiles consistently; adjust heightmap.out_res or heightmap.resample in geodata_config.json if your AOI or target resolution changes.
• _tmp.tif files in work/ are transient; you can delete work/ to force a clean rebuild.
• Keep file names stable to avoid churn in Unity scenes; re-exports overwrite in place.
• Large raw datasets are intentionally excluded from version control—document download sources or scripts instead of committing data.
• Additional inputs: download helper lives in scripts/dlscript_dop20.sh and pulls JP2/J2W/XML orthophotos listed in archive/dop20/filelist.txt (one URL per line); archive/ can hold zipped 3D building tiles for future use.
• Handoff to Unity: copy/sync export_unity/height_png16/ and export_unity/tile_index.csv into DTrierFlood/Assets/GeoData/ before running the Unity-side importer. Keep heightmap.out_res aligned with the importer’s expected resolution (currently 1025).

Orthophotos (textures)

1. Ensure DOP assets are present in raw/dop20/jp2/, raw/dop20/j2w/, and raw/dop20/meta/; use scripts/dlscript_dop20.sh to fetch JP2/J2W/XML entries listed in archive/dop20/filelist.txt (one URL per line).
2. From GeoData/, run:

   uv run python geodata_to_unity.py --export textures
   

   This builds work/dop.vrt if missing and writes export_unity/ortho_jpg/<tile>.jpg + .jgw aligned to tile_index.csv.
   • If you see Computed -srcwin ... falls partially outside source raster extent warnings, the DOP coverage is slightly smaller than the tile footprint; edge pixels will be filled with NoData/zeros. Add adjacent JP2s or shrink the requested window if you need to avoid the warning.
   • The download script relies on a Linux/OpenSSL toolchain with system CA bundle at /etc/ssl/certs/ca-certificates.crt; it builds a trust chain by fetching the geobasis intermediate. macOS/Windows users should either provide a combined CA via CURL_CA_BUNDLE or download with a browser/wget and place files manually.

Buildings

The building export pipeline is temporarily disabled while we choose a mesh conversion approach (GDAL lacks a native OBJ writer). CityGML LoD2 sources remain in raw/citygml/lod2/ locally (ignored in git); consider CityGML→glTF/OBJ tools (e.g., citygml-tools + cityjson2gltf) for future integration.