From 27ec81f53bd20a83205e08a0ce294ba914ab0c74 Mon Sep 17 00:00:00 2001 From: "s0wlz (Matthias Puchstein)" Date: Wed, 11 Feb 2026 23:56:32 +0100 Subject: [PATCH] README update --- README.md | 133 ++++++++++++++++++++---------------------------------- 1 file changed, 49 insertions(+), 84 deletions(-) diff --git a/README.md b/README.md index 3f1febe..4ba6a56 100644 --- a/README.md +++ b/README.md @@ -1,96 +1,61 @@ -## GeoData Toolkit +# GeoData -This repository converts DGM1 elevation tiles into Unity-ready 16-bit PNG heightmaps and a placement manifest. It relies on GDAL for mosaicking, resampling, and scaling to UInt16 ranges Unity expects. +Toolkit zum Export von Geodaten in Unity-kompatible Assets (`export_unity/...`). -### Prerequisites -- `uv` installed. Use Python 3.10–3.12 for now (`triangle2` from `cjio[export]` has no CPython 3.13 wheels). -- Optional: Java 17+ if you want to experiment with the bundled `citygml-tools` utilities (not needed for heightmaps/orthophotos). +## 1) Wie startet man das Projekt? -### Environment setup (uv) -- Install deps (creates `.venv` if missing): `uv sync`. You can skip manual activation by prefixing commands with `uv run ...`; if you prefer activation, run `uv venv && source .venv/bin/activate`. -- `uv run ` executes with the project environment (e.g., `uv run python geodata_to_unity.py --setup`). Use `--directory` to target another path if needed; `--offline` disables network fetches. -- If wheels fail to resolve, ensure system GDAL is present (e.g., `brew install gdal` or `apt-get install gdal-bin libgdal-dev`), then rerun `uv sync`. -- Create the default directory tree and config: `uv run python geodata_to_unity.py --setup` (or `bash scripts/setup_dirs.sh` for directories only). +**Voraussetzungen** +- Python `>=3.11,<3.13` +- `uv` +- GDAL (inkl. `osgeo`) +- Optional fuer Gebaeude: `cjio`, `citygml-tools` -### Repository Layout -- `raw/` — working inputs (not versioned): `raw/dgm1/`, `raw/dop20/jp2/`, `raw/citygml/lod1/`, `raw/citygml/lod2/`. -- `archive/` — offline storage for untouched downloads (e.g., zipped DOP/CityGML tiles, dop20 filelist). -- `work/` — intermediates such as `dgm.vrt` and `_tmp.tif` files; safe to delete/regenerate. -- `export_unity/height_png16/` — final 16-bit PNG heightmaps for Unity import. -- `export_unity/tile_index.csv` — manifest mapping tile IDs to world bounds and scaling ranges (`global_min/global_max` plus per-tile `tile_min/tile_max`), plus `tile_key`; built during heightmap export and required by orthophotos. -- `export_unity/ortho_jpg/` — cropped orthophoto tiles aligned to the terrain grid (JPEG + worldfiles). -- `geodata_to_unity.py` — main CLI (uses `geodata_pipeline/` library modules). -- `scripts/` — helpers to create the directory tree and fetch DOP20 inputs. -- `geodata_config.toml` — generated config (see `geodata_config.example.toml` for defaults). -- `AGENTS.md` — contributor guide. +**Schnellstart** +```bash +cd GeoData +uv sync +uv run python geodata_to_unity.py --setup +uv run python geodata_to_unity.py --export all +``` -### Quick Start -1. Activate the uv venv (`source .venv/bin/activate`) or prefix commands with `uv run`. -2. Initialize config + directories: `uv run python geodata_to_unity.py --setup`. -3. Export assets (builds VRTs automatically if missing): - ```bash - uv run python geodata_to_unity.py --export all - # heightmaps only: uv run python geodata_to_unity.py --export heightmap - # textures only: uv run python geodata_to_unity.py --export textures - # buildings: uv run python geodata_to_unity.py --export buildings - # trees (CSV+GLB): uv run python geodata_to_unity.py --export trees - ``` -4. Import the PNGs into Unity Terrains using `tile_index.csv` for placement and height scaling (0–65535, per-tile by default). +**Alternative mit Archivdaten** +```bash +uv run python geodata_to_unity.py --build-from-archive --export all +``` -### How the export works -- Heightmaps: the pipeline builds `work/dgm.vrt` from all `raw/dgm1/*.tif`, computes a global min/max once (legacy fallback), and warps each tile footprint to `heightmap.out_res` with `srcNodata=-9999`. Per-tile min/max are computed from the warped tile and used to scale PNGs to `[0, 65535]` by default (`heightmap.use_tile_minmax=false` keeps global scaling). `export_unity/tile_index.csv` records `global_min/global_max`, `tile_min/tile_max`, and `tile_key = f"{floor((xmin + overlap_x) / tile_size_x)}_{floor((ymin + overlap_y) / tile_size_y)}"` (defaults: `tile_size_x=1000.0`, `tile_size_y=1000.0`, `overlap_x=0.5`, `overlap_y=0.5` in `[tile_key]`). -- Orthophotos: `work/dop.vrt` is built from `raw/dop20/jp2/*.jp2`; the manifest drives the cropping bounds. JPEG tiles are written to `export_unity/ortho_jpg/` with matching `.jgw` worldfiles. If the manifest is missing, the orthophoto export aborts—run the heightmap export first or use `--export all`. -- Archives: `--build-from-archive` supports a monolithic ZIP (`archive/archive_raw.zip`) and expands every `*.zip` under `archive/*` into the matching `raw/*` directories; dataset zips overlay the monolithic data. It also copies `archive/dop20/filelist.txt` next to `raw/dop20/` for the downloader. -- Cleanup: temporary `_tmp.tif` and GDAL aux XML files under `work/` and `raw/dgm1/` are removed at the end of the heightmap export; avoid storing non-GDAL metadata in those folders. +## 2) Welche Funktionen umfasst das Projekt und wie werden diese genutzt? -### Key Commands -- Refresh VRT: `gdalbuildvrt work/dgm.vrt raw/dgm1/*.tif` -- Run export pipeline: `uv run python geodata_to_unity.py --export all` -- Inspect an output tile: `gdalinfo export_unity/height_png16/.png | head` -- Override config paths: use `--config `, `--raw-dgm1-path `, `--raw-dop20-path `. -- Build raws from archives: `uv run python geodata_to_unity.py --build-from-archive --export all` (uses `archive/archive_raw.zip` when present, then overlays `archive/*/*.zip`). -- Deterministic submission rebuild: `uv run python geodata_to_unity.py --build-from-archive --clean-raw --validate --export all --force-vrt`. -- Rebuild VRTs after moving data: add `--force-vrt`. +```bash +# Hoehenmodelle (PNG16) + tile_index.csv +uv run python geodata_to_unity.py --export heightmap -### Workflow Notes -- Heightmaps normalize per tile using `tile_min/tile_max` by default; set `heightmap.use_tile_minmax=false` to restore global scaling across tiles. Adjust `heightmap.out_res` or `heightmap.resample` in `geodata_config.toml` if your AOI or target resolution changes. -- `tile_key` config controls the tile grouping key in the manifest; defaults are `tile_size_x=1000.0`, `tile_size_y=1000.0`, `overlap_x=0.5`, `overlap_y=0.5` with `enabled=true`. -- `_tmp.tif` files in `work/` are transient; you can delete `work/` to force a clean rebuild. -- Keep file names stable to avoid churn in Unity scenes; re-exports overwrite in place. -- Large raw datasets are intentionally excluded from version control—document download sources or scripts instead of committing data. -- Additional inputs: download helper lives in `scripts/dlscript_dop20.sh` and pulls JP2/J2W/XML orthophotos listed in `archive/dop20/filelist.txt` (one URL per line); `archive/` can hold zipped 3D building tiles for future use. -- `--clean-raw` only removes managed ingestion dirs (`raw/dgm1`, `raw/dop20/jp2`, `raw/dop20/j2w`, `raw/dop20/meta`, `raw/citygml/lod1`, `raw/citygml/lod2`) and intentionally keeps custom masks. -- `--validate` writes `work/archive_materialize_report.json` and fails only when core datasets are missing (`dgm1 tif`, `dop20 jp2`, `citygml lod2`); optional sidecar gaps are warnings. -- Handoff to Unity: copy/sync `export_unity/height_png16/` and `export_unity/tile_index.csv` into `DTrierFlood/Assets/GeoData/` before running the Unity-side importer. Keep `heightmap.out_res` aligned with the importer’s expected resolution (currently 1025). +# Orthofotos (JPG), nutzt tile_index.csv +uv run python geodata_to_unity.py --export textures -### Orthophotos (textures) -1. Ensure DOP assets are present in `raw/dop20/jp2/`, `raw/dop20/j2w/`, and `raw/dop20/meta/`; use `scripts/dlscript_dop20.sh` to fetch JP2/J2W/XML entries listed in `archive/dop20/filelist.txt` (one URL per line). -2. From `GeoData/`, run: - ```bash - uv run python geodata_to_unity.py --export textures - ``` - This builds `work/dop.vrt` if missing and writes `export_unity/ortho_jpg/.jpg` + `.jgw` aligned to `tile_index.csv`. - - If you see `Computed -srcwin ... falls partially outside source raster extent` warnings, the DOP coverage is slightly smaller than the tile footprint; edge pixels will be filled with NoData/zeros. Add adjacent JP2s or shrink the requested window if you need to avoid the warning. - - The download script relies on a Linux/OpenSSL toolchain with system CA bundle at `/etc/ssl/certs/ca-certificates.crt`; it builds a trust chain by fetching the geobasis intermediate. macOS/Windows users should either provide a combined CA via `CURL_CA_BUNDLE` or download with a browser/wget and place files manually. - - Place companion `.j2w` and `.xml` files under `raw/dop20/j2w/` and `raw/dop20/meta/` if available; they are not required for the VRT but help provenance. +# Gebaeude (GLB) aus CityGML LoD2 +uv run python geodata_to_unity.py --export buildings -### Downloads (raw data) -- Run: `uv run python geodata_to_unity.py --download` (uses `geodata_download.toml`). -- Shows a progress bar while downloading and exits cleanly on Ctrl+C (exit code 130). +# Baeume (CSV + GLB) +uv run python geodata_to_unity.py --export trees +``` -### Buildings (automated exporter) -- Run: `uv run python geodata_to_unity.py --export buildings` -- What it does per tile: - - Converts LoD2 CityGML → CityJSON (citygml-tools), triangulates with cjio, rebases to tile-local XY using `tile_index.csv`. - - Merges all buildings into one GLB (1 mesh with roof/wall primitives), decimates to the configured triangle budget. - - Roofs: planar UVs from tile-local XY, embedded per-tile DOP20 orthophoto as base color (unlit by default). - - Walls: vertex colors sampled from the ortho as a fallback (neutral otherwise). - - Coordinates: glTF-friendly (x=east, y=height, z=-north) so glTFast instantiates one GameObject with two submeshes. -- Requirements: LoD2 GMLs under `raw/citygml/lod2/`, per-tile orthos in `export_unity/ortho_jpg/`, tools on PATH (`tools/citygml-tools-2.4.0/citygml-tools`, `cjio` or override via `CJIO` env). -- Open items: richer wall coloring from BDOM/LPO facades, better simplification, footprint-aware ground snapping beyond the current clamp-to-ground. +**Hauptausgaben** +- `export_unity/height_png16/` und `export_unity/tile_index.csv` +- `export_unity/ortho_jpg/` +- `export_unity/buildings_tiles/` +- `export_unity/trees*/` -### Troubleshooting -- Empty raw directories cause VRT creation to fail fast (`No sources available to build VRT`); populate inputs or adjust `--raw-*` overrides. -- If you moved raw data or deleted `work/`, add `--force-vrt` to rebuild VRTs before exporting. -- Orthophoto export warnings like `Computed -srcwin ... falls partially outside source raster extent` indicate coverage gaps; add neighboring JP2s or accept the NoData edge fill. -- If GDAL Python bindings are missing, install system GDAL first and re-run `uv sync` so `osgeo` imports succeed. +## 3) Welche externen Bibliotheken und Quellen wurden verwendet? + +**Bibliotheken/Tools** +- Python +- uv +- GDAL +- cjio +- citygml-tools + +**Externe Datenquellen (Kategorien)** +- DGM1 (GeoTIFF) in `raw/dgm1/` +- DOP20 (JP2) in `raw/dop20/jp2/` +- CityGML LoD2 in `raw/citygml/lod2/` +- Optional: DOM1, BDOM, LPO/LPG fuer Baum-/Masken-Workflows