16 KiB
16 KiB
GeoData Pipeline Architecture
Visual documentation showing how the GeoData pipeline transforms raw geospatial data into Unity-ready assets.
Complete Pipeline Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ INPUT LAYER │
├──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┤
│ DGM1 │ DOM1 │ DOP20 │ CityGML │ LPG │ LPO │ BDOM20 │
│ TIFF │ TIFF │ JP2 │ GML │ XYZ │ XYZ │ LAZ │
│ terrain │ surface │ ortho │ buildings│ ground │ object │ RGB │
│ 10m │ 10m │ 20cm │ LoD2 │ points │ points │ points │
├──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┤
│ raw/ directory (inputs) │
└────────────────────────────────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ PROCESSING LAYER │
├───────────────────┬────────────────────┬────────────────────────────────────┤
│ │ │ │
│ HEIGHTMAPS │ BUILDINGS │ VEGETATION & OBJECTS │
│ │ │ │
│ ┌─────────────┐ │ ┌──────────────┐ │ ┌──────────────┐ ┌─────────────┐ │
│ │ DGM1 → VRT │ │ │GML→CityJSON │ │ │ DOM1-DGM1 │ │nDSM detect │ │
│ │ ↓ │ │ │ ↓ │ │ │ = CHM │ │ ↓ │ │
│ │ Warp 1025² │ │ │ Triangulate │ │ │ ↓ │ │ Classify │ │
│ │ ↓ │ │ │ ↓ │ │ │Local maxima │ │(lamp/bench │ │
│ │ Scale 16bit│ │ │ Decimate │ │ │ ↓ │ │ /sign) │ │
│ │ ↓ │ │ │ ↓ │ │ │ Tree peaks │ │ │ │
│ │ [LPG valid]│ │ │ Ground-snap │ │ │ [LPO refine] │ │ [LPO ref] │ │
│ └─────────────┘ │ │ [RANSAC roof]│ │ │ [DOP20 RGB] │ └─────────────┘ │
│ │ └──────────────┘ │ └──────────────┘ │
├───────────────────┴────────────────────┴────────────────────────────────────┤
│ work/ directory (temp) │
└────────────────────────────────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ OUTPUT LAYER │
├─────────────────┬─────────────────┬─────────────────┬───────────────────────┤
│ height_png16/ │ ortho_jpg/ │ buildings_tiles/│ trees/ + furniture/ │
│ *.png (16-bit) │ *.jpg (2048²) │ *.glb │ *.csv │
│ *.pgw │ *.jgw │ │ trees_tiles/*.glb │
├─────────────────┴─────────────────┴─────────────────┴───────────────────────┤
│ tile_index.csv (manifest with bounds + elevation range) │
├─────────────────────────────────────────────────────────────────────────────┤
│ export_unity/ directory │
└────────────────────────────────────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ UNITY IMPORT │
│ GeoTileImporter.cs reads tile_index.csv, creates: │
│ • Terrain tiles (heightmaps + ortho textures) │
│ • Building GLB instances │
│ • Tree proxies with canopy colors │
│ • Street furniture placeholders │
└─────────────────────────────────────────────────────────────────────────────┘
Pipeline Dependency Graph
┌─────────────────┐
│ --setup │
│ materialize │
│ archives │
└────────┬────────┘
│
▼
┌─────────────────┐
│ HEIGHTMAP │◄──── MUST RUN FIRST
│ export_height │ Creates tile_index.csv
│ maps() │ Creates work/dgm.vrt
└────────┬────────┘
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
┌────────────┐ ┌────────────┐ ┌────────────┐
│ TEXTURES │ │ BUILDINGS │ │ TREES │
│ ortho │ │ citygml │ │ dom1-dgm1 │
│ photos │ │ → glb │ │ → csv+glb │
└────────────┘ └────────────┘ └─────┬──────┘
│
▼
┌───────────────────┐
│ STREET FURNITURE │
│ ndsm detection │
│ → csv │
└─────────┬─────────┘
│
┌──────────────────────────┼──────────────┐
│ │ │
▼ ▼ ▼
┌────────────────┐ ┌────────────────┐ ┌─────────────────┐
│ HEIGHTMAP- │ │ BUILDINGS- │ │ TREES- │
│ ENHANCED │ │ ENHANCED │ │ ENHANCED │
│ +LPG valid │ │ +RANSAC roofs │ │ +LPO heights │
│ +quality map │ │ +LPO heights │ │ +DOP20 colors │
└────────────────┘ └────────────────┘ │ -furniture mask │
└─────────────────┘
Data Flow Per Export Type
1. Heightmap (Standard)
raw/dgm1/*.tif ──► work/dgm.vrt ──► Per-tile warp (1025×1025)
│
▼
Scale to UInt16
│
▼
export_unity/height_png16/{tile}.png
export_unity/tile_index.csv (manifest)
2. Orthophoto (Textures)
raw/dop20/jp2/*.jp2 ──► work/dop.vrt ──► Per-tile extract
│ │
│ tile_index.csv ────────────►│ (read bounds)
│ │
▼ ▼
export_unity/ortho_jpg/{tile}.jpg (2048×2048)
3. Buildings (Standard)
raw/citygml/lod2/*.gml ──► citygml-tools ──► work/cityjson/*.json
│
▼
cjio triangulate
│
▼
work/cityjson_local/*.tri.json
│
work/dgm.vrt ──────────────────────────►│ (ground-snap)
│
▼
export_unity/buildings_tiles/{tile}.glb
4. Trees (Standard)
raw/dom1/*.tif ─────────► CHM = DOM1 - DGM1
raw/dgm1/*.tif ─────────► │
▼
Local maxima detection
│
raw/citygml/lod2/*.gml ─────────►│ (building mask)
│
▼
export_unity/trees/{tile}.csv
export_unity/trees_tiles/{tile}_{chunk}.glb
5. Street Furniture
raw/dom1/*.tif ─────────► nDSM = DOM1 - DGM1
raw/dgm1/*.tif ─────────► │
▼
Blob detection (0.3-8m)
│
raw/lpo/*.xyz ──────────────────►│ (optional LPO refinement)
│
▼
Classify: lamp/bench/sign/bollard
│
▼
export_unity/street_furniture/{tile}.csv
6. Enhanced Pipelines
HEIGHTMAP-ENHANCED:
heightmap output + raw/lpg/*.xyz ──► LPG validation ──► quality map
BUILDINGS-ENHANCED:
buildings output + raw/bdom20rgbi/*.laz ──► RANSAC roof extraction
+ raw/lpo/*.xyz ──────────► height refinement
TREES-ENHANCED:
trees output + raw/lpo/*.xyz ──────► height refinement
+ raw/dop20/*.jp2 ──────► canopy RGB sampling
+ street_furniture/*.csv ► exclusion mask
File Format Reference
| Output | Format | Resolution | Purpose |
|---|---|---|---|
height_png16/*.png |
16-bit PNG | 1025×1025 | Unity terrain heightmap |
ortho_jpg/*.jpg |
JPEG Q90 | 2048×2048 | Terrain texture |
buildings_tiles/*.glb |
glTF Binary | 200-350k tris | 3D building models |
trees/*.csv |
CSV | - | Tree positions (x,y,z,h,r) |
trees_tiles/*.glb |
glTF Binary | - | Proxy geometry chunks |
street_furniture/*.csv |
CSV | - | Furniture positions |
tile_index.csv |
CSV | - | Manifest (bounds, min/max) |
CLI Commands
# Setup (extract archives, create directories)
uv run python geodata_to_unity.py --setup --build-from-archive
# Standard exports
uv run python geodata_to_unity.py --export heightmap # First!
uv run python geodata_to_unity.py --export textures
uv run python geodata_to_unity.py --export buildings
uv run python geodata_to_unity.py --export trees
uv run python geodata_to_unity.py --export all # All standard
# Enhanced exports (requires point cloud data)
uv run python geodata_to_unity.py --export heightmap-enhanced
uv run python geodata_to_unity.py --export buildings-enhanced
uv run python geodata_to_unity.py --export trees-enhanced
uv run python geodata_to_unity.py --export street-furniture
uv run python geodata_to_unity.py --export all-enhanced # All enhanced
Config Classes
Config
├── raw: RawConfig # Input directories
├── archives: ArchiveConfig # Archive staging
├── work: WorkConfig # Temp/intermediate
├── export: ExportConfig # Output directories
├── heightmap: HeightmapConfig # out_res=1025, tile_size_m=1000
├── ortho: OrthoConfig # out_res=2048, quality=90
├── buildings: BuildingConfig # triangle budget 200-350k
├── trees: TreeConfig # max_trees=5000, chunk_grid=4x4
├── pointcloud: PointCloudConfig # LPG/LPO/BDOM directories
├── heightmap_enhanced: EnhancedHeightmapConfig
├── buildings_enhanced: EnhancedBuildingConfig
├── trees_enhanced: EnhancedTreeConfig
└── street_furniture: StreetFurnitureConfig
Key Files
| Module | Purpose |
|---|---|
geodata_to_unity.py |
CLI entry point |
geodata_pipeline/config.py |
Configuration dataclasses |
geodata_pipeline/heightmaps.py |
DGM1 → PNG16 |
geodata_pipeline/orthophotos.py |
DOP20 → JPEG |
geodata_pipeline/buildings.py |
CityGML → GLB |
geodata_pipeline/trees.py |
CHM → CSV + GLB |
geodata_pipeline/street_furniture.py |
nDSM → CSV |
geodata_pipeline/pointcloud.py |
XYZ/LAZ utilities |
geodata_pipeline/*_enhanced.py |
Enhanced pipelines |
GeoTileImporter.cs |
Unity importer |