Restore DOP download helpers in raw layouts

This commit is contained in:
2025-12-15 21:23:43 +01:00
parent 4e7697e73a
commit 4a85b63f28
7 changed files with 78 additions and 5 deletions

13
.gitignore vendored
View File

@@ -39,7 +39,13 @@ export_unity/buildings_obj/
# --- Raw downloaded source data (usually too big for git) ---
*.zip
raw/
# Prefer raw/ layout; keep data ignored but allow helper files
raw/**
!raw/dop/
raw/dop/**
!raw/dop/dlscript.sh
!raw/dop/filelist.txt
raw/dop/jp2/**
archives/
# If you DO want to keep small sample tiles in git for CI/tests,
@@ -53,7 +59,10 @@ tmp/
temp/
.cache/
*.gfs
raw_dop/
raw_dop/**
!raw_dop/dlscript.sh
!raw_dop/filelist.txt
raw_dop/jp2/**
raw_dgm1/
raw_3dgeb_lod1/
raw_3dgeb_lod2/

View File

@@ -44,11 +44,11 @@ This repository converts DGM1 elevation tiles into Unity-ready 16-bit PNG height
- `_tmp.tif` files in `work/` are transient; you can delete `work/` to force a clean rebuild.
- Keep file names stable to avoid churn in Unity scenes; re-exports overwrite in place.
- Large raw datasets are intentionally excluded from version control—document download sources or scripts instead of committing data.
- Additional inputs: `raw_dop/` contains an HTTPS download helper (`raw_dop/dlscript.sh`) that fetches JP2/J2W/XML orthophotos listed in `filelist.txt`; `3dgeblod1/` and `3dgeblod2/` hold zipped 3D building tiles with object lists for future use.
- Additional inputs: download helpers live in `raw/dop/dlscript.sh` (legacy: `raw_dop/dlscript.sh`) which fetch JP2/J2W/XML orthophotos listed in `filelist.txt`; `3dgeblod1/` and `3dgeblod2/` hold zipped 3D building tiles with object lists for future use.
- Handoff to Unity: copy/sync `export_unity/height_png16/` and `export_unity/tile_index.csv` into `DTrierFlood/Assets/GeoData/` before running the Unity-side importer. Keep `OUT_RES` aligned with the importers expected resolution (currently 1025).
### Orthophotos (textures)
1. Ensure DOP JP2s are present in `raw/dop/jp2/` (legacy: `raw_dop/jp2/`; download helper currently lives in `raw_dop/dlscript.sh`).
1. Ensure DOP JP2s are present in `raw/dop/jp2/` (legacy: `raw_dop/jp2/`; download helper lives in `raw/dop/dlscript.sh` or `raw_dop/dlscript.sh`).
2. From `GeoData/`, run:
```bash
uv run python export_ortho_tiles.py

View File

@@ -49,7 +49,9 @@ def export_orthos(args: argparse.Namespace) -> int:
jp2_paths = sorted(glob.glob(os.path.join(raw_ortho_dir, "*.jp2")))
if not jp2_paths:
raise SystemExit(f"No JP2 files found in {raw_ortho_dir}. Run raw_dop/dlscript.sh first.")
raise SystemExit(
f"No JP2 files found in {raw_ortho_dir}. Run raw/dop/dlscript.sh (or legacy raw_dop/dlscript.sh) first."
)
build_vrt(args.vrt_path, jp2_paths)
vrt_ds = open_dataset(args.vrt_path, f"Could not open VRT at {args.vrt_path}")

28
raw/dop/dlscript.sh Executable file
View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Download DOP JP2/J2W/XML tiles listed in filelist.txt.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
LIST="$ROOT/filelist.txt"
OUT="$ROOT/jp2"
if [[ ! -f "$LIST" ]]; then
echo "Missing filelist: $LIST" >&2
exit 1
fi
mkdir -p "$OUT"
while IFS= read -r url; do
[[ -z "$url" || "$url" =~ ^# ]] && continue
fname="${url##*/}"
dest="$OUT/$fname"
if [[ -f "$dest" ]]; then
echo "Exists: $fname"
continue
fi
echo "Downloading $fname"
curl -fL "$url" -o "$dest"
done < "$LIST"
echo "Done. Files in $OUT"

3
raw/dop/filelist.txt Normal file
View File

@@ -0,0 +1,3 @@
# List JP2 download URLs, one per line. Lines starting with # are ignored.
# Example:
# https://example.com/path/to/dop20rgb_32_328_5512_2_rp_2023.jp2

28
raw_dop/dlscript.sh Executable file
View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
# Download DOP JP2/J2W/XML tiles listed in filelist.txt.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
LIST="$ROOT/filelist.txt"
OUT="$ROOT/jp2"
if [[ ! -f "$LIST" ]]; then
echo "Missing filelist: $LIST" >&2
exit 1
fi
mkdir -p "$OUT"
while IFS= read -r url; do
[[ -z "$url" || "$url" =~ ^# ]] && continue
fname="${url##*/}"
dest="$OUT/$fname"
if [[ -f "$dest" ]]; then
echo "Exists: $fname"
continue
fi
echo "Downloading $fname"
curl -fL "$url" -o "$dest"
done < "$LIST"
echo "Done. Files in $OUT"

3
raw_dop/filelist.txt Normal file
View File

@@ -0,0 +1,3 @@
# List JP2 download URLs, one per line. Lines starting with # are ignored.
# Example:
# https://example.com/path/to/dop20rgb_32_328_5512_2_rp_2023.jp2