Commit Graph

161 Commits

Author SHA1 Message Date
b222d5fbc8 fix(discovery): use interval multiplication for forward-window query
pgx cannot implicitly encode int arg into text for the `$1 || ' month'`
concatenation pattern (error: "unable to encode 12 into text format for
text (OID 25): cannot find encode plan"). Multiplication with a known
interval works directly with the int parameter and is semantically
equivalent.

Discovered during the T19 smoke test — the tick endpoint returned 500
on every call before this fix.
2026-04-18 08:08:27 +02:00
31ce937f55 feat(helm): add discovery CronJob + token secret + env wiring
Adds a batch/v1 CronJob that POSTs to /api/v1/admin/discovery/tick on a
configurable schedule (default every 4h). Wires DISCOVERY_TOKEN into the
ci-secrets Secret and projects discovery/AI env vars into the backend
Deployment.
2026-04-18 07:57:18 +02:00
2a1fb1eece feat(web): admin discovery review queue with accept/reject 2026-04-18 07:54:18 +02:00
3d785d711b feat(discovery): wire domain into API server
Construct discoveryRepo, discoveryAgent, discoveryService, and
discoveryHandler in registerRoutes(); register all 4 discovery routes
on /api/v1 with bearer-token guard on /tick and admin-session guard on
queue management endpoints.
2026-04-18 07:50:14 +02:00
f0ba134514 feat(discovery): add HTTP handlers and route registration 2026-04-18 07:43:38 +02:00
4a9d1ff908 feat(middleware): add constant-time bearer token auth for machine routes 2026-04-18 07:39:31 +02:00
540298fb88 feat(discovery): implement Accept + Reject with transactional state changes 2026-04-18 07:37:39 +02:00
4e7120e958 feat(discovery): implement Tick with dedup + queue write 2026-04-18 07:34:14 +02:00
aa14724947 feat(discovery): add service skeleton with bucket picker + series matcher 2026-04-18 07:30:47 +02:00
92a5b05875 feat(discovery): add Pass 0 agent client + parser (Mistral) 2026-04-18 07:28:25 +02:00
e245f3ec22 feat(discovery): add repository with bucket/queue/rejection queries 2026-04-18 07:25:54 +02:00
f7e4bab2c3 feat(discovery): add domain types (Bucket, DiscoveredMarket, Pass0*) 2026-04-18 07:23:25 +02:00
a9462d1695 feat(discovery): add DiscoveryConfig (token, batch, forward months) 2026-04-18 07:22:13 +02:00
174635e241 feat(discovery): seed 24-month DACH bucket window 2026-04-18 07:20:32 +02:00
fa15810dbb feat(discovery): add rejected_discoveries sticky-reject table migration 2026-04-18 07:17:21 +02:00
cddd0196c0 feat(discovery): add discovered_markets queue table migration 2026-04-18 07:14:19 +02:00
d0b2ad6362 feat(discovery): add discovery_buckets table migration 2026-04-18 07:11:53 +02:00
c78e24e4f9 fix(ai): address review feedback on rate limiter (sort/tolerance, ctx-cancel TODO) 2026-04-18 07:06:22 +02:00
c95261d747 feat(ai): add process-wide 1 req/s rate limiter to Mistral client 2026-04-18 07:00:35 +02:00
aa965d292a fix(discovery): address review feedback on name/city normalization 2026-04-18 06:55:38 +02:00
c95ce55318 feat(discovery): add market name normalization with stripword guard
Normalizes market names for dedup matching: lowercase, umlaut expansion,
punctuation stripping, whitespace collapse, and leading/trailing filler
word removal. Guards stripping so edge fillers are preserved when the
remaining content is purely numeric (e.g. 'Markt 2026' stays 'markt 2026').
2026-04-18 06:48:33 +02:00
abe5f53a54 Merge branch 'fix/market-search-plz-filter' into 'main'
fix(search): support PLZ filter on home page

See merge request vikingowl/marktvogt.de!1
2026-04-18 03:32:59 +00:00
79001011df fix(search): support PLZ filter on home page
The home page dropped `plz` server-side, so /markets was called with
radius but no center (unfiltered) and the PLZ input rendered empty
after reload. +page.server.ts now reads plz, geocodes via /geocode,
and echoes plz back in searchParams for form rehydration.

Relaxes /geocode DTO + guard to accept PLZ without city — Nominatim
already supports postal-only lookups. URL lat/lon (GPS flow) take
priority over plz on tie-break; geocode failures fall through to no
geo-filter so the page always renders.
2026-04-18 05:32:28 +02:00
df5b0563c9 refactor(web): bundle server with esbuild, slim runtime to alpine+node
Post-process adapter-node output into a single self-contained
build/bundle.mjs (614 KB) via esbuild, then ship it on a fresh
alpine:3.21 base with just the node binary copied in. Drops the
node_modules + package manager baggage that comes with node:25-alpine.

- Add esbuild devDep + `bundle` script (scripts/bundle.mjs)
- Dockerfile: drop `deps` stage; final stage is alpine + node binary +
  bundle + static client assets
- Uncompressed image: 177 MB -> 149 MB (-16%)
- Verified: /, /healthz, static assets all respond identically;
  outbound TLS to api.marktvogt.de works via node's built-in CA bundle
2026-04-18 05:05:07 +02:00
f9b77f362f chore(helm): right-size resource requests/limits per cluster telemetry
Drop requests to match observed peak usage and widen CPU limits for
burst headroom (Burstable QoS). Backend, web, Postgres, and Dragonfly
all had requests == limits pinned at defaults well above measured
7-day peaks.

- backend: req 100m/128Mi -> 50m/64Mi, lim 100m/128Mi -> 200m/128Mi
- web:     req 100m/128Mi -> 50m/96Mi, lim 100m/128Mi -> 200m/128Mi
- postgres (CNPG): req 50m/256Mi -> 15m/128Mi, lim 200m/512Mi -> 100m/256Mi
- dragonfly: req 100m/128Mi -> 100m/72Mi, lim 100m/128Mi -> 150m/128Mi

RAM limits unchanged where reasonable to preserve OOM protection;
Dragonfly CPU request kept at 100m (peak 74m) but limit raised to
avoid throttling under brief bursts.
2026-04-18 04:36:12 +02:00
0b797aec66 chore: sync CLAUDE.md with GitLab migration; add SOURCE_DATE_EPOCH to builds
- CLAUDE.md: correct stale Woodpecker/submodules claims; note
  registry.itsh.dev's attestation requirement and the docker-container
  driver prerequisite so future contributors don't rediscover the issue.
- .gitlab-ci.yml: export SOURCE_DATE_EPOCH from commit time so the
  rewrite-timestamp=true buildx flag produces genuinely reproducible
  images (layer digests stable across rebuilds of the same commit).
2026-04-18 04:16:13 +02:00
cd7ea0b47b fix(ci): override alpine/helm entrypoint so shell wrapper runs
The alpine/helm:4.1 image has 'helm' as ENTRYPOINT. GitLab CI wraps
step_script in sh, which the container resolves to 'helm sh ...',
failing with 'unknown command sh for helm'. Setting entrypoint: [""]
lets GitLab's sh wrapper execute normally before invoking helm.
2026-04-18 03:57:50 +02:00
ad846be2c7 fix(ci): create docker context before buildx to handle dind TLS
buildx create --driver docker-container cannot inherit TLS env vars
from docker:dind directly; it needs a named context. Create 'tls-env'
from the ambient DOCKER_HOST/DOCKER_CERT_PATH, then point buildx at it.
2026-04-18 03:51:33 +02:00
faa63de3c7 fix(ci): use docker-container driver so buildx emits attestations
The default buildx driver inside docker:29-dind is 'docker' (host daemon),
which cannot produce attestations. Even with default provenance enabled,
the docker driver silently drops attestation-related flags and emits a
bare single-image manifest. registry.itsh.dev (Zot with strict attestation
policy) rejects these with 'manifest invalid'.

Creating a 'docker-container' driver builder before each build gives
buildkit full export capabilities, matching both the Woodpecker plugin's
behavior and what works from local development machines.
2026-04-18 03:48:37 +02:00
83b369e339 fix(ci): re-enable buildx attestations (registry requires them)
Empirical finding: registry.itsh.dev rejects pushes that contain zero
attestations ("manifest invalid"). Attestations cause buildx to emit
an OCI image index; without any, it emits a bare single-image manifest
which this Zot instance refuses in this namespace.

Remove --provenance=false --sbom=false, switch to the explicit output
form matching woodpeckerci/plugin-docker-buildx (proven working against
the same registry for the infinity-tales project).

Root cause of the original 610ca91 regression: that commit added
--provenance=false, which stripped the last required attestation once
SBOM generation was disabled in a later buildkit default change.
2026-04-18 03:43:38 +02:00
29ec565206 test(ci): bump docker:27 -> docker:29 to match Woodpecker plugin env
The woodpeckerci/plugin-docker-buildx image that successfully pushes to
this same registry uses docker:29.4.0-dind with buildx 0.33.0. Version
skew between docker:27's bundled buildkit and the registry is the next
hypothesis after ground-truth manifest inspection ruled out OCI format
and attestations as the cause.
2026-04-18 03:29:52 +02:00
bb72b4ce94 chore(ci): trigger rebuilds when .gitlab-ci.yml changes
Previously only backend/** or web/** file changes would trigger the
respective docker+deploy jobs. CI config changes now also trigger
rebuilds of both services, ensuring any pipeline-definition changes
are verified end-to-end on the same commit that introduces them.
2026-04-18 03:19:23 +02:00
dd7d52e249 fix(ci): disable SBOM attestations on buildx to unblock registry push
Matches woodpeckerci/plugin-docker-buildx defaults. Without --sbom=false
buildkit emits an OCI image index with SBOM attestation that the itsh.dev
registry rejects with 'manifest invalid'. Provenance was already disabled.
2026-04-18 03:17:20 +02:00
808f4ddda6 chore(deps): bump Kit 2.57.1, Vite 7.3.2, quic-go 0.57.0; override cookie 0.7.2
Resolves 11 Semgrep Supply Chain findings (4 reachable HIGH, 3 unreachable HIGH,
4 moderate/low). Build verified on web (pnpm build) and backend (go build ./...).
2026-04-18 02:53:15 +02:00
610ca91fdb fix: use buildx --push --provenance=false to avoid OCI manifest rejection 2026-04-07 03:29:52 +02:00
13c6cadb99 fix: disable BuildKit to avoid OCI manifest rejection by registry 2026-04-07 03:21:47 +02:00
ca11ecb112 fix: svelte-check needs web/.env; only fail on errors not warnings 2026-04-07 03:09:01 +02:00
32e3aaa2b8 fix: hooks-install sets up golangci-lint with Go 1.26 and web node_modules 2026-04-07 03:06:43 +02:00
e175d3c43c chore: remove lint/test from CI — handled by pre-commit hooks 2026-04-07 03:05:11 +02:00
ca86001e7b fix: pre-commit hooks for monorepo (golangci-lint cd, web deps, branch skip) 2026-04-07 03:04:06 +02:00
7bddf018e6 chore: add root Justfile for local lint, test, and dev commands 2026-04-07 03:01:42 +02:00
d802de6d9e fix: web:lint cd already in web/ after before_script 2026-04-07 02:56:01 +02:00
7d07d47ec5 chore: convert to GitLab monorepo
- Merge backend, web, app, planning histories (118 commits preserved)
- Replace Woodpecker CI with .gitlab-ci.yml (path-based triggering)
- Switch mistral-go-sdk from somegit.dev to github.com/VikingOwl91/mistral-go-sdk v1.3.0
- Consolidate .pre-commit-config.yaml and .gitignore at repo root
- Remove per-service .woodpecker.yml files
2026-04-07 02:53:03 +02:00
d9e48ee007 merge: root history into monorepo 2026-04-07 02:45:51 +02:00
ca2e35113f merge: app history into monorepo 2026-04-07 02:45:51 +02:00
7b6bc90633 merge: web history into monorepo 2026-04-07 02:45:51 +02:00
01881e56bc fix(helm): update imagePullSecret to itsh-registry 2026-04-06 20:01:32 +02:00
a95d24876d fix(helm): update imagePullSecret to itsh-registry 2026-04-06 20:01:10 +02:00
e454e31472 fix(ci): switch container registry to registry.itsh.dev 2026-04-06 19:49:06 +02:00
cb996f46bb fix(ci): switch container registry to registry.itsh.dev 2026-04-06 19:48:56 +02:00