Commit Graph

86 Commits

Author SHA1 Message Date
vikingowl b222d5fbc8 fix(discovery): use interval multiplication for forward-window query
pgx cannot implicitly encode int arg into text for the `$1 || ' month'`
concatenation pattern (error: "unable to encode 12 into text format for
text (OID 25): cannot find encode plan"). Multiplication with a known
interval works directly with the int parameter and is semantically
equivalent.

Discovered during the T19 smoke test — the tick endpoint returned 500
on every call before this fix.
2026-04-18 08:08:27 +02:00
vikingowl 31ce937f55 feat(helm): add discovery CronJob + token secret + env wiring
Adds a batch/v1 CronJob that POSTs to /api/v1/admin/discovery/tick on a
configurable schedule (default every 4h). Wires DISCOVERY_TOKEN into the
ci-secrets Secret and projects discovery/AI env vars into the backend
Deployment.
2026-04-18 07:57:18 +02:00
vikingowl 3d785d711b feat(discovery): wire domain into API server
Construct discoveryRepo, discoveryAgent, discoveryService, and
discoveryHandler in registerRoutes(); register all 4 discovery routes
on /api/v1 with bearer-token guard on /tick and admin-session guard on
queue management endpoints.
2026-04-18 07:50:14 +02:00
vikingowl f0ba134514 feat(discovery): add HTTP handlers and route registration 2026-04-18 07:43:38 +02:00
vikingowl 4a9d1ff908 feat(middleware): add constant-time bearer token auth for machine routes 2026-04-18 07:39:31 +02:00
vikingowl 540298fb88 feat(discovery): implement Accept + Reject with transactional state changes 2026-04-18 07:37:39 +02:00
vikingowl 4e7120e958 feat(discovery): implement Tick with dedup + queue write 2026-04-18 07:34:14 +02:00
vikingowl aa14724947 feat(discovery): add service skeleton with bucket picker + series matcher 2026-04-18 07:30:47 +02:00
vikingowl 92a5b05875 feat(discovery): add Pass 0 agent client + parser (Mistral) 2026-04-18 07:28:25 +02:00
vikingowl e245f3ec22 feat(discovery): add repository with bucket/queue/rejection queries 2026-04-18 07:25:54 +02:00
vikingowl f7e4bab2c3 feat(discovery): add domain types (Bucket, DiscoveredMarket, Pass0*) 2026-04-18 07:23:25 +02:00
vikingowl a9462d1695 feat(discovery): add DiscoveryConfig (token, batch, forward months) 2026-04-18 07:22:13 +02:00
vikingowl 174635e241 feat(discovery): seed 24-month DACH bucket window 2026-04-18 07:20:32 +02:00
vikingowl fa15810dbb feat(discovery): add rejected_discoveries sticky-reject table migration 2026-04-18 07:17:21 +02:00
vikingowl cddd0196c0 feat(discovery): add discovered_markets queue table migration 2026-04-18 07:14:19 +02:00
vikingowl d0b2ad6362 feat(discovery): add discovery_buckets table migration 2026-04-18 07:11:53 +02:00
vikingowl c78e24e4f9 fix(ai): address review feedback on rate limiter (sort/tolerance, ctx-cancel TODO) 2026-04-18 07:06:22 +02:00
vikingowl c95261d747 feat(ai): add process-wide 1 req/s rate limiter to Mistral client 2026-04-18 07:00:35 +02:00
vikingowl aa965d292a fix(discovery): address review feedback on name/city normalization 2026-04-18 06:55:38 +02:00
vikingowl c95ce55318 feat(discovery): add market name normalization with stripword guard
Normalizes market names for dedup matching: lowercase, umlaut expansion,
punctuation stripping, whitespace collapse, and leading/trailing filler
word removal. Guards stripping so edge fillers are preserved when the
remaining content is purely numeric (e.g. 'Markt 2026' stays 'markt 2026').
2026-04-18 06:48:33 +02:00
vikingowl 79001011df fix(search): support PLZ filter on home page
The home page dropped `plz` server-side, so /markets was called with
radius but no center (unfiltered) and the PLZ input rendered empty
after reload. +page.server.ts now reads plz, geocodes via /geocode,
and echoes plz back in searchParams for form rehydration.

Relaxes /geocode DTO + guard to accept PLZ without city — Nominatim
already supports postal-only lookups. URL lat/lon (GPS flow) take
priority over plz on tie-break; geocode failures fall through to no
geo-filter so the page always renders.
2026-04-18 05:32:28 +02:00
vikingowl f9b77f362f chore(helm): right-size resource requests/limits per cluster telemetry
Drop requests to match observed peak usage and widen CPU limits for
burst headroom (Burstable QoS). Backend, web, Postgres, and Dragonfly
all had requests == limits pinned at defaults well above measured
7-day peaks.

- backend: req 100m/128Mi -> 50m/64Mi, lim 100m/128Mi -> 200m/128Mi
- web:     req 100m/128Mi -> 50m/96Mi, lim 100m/128Mi -> 200m/128Mi
- postgres (CNPG): req 50m/256Mi -> 15m/128Mi, lim 200m/512Mi -> 100m/256Mi
- dragonfly: req 100m/128Mi -> 100m/72Mi, lim 100m/128Mi -> 150m/128Mi

RAM limits unchanged where reasonable to preserve OOM protection;
Dragonfly CPU request kept at 100m (peak 74m) but limit raised to
avoid throttling under brief bursts.
2026-04-18 04:36:12 +02:00
vikingowl 808f4ddda6 chore(deps): bump Kit 2.57.1, Vite 7.3.2, quic-go 0.57.0; override cookie 0.7.2
Resolves 11 Semgrep Supply Chain findings (4 reachable HIGH, 3 unreachable HIGH,
4 moderate/low). Build verified on web (pnpm build) and backend (go build ./...).
2026-04-18 02:53:15 +02:00
vikingowl 7d07d47ec5 chore: convert to GitLab monorepo
- Merge backend, web, app, planning histories (118 commits preserved)
- Replace Woodpecker CI with .gitlab-ci.yml (path-based triggering)
- Switch mistral-go-sdk from somegit.dev to github.com/VikingOwl91/mistral-go-sdk v1.3.0
- Consolidate .pre-commit-config.yaml and .gitignore at repo root
- Remove per-service .woodpecker.yml files
2026-04-07 02:53:03 +02:00
vikingowl a95d24876d fix(helm): update imagePullSecret to itsh-registry 2026-04-06 20:01:10 +02:00
vikingowl e454e31472 fix(ci): switch container registry to registry.itsh.dev 2026-04-06 19:49:06 +02:00
vikingowl 53d7faae24 fix(helm): guaranteed QoS, config checksum, migration retry limit
- Set resources req=limit (100m/128Mi) for Guaranteed QoS class
- Add ConfigMap checksum annotation to trigger rollouts on config changes
- Add retry limit (60 attempts) to migration init container
- Use TARGETARCH in Dockerfile for multi-arch build support
2026-04-01 23:44:50 +02:00
vikingowl 482fcd180a feat(helm): add Go runtime tuning, startup probe, upgrade to Helm 4
- Set GOMAXPROCS and GOMEMLIMIT from cgroup limits to prevent
  thread oversubscription and unbounded GC memory growth
- Add startup probe (60s budget) to gate liveness/readiness during
  connection pool initialization
- Increase liveness failureThreshold to 5 to avoid restarts on
  transient issues
- Remove initialDelaySeconds (startup probe replaces this)
- Upgrade CI from alpine/helm:3.17 to alpine/helm:4.1
- Replace deprecated --atomic with --rollback-on-failure + --wait=watcher
2026-04-01 00:07:01 +02:00
vikingowl 74ee825039 fix(helm): switch migrate init container from busybox to alpine
busybox:1.37 nc -z is broken (outputs "punt!" and hangs).
Alpine 3.21 ships a working nc -z implementation.
2026-03-31 23:38:50 +02:00
vikingowl 08d83bc57e fix(helm): replace broken nc -z in migrate job init container
BusyBox 1.37 nc -z outputs "punt!" and hangs. Use nc -w 2 with
stdin redirect instead, which correctly tests TCP connectivity.
2026-03-31 23:06:51 +02:00
vikingowl ab2484474e fix(helm): remove busybox init container blocking backend startup
BusyBox 1.37 nc -z is broken (outputs "punt!" and never exits),
causing the wait-for-cache init container to loop indefinitely.
The cache is healthy — the backend should handle reconnects itself.
2026-03-31 23:02:26 +02:00
vikingowl 1b329b8222 fix(ci): update deploy namespace to tenant-2 2026-03-09 16:19:27 +01:00
vikingowl 9c051df350 feat(helm): add wait-for-cache init container to backend deployment
Prevents the backend from starting before the DragonflyDB operator
has the cache pod ready and reachable. Mirrors the existing
wait-for-postgres pattern in the migration job.
2026-03-08 20:00:52 +01:00
vikingowl 3d17e25764 fix(helm): bump dragonfly memory limit to 512Mi
DragonflyDB requires 256MiB minimum per thread. With container
overhead the 256Mi limit is insufficient, causing immediate exit.
2026-03-08 19:43:05 +01:00
vikingowl b00e8df6db fix(helm): lower resource limits to fit within tenant-quota (1 CPU)
Set backend and cache limits to 200m/256Mi to stay within the
tenant-1 ResourceQuota of 1 CPU total.
2026-03-08 19:29:50 +01:00
vikingowl 2e1eed543d feat(helm): use DragonflyDB operator CRD, add HTTPRoute sectionName and HTTP→HTTPS redirects
Replace manual Valkey Deployment+Service with DragonflyDB operator CRD.
Add sectionName to HTTPRoute for HTTPS listener pinning and a separate
HTTP→HTTPS 301 redirect route. Update resources from req=limit to
request/limit separation for pay-as-you-go billing. Fix NetworkPolicy
cache pod selector to match operator-managed labels.
2026-03-08 19:01:34 +01:00
vikingowl 41876cd698 fix: remove completion_args from agent conversation request
Mistral rejects completion_args when agent_id is set. The agent's
own configuration (set in Mistral UI) controls response format.
2026-03-05 21:37:37 +01:00
vikingowl 209efe5309 fix: override response format to text for agent conversations
Mistral rejects json response format when tools (web search) are
active. Override to text since the prompt already requests JSON
output in the instructions.
2026-03-05 21:32:03 +01:00
vikingowl c7085e5337 chore: bump Go to 1.26 in CI and Dockerfile
Required by mistral-go-sdk which targets go 1.26.
2026-03-05 21:25:58 +01:00
vikingowl 340132626c refactor: replace hand-rolled Mistral HTTP client with mistral-go-sdk
Use the Conversations API (POST /v1/conversations) via the SDK for
Pass 1 agent calls instead of /agents/completions which doesn't
support built-in web search connectors. Pass 2 uses the SDK's
ChatComplete method.
2026-03-05 21:21:41 +01:00
vikingowl 02a03c3d41 feat: pass AI and Turnstile secrets via Helm deploy pipeline
Add Woodpecker secrets for AI_API_KEY, AI_AGENT_SIMPLE, and
TURNSTILE_SECRET_KEY. Create ci-secrets.yaml template and wire
them through the deploy step alongside existing SMTP secrets.
2026-03-05 18:41:42 +01:00
vikingowl 7937f43a06 fix: resolve golangci-lint dupl and goconst violations
Add nolint:dupl to Update handler (structurally similar to CreateEdition
by design). Extract "NULL" string literal to sqlNull constant.
2026-03-05 18:31:53 +01:00
vikingowl bec253506e chore: normalize resources to 100m/100Mi and enable zero-downtime deploys
Set CPU and memory requests equal to limits (100m/100Mi) for backend,
cache, and web. Switch rolling update strategy to maxSurge=1,
maxUnavailable=0 so new pods start before old ones terminate.
Add readiness probe to cache deployment.
2026-03-05 18:24:58 +01:00
vikingowl d6e6d35ae5 feat: series-grouped market display with edition switching
- Public search dedup: CTE with ROW_NUMBER picks best edition per series
  (nearest future, fallback most recent past), COUNT(*) OVER for edition_count
- Add EditionCount to Market model and MarketSummary DTO
- GetBySlug returns nearest-future edition by default, supports ?year= override
- GetBySlug returns sibling edition briefs for edition switcher UI
- New GET /admin/markets/grouped endpoint returns AdminSeriesGroup[] with
  series-level pagination and all editions per group
- Add AdminSearchGrouped to repository, service, handler, and routes
2026-03-05 18:11:27 +01:00
vikingowl 6f743210f5 feat: add edition creation for existing series, auto-match on submit
Admin endpoint POST /admin/series/:id/editions creates a new yearly
edition for an existing series, inheriting series-level defaults.

Public market submissions now auto-match against existing series using
trigram similarity (>0.6) on name + exact city match. Matched
submissions link to the existing series instead of creating duplicates.

Fix Gin route tree conflict by nesting markets and series groups under
a shared /admin parent group.
2026-03-05 17:08:13 +01:00
vikingowl b5ac2620bf feat: market series/editions model, two-pass AI research, source-derived confidence
Split markets into market_series (persistent identity) + market_editions
(yearly instances). Migration preserves edition UUIDs from old market IDs
for URL backward compatibility. Edition statuses: rumored, confirmed,
active, completed, cancelled, archived.

AI research now uses two-pass pipeline: Pass 1 via pre-created Mistral
agent with web search for structured extraction, Pass 2 via
mistral-large-latest for description writing and field retry. Confidence
scoring is source-derived (source count + extraction type) instead of
LLM self-reported.
2026-03-05 16:38:43 +01:00
vikingowl 10da87735f feat: add sort and order params to admin market list
Supports sorting by name, city, date, created, status with asc/desc order.
2026-03-05 15:22:01 +01:00
vikingowl aa7a982caf feat: add AI research, geocoding, duplicate detection, enriched market submit
- AI research via Mistral API for admin market editing
- Auto-geocoding via Nominatim OSM with rate limiting
- Public geocode endpoint (POST /api/v1/geocode)
- Duplicate detection using pg_trgm trigram similarity
- Extended SubmitMarketRequest with street, opening_hours, admission_info
- pg_trgm migration for fuzzy name matching
2026-03-05 15:18:44 +01:00
vikingowl f839f32ddc chore: disable OAuth routes until provider apps are configured 2026-02-27 14:54:22 +01:00
vikingowl 841a69c59a feat: add password change endpoint, random medieval display names
- Add PUT /auth/password for setting/changing passwords (handles both
  first-time set for magic link/OAuth users and change for password users)
- Generate random medieval display names (e.g. Gaukler1025) for new
  magic link and OAuth users instead of leaving display_name empty
- Add has_password field to ProfileData response
2026-02-27 14:38:02 +01:00