depends_on: [backend] in web.yaml blocked web from triggering on
single-subtree web commits (backend's path filter excluded it from the
pipeline, so web's dependency was never satisfied; web never ran).
Switching to retry loop: 3 attempts with 30s backoff. When a cross-subtree
commit triggers both pipelines, the loser of the helm release-lock race
sleeps and retries — works whether or not the other workflow ran.
helm 4.1 in alpine/helm:4.1 errored 'release: already exists' on the latter
flag despite the release being deployed. --reuse-values is the older,
universally-supported variant — preserves the other service's image tag
between pipeline runs the same way.
CI deploy steps now target helm/marktvogt with --reset-then-reuse-values,
preserving the other service's image tag across pipeline runs. Each pipeline
sets only its own X.image.tag.
App-level secrets (smtp/turnstile/discovery/ai/JWT/oauth) moved out of CI's
--set chain in the previous phase — now pre-created via
scripts/k8s-secrets-sync.sh from .env.helm. The chart's conditional secret
templates remain for backward-compat with the live release's stored values
but will be removed in a follow-up once those values are cleared.
Old per-service chart directories deleted; only the monolithic
helm/marktvogt/ remains.
MIGRATION.md updated with the actual procedure that worked, including the
several pitfalls hit during the live tenant-2 migration on 2026-04-28
(helm uninstall trap, SSA field-manager swap for CRDs, kyverno hostname
allowlist for new subdomains).
New unified helm chart at helm/marktvogt/ that combines backend (Go API,
Postgres, Dragonfly, migrate hook, discovery cron) and web (SvelteKit SSR)
into a single release. Replaces the per-service charts at backend/deploy/helm
and web/deploy/helm — kept in place until the live migration is verified
(see helm/marktvogt/MIGRATION.md).
Selector labels and resource names match the existing per-service charts
exactly so migration is by re-annotation rather than recreate; CNPG cluster
and Dragonfly survive the cutover with no data loss.
Adds scripts/k8s-secrets-sync.sh + .env.helm.example for reproducible
out-of-band secret creation. .env.helm itself is gitignored.
Nil slices in MergePlan.AutoApply/ReviewRequired/Rejected serialized to
JSON null, causing the admin research panel to crash with
"can't access property 'map', plan.review_required is null". Initialize
the buckets as empty slices so the wire contract is always an array.
Tightened the empty-buckets test to assert the JSON shape.
Adds the native (Go) TypeScript compiler as a devDep and routes
svelte-check through it via --tsgo. Local pnpm run check goes from
~5s to ~3s on this codebase; pre-commit hook inherits the speed
automatically.
The linux-x64 prebuilt is a statically-linked Go binary (~25MB), so
the alpine builder in web/Dockerfile installs it cleanly even though
it never invokes svelte-check during the image build.
Re-prices every existing ai_usage row using the correct $/1M token rates
per model family. CASE clauses ordered specific-first (flash-lite before
flash) to mirror the longest-prefix-match in priceFor(). Aliases
(gemini-*-latest) resolve to the 2.5 family, the only one in production
during the affected window.
The grounding-fee component ($35/1k above 1500/day free tier) is not
recomputed: historical traffic shows zero grounded calls in the window,
so the bumper would be 0. Down is a no-op (irreversible by design — the
original miscalculated values are not preserved).
estimateCost ignored the model name and billed every Gemini call at
hardcoded flash-lite rates ($0.10 / $0.40 per 1M), under-counting Pro
calls by ~12-25x. Switch to priceFor(model) and prefer resp.ModelVersion
so aliases like gemini-pro-latest resolve to their concrete family.
Capture ThoughtsTokenCount as a separate ThinkingTokens column on
ai_usage (migration 000030) and bill it at the output rate.
Add a global thinking on/off toggle that mirrors the grounding pattern:
provider holds an in-memory cache (read at startup from settings.Store),
handler keeps it in sync, Chat() applies ThinkingConfig.ThinkingBudget=0
only when disabled. Default true preserves SDK behavior. Grounding+
thinking get/set helpers folded into shared getBool/setBool to keep
goconst happy.
Web admin settings: new "Modell-Reasoning" toggle card; usage panel sums
include thinking tokens. Types are optional with `?? 0` defaults so a
brief web-before-backend rollout window cannot render NaN.
The original sessions table has expires_at TIMESTAMPTZ NOT NULL with no
default. Migration 000027 added the new columns but did not drop this one,
so CreateSession must still supply a value. Using AbsoluteExpiresAt.
Replace HS256 JWT access tokens with two opaque 32-byte random tokens
(access + refresh), both stored as SHA-256 hashes in sessions + Valkey.
Key changes:
- GenerateOpaqueToken() replaces JWT issuance; TokenService removed
- Sessions now carry access_token_hash, refresh_token_hash, family_id,
parent_session_id, access_expires_at, absolute_expires_at, last_used_at,
revoked_at — per migration 000027 (updated to add access_expires_at)
- Refresh rotation is atomic (UPDATE...RETURNING); reuse detection kills
the entire token family and returns auth.refresh_reuse_detected
- RequireAuth/OptionalAuth now take SessionLookup (Valkey→Postgres) instead
of *TokenService; sets session_id in context alongside user_id
- last_used_at is bumped on each request, throttled to writes >60s old
- AuthConfig{AccessTTL,RefreshIdleTTL,RefreshAbsoluteTTL} replaces JWT TTL env
vars (AUTH_ACCESS_TTL=30m, AUTH_REFRESH_IDLE_TTL=168h, AUTH_REFRESH_ABSOLUTE_TTL=720h)
- JWT_SECRET kept for AI-settings key derivation (drops from auth flow)
Forced logout on deploy (D3 behaviour); pre-launch so acceptable.
structuredClone on a Svelte 5 reactive Proxy throws DataCloneError during
component init, causing MergeProposalPanel to silently fail to mount.
Replace with \$state.snapshot which is the documented way to deep-copy a
reactive prop into a local editable state.
Frontend budget was 180s — equal to the backend goroutine cap — so a race
determined which side timed out first. Bumped to 270s to guarantee the frontend
outlasts the backend's 3-minute window.
Added explicit null guard on result.proposal: if the LLM ever returns a
done-status without a proposal body the UI now surfaces a clear error instead
of silently assigning undefined (which kept the panel hidden with no feedback).
Also guards field_merges ?? {} in MergeProposalPanel to avoid Object.keys(null)
if the model returns a null map.
POST /admin/markets/:id/merge-plan now returns 202 + job_id immediately
and runs the Gemini advisor in a detached goroutine. Frontend polls
GET .../merge-plan/:job_id until done, with backoff up to 3 minutes.
Adds in-memory job registry (keyed map + RWMutex, 5-min TTL sweep) and
handler tests covering the full pending→done and error paths.
All serverFetch calls were going to https://api.marktvogt.de (public
gateway), creating a second nginx hop for every SSR operation. Slow LLM
calls (merge-plan, research-plan) hit the 60s proxy_read_timeout.
- Add PRIVATE_API_BASE_URL=http://marktvogt-backend to web Helm config
- serverFetch now builds SERVER_API_BASE from PRIVATE_API_BASE_URL at
runtime (falls back to PUBLIC_API_BASE_URL when not set)
- apiFetch accepts optional baseURL param; client-side calls unchanged
Merge-plan and research-plan both call Gemini which can take >60s.
The default gateway timeout was killing connections with 504.
- Web HTTPRoute: add /admin/ rule with 120s request+backendRequest timeout
- Backend HTTPRoute: add /api/v1/admin/markets/ rule with 120s timeout
- MergePlan handler: add 110s context deadline for graceful degradation
before the gateway cuts the upstream connection
Gemini returned field_merges as an array without structure constraint,
causing json.Unmarshal to fail with "cannot unmarshal array into Go struct
field of type map[string]mergeFieldDecision".
- Pass merge_advisor_schema.json via JSONSchema instead of bare JSONMode
- Add parseFieldMerges() that accepts both object and array LLM formats
- Validate target_id is one of the two input market IDs after parsing
- Fix schemaFromMap: minimum/maximum are supported by genai.Schema v1.54
LLM tiebreaker can take several seconds; return the duplicates fetch
as an unawaited Promise so the page renders immediately with market
data. Template uses {#await} to render the panel when it resolves.
The Plan handler returns {plan, research_result} directly without a
data wrapper. apiFetch casts the body to ApiResponse<T>, so res.data
was undefined, json(undefined) produced an empty response, and the
client either crashed (JSON.parse) or silently got a null plan.
H1: Drop empty string from enricher_schema.json category enum —
Gemini rejects enum[7]: cannot be empty (Error 400). Remove category
from required so the model can omit it when no category fits.
H2: Research-plan/apply client reads response as text before
JSON.parse; empty or HTML error bodies now surface the actual HTTP
status instead of crashing with "unexpected end of data".
I: Dedup UI for approved markets:
- DuplicatesPanel: LLM verdict pills (same/not-same, confidence),
llm_reason, per-candidate Merge-planen button
- MergeProposalPanel: summary, confidence, flags, per-field
decisions with editable source radio (a/b/combined), current
value context, confirm() before destructive apply
- Two SvelteKit proxy routes: merge-plan/ and merge-into/[targetId]/
- [id]/+page.svelte: wired with full state; navigates to survivor
after successful merge
- [id]/+page.server.ts: load duplicates for all non-merged editions
(was gated to status=rumored only)
- types.ts: DuplicateMarket gains llm_same/llm_confidence/llm_reason;
add MarketMergeProposal + MergeFieldDecision; add merged to
EditionStatus
MergeAdvisor calls Gemini with a German system prompt to propose how to merge
two duplicate market editions. It guards against confident non-duplicates via
ErrNotDuplicate (same=false AND confidence>0.5).
POST /:id/merge-plan generates a MarketMergeProposal (read-only).
POST /:id/merge-into/:target_id applies the merge: updates target fields,
marks source as status=merged with merged_into_id set, reparents discovered_markets,
and writes a market_merge_log audit row — all in one transaction.
AdminHandler gains advisor and updated constructor. VersionMergeAdvisor added
to pkg/ai versions.
Migration 000026 adds merged_into_id + merged_at to market_editions and
extends the status CHECK constraint to include 'merged'. FindSimilar now
excludes merged editions from candidates.
AdminHandler gains a SimilarityClassifier field; FindDuplicates enriches
the top 5 pg_trgm candidates with LLM same/confidence/reason verdicts.
simClassifier from routes.go is passed through to avoid a second instance.
schemaFromMap now logs a warning when keys genai.Schema ignores
(pattern, minLength, $ref, etc.) are present, keeping the workaround
visible. LLMEnricher skips Google Search grounding when total scraped
chars >= 1500, conserving free-tier quota on content-rich pages.
promptHashShort(system+"\x00"+user)[:12] computed on ErrSchemaViolation
and attached to ProviderError.PromptHash. research.go schema-violation
log now includes prompt_hash for cross-referencing ai_usage rows.
Migration 000024 adds prompt_version column + partial index.
PromptVersion plumbed through ChatRequest -> UsageEvent ->
buildUsageEvent -> settings INSERT/SELECT. Version constants
defined in ai/versions.go and wired at all three call sites.
- Add confidence scale (0.95-1.00 / 0.70-0.90 / 0.50-0.70 / 0.00-0.50)
with four annotated few-shot examples to the similarity system prompt
- Add two Ronneburg real-world pairs to similarity.json: descriptive-prefix
swap and low-trigram-overlap rename, both expected same=true
- Replace 3-example inline comment with 7-label taxonomy block so the
model knows all valid categories instead of guessing from partial hints
- Tighten description constraint to 60-220 chars with explicit word bans
- Mark opening_hours as a rough guide, not authoritative for booking
Adds ListEnrichedNeedingLLM to the Repository interface and RunLLMEnrichBacklog
to Service, then wires RunLLMEnrichBacklog into the post-crawl goroutine so
LLM enrichment runs automatically after every crawl without manual triggers.
