fix(agent): bound postgres metrics payload to fix large-upload stall #23

Merged

gmackie merged 1 commit from fix/pg-metrics-lean-payload into main

2026-06-26 00:10:38 +00:00

gmackie commented

2026-06-25 23:42:29 +00:00

Owner

Root cause (finally) of master's metrics timeout. Captured via GODEBUG=http2debug on the live agent: the metrics POST body was ~95KB (28 DBs x table stats + full slow-query text). Go's HTTP client stalls mid-upload on a body that large to the Cloudflare edge — the trace shows it sending the body, receiving WINDOW_UPDATE frames, then a 15s timeout. A ~4KB db-level-only payload always succeeds (and curl handles 80KB fine), so it's a Go large-body-upload interaction, triggered purely by payload size. Earlier fixes (h2 pings, fresh conn, backoff in #22) didn't address the size, so the hang persisted.

Fix: keep lightweight db-level metrics (size/connections/cache — what GET /api/fg/databases and the view use) for every DB each cycle, but collect the heavy per-table + slow-query detail for only a rotating 2 DBs per cycle; cap slow queries 10->5 and truncate query text to 1000 chars. Every payload now stays ~10KB, well under the h2 flow-control window; all DBs' detail still refreshes within a few minutes, and the per-app postgres-metrics UI keeps its data. nil detailDBs = full detail (on-demand CLI path).

Verified on master (0.1.15): 0 report errors + 0 collection errors over 175s (was every cycle); clean 60s cadence (66 rows / 3 min = 2 cycles x 33 DBs, no retry churn); GET /api/fg/databases shows all 33 DBs live. New tests for detail-gating + truncate; full suite green.

🤖 Generated with Claude Code

**Root cause (finally) of master's metrics timeout.** Captured via `GODEBUG=http2debug` on the live agent: the metrics POST body was **~95KB** (28 DBs x table stats + full slow-query text). Go's HTTP client stalls mid-upload on a body that large to the Cloudflare edge — the trace shows it sending the body, receiving WINDOW_UPDATE frames, then a 15s timeout. A ~4KB db-level-only payload always succeeds (and curl handles 80KB fine), so it's a Go large-body-upload interaction, triggered purely by payload size. Earlier fixes (h2 pings, fresh conn, backoff in #22) didn't address the size, so the hang persisted. **Fix:** keep lightweight db-level metrics (size/connections/cache — what `GET /api/fg/databases` and the view use) for **every** DB each cycle, but collect the heavy per-table + slow-query detail for only a **rotating 2 DBs per cycle**; cap slow queries 10->5 and truncate query text to 1000 chars. Every payload now stays ~10KB, well under the h2 flow-control window; all DBs' detail still refreshes within a few minutes, and the per-app postgres-metrics UI keeps its data. nil `detailDBs` = full detail (on-demand CLI path). **Verified on master (0.1.15):** 0 report errors + 0 collection errors over 175s (was every cycle); clean 60s cadence (66 rows / 3 min = 2 cycles x 33 DBs, no retry churn); `GET /api/fg/databases` shows all 33 DBs live. New tests for detail-gating + truncate; full suite green. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

gmackie added 1 commit

2026-06-25 23:42:29 +00:00

fix(agent): bound postgres metrics payload to fix large-upload stall

AI Code Review / review (pull_request) Failing after 7s

Details

CI / ci (pull_request) Successful in 18m3s

Details

50d8e89b21

The metrics POST payload reached ~95KB on hetzner-master (28 DBs x table
stats + full slow-query text). Go's HTTP client stalls mid-upload on a body
that large to the Cloudflare edge (verified via http2debug: body send + window
updates, then 15s timeout), while the small db-level-only payload (~4KB) always
succeeds. Keep db-level metrics for every DB each cycle, but collect the heavy
table + slow-query detail for only a rotating subset of DBs per cycle, and cap
slow-query text + count. Every payload now stays well under the h2 flow-control
window; all DBs' detail still refreshes within minutes.

gmackie merged commit 47c447dd22 into main

2026-06-26 00:10:38 +00:00

gmackie referenced this pull request from a commit

2026-06-26 00:10:39 +00:00

fix(agent): bound postgres metrics payload to fix large-upload stall (#23)

gmackie referenced this pull request from a commit

2026-08-25 16:21:59 +00:00

Let a path-scoped route keep its prefix (+ field report on route update --dns-only) (#23)