gmackie/ForgeGraph
1
0
Fork 0
Code Issues 8 Pull requests 10 Projects Releases Packages Wiki Activity Actions

feat(previews): run verification suites against the preview URL as a PR status #428

Merged
gmackie merged 2 commits from feat/preview-verification into main 2026-08-25 03:26:30 +00:00
Conversation 1 Commits 2 Files changed 10 +661 -9
gmackie commented 2026-08-24 23:21:34 +00:00
Owner
Copy link

Phase 3 of PR previews plus the fix for tonight's habitplay.io outage (both found/caused by the live proof).

Phase 3 — suites against the preview URL as a PR status

  1. Preview flips live (tickPreviews) → one verification run per active suite (#405 machinery) under pipelineNodeKey: "preview.verify", plus a pending fg/preview-verify commit status on the PR head. Enqueued at the live transition, NOT at agent report time — the custom domain isn't routable until it propagates (the agent report route now skips trigger='preview' deployments for the same reason).
  2. The runner claim route overrides the suite config's baseUrl with the preview hostname; runners already export it as FG_BETA_BASE_URL, so existing suites work against previews unchanged.
  3. The verification report route rolls finished runs into ONE fg/preview-verify status: fail-fast failure on a required-suite failure, success with the test tally once everything is terminal.

Runs attach to the preview's own deployment id, so betaVerificationGate / promotion-path summaries (per-deployment) never see them.

Domain-theft fix (habitplay.io outage, 2026-08-25 ~02:29 UTC)

wrangler attaches the config's routes/custom domains to whatever worker it uploads: the pr-23 preview deploy moved habitplay.io+www onto habit-app-pr-23, and destroying that worker deleted the domains with it (~15 min outage; restored via POST /api/fg/dns/workers-domains → habit-app-router).

  • agent: on a --name override where the config's name differs, strip route/routes (JSONC + TOML, per-env) from the wrangler config before deploying — the platform owns hostname attachment.
  • server: destroyPreview refuses to delete a worker still serving hostnames other than the preview's own, and records which ones.

24/24 preview tests, agent cmd/agent suite green, tsc clean. NOTE: the agent half needs the next agent release to take effect; until then the server guard alone protects production domains.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CMpzX1b6swjezEptw3T71f

Phase 3 of PR previews plus the fix for tonight's habitplay.io outage (both found/caused by the live proof). ## Phase 3 — suites against the preview URL as a PR status 1. Preview flips `live` (tickPreviews) → one verification run per active suite (#405 machinery) under `pipelineNodeKey: "preview.verify"`, plus a **pending** `fg/preview-verify` commit status on the PR head. Enqueued at the live transition, NOT at agent report time — the custom domain isn't routable until it propagates (the agent report route now skips `trigger='preview'` deployments for the same reason). 2. The runner claim route overrides the suite config's `baseUrl` with the preview hostname; runners already export it as `FG_BETA_BASE_URL`, so existing suites work against previews unchanged. 3. The verification report route rolls finished runs into ONE `fg/preview-verify` status: fail-fast `failure` on a required-suite failure, `success` with the test tally once everything is terminal. Runs attach to the preview's own deployment id, so `betaVerificationGate` / promotion-path summaries (per-deployment) never see them. ## Domain-theft fix (habitplay.io outage, 2026-08-25 ~02:29 UTC) wrangler attaches the config's routes/custom domains to whatever worker it uploads: the pr-23 preview deploy moved `habitplay.io`+`www` onto `habit-app-pr-23`, and destroying that worker deleted the domains with it (~15 min outage; restored via POST /api/fg/dns/workers-domains → `habit-app-router`). - **agent**: on a `--name` override where the config's `name` differs, strip `route`/`routes` (JSONC + TOML, per-env) from the wrangler config before deploying — the platform owns hostname attachment. - **server**: `destroyPreview` refuses to delete a worker still serving hostnames other than the preview's own, and records which ones. 24/24 preview tests, agent `cmd/agent` suite green, tsc clean. NOTE: the agent half needs the next agent release to take effect; until then the server guard alone protects production domains. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01CMpzX1b6swjezEptw3T71f
feat(previews): run verification suites against the preview URL as a PR status
All checks were successful
CI / gitleaks (pull_request) Successful in 7s
Details
CI / storybook (pull_request) Successful in 1m24s
Details
forgegraph/ci CI passed
CI / ci (pull_request) Successful in 9m36s
Details
680642bd08 
Phase 3 of PR previews: when a preview flips live, the repo's active
verification suites (#405 machinery) are queued against its deployment
under pipelineNodeKey preview.verify — at the live transition, not at
report time, so the custom domain has propagated before a runner probes
it. The claim route overrides the suite config's baseUrl with the
preview hostname (runners already export it as FG_BETA_BASE_URL, so
suites need no changes), and the verification report route rolls the
finished runs up into one fg/preview-verify commit status on the PR
head: fail-fast on a required failure, success with the test tally once
everything is terminal. Preview deployments are excluded from the
report-route enqueue that serves beta.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CMpzX1b6swjezEptw3T71f
gmackie force-pushed feat/preview-verification from 680642bd08
All checks were successful
CI / gitleaks (pull_request) Successful in 7s
Details
CI / storybook (pull_request) Successful in 1m24s
Details
forgegraph/ci CI passed
CI / ci (pull_request) Successful in 9m36s
Details
to 812469634f
All checks were successful
CI / gitleaks (pull_request) Successful in 6s
Details
CI / storybook (pull_request) Successful in 1m21s
Details
forgegraph/ci CI passed
CI / ci (pull_request) Successful in 9m41s
Details
2026-08-25 02:41:26 +00:00 Compare
gmackie commented 2026-08-25 02:49:58 +00:00
Author
Owner
Copy link

Preview environment is live: https://pr-428-forgegraph.forgegraf.com

Deployed 81246963 with the beta stage's environment. It redeploys on every push and is destroyed when this PR closes.

Preview environment is live: https://pr-428-forgegraph.forgegraf.com Deployed `81246963` with the beta stage's environment. It redeploys on every push and is destroyed when this PR closes.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
community
Community process, governance, contribution UX

  
docs
Documentation improvements

  
forgegraph:public-update
Publish approved PR lifecycle updates to the ForgeGraph public feed.

  
good first issue
Suitable for first-time contributors

  
help wanted
Extra attention is needed; maintainers welcome a PR

No labels
community
docs
forgegraph:public-update
good first issue
help wanted
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
gmackie/ForgeGraph!428
No description provided.