CI health · week 42 · 12–18 Oct

tidewater-api: flaky tests cost us 9.4 hours of CI this week

Generated by a coding agent from 1,390 CI runs on main and open PRs. Three tests caused 71% of the reruns.

First-try pass rate
83.2%
▼ 4.1 pts
Flaky reruns
173
▲ 58 vs wk 41
Median pipeline
11m 40s
▲ 0m 50s
p95 API latency (staging)
412 ms
▼ 38 ms

Runs per day

0100200300 MonTueWedThuFriSatSun
PassedPassed after rerun (flaky)Failed

Where the time went

9.4 h CI time lost
Integration 62%E2E 24%Unit 9%Lint 5%

Flakiest tests

TestFlake rateRerunsStatus
orders/refund.int.test.ts
“refunds a partially shipped order”
14.8%
61Quarantine
auth/session.e2e.spec.ts
“refresh token survives tab sleep”
9.3%
38Quarantine
search/index.int.test.ts
“reindex is idempotent”
6.1%
24Fix this week
billing/webhook.int.test.ts
“retries on 503 from provider mock”
2.4%
9Watch
utils/date.test.ts
“formats week numbers across DST”
0.8%
3Fixed

Likely causes, and what the agent suggests

  1. Shared database between parallel integration shards. refund.int and index.int both fail only when shard 3 and 4 overlap. Suggest: one schema per shard.
  2. Fixed sleep in the session E2E test. A wait(2000) races the token refresh on slow runners. Suggest: wait for the refresh request instead.
  3. Mock provider returns 503 without a Retry-After header. The retry path is untested in one branch. Suggest: add the header to the mock and a test for the missing-header case.

Do you agree with the quarantine list? Say so in a comment and the agent will update this page.

Fictional example data. Repository, tests, numbers and PR references are invented for demonstration. Self-contained page: no network requests, no external fonts or images.