What it does
Flaky Test Detective takes a repository's recent CI run history and measures how unstable each test actually is, from how often its status flips across time-ordered runs. The important part is what it does with that number: it separates genuine flakes from tests that fail consistently, because those are regressions wearing a flake's clothing. Each flake gets a likely root cause — timing, shared state, network or unknown — inferred from its failure messages, and flakes above your threshold go into a quarantine plan. The quarantine is deliberately time-limited and auto-expires after fourteen days by default, so a real regression can never be permanently hidden behind a quarantine nobody revisits. Consistently failing tests are surfaced to be fixed and are never quarantined at all. Both rules are stated guardrails in the repo, not settings you have to remember to check. Every step is callable on its own. `detect_regressions` answers "which of these are actually broken" without touching the flakes. `classify_root_causes` gives you the timing-versus-shared-state breakdown when you already know which tests are unstable. `build_quarantine_plan` and `draft_fix_pr` each stand alone. `run_full` is the orchestrated path when you want the whole report. The fix PR comes with a title, branch, labels, a markdown body containing the findings table, and the file changes. It is a draft for human review; the agent never merges it. The report leads with the numbers that matter — flaky count, quarantined, regressions, runs analysed and average instability — then the per-test findings with severity, flakiness rate, root cause and quarantine expiry. All of it is computed deterministically from the history you supply. The language model only narrates, and test error messages are sanitised before they reach it, so raw error text never leaks into the report. No connectors and no credentials: export your CI history and pass it in. Built for QA engineers and DevOps tired of re-running the pipeline and hoping.
Example prompts
- Here is our CI history — which tests are flaky?
- Which of these failing tests are real regressions rather than flakes?
- Classify the root causes: timing, shared state or network?
