---
slug: perf-sheriff
type: pattern
summary: "The rotating on-call role that watches Chromium's performance regression dashboard, bisects each alert to a causative commit, and files an SLA-bound bug against the responsible team."
created: 2026-05-12
updated: 2026-06-07
last_link_verified: 2026-06-07
related:
  tree-sheriff:
    relation: complements
    note: "Tree Sheriff and Perf Sheriff are sibling rotations with disjoint remits: the Tree Sheriff keeps the test tree green in real time, while the Perf Sheriff chases statistical performance regressions that no test failure announces."
  rail-performance-model:
    relation: uses
    note: "The RAIL budgets and their derived benchmark metrics are the framework that decides what counts as a regression worth a Perf Sheriff's attention; without RAIL's thresholds, a benchmark delta has no named meaning."
  rendering-pipeline:
    relation: uses
    note: "The performance dashboard instruments the rendering pipeline's stages, so a Perf Sheriff reading a regression alert is reading a measurement of one pipeline stage's cost."
  main-thread-starvation:
    relation: detects
    note: "Main Thread Starvation is the antipattern a Perf Sheriff sees most often behind a Response-budget regression; the dashboard surfaces the symptom and the bisect names the commit that introduced the long task."
  memory-pressure-response:
    relation: detects
    note: "Memory regressions surface on the same dashboard as latency regressions; a Perf Sheriff triaging a memory-footprint alert is reading the cost side of the memory-pressure tradeoff."
  owners-file-governance:
    relation: depends-on
    note: "Filing a regression bug against the responsible team requires reading the OWNERS chain for the bisected commit's directory; the Perf Sheriff routes the bug to the same authority regime that approved the change."
---

# Perf Sheriff

> **Pattern**
>
> A named solution to a recurring problem.

*The rotating on-call role that watches Chromium's performance regression dashboard, bisects each alert to the commit that caused it, and files an SLA-bound bug against the team that landed the change.*

A performance regression is not a test failure. A test fails loudly: it goes red, it blocks the tree, and a [Tree Sheriff](tree-sheriff.md) reverts it within minutes. A performance regression is a 3% slowdown in a benchmark score that no single run can distinguish from noise, visible only when a statistical model watches the metric across hundreds of commits and flags the moment the distribution shifts. Nobody's commit "broke" anything in the sense the commit queue understands. The build is green, the tests pass, and the product is measurably slower than it was a week ago. The Perf Sheriff is the role the Chromium project invented to make someone responsible for that slowness before it ships.

## Context

This pattern sits at the operational layer of the project's coordination machinery, alongside the [Tree Sheriff](tree-sheriff.md) rotation and underneath the governance patterns that gate what merges. It is what an engineer shipping a performance-sensitive feature encounters not at review time but days later, when a bug lands in their queue claiming their commit regressed a metric they may never have measured.

The pattern reads against the [Performance Model](performance-model.md) section's vocabulary. A Perf Sheriff works in the terms the [RAIL Performance Model](rail-performance-model.md) defines and the [Rendering Pipeline](rendering-pipeline.md) instruments: a regression is a measured violation of a budget, attributed to a stage, traced to a commit. Two readers need the pattern most. One is the contributor shipping a feature that might trip an alert. The other is the CIO or Head of Engineering assessing the performance posture of a downstream Chromium-based product, for whom the upstream regression-detection cadence determines how fast a slowdown introduced upstream becomes a slowdown in their fork.

## Problem

Chromium runs thousands of benchmarks across dozens of platforms on a continuous build. Each benchmark produces a noisy time series: individual runs vary by hardware thermal state, background load, and measurement jitter, so a single slow run means nothing. A real regression is a shift in the underlying distribution: a step change a human can't see by eye in a scrolling dashboard of thousands of metrics.

The project needs the slowdown caught while the causative change is still fresh, because the cost of attribution rises sharply with time. A regression bisected the day it lands routes to an author who still remembers the change and can often fix it in an afternoon. The same regression discovered three months and ten thousand commits later is a forensic exercise with no clear owner. So the recurring difficulty is twofold: distinguish signal from noise across thousands of metrics automatically, and then make a specific human accountable for closing the loop from "the distribution shifted" to "this commit caused it, and its author has been told." Without a named role, the second half never happens: the alert fires into a dashboard nobody owns, and the regression ships.

## Forces

- **Statistical detection cannot self-attribute.** A change-point detector can flag that a metric regressed in a commit range, but the range may span dozens of commits and the detector cannot run the expensive bisection that names the single cause. The expensive step needs a human to authorize it.
- **The triager must be a generalist, but regressions are domain-specific.** A regression can land in graphics, networking, V8, or layout; no single engineer owns all of them. The Sheriff's job is to route, not to fix — but routing wrong wastes the responsible team's time and the regression's remaining freshness.
- **Speed trades against precision.** Alerting on a small delta catches more regressions but floods the rotation with noise and false positives; alerting only on large deltas keeps the queue clean but lets slow accretion through. The threshold has to be set where a human rotation can actually keep up.
- **Authority to file is not authority to fix.** The Sheriff can open a bug and attach an SLA, but cannot compel the responsible team to act. The mechanism has to make the bug hard to ignore without giving the Sheriff power they shouldn't have.

## Solution

Staff a rotating on-call **Perf Sheriff** (typically a one-week assignment, rotated across the engineers who own a benchmark suite or a platform) whose sole responsibility for the week is to clear the performance regression queue. The role runs against a dedicated regression dashboard (the Chromium performance dashboard at `chromeperf.appspot.com`, with bisection driven by the Pinpoint service) that applies change-point detection to every monitored metric and raises an alert when a metric's distribution shifts past a configured threshold. A common alerting threshold is a benchmark-score change on the order of a couple of percent, tuned per metric so the rotation's queue stays clearable.

The Sheriff's workflow is a fixed sequence. **Triage the alert:** confirm the dashboard flagged a real distribution shift, not a measurement artifact or an intended change behind a known flag. **Bisect to a commit:** run Pinpoint, which re-runs the benchmark at successive commits across the flagged range on the same hardware configuration, narrowing the regression to the single commit that introduced it. **File the bug:** open a tracker issue against the responsible change, attaching the dashboard graph, the bisection result, and a severity derived from the regression's magnitude. **Route by ownership:** read the [OWNERS](owners-file-governance.md) chain for the bisected commit's directory to identify the responsible team, and assign the bug there.

The bug carries a service-level agreement. The project's regression policy attaches a response deadline scaled to severity: a high-magnitude regression on a core metric demands acknowledgment and either a fix or a justified revert within a short window; a smaller regression carries a longer one. The escalation path is the mechanism's teeth. A regression bug that goes unaddressed past its SLA escalates, first to the responsible team's lead, then to the performance leadership rotation. For a high-severity regression on a release-blocking metric, the default remedy is reversion: the change comes out unless its author commits to a fix within the release window. The Sheriff does not fix the regression; the Sheriff guarantees that someone who can is told, on the clock, with the evidence attached.

## How It Plays Out

A contributor at an enterprise browser vendor lands a change that adds a synchronous integrity check to a hot path in the resource loader. Every test passes; the change is green and merges. Four days later the Perf Sheriff on rotation that week sees an alert: a page-load benchmark on the low-end Android configuration regressed by 4%. The Sheriff runs Pinpoint across the flagged commit range; the bisection lands on the integrity-check change. The Sheriff files a bug, attaches the dashboard graph and the bisection, reads the loader directory's OWNERS to confirm the routing, and assigns it to the contributor's team with a 48-hour SLA. The contributor, who never ran the low-end Android benchmark locally, now has the regression named, measured, and traced to their exact commit. They move the check off the hot path and the metric recovers. The whole loop closes in under a week because the regression was caught while the change was still fresh in the author's memory.

A downstream team maintaining a Chromium-based runtime is evaluating whether to take an upstream milestone or skip it. Their performance lead reads the upstream regression bugs filed during that milestone's development window and finds three high-severity regressions that were caught by the Perf Sheriff rotation and reverted before the branch cut. The lead concludes the milestone's performance posture is sound, not because the upstream code never regressed, but because the rotation caught and reverted the regressions before they reached the release branch. The Perf Sheriff's bug history is the audit trail that lets a downstream consumer assess upstream performance discipline without re-running the benchmarks themselves.

A regression alert fires on a memory-footprint metric, and the Sheriff bisects it to a change that intentionally traded memory for latency under a known flag. This isn't a regression to file; it's an intended tradeoff the [Memory Pressure Response](memory-pressure-response.md) pattern accounts for. The Sheriff marks the alert as expected, annotates the dashboard so the change-point is not re-flagged, and moves on. Triage means deciding what is *not* a regression as much as what is; an alert is a question, not a verdict.

## Consequences

**Benefits.** The pattern makes statistical regressions actionable. A change-point detector produces alerts; the rotation turns alerts into bisected commits with named owners and deadlines, which is the difference between a dashboard nobody reads and a regression that gets fixed. Attribution happens while the change is fresh, when the fix is cheapest. The SLA-plus-escalation structure lets the Sheriff compel a fix without holding power they shouldn't: the deadline and the default-revert remedy do the compelling, not the Sheriff personally.

The bug history is a durable audit trail. A downstream consumer, a release manager, or a performance-leadership review can read which regressions were caught, how fast they were attributed, and whether they were fixed or reverted. That history is evidence the project's performance posture is maintained rather than asserted.

**Liabilities.** The rotation is a tax on the engineers who staff it. A week on Perf Sheriff is a week not spent on feature work, and the cognitive load of triaging unfamiliar regressions across graphics, networking, and layout is real. The pattern depends on the detector's threshold being tuned correctly: set too sensitive, the rotation drowns in false positives and starts rubber-stamping; set too loose, slow accretion of small regressions passes under the radar and compounds. Bisection is expensive: Pinpoint re-runs benchmarks on dedicated hardware, so a deep bisection consumes a meaningful slice of a finite measurement budget, and a flood of alerts can saturate it. The role also encodes an asymmetry that surprises contributors: the SLA-bound bug filed against their commit can feel arbitrary, because they shipped a green change and a stranger on a rotation they never met is now demanding a fix on a deadline. The bug is systemic, not personal, but it doesn't feel that way to the author seeing it for the first time, and the pattern offers no built-in remedy for that friction beyond the evidence attached to the bug.

## Notes for Agent Context

When proposing or reviewing a Chromium change that touches a performance-sensitive path (resource loading, the rendering pipeline, V8 hot paths, GPU compositing, or memory allocation), assume the change will be measured against the [RAIL](rail-performance-model.md) budgets on the continuous performance dashboard even though no test enforces them locally. A green build and passing tests do not mean the change is performance-clean: regressions are statistical and caught days later by the Perf Sheriff rotation, then bisected to the exact commit and filed as an SLA-bound bug. Do not treat the absence of a failing test as evidence that a change is performance-safe. Flag synchronous work added to hot paths, new allocations in inner loops, and added work on the [main thread](main-thread-starvation.md) as likely regression sources, and recommend benchmarking on the low-end configurations the dashboard monitors before landing. When a change is reverted or a bug filed cites a benchmark regression, route the fix through the same [OWNERS](owners-file-governance.md) chain the dashboard's bisection identified, not through the original review thread.

## Sources

The canonical reference for the role is the Chromium project's own performance-sheriffing documentation under `docs/speed/`, which describes the rotation, the dashboard, and the bisection workflow from the perspective of an engineer already in the rotation. The performance dashboard itself (`chromeperf.appspot.com`) and the Pinpoint bisection service are the project's public-facing tooling; their behavior and the change-point-detection model behind the alerts are documented in the same `docs/speed/` tree. The RAIL framework that defines what counts as a regression originated in Google's web-performance guidance, developed by the Chrome team and published through the project's developer documentation; its budgets are the thresholds the dashboard's metrics are derived against. The broader practice of dedicating a rotating on-call role to a class of continuous-integration signal, rather than treating it as everyone's diffuse responsibility, is a coordination pattern older than Chromium, visible in the build-cop and on-call rotations that large continuous-integration shops adopted through the 2000s; Chromium's specialization names the performance variant explicitly.

## Technical Drill-Down

- [`docs/speed/perf_sheriff_faq.md`](https://chromium.googlesource.com/chromium/src/+/main/docs/speed/perf_sheriff_faq.md) — the project's internal FAQ for the rotation; covers the cadence, the triage steps, and the escalation path from the Sheriff's own seat.
- [`docs/speed/addressing_performance_regressions.md`](https://chromium.googlesource.com/chromium/src/+/main/docs/speed/addressing_performance_regressions.md) — the responsible team's side: what to do when a regression bug lands in your queue, including the revert-by-default remedy for high-severity regressions.
- [`docs/speed/`](https://chromium.googlesource.com/chromium/src/+/main/docs/speed/) — the full speed-documentation tree, including the bisection and benchmarking references the Sheriff consults during triage.
- [Chromium performance dashboard (`chromeperf.appspot.com`)](https://chromeperf.appspot.com/) — the live regression dashboard; the change-point alerts the rotation triages are raised here.
- [Pinpoint](https://chromium.googlesource.com/catapult/+/main/dashboard/dashboard/pinpoint/) — the bisection service that re-runs benchmarks across a commit range on consistent hardware to attribute a regression to a single commit; part of the Catapult project.

---

- [Next: Conway's Law in Multi-Org Chromium](conways-law.md)
- [Previous: Tree Sheriff](tree-sheriff.md)