Analytics

Analytics shows how a project’s tests develop over a period: whether runs pass more often, whether they get slower, and where failures come from. Open it from the sidebar, under the Error explorer.

It is one page. Under the filter sentence, three links jump to its three groups - Reliability · Speed · Where. Five tiles come first, then the groups:

Group	What it holds
Reliability	Pass rate over time, and tests failing more often, flaky more often, and better than before
Speed	Run duration over time, the slowest tests and the tests getting slower
Where	where failures and test time come from: by spec file, by error category, by tag and by project

A sentence under the title sums up the period in its own numbers, for example “56.0% of runs passed in the last 30 days, down from 71.4%. 40.0% of the runs needed a retry.”

A project that has never reported a run shows Nothing to chart yet - “Trends appear here once runs arrive.” - with a link to the Overview.

What it counts

The line under the title is the Overview’s:

Showing CI and local runs on all branches from the last 30 days

• Source - “CI and local runs” (the default), “CI runs” or “local runs”.
• Branch - “all branches” (the default), or one or more branches.
• Period - the last 7, 14, 30 (the default) or 90 days.
• + Filter - narrow by Project (a Playwright project) or by Tag. A chosen filter joins the sentence - “in chromium”, “tagged @smoke” - with a × to remove it.

Runs count as on the Overview: once all their shards have finished, a sharded run as one run. Charts and lists compare shares, not counts - a day with five runs is not worse than a day with one - and show the counts beside them.

The lists of tests - Failing more often, Flaky more often, Better than before, Slowest tests and Getting slower - count one test across Playwright projects, like the Overview’s Most failing: “adds an item” in chromium and in firefox is one row, its second line names both projects, and a run is one run for it.

Project and tag filters

The ⓘ says what a filter narrows:

• Run figures - pass rate, runs with a retry, run duration - count the runs that carry the tag or in which the project ran, with the run’s own outcome. Their tiles say so under the number, for example “runs with @smoke”.
• Every other figure counts only the tagged results of the project.

So under a tag, the pass rate can show runs that failed while By spec file shows no failed test: the runs carried the tag, but the tests that failed did not.

Every link on the page keeps the period, branches, source, tag and project.

Tiles

Tile	What it shows
Pass rate	finished runs without a failed test - passed or flaky - out of all finished runs, for example “56.0%” with “42 of 75 runs”. Cancelled and incomplete runs are in neither number
Runs with a retry	finished runs in which some test passed only after a retry
Run duration · median	half of the finished runs took less, half took more - first shard start to last shard finish. Cancelled and incomplete runs are left out
Run duration · p95	19 of 20 finished runs took less than this - the slow end of a normal period
Suite size	tests in the largest run of the period - a partial run does not shrink it

Hover a tile’s title for its definition.

Comparison with the period before

Every tile compares with the period before, of the same length, on the same source, branch and filters: “↓ from 71.4%”, “same as before”, or “nothing before”. The arrow only says the direction.

The comparison is skipped when the period before was mostly empty - when the project’s first run lies in its second half. A project whose history starts two days before the period would otherwise read “up from 0.0%” against two days of runs. Then:

• every tile’s comparison reads nothing before, and the sentence names no “up from” or “down from”;
• the lists that compare with the period before - Failing more often, Flaky more often, Better than before and Getting slower - say Not enough history before this period to compare. A longer period would only move the period before further back, so there is no link to one;
• By error category’s last column reads “—”.

Charts

Pass rate over time and Run duration over time each have a Day / Week switch; the choice applies to both. Week is offered from 14 days; a week is seven days counted from the period’s first day, so the last one can be shorter, and its tooltip reads “Week of Oct 7”.

The period is counted in 24-hour days back from now, the charts in calendar days, so the first point also holds the hours before its day. Its tooltip says so: “Oct 7, 14:00 – Oct 8”.

A line joins only neighbouring days (or weeks) that each have at least three runs. A day with one or two runs is a hollow, faded point, never joined - one run makes a 0% or a 100% (“fewer than 3 runs” in the legend). A day without runs is a gap with a short mark on the baseline (“no runs”), not a zero, and no line is drawn across it. Hover a point for its value and its runs.

Reliability

Pass rate over time draws the share of finished runs that passed, per day or week, on a 0-100% scale.

Three lists side by side compare each test’s share of runs with the period before - each says so under its heading: “share of runs, against the 30 days before”. The largest change comes first, at most five rows each:

List	What it shows
Failing more often	tests whose share of failed runs rose - the share now (“11.6%”) and before (“before 1.6%”, or “0% before” for a test that ran and never failed)
Flaky more often	tests whose share of runs that needed a retry rose, the same way
Better than before	tests that failed or were flaky less often - the five largest improvements of both kinds. Line 2 starts with what the test was: was failed or was flaky. On the right the share now with its direction - “↓ 8.4%” - and “before 62.3%”

A run counts as failed for a test when the test failed in any of its projects, and as flaky when it passed only after a retry and failed in none. A test needs at least three runs in both periods to move. A test that improved both ways is listed once, by its larger improvement.

New tests are not in these lists. A test that did not run at all in the period before has nothing to move from. When new tests failed in this period, a grey line under Failing more often counts them - “2 new tests failing · see Most failing” - and links to the Overview on the same period, source and branches, where Most failing ranks them; under Flaky more often, “1 new test flaky · see Most flaky”. The line counts new tests with at least three runs, Most failing’s own minimum, and is left out when there are none.

An empty list says why, in this order:

• Not enough history before this period to compare. - the period before is mostly empty (see above). No link, and no line about new tests.
• Needs at least 3 runs in this and the previous period. - no test ran three times in both. When the project has runs before this period, Show the last 14 days (the next longer period) follows.
• “No test failed more often than before.”, “No test was flaky more often than before.” or “No test failed or was flaky less often than before.” - tests could move, and none did.

Speed

Run duration over time draws the median as a solid line and the p95 as a dashed one, per day or week. Hover or focus a legend entry to bring its line forward; the other fades.

Section	What it shows
Slowest tests	up to five tests with the slowest median, among tests with at least three runs, with how many runs that is (“83 runs”)
Getting slower	tests whose median rose against the period before - “+4.0s” with “16s, was 12s” - the largest rise first, among tests with at least three runs in both periods. A test that did not get slower is not listed, however slow it is

Both count one test across projects, per run: in each run the test’s longest final attempt over its projects, then the median over the runs. Line 2 names the test’s file and its projects.

Getting slower is empty with Not enough history before this period to compare. when the period before is mostly empty, “Nothing to compare with: no test ran 3 times in the period before.” when no test had enough runs there, and “No test got slower than in the period before.” otherwise. It never links to a longer period.

Where

By spec file

One table for each spec file’s failures and the time it costs:

Column	What it shows
Spec file	the file, “no file” for results without one
Tests	the tests in the file - a test in two Playwright projects counts twice here, as in the Test explorer
Failed	the share of the file’s test results that failed
Flaky	the share that passed only after a retry
Test time	the time the file’s tests took - every attempt, retries included - and its share of all test time: “34min 3s · 24%”

The table opens on Failed, the highest share first. Click a column’s header to sort by it - names A to Z, numbers largest first - and again to turn the order around. A share of 0% is drawn grey; “—” means there is no result to compute it from, and such a row sorts last either way.

Five files are shown; the rest are one row, “2 other files”, with their sums - their tests, the shares of their results, their test time. Show all 7 files under the table lists every file, sorted the same way, without the rest row; Show fewer folds them again. Sorting applies to all files first, so sorted by Test time the five shown are the five that cost the most time.

By error category

Column	What it shows
Category	the Error explorer’s category - Assertion, Timeout, Element not found, Network or Other. It opens the Error explorer filtered by that category, with the page’s period, branches, source, tag and project
Attempts	the failed attempts of that category, retries included - the Error explorer’s occurrences
% of failures	its share of all failed attempts of the period
Tests	how many tests failed with it - each Playwright project’s variant on its own, as in By spec file
Against the period before	the share then - “↑ from 21%” in red when it rose, “↓ from 47%”, “same as before”, or “↑ from 0%” when the category did not occur then. “—” when the period before is not comparable

The category with the most attempts comes first; the attempts add up to every failed attempt of the period. Without one, the card says “No failed attempt in this period.”

By tag

The same columns as By spec file, per tag - every tag, whatever the tag filter - with the same sorting, rest row (“3 other tags”) and Show all. The row (untagged) holds the results without a tag of their own and stays the last row whatever the sort. A result’s tags include the tags your Playwright config gives every test; a result whose tags are all such config tags counts as (untagged).

By tag appears only when some test carries a tag of its own. Without one, the section is left out. See tags.

By project

Per Playwright project, the share of test results that Failed and that were Flaky, Fails only here - the tests that failed in that project and in none of the others they ran in - and the Median duration.

By project needs two or more Playwright projects in the period. With one, the card says so instead: “By project appears when the period has runs from 2 or more Playwright projects. This period has one: chromium.” With a project filter the table stays, without Fails only here - with one project in view, nothing can fail “only here”.