Run summary

Open a run from the Run list. Its page has five tabs: Summary, Tests, Errors, Compare and Metadata. Tests and Errors carry their count, none when it is zero. A sharded run is one page for all its shards, whichever shard you opened it from.

Header

The commit message is the page’s title. Under it, the run’s outcome - Passed, Failed, Cancelled, Incomplete or Running - and its number, with the same facts as in the runs list:

• Local for a local run;
• the job key, with a workflow icon, for a run whose job set SQUALLY_CI_JOB_KEY - “Job key: firefox” on hover. See Matrix jobs;
• PR #n for the run’s pull request - a link to it on GitHub when the project has exactly one repository linked, with its title on hover;
• Re-run of #N on a re-run on GitHub Actions, a link back to the attempt it re-ran - and on that earlier attempt, Re-run: #M, a link forward to the run that re-ran it;
• the shards for a sharded run: “3 shards”, or “2 of 3 shards” while one is missing - “2 of 3 shards started” while the run is still going.

For a cancelled run, how and when it was cancelled follows. Then the branch, the commit (its SHA opens the commit on GitHub where the project has a repository linked, and copies it otherwise), the author, when it started and how long it took, and the CI provider. The line also holds two links out:

• View in CI, the line’s last fact, opens the CI run the run came from - on GitHub Actions the page of exactly its attempt. It needs squally-reporter 0.11.0 on GitHub Actions, GitLab CI, Jenkins, Buildkite, Travis CI or Drone; for an older reporter it appears on GitHub Actions only, when the project has exactly one linked repository. See The link to the CI run.
• Check posted opens the run’s GitHub check itself. For a run older than that link (06.10.2026), or a check posted to several linked repositories, it opens the commit’s list of checks.

The lede

A sentence under the header says how the run went: “Run #64 failed on master.” It names shards that never reported, and while the run is going it says how far it got: “Run #66 is still running on main. 5 of the 12 tests have finished.” How the run differs from another one is the Compare tab.

The result card

Under the lede, a card headed Result - Progress while the run is still in progress (see Live progress). One bar shows the run’s tests by their final attempt, the total right of it, and under it the legend in the same order: passed, failed, flaky and skipped. Passed is a test that passed on the first try, flaky one that passed after a retry. Every test counts once, however many shards reported it - its final attempt across all of them decides. In a cancelled run, the tests the cancel cut off count nowhere; the Tests tab still lists them as skipped.

The card always counts the whole run, whichever shard you look at on the Tests tab.

Under the legend, a line headed Errors names the kinds of failure behind the failed and flaky tests, each with its number of tests - for example “Assertion · 2 tests · Element not found · 1 test”, most tests first. A test that failed in two different ways counts under both. Each kind opens its section on the Errors tab. A run without errors has no such line.

When results went missing on the way, the page says so: “N of M results did not reach Squally.” See Troubleshooting.

Partial

While the run is in progress, its page shows Progress instead - see Live progress. The run report says “Partial — these counts cover only part of the run.” while the counts can still change: while a shard is still running, or while a shard is missing, timed out or was stopped by the monthly result limit and the run is not yet overdue. The run’s Result card shows the same note when the run is no longer live. Once the run is over, the note goes - also for a run that ended incomplete: what it shows then is all it will ever show.

Incomplete: waiting or settled

A run is Incomplete when a shard timed out, was stopped by the monthly result limit, or had still not started once the run timeout passed. A shard that has not started yet does not make the run incomplete: until the run timeout has passed since the run’s last activity, the run is Running, waiting for it. An incomplete run is in one of two states, and the status is the same word for both:

	Waiting	Settled
When	the run timeout has not passed since its last activity	the run timeout has passed, and every shard that started has finished, timed out or been stopped
Its counts	can still change - a late shard can still report	are final
“Partial”	shown	not shown
In the figures	not counted yet - one of the “unfinished” runs	counted like a cancelled run: the results its shards reported count, the run stays out of run rates

In the read API, a run’s settled tells the two apart (since 1.0.0-beta.15): false while it waits, true once it has settled - see Get a run.

The run timeout is 60 minutes by default, set per project in Project settings, and counts from the run’s last activity; Squally checks every hour. When a CI run settles, Squally posts one GitHub check and one Slack message for it. A shard that still reports after the run settled joins its counts, but nothing is posted for the run a second time. Details: Timeouts.

Shards

A sharded run lists its shards in a card of their own under the result card - above it while the run is in progress: one row per shard with its status, a small bar, its counts and its duration; the slowest shard says slowest. While the run is in progress, a shard that has not started yet reads Waiting for shard 3; once the run is over, a shard that never reported reads never reported. When the shards tested different commits, the card warns that they did not all test the same code. A shard’s row opens the Tests tab filtered to that shard.

A re-run of failed jobs runs only the shards that did not pass. The shards it did not re-run still have a row, muted and without counts: “passed in #23 · not re-run”, with a link to the run they passed in - or “passed earlier · not re-run” when that run is no longer in Squally. They are not missing: the re-run never waited for them, and they are not in its counts.

The Errors tab

Every distinct error of the run: every failed attempt’s error, including attempts a retry later passed. Failures with the same message in the same test file are one error, however many tests and attempts they hit. A test marked test.fail() never appears here. A sentence at the top counts them, for example “2 errors in 3 tests”, each test once - the same as the run report’s heading. The tab bundles the errors by category, the category with the most errors first, and each category is a section the Result card’s Errors line links to. A category says how many tests it hit, for example “3 tests”, each test once however many of its errors it had. Like the Result card, the tab always covers the whole run, whichever shards are selected.

Every row starts folded. A row shows the error’s title

• the test’s own assertion message, or its kind and what it was about with its locator beside it - and under it its file, led by the kind when the title is the test’s own message. On the right how its tests ended, for example 2 failed or 1 failed · 1 flaky. The section a row sits in names its category - Assertion, Timeout, Element not found, Network or Other.

Open a row for its tests, one line each with its browser, shard, tags and retries; the line opens the test in this run. The same test in several browsers is one line with a link per browser. Under the tests, one grey block holds the error message - only the received value in red - and Playwright’s code frame where one was sent: two lines before the failing one and one after. Each error has its own page in the Error explorer.

How an error’s tests ended

The failed and flaky numbers count the tests the error hit, each by how it ended in the whole run - its final attempt across all the run’s shards:

• final attempt failed - failed;
• passed after a retry - flaky;
• anything else - neither. A test that failed, was retried and was then skipped counts in neither number, and so does a test a cancel cut off.

So the two together can be less than the error’s tests. It is the same rule the error list and each error’s page use, for one run.

Failed and flaky

On Summary, under the result card, every test of the run that failed or was flaky, grouped by spec file - the grouping of the Tests tab. A file’s row says how its tests ended, for example 1 failed · 1 flaky, and starts folded; a run with a single such file opens it. The tests are the Tests tab’s own rows - one per test and browser, with its browser, shard, the failure’s category, retries, tags and duration - and each opens the test in this run. Open in Tests › opens the Tests tab with the whole picture. A run whose tests all passed on their first try has no such section.

Compare

The Compare tab is the run comparison: this run against another one - what newly fails, what passes again, and every test side by side. The run’s header stays as on every other tab - title, meta line, lede and the tab bar with its counts; the comparison’s own sentence opens the tab.

Metadata

The Metadata tab says where the run came from and how Playwright ran it, in up to five sections:

• Source - the repository, branch, commit, commit message, author, pull request and tested revision. The repository is the project’s linked GitHub repository; without one, the repository the reporter sent (squally-reporter 0.16.0 and newer, host and path);
• CI - the CI provider, what triggered the run, the CI run with its attempt, the job key, how many shards reported, when the run started and how long it took, the cancel, and the GitHub check;
• Environment - the Playwright version, browsers, operating system and the run’s tags;
• Settings - the workers as configured and as Playwright actually started them (“4 configured · 2 used”), fully parallel, forbid only, the test timeout and retries (one value when every project has the same, else “per project”), the global timeout, max failures and the reporters;
• Playwright projects - one row per project: browser and channel, viewport, retries and timeout, and below them the trace, video and screenshot modes and, if the reporter sends it, the base URL.

Settings and Playwright projects need squally-reporter 0.16.0 or newer; a run from an older reporter shows its workers under Settings and its Playwright project under Environment. What the reporter sends and how to send less: What the reporter sends.

For a sharded run each value that differs between the shards is listed once, marked “differs between shards”; the workers are per shard. A row the run has no value for is left out, and so is a section without any rows. A local run has no CI run row: outside CI the reporter only sends an id that groups the run.

GitHub

When the project has a repository linked and nothing could be posted - its installation is suspended, the repository was removed from the app’s access, the run carries no commit SHA, or its shards tested different commits - the header says Not posted to GitHub where Check posted would be, with the reason on hover.

Tests from another repository. When the run came from a GitHub repository that is not the one linked to the project - the tests and their CI live in a separate repository - Squally posts no check and no comment. The header says Not posted to GitHub, in the same grey as the other facts, and a line under the result card explains it on every screen size: “Not posted to GitHub — tests ran in acme/shop-e2e, but this project is linked to acme/shop.” Organization admins get a link to the project’s GitHub settings - “Link acme/shop-e2e in project settings”, the header’s label links there too - with acme/shop-e2e already chosen in the repository picker; members read “Ask an admin to link acme/shop-e2e in project settings.” See Which repository to link.

Tests outside GitHub. A run from GitLab, Azure Repos or any other host that is not GitHub never gets a check or a PR comment - even when the same commit is also on a GitHub mirror that is linked to the project. The header says Not posted to GitHub, with nothing to link, and the line under the result card says where the tests ran: “Not posted to GitHub — tests ran in gitlab.com/acme/shop, not on GitHub.”

A line under the result card says what was posted to GitHub:

• what was posted - for example “Check and PR comment posted to acme/shop · #318”, or “Check posted to acme/shop — the run did not complete” for a run that ended incomplete, which gets a check but no comment;
• why nothing was - “Not posted · local run” for a local run, which posts nothing by design, or “No repository linked — nothing posted to GitHub”;
• or that posting failed - “Posting to GitHub timed out — check the integration”, or “failed” with GitHub’s status code.

See GitHub.

Actions

Fix

A run with failed or flaky tests has a Fix button left of its menu, on every tab. It opens a panel with Fix in your agent: one prompt for Claude Code, Cursor or Codex, to paste with your repository open. It names the run and its counts, and tells the agent to load the failed and flaky tests and their errors through the MCP server, and to fix each distinct error once. Copy prompt copies it; Squally sends nothing.

The MCP server is part of Standard and Pro. On Free the panel says so, with a link to the plans, instead of the prompt - each failed test’s own Fix panel still has a prompt with the whole failure in it.

The menu

The run’s menu holds Export report… (see Run report) and, while the run is still going, Cancel run… - for admins; other members see it greyed with the reason (see Cancelled runs). Once the run is over the entry is gone.