Skip to content
squallyDocs
Runs

Tests tab

Every test of a run, grouped by spec file - filtered by status, name, file, shard, browser or tag, with what each row says about its attempts, and how to tag tests without changing their names.

The Tests tab of a run lists every test it ran, grouped by spec file. A sharded run lists the tests of all its shards; a test that two shards both reported is one row.

Spec files

Section titled “Spec files”

Each spec file is a row with its status - failed when any of its tests failed - its tests and its duration. Expand it to see its tests.

Filters

Section titled “Filters”
  • Status: All, Passed, Failed, Flaky and Skipped, each with its count - the same four as the run’s result card: Passed means passed on the first try, Flaky passed after a retry. The four add up to the run’s tests.
  • Search: part of a test’s name, of its file or of one of its tags.
  • Shard, on a sharded run: one shard, several, or All shards, each with its number of tests. The selection is part of the URL (?shards=1,3). A shard’s row on the summary lands here with that shard selected.
  • Browser, when the run ran more than one browser: one or several, each with its number of tests.
  • Tag, when the run’s tests carry tags: one or several, each with its number of tests. Several tags show the tests that carry any of them; together with a browser, the tests of that browser that carry one of them. The selection is part of the URL (?tag=@smoke&tag=@checkout), so a filtered list can be shared.
  • Clear filters goes back to all tests.

Within a spec file, failed tests come first, then flaky ones.

What a row says

Section titled “What a row says”
  • The test’s status by its final attempt, and “1 retry” or “N retries” when it ran more than once.
  • Its browser, and on a sharded run its shard.
  • Its tags, exactly as Playwright reports them, @ included.
  • Its annotations by type - issue, slow, fixme, your own - at most two, then “+N” for the rest. fail is not repeated here; the expected failure mark says it. The descriptions and links are on the test’s page in the run (squally-reporter 0.19.0 and newer).
  • On a failed or flaky test, the error’s category - a test that ran into its time limit says so with the limit, for example “Timeout · 1min”. See Timeouts.
  • On a flaky test, which attempts failed before it passed: “Failed in attempt 1 · passed on retry”.
  • A test marked test.fail() is marked as an expected failure - see Expected failures.

A row opens the test in this run, with every attempt and its evidence - see Test in a run. A skipped test has nothing to show and is not a link.

In a cancelled run, the tests the cancel cut off are listed here as skipped, though the result card leaves them out. See Cancelled runs.

Tags

Section titled “Tags”

Squally shows the tags Playwright gives each test. Tags set in the Playwright config (tag in playwright.config.ts) apply to every test, so they stand once in the run’s header instead of on every row.

Tag with the tag option

Section titled “Tag with the tag option”

Squally recognises a test by its name - its Playwright project, file, describe blocks and title. Two ways of tagging change that name:

  • a tag written into the title (test("checkout works @smoke", …)) stays part of the title;
  • test.describe("cart", { tag: "@cart" }, …) adds the describe title to every test inside it.

Tagging an existing test either way makes it a new test in Squally: its history - flaky rate, trends, a test.fail() marker - starts again from nothing. Tag through the option on the test instead, which leaves the name alone:

test("checkout works", { tag: ["@smoke", "@checkout"] }, async ({ page }) => {
  // …
});