Error explorer

The Error explorer lists the distinct errors in your project’s runs. Squally groups failures by their error, so one broken selector is one row, not forty red tests.

What an error is

Failures with the same error message in the same test file are one error. Before comparing, Squally removes the parts of a message that are values - durations, IDs, timestamps, numbers - so “Timeout 30000ms exceeded” and “Timeout 30001ms exceeded” are the same error, while locator('#submit') and locator('#cancel') stay two.

An occurrence is one failed attempt with that error, retries included. Tests marked test.fail() are left out.

The title

Every error has one title, the same in the Error explorer, on the error’s page, in the Overview’s Top errors, on a test’s page and in a run’s Errors tab:

1. The test’s own assertion message, when it wrote one - the second argument of expect:

   await expect(page.getByTestId("checkout"), "Checkout button never appeared").toBeVisible();

   The title is then “Checkout button never appeared”. expect.soft and expect.poll with a message count too. The kind of failure moves to the second line, in front: “Not visible · tests/cart.spec.ts · Element not found”.

2. Otherwise the kind and what it was about - “Timeout · test limit 30s”, “Timeout · page.waitForURL”, or “Wrong value” with the value the test expected - or the kind alone, “Not visible”, when what it was about is a locator. The locator then sits beside the title in code font.

3. When Squally recognises no kind, the first line of the message.

A leading “Error:” is never part of a title. A thrown error with a message of its own - throw new Error("Autocomplete returned no suggestions") - is not an assertion message: it has no kind Squally recognises, so rule 3 makes its first line the title.

The read API’s title field keeps the second form, the kind and what it was about, whatever the test’s own message.

The filter sentence

Under the title, the filters read as one sentence:

Showing CI and local runs from the last 30 days in all categories

Each bold word opens a menu:

• Source - CI and local runs (the default), CI runs, or local runs. See local runs.
• Period - the last 7, 14, 30 or 90 days, 30 by default, by the start of the run’s first shard.
• Category - all categories, or one of the categories the period holds.

A word you changed from its default is filled in dark. The ⓘ at the end of the sentence says which runs are counted, for example “All counted runs · 2 unfinished”, or “Counted CI runs only” under CI runs.

The search field at the end of the sentence finds errors by title, locator or message.

The lede

Above the filters, one sentence sums up the list:

14 distinct errors in CI runs of the last 30 days. 3 of them occurred in the latest run with errors, #69.

The second sentence is left out while a search or a category narrows the list.

Which runs count

The same runs as everywhere else in Squally, with one difference:

• Finished runs only. A run counts once all its shards have finished, and a sharded run is one run. A run whose shard died, never started or hit the monthly result limit counts once the run timeout has passed, with the results its shards reported. Runs still going are in no figure; the ⓘ says how many there are.
• Mass failures count. Unlike the test metrics, the error figures include runs where most of the suite failed at once - an outage is still an error you want to see. Such a run is counted once, like any other, and the row is marked mass failure; hover the mark to see how many of the error’s runs were one.

Columns

Column	Shows
Error	the error’s title - the test’s own assertion message, or its kind and what it was about (“Timeout · page.waitForURL”, “Timeout · test limit 30s”) with a locator beside it in code font. Under it the spec file, with a line number where two errors would otherwise read the same, and the category - Assertion, Timeout, Element not found, Network or Other. When the title is the test’s own message, the kind leads that line: “Not visible · tests/cart.spec.ts · Element not found”
Runs	how many runs the error appeared in
Tests	how many tests it hit
Failed %	the share of those runs that failed
Flaky %	the share that were flaky
Last seen	the newest run with this error, and under it since when you have it (“since Sep 30”)

A run failed when a test the error hit failed in it, by the test’s final attempt across all the run’s shards. It was flaky when none of them failed and at least one passed after a retry - the error was real, the retry hid it. Each run counts once, however many tests the error hit in it.

First seen is the earliest counted run with this error that is still stored, whatever the period.

On a phone each error is a card: its title and second line, then “12 runs · 2 tests · 25.0% failed · 8.3% flaky” and when it was last seen.

A timeout the reporter reported decides the category, whatever the message says.

Click Runs, Failed % or Flaky % in the header to sort by that column, highest first; click it again to turn the order around. Last seen opens a menu: sort by last seen or by first seen, newest first, and again to turn it around.

Show more

The list shows 20 errors. Under it, “20 of 54” and Show 20 more add the next 20. A change of filter starts again at 20.

When some failed results had no error to group by, a line above the list says how many are missing from it.

Empty list

• No errors at all: “No errors in the last 30 days” - no test failed with an error in this period.
• A search or category hides everything: “No errors match …” or “No errors in this category in the last 30 days”, with Clear filters.
• A project that has never reported a run says “No errors found”, with Go to Overview - the setup steps are on the Overview.

An error opens its own page: Error detail. It opens on the list’s source, period, category and order, and Error explorer there brings you back to the list as you left it.

For the errors of one run, see the run summary’s Errors tab.