Timeouts

Squally knows two timeouts:

	Test timeout	Run timeout
Set by	Playwright	Squally
Applies to	one attempt of one test	a whole run
Default	Playwright’s, e.g. 30 s	60 minutes, per project 30-1440
Shown as	“Timeout · 30s” on the attempt	the run is Incomplete

What the reporter sends

A test that runs out of time is reported as failed, as before. Since squally-reporter 0.10.0 each result also says whether the attempt ran into its time limit (timedOut) and what that limit was (timeoutMs) - including a limit set by test.setTimeout() or test.slow() inside the test. A limit of 0 means the test had none.

Results from a reporter before 0.10.0 carry neither: the API returns null for both, and Squally shows nothing new for them.

The Timeout mark

An attempt that ran into its limit is marked Timeout in Squally, together with its limit - for example “Timeout · 30s”. The Copy-for-AI prompt for such an attempt says the same in its own words: “Timed out at its N ms limit”, right after the attempt’s duration.

The Timeout category

Squally sorts errors into categories. For an error whose example occurrence ran into its time limit, the category is Timeout - Timeout Issues in the read API - and the error’s title starts with “Timeout”, whatever the message says. The flag decides, not the wording: a test that timed out while waiting for an element is a timeout, even if its message reads like a missing element.

This applies to results from squally-reporter 0.10.0 on; errors recorded before keep their category.

The metrics are unchanged

A timed-out attempt is a failed attempt. When it is a test’s final attempt, that run counts as failed for the test - in its failure rate and its time lost, exactly like any other failure. See How Squally measures tests.

Two cases that look similar and are not timeouts:

• A test Playwright cuts off because the whole run is stopping - maxFailures reached, or a Ctrl+C - is reported as skipped. It did not fail and does not count against the test. See Cancelled runs.
• A test marked test.fail() that times out counts as a failure, not as the expected failure - Playwright reads it that way too.

The run timeout

A run that never reports its end - its CI job was killed, or the reporter stopped - would stay Running forever. Squally ends it instead:

• The run timeout counts from the run’s last activity - when it began, last sent results or reported its end - not from its start. A long suite that keeps sending results stays running, however long it takes; a run that goes silent ends one timeout after it last sent anything.
• It is 60 minutes by default. Admins set it per project, from 30 to 1440 minutes, in Project settings → General. Pick more than the longest stretch your suite runs without finishing a test - a single very long test sends nothing until it ends.
• Squally checks every hour. A run silent for longer than the timeout is then marked Incomplete. Until that check, a run whose reporter died shows as running - for up to an hour more.
• A sharded run is overdue once the timeout has passed since the last activity of any of its shards. If every shard that started has then finished, timed out or been stopped by the monthly result limit, the run is over - a shard that never started at all no longer keeps it waiting. It is Incomplete.
• A run stopped by the monthly result limit - shown as Incomplete the moment the limit is reached - ends the same way once it is overdue.

A run waiting for a shard that has not started is Running until it is overdue - only then does the missing shard make it incomplete. Until it is overdue, an incomplete run - a shard timed out or was stopped by the limit - is waiting: its counts can still change, and it counts nowhere yet. Once it is overdue and over, it is settled - see Incomplete: waiting or settled. The runs list shows both under Incomplete in the result menu of its filter sentence (any result), and runs still going under Running.

An incomplete run counts like a cancelled one once it has settled. The results its shards reported count in the test metrics and the errors. The run itself is a run, but what it came to is not the suite’s result, so it stays out of the run rates: a branch’s pass rate and median duration on the Overview’s Branches and Analytics’ pass rate and run duration. The Overview’s Runs chart counts it as Cancelled or incomplete. A shard that still reports later joins the counts; nothing is delivered for the run a second time. A cancelled run is different - it reported its end; see Cancelled runs.

When a CI run settles, Squally tells GitHub and Slack once - for every cause, a timed-out shard, a shard that never started or the monthly result limit: a GitHub check titled with the run’s counts and “incomplete”, whose conclusion is timed out - neutral when the monthly result limit stopped the run - and one Slack message with an “Incomplete” line. A local run sends neither.

In the read API

timedOut and timeoutMs are on each test of Get a run (the final attempt’s), on each attempt of Get test attempts, and on each occurrence of Get test history, all since read API 1.0.0-beta.4 (API changelog).