Local runs

What a local run is

A run is local when the reporter finds no CI system: none of the supported providers is recognised, and CI is not true or 1. That is a run from your own machine - and also a run from a CI system the reporter does not know that sets no CI variable, or one from squally-reporter older than 0.10.0 on Jenkins, Azure Pipelines, TeamCity or AWS CodeBuild. How the reporter recognises CI is on CI setup.

A sharded run is a CI run when any of its shards reported a CI provider.

Send local runs

The setup is the same as in CI: install the reporter, add it to playwright.config.ts - see Reporter setup - and give it the project’s ingest key in the environment of the terminal you run the tests from. macOS and Linux (bash, zsh):

export SQUALLY_INGEST_KEY=your-project-ingest-key
npx playwright test

Windows (PowerShell) - it asks for the key; paste it and press Enter:

$env:SQUALLY_INGEST_KEY = Read-Host "Ingest key"
npx playwright test

The variable lives only in that terminal. Without it the reporter warns and sends nothing, so a machine without it is not reported at all.

• Branch and commit come from git: the checked-out branch and HEAD.
• Each npx playwright test is its own run. If you shard locally across several invocations and want them to be one run, set SQUALLY_CI_RUN_ID to the same value for each - see Shards, matrix jobs and re-runs.
• Retries are often off locally (retries: process.env.CI ? 2 : 0), and without a retry no run can be flaky - see Turn on retries.

Where they appear

• In the runs list, marked Local. The list shows CI and local runs together by default - it counts nothing, and a run you just started should be there without switching; its All · CI · Local switch narrows it.
• On the run’s own page, with Local beside the run number, and in its run report.
• In the comparison, marked Local on its card and in the picker. A local run is compared with a local run by default, a CI run with a CI run; the picker’s All offers both.
• On a test’s page, where its run table marks each local run Local, and in the row of runs around a test in a run.
• In the numbers, under Local. The Overview, the Test explorer, each test’s page, the Error explorer and each error’s page have an All · CI · Local switch. All is the default and counts both; CI counts CI runs only, Local local runs only.
• In the read API, counted by default on the overview, tests, test metrics, test history, errors and error endpoints; source=ci or source=local narrows them (the default was ci before 1.0.0-beta.18). The response’s population names the runs it counted - see Conventions. List runs lists both kinds unless source narrows it, and every run there and in Get a run says local (since 1.0.0-beta.14).

Local runs count with the CI ones by default, so a project that has only run on a laptop so far shows its numbers. They are a different kind of run, though: you start single tests, let a test fail on purpose while writing it, usually run without retries, and on a machine whose durations do not compare with CI’s. To read your CI on its own, pick CI (or send source=ci).

What they leave out

• GitHub and Slack. A local run posts no GitHub check, no pull-request comment and no Slack message - not even when its commit is on GitHub. The GitHub line on its run page says Not posted · local run. A sharded run with one shard from CI is a CI run, and posts as one.
• The weekly email report counts CI runs only, whatever you send from your machine - and says so in its header, “CI runs only”.