Reporter setup
Install squally-reporter, point it at your project, and see what it reads from your CI - provider, branch, commit and shards.
Install
Section titled “Install”The reporter needs Node.js 20.18.1 or newer and @playwright/test 1.30.0 or
newer.
Squally accepts squally-reporter 0.5.0 or newer; use the latest version - the cancel step needs 0.14.0.
Older reporters send to an ingest route that has been removed. Squally answers
them with HTTP 410, code ingest_v1_removed and the message “Ingest v1 was
removed - upgrade squally-reporter to 0.10 or later”, and stores none of their
results.
npm install -D squally-reporterGive it your project’s ingest key through the environment variable
SQUALLY_INGEST_KEY - in CI as a secret (below), locally in the terminal you
run the tests in. Squally shows the key once, when the project is created;
later an admin can only generate a new one, not look the old one up (see
Keys).
macOS and Linux (bash, zsh):
export SQUALLY_INGEST_KEY=your-project-ingest-keyWindows (PowerShell) - it asks for the key; paste it and press Enter:
$env:SQUALLY_INGEST_KEY = Read-Host "Ingest key"Run the tests in the same terminal: the variable lives only there. Neither
Playwright nor the reporter reads a .env file.
Since 0.12.0 the reporter sends to https://app.squally.dev unless
SQUALLY_INGEST_URL says otherwise; older versions need both variables. A
read key (sqly_ro_…) is refused before anything is sent - reporting needs
the ingest key.
Then add it to playwright.config.ts:
import { defineConfig } from "@playwright/test";
export default defineConfig({ reporter: [["html"], ["squally-reporter"]],});Replace your reporter line - don’t add a second one. npm init playwright writes reporter: 'html',, and with two reporter: keys the later
one wins without a warning, so either the HTML report or Squally silently goes.
Using other reporters? Keep them in the list and add ["squally-reporter"].
Turn on retries in CI: Playwright’s
template already has retries: process.env.CI ? 2 : 0, - make sure your config
does. Without retries Squally cannot tell a flaky test from a broken one.
Run your tests as usual, with npx playwright test. Results stream to Squally
while the suite runs, in small batches.
The reporter also takes apiUrl and apiKey as options in the config, but
prefer the environment variables for the key: Playwright’s json and blob
reporters write your resolved config, options included, into their output
file - an apiKey passed as an option ends up there in plain text. The
reporter warns when it sees this combination.
The reporter never changes your test run’s exit code. If Squally cannot be reached, what was already sent stays, only what could not be delivered is lost, and the run itself is unaffected - see When Squally cannot be reached for a while.
A run appears in Squally once its first test starts. Commands that never run a
test create nothing: --list, a globalSetup or webServer that fails to
start, a --grep that matches nothing. A run that runs and contains no tests -
an empty shard, for example - is reported with zero results.
GitHub Actions
Section titled “GitHub Actions”This is the workflow npm init playwright writes to
.github/workflows/playwright.yml. The changes for Squally are the job’s
env: block with the key and, after the test step, the step “Mark Squally run
cancelled”:
name: Playwright Testson: push: branches: [ main, master ] pull_request: branches: [ main, master ]jobs: test: timeout-minutes: 60 runs-on: ubuntu-latest env: SQUALLY_INGEST_KEY: ${{ secrets.SQUALLY_INGEST_KEY }} steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: lts/* - name: Install dependencies run: npm ci - name: Install Playwright Browsers run: npx playwright install --with-deps - name: Run Playwright tests run: npx playwright test - name: Mark Squally run cancelled if: ${{ cancelled() }} run: npx squally-reporter cancel - uses: actions/upload-artifact@v4 if: ${{ !cancelled() }} with: name: playwright-report path: playwright-report/ retention-days: 30Create the secret in your repository under Settings → Secrets and variables
→ Actions → New repository secret, named SQUALLY_INGEST_KEY.
No Squally upload step – the reporter sends results while tests run. The
upload-artifact step is Playwright’s own: it keeps the HTML report and has
nothing to do with Squally. Sharded with --shard? Nothing to add - the shards
of one CI run become one Squally run (see
Shards, matrix jobs and re-runs).
The key sits on the job, not on the test step, because the cancel step reads it too. That step is optional: when the workflow is cancelled, GitHub stops the test step’s shell, not Playwright, so the reporter never reports the end. The step marks the run cancelled in Squally - see Reporter CLI. Without it a cancelled workflow shows as incomplete once the run timeout has passed. Two things to know:
- Set
SQUALLY_CI_JOB_KEYat job level, not only on the test step, if you use one. The cancel step builds the run’s id the same way the reporter did; without the key it names another run, or none. - A fail-fast matrix marks the whole run cancelled. With GitHub’s default
fail-fast: true, one failing shard cancels the others, their cancel steps run, and the run shows as Cancelled instead of Failed. Setfail-fast: falseon a sharded or matrix job.
Other CI
Section titled “Other CI”Add SQUALLY_INGEST_KEY as a secret variable in your CI’s settings (GitLab:
Settings → CI/CD → Variables) and mark it masked. On GitLab, masked but not
protected: GitLab may tick Protect variable for you, and a protected
variable reaches only pipelines on protected branches and tags - see
GitLab CI. Many CIs hand secret
variables to every job, so the test command needs no change - it stays
npx playwright test. Not all do: Azure Pipelines hands a secret variable
to a script only when the step maps it in env: - see
Azure Pipelines.
If yours is one of them, pass SQUALLY_INGEST_KEY to the test step.
Runs, shards and results arrive as CI runs. Pull request checks and comments go to GitHub repositories only, through the Squally GitHub App.
Supported CI providers
Section titled “Supported CI providers”Squally tells CI runs and local runs apart - its pages count both by default, and CI or Local narrows them - so the reporter works out which CI system it runs on from that system’s own environment:
| Provider | Recognised by | Shards grouped by |
|---|---|---|
| GitHub Actions | GITHUB_ACTIONS | GITHUB_RUN_ID-GITHUB_RUN_ATTEMPT |
| GitLab CI | GITLAB_CI | CI_PIPELINE_ID |
| CircleCI | CIRCLECI | CIRCLE_WORKFLOW_ID |
| Jenkins | BUILD_TAG (jenkins-…) | BUILD_TAG |
| Azure Pipelines | TF_BUILD | BUILD_BUILDID |
| Bitbucket Pipelines | BITBUCKET_BUILD_NUMBER | BITBUCKET_BUILD_NUMBER |
| Buildkite | BUILDKITE | BUILDKITE_BUILD_ID |
| TeamCity | TEAMCITY_VERSION | - (set SQUALLY_CI_RUN_ID) |
| AWS CodeBuild | CODEBUILD_BUILD_ID | CODEBUILD_BUILD_ID |
| Travis CI | TRAVIS | TRAVIS_BUILD_ID |
| Drone | DRONE | DRONE_BUILD_NUMBER |
| any other CI | CI=true or CI=1 | - (set SQUALLY_CI_RUN_ID) |
CI counts only as true or 1: CI=false or CI=0 is a local run, unless
one of the providers above is recognised, which always wins. A run that is none
of these is local. Local runs count together with CI runs by default; pick
CI to leave them out, or Local to see them on their own - see
Local runs.
Since 0.10.0, Jenkins, Azure Pipelines, TeamCity and AWS CodeBuild count as CI.
None of them sets a CI variable, so before that their runs were taken for
local runs and counted as local, not CI.
Branch and commit
Section titled “Branch and commit”Each provider’s own variables name the branch and commit a run is reported under; git answers where a provider says nothing.
| Provider | Branch | Commit |
|---|---|---|
| GitHub Actions | GITHUB_REF_NAME; pull requests: GITHUB_HEAD_REF | GITHUB_SHA; pull requests: the PR’s head commit |
| GitLab CI | CI_COMMIT_BRANCH; merge requests: CI_MERGE_REQUEST_SOURCE_BRANCH_NAME | CI_COMMIT_SHA; merged-results pipelines: CI_MERGE_REQUEST_SOURCE_BRANCH_SHA |
| CircleCI | CIRCLE_BRANCH | CIRCLE_SHA1 |
| Jenkins | CHANGE_BRANCH, else BRANCH_NAME, else GIT_BRANCH | GIT_COMMIT |
| Azure Pipelines | BUILD_SOURCEBRANCH; pull requests: SYSTEM_PULLREQUEST_SOURCEBRANCH | BUILD_SOURCEVERSION; pull requests: SYSTEM_PULLREQUEST_SOURCECOMMITID |
| Bitbucket Pipelines | BITBUCKET_BRANCH | BITBUCKET_COMMIT |
| Buildkite | BUILDKITE_BRANCH | BUILDKITE_COMMIT |
| TeamCity | from git | BUILD_VCS_NUMBER |
| AWS CodeBuild | CODEBUILD_WEBHOOK_HEAD_REF | CODEBUILD_RESOLVED_SOURCE_VERSION |
| Travis CI | TRAVIS_BRANCH; pull requests: TRAVIS_PULL_REQUEST_BRANCH | TRAVIS_COMMIT; pull requests: TRAVIS_PULL_REQUEST_SHA |
| Drone | DRONE_BRANCH; pull requests: DRONE_SOURCE_BRANCH | DRONE_COMMIT_SHA |
| any other CI | from git | from git |
A tag build has no branch. A detached checkout never reports HEAD as a branch
name, and refs/heads/ or origin/ prefixes are removed.
Pull requests
Section titled “Pull requests”A pull request’s run is reported under its own head commit and its real
source branch - not the merge commit CI checks out, and not a branch like
123/merge. That head commit is the one Squally’s GitHub check and
pull-request comment have to find. The commit message and author are the head
commit’s too, never the “Merge … into …” message of the checkout.
The head commit is fetched when the checkout lacks it. With
actions/checkout’s default fetch-depth: 1 it is not in the clone, so the
reporter fetches that one commit from origin (git fetch --depth=1 --no-tags), waits at most 3 seconds, and reads its message and author. It does
so only in a shallow checkout - fetching into a full clone would make it
shallow for the rest of your job - and never changes HEAD or any branch. If
the fetch fails or takes longer, message and author stay empty and the reporter
says so once; the run and its exit code are unaffected.
To avoid the fetch, set fetch-depth: 2 on the checkout step, which puts the
head commit in the clone; SQUALLY_DISABLE_HEAD_COMMIT_FETCH=true turns the
fetch off.
The commit the tests actually ran against - the merge - is the run’s tested revision, if you opt in to one.
The link to the CI run
Section titled “The link to the CI run”Since 0.11.0 the reporter sends the page of the CI run, and the run’s page in Squally links to it as View in CI - see Run summary. It is read from each provider’s own variables:
| Provider | Link |
|---|---|
| GitHub Actions | GITHUB_SERVER_URL/GITHUB_REPOSITORY/actions/runs/GITHUB_RUN_ID/attempts/GITHUB_RUN_ATTEMPT - the page of exactly this attempt |
| GitLab CI | CI_PIPELINE_URL |
| Jenkins | BUILD_URL (only with Jenkins served over https) |
| Buildkite | BUILDKITE_BUILD_URL |
| Travis CI | TRAVIS_BUILD_WEB_URL |
| Drone | DRONE_BUILD_LINK |
Other providers send no link. The link is the CI run’s page, never one job’s,
and it is the same with SQUALLY_CI_RUN_ID set. It is sent only as an
https:// URL of at most 2,048 characters - anything else is left out, with
a line under SQUALLY_DEBUG=1 - and Squally never opens it, it only shows it.
For a run from an older reporter, Squally builds the link itself on GitHub
Actions only: when the project has exactly one linked repository and the
run’s id has the reporter’s default shape, digits on both sides:
<GITHUB_RUN_ID>-<GITHUB_RUN_ATTEMPT>. Any other run of an older reporter
has no View in CI.
Shards, matrix jobs and re-runs
Section titled “Shards, matrix jobs and re-runs”Shards are grouped automatically. Each shard reports on its own; Squally groups all shards of one CI run by the id in the “Shards grouped by” column above and shows them as one run. The run completes once every shard has reported - or, once the run timeout has passed, it ends incomplete and counts with the results its shards reported.
On TeamCity, on a CI system not in the table, or when you shard locally across
several --shard invocations, set SQUALLY_CI_RUN_ID to the same value for
every shard of one run - anything unique per CI run and shared by its shards,
such as a build number your CI already exposes. It must be different for
every CI run: a fixed value - a constant, or the commit SHA, which a re-run
of the same commit repeats - lets only the first run in, and Squally refuses
every later one at its start (ci_run_reused). If your CI re-runs a build
under the same number, add the attempt.
Several jobs in one CI run need a job key (0.11.0). The id is shared by
everything in one CI run, so two jobs that are not shards of each other - a
GitHub Actions matrix over browsers without --shard, or an api and a ui
suite - would report into the same Squally run. Give each job its own
SQUALLY_CI_JOB_KEY, shared only by that job’s own shards:
jobs: test: strategy: fail-fast: false matrix: browser: [chromium, firefox] env: SQUALLY_INGEST_KEY: ${{ secrets.SQUALLY_INGEST_KEY }} SQUALLY_CI_JOB_KEY: ${{ matrix.browser }} steps: - run: npx playwright test --project=${{ matrix.browser }} - name: Mark Squally run cancelled if: ${{ cancelled() }} run: npx squally-reporter cancelThe reporter puts the key into the id - <GITHUB_RUN_ID>-<key>-<GITHUB_RUN_ATTEMPT>
on GitHub Actions, <provider id>-<key> on every other provider - so each job
is a run of its own in Squally, with the key as a chip beside its number and a
GitHub check of its own. Without a
key the id is exactly what it was before 0.11.0.
A suite that is sharded and split into jobs takes one key per suite, shared by that suite’s shards:
strategy: fail-fast: false matrix: suite: [api, ui] shard: [1, 2]env: SQUALLY_INGEST_KEY: ${{ secrets.SQUALLY_INGEST_KEY }} SQUALLY_CI_JOB_KEY: ${{ matrix.suite }}steps: - run: npx playwright test tests/${{ matrix.suite }} --shard=${{ matrix.shard }}/2 - name: Mark Squally run cancelled if: ${{ cancelled() }} run: npx squally-reporter cancelEach suite is then one run of two shards.
The key’s rule. Letters, digits, ., _ and -, at most 64 characters;
spaces around it are trimmed. An invalid key prints one warning and is not
used - the job reports without a key, as before 0.11.0 - and your test run is
unaffected.
It replaces the hand-built id. Until 0.11.0 this page suggested
SQUALLY_CI_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}-${{ matrix.browser }}.
It still groups each job, but Squally cannot tell its attempts apart, so a
re-run is never linked to the attempt before it. Switch to
SQUALLY_CI_JOB_KEY. An explicit SQUALLY_CI_RUN_ID still wins for the id;
the key is then still sent - for the chip and the check’s name - but the
reporter sends no attempt with an id it did not build itself, so the job’s
re-runs are not linked. The one exception is an id of exactly the default
shape, <GITHUB_RUN_ID>-<GITHUB_RUN_ATTEMPT>, which Squally still reads on
GitHub Actions.
When two jobs collide anyway, Squally refuses the second one at its start (reporter 0.11.0). The reporter prints one block that names the reason and the fix, sends nothing more for that job, and leaves your test run and its exit code alone:
ci_run_collision- another job already reports under this CI run id (and shard): two jobs without a key, or two with the same key.ci_run_mixed- the id would put an unsharded job and a sharded one, or shards of different totals, into one run.ci_run_reused- the id belongs to a run that has already ended: a fixedSQUALLY_CI_RUN_ID, or a retry that keeps the id (below). The fix is an id that differs per CI run, not a job key.
The fix for the first two is a SQUALLY_CI_JOB_KEY per job - see
Troubleshooting. A
reporter older than 0.11.0 is not refused at the start: its second job is
handed the first job’s run, its results are refused once that run has
completed, and it prints “Squally has already completed run …”.
On Bitbucket, BITBUCKET_BUILD_NUMBER is per repository. Runs from two
repositories that report into the same Squally project can therefore collide
in the same way; use one Squally project per repository.
Re-runs. On GitHub Actions each re-run attempt is its own run, linked to the attempt before it (“Re-run of #N”) - with a job key, to that job’s own earlier attempt. For a sharded run, “Re-run failed jobs” runs only the shards that did not pass; the attempt’s run waits for those shards alone and completes once they have reported, counted with the tests they ran. See GitHub Actions.
Other CI systems re-run a job under the id of a run that already exists: a
GitLab job retry, an Azure Pipelines re-run, a Bitbucket re-run of failed
steps, a Buildkite retry. A job key does not change that id, and Squally
refuses those results - at the job’s start when both attempts ran reporter
0.11.0 (ci_run_reused once the first attempt’s run has ended,
ci_run_collision while it still runs), otherwise once the run has
completed, with the “already completed” message.
playwright merge-reports is not the way to get one run in Squally - the
grouping above is. You can still use it for Playwright’s own HTML report; just
do not point this reporter at the merged result.
Annotations and the test’s line
Section titled “Annotations and the test’s line”Since 0.19.0 every attempt also sends its
annotations - from
test.info().annotations, the { annotation } option, and the ones
test.skip(), test.fixme(), test.slow() and test.fail() add:
test("applies the coupon", { annotation: { type: "issue", description: "https://github.com/acme/shop/issues/412" },}, async ({ page }) => { // …});Squally shows them on the run’s Tests tab,
the test’s page in the run and the
Test detail. An issue whose description is a link
becomes that link.
- Masked like error messages, before they leave your runner and again in
Squally - a token in a description arrives as
[redacted]. - At most 20 per attempt, descriptions cut at 1,000 characters and types at 64; Squally is told how many were left out.
- With them go the test’s line and column, for Open in repository,
and once per run where in the repository the run started (
e2e/in a monorepo) - so the link opens the right file.SQUALLY_SEND_REPOSITORY=falseturns that off with the repository. - The worker, its slot and when the attempt started are stored for a later timeline of the run; nothing shows them yet.
Nothing to configure. See What the reporter sends.
Secrets in test output
Section titled “Secrets in test output”Squally masks common secrets before it stores a result: in error messages,
stack traces, error snippets, page snapshots (errorContext) and test output
(the console output of your test process). A masked secret is replaced with
[redacted]; where a secret follows a label - Bearer, an Authorization: Basic header, the user name in a URL - the label stays, so the line still
reads as what it was.
It masks:
- Bearer tokens
- Basic auth headers
- GitHub tokens, in all formats - including
ghs_installation tokens, such as a GitHub ActionsGITHUB_TOKEN, andgithub_pat_fine-grained tokens - Stripe-style
sk_keys - Squally keys (
sqly_) - AWS access key IDs (
AKIA…) - passwords in URLs (
https://user:password@host)
Masking happens before errors are grouped, so two errors that differ only by a token count as the same error.
squally-reporter 0.10.2 and newer mask in your CI first, with the same rules: error messages, stack traces and snippets, page snapshots, test output, the commit message and (since 0.19.0) annotation descriptions are masked before the reporter uploads them, so a masked secret never leaves the runner. Squally masks again when it stores a result, which covers reporters older than 0.10.2. The reporter also masks the browser console text it uploads (since 0.8.1).
Squally does not open the files the reporter uploads - screenshots, traces and the browser console - so their content is stored as uploaded.
Masking is best effort, not a guarantee: it catches these known formats, not every secret. Do not print secrets in your tests.
When Squally cannot be reached for a while
Section titled “When Squally cannot be reached for a while”Since squally-reporter 0.17.0, failures that pass on their own are waited out instead of costing results:
- a network error, or no answer within 30 seconds
- an HTTP 5xx
- HTTP 429 - Squally’s rate limit
- HTTP 403 from Vercel’s firewall in front of Squally, marked
x-vercel-mitigated: challenge(ordeny). The firewall can challenge one runner for a few minutes; it says nothing about your key.
Each request is retried after 1 second, then 2, 4, 8 and so on up to 30
seconds between tries - each wait shortened by a random part, so shards that
were refused together do not come back together - for up to 90 seconds. A
Retry-After header is honoured. The start of the run gets the same 90
seconds: your tests run meanwhile, and their results are held until Squally
answers. While Squally keeps refusing, results go out one request at a time.
Not retried, because waiting does not change them: Squally’s own error answers (a refused key, an exhausted quota, an unknown or finished run), an HTTP 403 without the firewall’s header, and a redirect. Those print their own line.
When your suite ends, the reporter normally waits up to 5 seconds for the last results. After one of the failures above it waits up to 60 seconds, for the last results and for marking the run complete.
What did not arrive is said once, at the end of the CI log - not once per batch:
[squally-reporter] 37 of 412 results did not reach Squally - Vercel's firewall in front of Squally challenged this runner (HTTP 403, x-vercel-mitigated: challenge), and the retries ran out. Your test run and its exit code are not affected. The run page in Squally shows how many are missing.If the run could not be marked complete either, the line says so, and the run
shows as incomplete in Squally once the project’s run
timeout has passed, with the results that did arrive. If the run could not be
started at all, the line begins Squally did not start this run, and nothing
was sent.
The exit code of your test run is never changed - not by a retry, not by a wait, not by a lost result. Up to 0.16.0 a 5xx or a network error got one retry, a 429 five tries, and a 403 none, with a line for every dropped batch.
Artifacts, since 0.18.0, are uploaded while your suite runs - each as soon as Squally has the result it belongs to - and retried by the same rules: a network error, an HTTP 5xx or 429. When the suite ends, the reporter waits at most 3 minutes for the uploads still running. What did not arrive is one line of its own:
[squally-reporter] 2 of 9 artifacts did not reach Squally - the storage answered HTTP 503, and the retries ran out. Your test run and its exit code are not affected.Until 0.17.0 every upload waited for the end of the run and was tried once.
The end of a run
Section titled “The end of a run”Since squally-reporter 0.18.0, when every result arrived and the run is complete, the last line of the CI log says where it is:
[squally-reporter] Reported to Squally: run #128 · 412 results, 9 artifacts · https://app.squally.dev/o/…/runs/…If something did not arrive, the line about what is missing takes its place.
Set SQUALLY_OUTPUT_FILE to a path to also get the run’s link and counts as
JSON - for a later step that posts the link to a chat or a job summary:
SQUALLY_OUTPUT_FILE=test-results/squally.json{ "version": 1, "runUrl": "https://app.squally.dev/o/…/runs/…", "runNumber": 128, "runId": "…", "shard": null, "complete": true, "tests": { "passed": 40, "failed": 1, "flaky": 1, "skipped": 0 }, "results": { "produced": 44, "delivered": 44 }, "artifacts": { "uploaded": 3, "notUploaded": 0 }}- The file is written at the very end of every run, also one Squally refused
or could not be reached for - then with
"runUrl": nulland"complete": false. Missing directories are created. testscounts the tests the way Playwright does at the end;resultscounts attempts, produced and accepted by Squally.- One file per process: under
--shard, give each shard its own path;shardsays which one it was. - A path that cannot be written is one warning - the exit code stays yours.
- Fields may be added;
versionchanges only if one changes its meaning.
Behind a proxy
Section titled “Behind a proxy”Since squally-reporter 0.18.0, every request goes through the proxy in
HTTPS_PROXY (for https:// addresses, such as Squally) or HTTP_PROXY
(for http://) - the reports, the artifact uploads and
npx squally-reporter cancel. Lower-case names are read first, as curl
does. Hosts in NO_PROXY are reached directly:
HTTPS_PROXY=http://proxy.internal:3128NO_PROXY=localhost,127.0.0.1,.internalNO_PROXYis comma-separated; a name covers all its subdomains (internaland.internalboth coversqually.internal),host:portonly that port, and*everything. IP ranges are not supported.- The proxy is asked for a tunnel (
CONNECT), so the connection to Squally stays encrypted end to end - the proxy sees the host name, not the data. - A login in the proxy URL (
http://user:password@host:port) is sent to the proxy asProxy-Authorization. - A value that is not a URL is one warning, naming the variable but not its value, and the reporter connects directly.
Without these variables nothing changes. Up to 0.17.0 they were ignored, so a runner that reaches the internet only through a proxy sent nothing.
Environment variables
Section titled “Environment variables”| Variable | What it does |
|---|---|
SQUALLY_INGEST_URL | Your Squally app. Optional since 0.12.0, where it defaults to https://app.squally.dev; required before. |
SQUALLY_INGEST_KEY | Your project’s ingest key. Required. (Until 0.9.0 these two were SQUALLY_API_URL and SQUALLY_API_KEY; the old names still work, with a warning - and a set but unused SQUALLY_API_URL keeps 0.12.0 from falling back to the default.) |
SQUALLY_CI_JOB_KEY | Since 0.11.0: one key per job of a CI run - the matrix browser, the suite - shared by that job’s shards, so each job is its own run. Letters, digits, ., _ and -, at most 64 characters - see Shards, matrix jobs and re-runs. |
SQUALLY_CI_RUN_ID | The id that groups the shards of one run - different for every CI run, or later runs are refused (ci_run_reused). Wins over every provider’s own id; a job key is then not part of the id - see Shards, matrix jobs and re-runs. |
SAME_REPO_AS_APP | true: the commit the job checked out is the tested revision. |
TESTED_REVISION | The tested revision itself - a commit SHA, tag or branch. Wins over SAME_REPO_AS_APP. |
SQUALLY_SEND_BASE_URL | Since 0.16.0, true: send each Playwright project’s base URL (scheme, host and path only) for the run’s Metadata tab. Off by default. |
SQUALLY_SEND_REPOSITORY | Since 0.16.0, false: do not send the repository’s host and path - nor, since 0.19.0, where in it the run started. On by default - see What the reporter sends. |
SQUALLY_DISABLE_HEAD_COMMIT_FETCH | true: never fetch a pull request’s head commit - see Pull requests. |
SQUALLY_OUTPUT_FILE | Since 0.18.0, a path: write the run’s link and counts there as JSON at the end of the run - see The end of a run. |
HTTPS_PROXY, HTTP_PROXY, NO_PROXY | Since 0.18.0: the proxy for every request, and the hosts reached without it - see Behind a proxy. |
SQUALLY_DEBUG | Any non-empty value - 1, but also 0 or false - prints detail that is otherwise kept quiet, such as the browser-console extraction, a CI run link that was not sent, and since 0.18.0 the proxy variable in use. |
Which key and URL the reporter takes
Section titled “Which key and URL the reporter takes”When more than one is set, the reporter takes the first of these that has a value (an empty variable counts as not set):
- The key: the
apiKeyoption inplaywright.config.ts, thenSQUALLY_INGEST_KEY, then the oldSQUALLY_API_KEY. WithSQUALLY_INGEST_KEYset,SQUALLY_API_KEYis not read at all - so a shell that holds a read key for the MCP server next to the ingest key reports as it should. - The URL: the
apiUrloption, thenSQUALLY_INGEST_URL, thenhttps://app.squally.dev. The oldSQUALLY_API_URLcounts only when the key came fromSQUALLY_API_KEYtoo; it is the read API’s variable now.
Three cases send nothing, each with one warning that names the variable:
- A read key (
sqly_ro_…) as the key - from the option,SQUALLY_INGEST_KEYorSQUALLY_API_KEY. Reporting needs the project’s ingest key (since 0.12.0 from every source; before, only the old variable was checked). SQUALLY_API_URLset while the key comes from the option orSQUALLY_INGEST_KEY, and noSQUALLY_INGEST_URL: someone meant another Squally, so the reporter does not fall back tohttps://app.squally.dev. SetSQUALLY_INGEST_URLinstead.- No key at all.
The old names still work, with a deprecation warning each.
Two things the reporter sends that have pages of their own: cancelled runs and timeouts.