Quickstart

You need Node.js 20.18.1 or newer, @playwright/test 1.30.0 or newer, and a Squally project. Sign up at app.squally.dev/sign-up with a permanent email address - disposable addresses are refused at sign-up. Sign-up then asks for your organization’s name and your first project’s name, and shows the project’s ingest key once - copy it before you continue. The project’s Overview then shows these steps, with copy buttons, until the first run arrives (No runs yet).

1

Install the reporter

npm install -D squally-reporter

2

Set the ingest key

Squally shows your project’s ingest key when you create the project. It starts with sqly_. The reporter reads it from the environment variable SQUALLY_INGEST_KEY - in CI from a secret (step 4), locally from the terminal you run the tests in.

macOS and Linux (bash, zsh):

export SQUALLY_INGEST_KEY=your-project-ingest-key

Windows (PowerShell) - it asks for the key; paste it and press Enter:

$env:SQUALLY_INGEST_KEY = Read-Host "Ingest key"

Run the tests in the same terminal: the variable lives only there. The reporter sends to https://app.squally.dev; set SQUALLY_INGEST_URL only for another host. Without the key, the reporter warns and sends nothing; your tests run as usual.

3

Add it to playwright.config.ts

import { defineConfig } from "@playwright/test";

export default defineConfig({
  retries: process.env.CI ? 2 : 0,
  reporter: [["html"], ["squally-reporter"]],
  use: {
    // Optional: Squally shows them next to a failed attempt.
    screenshot: "only-on-failure",
    trace: "on-first-retry",
  },
});

The trace is recorded only when a test is retried - one per flaky or failing test instead of one per failed attempt, which keeps stored artifacts small. Locally, with retries off, you get the screenshot.

Replace your reporter line - don’t add a second one. npm init playwright writes reporter: 'html',, and with two reporter: keys the later one wins without a warning, so either the HTML report or Squally silently goes. Using other reporters? Keep them in the list and add ["squally-reporter"].

Retries in CI: Playwright’s template already has the retries line - make sure your config does. Without retries Squally cannot tell a flaky test from a broken one.

4

Run it in CI

Add SQUALLY_INGEST_KEY as a secret in your CI and give it to the job that runs the tests. CIs differ in how a secret reaches the test step: GitHub Actions and Azure Pipelines need it mapped in env: - without that line the tests run as usual and nothing is sent. GitLab CI and most other CIs hand a secret variable to every job.

On GitHub Actions, the key on the job, with an optional last step that marks a cancelled workflow’s run as cancelled (Reporter CLI):

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      SQUALLY_INGEST_KEY: ${{ secrets.SQUALLY_INGEST_KEY }}
    steps:
      # …checkout, setup-node, npm ci, playwright install
      - name: Run Playwright tests
        run: npx playwright test
      - name: Mark Squally run cancelled
        if: ${{ cancelled() }}
        run: npx squally-reporter cancel

Create the secret under Settings → Secrets and variables → Actions → New repository secret. See GitHub Actions.

Azure Pipelines hands a secret variable to a script only through the step’s env:. Map it on the step that runs npx playwright test:

- script: npx playwright test
  env:
    SQUALLY_INGEST_KEY: $(SQUALLY_INGEST_KEY)

See Azure Pipelines. On GitLab CI, add it as a masked CI/CD variable, not a protected one - see GitLab CI.

Sharded with --shard? Nothing to add - the shards of one CI run become one Squally run (Sharded runs). In the app, the project’s Run it in CI page shows these steps for GitHub Actions, GitLab CI, Azure Pipelines and other CIs (The setup page).

The reporter recognises GitHub Actions, GitLab CI, CircleCI, Jenkins, Azure Pipelines, Bitbucket Pipelines, Buildkite, TeamCity, AWS CodeBuild, Travis CI and Drone by themselves; see Reporter setup.

5

See the first run

Open app.squally.dev. The run appears as soon as its first test starts, and its results fill in while the suite runs. An Overview still showing the setup turns into the dashboard by itself. Nothing arrives? Look for the [squally-reporter] lines in your terminal or CI log - when nothing arrives, one of them says why. See Troubleshooting.

Keep the key in the environment

The reporter also takes apiUrl and apiKey as options in the config. Use the environment for the key anyway: Playwright’s json and blob reporters write your resolved config, options included, into their output file.

The reporter never changes your test run’s exit code. Squally’s pages count CI and local runs together by default, so a first run from your machine shows up in the numbers right away; CI or Local narrows them - see Local runs. What the numbers mean: How Squally measures tests.