What the reporter sends

Per run

• Git: the branch, the commit SHA, the commit message (up to 500 characters) and the author’s name (up to 100).
• CI: the CI provider, the CI run ID, the shard number and total, the event that started the run, and the tested revision if you opt in. Since 0.11.0 also the job key if you set one, the attempt on GitHub Actions, the link to the CI run, and a random value per reporter process that tells a retried start apart from a second job.
• Playwright: the Playwright version, the number of workers as configured, the operating system and its version, and the run’s tags.
• Playwright config (since 0.16.0), read field by field, never the config itself:
  • the workers Playwright actually started (newer Playwright versions only), fullyParallel, forbidOnly, globalTimeout and maxFailures;
  • the reporters by name - Playwright’s own and npm packages by package name; a reporter file of your own is sent as custom, never its path;
  • per project: its name, browserName, channel, viewport, the trace, video and screenshot mode, retries and timeout;
  • the project’s base URL only if you turn it on with SQUALLY_SEND_BASE_URL=true - and then only scheme, host and path, never a login, query or fragment.
• Repository (since 0.16.0): its host and path, such as github.com/acme/shop, from your CI’s variables or the git remote, with credentials, query and .git removed. Squally shows it on the run’s Metadata tab when the project links no GitHub repository. SQUALLY_SEND_REPOSITORY=false turns it off.
• Where in the repository the run started (since 0.19.0): the directory Playwright runs in, relative to the repository’s root - e2e/ in a monorepo, empty at the root - from git rev-parse --show-prefix. Squally needs it to link a test to its file. Not sent when git cannot say, and off with SQUALLY_SEND_REPOSITORY=false.
• The end: passed or failed, when it finished, how many results the reporter produced, and whether the run was interrupted.

Per test result

• The test’s name (with its Playwright project), file path, browser and tags.
• Its status, retry and repeat number, duration, timeout and whether it ran into it, and whether it is marked test.fail().
• For a failure: each error’s message, stack, location and code snippet - the lines of your test file around the failing line, as Playwright shows them.
• The page snapshot Playwright records at a failure - the page’s structure as text.
• The test output - what your test process printed to stdout and stderr - for every result, cut at 16 KB.
• Since 0.12.0, for an attempt that failed or was retried: its steps - the titles of the actions, assertions, hooks and fixtures Playwright recorded, their durations and file:line. At most 300 per attempt, 200 characters per title. Typed values are removed before sending: a fill, type or pressSequentially reads Fill "•••" getByLabel('Password'). An attempt that passed on its first try sends no steps.
• Since 0.19.0, for every attempt: its annotations, masked like error messages - each one’s type, its description and where it was added (file:line), at most 20, descriptions cut at 1,000 characters; the line and column where the test is declared; and the worker that ran it, its slot and when it started.

Files

Only for attempts that failed or were retried - never for a test that passed on its first try, whatever playwright.config.ts says:

• Screenshots Playwright took;
• Traces Playwright recorded;
• the browser console, read out of the trace.

Not sent

Your repository’s contents, your test files (beyond the snippet lines in an error) and your environment variables (beyond the CI values listed above). The file SQUALLY_OUTPUT_FILE writes (since 0.18.0) stays on the runner, and a proxy changes nothing about what is sent.

From the Playwright config, nothing but the fields listed above - in particular never the rest of use (extra HTTP headers, HTTP credentials, storageState, proxy, launch and context options), config.metadata, the command line, the config file and every absolute directory path, webServer, grep, global setup and teardown, or a project’s own metadata.

The GitHub App

If you connect GitHub, Squally’s GitHub App only reads repository metadata and writes checks and pull-request comments. It has no access to your code. While you connect it, Squally asks GitHub once whether you administer the account the app is installed on (for a GitHub organization: whether you are an owner, which is why the app reads organization members); the token GitHub hands over for that question is used for nothing else and not kept. See GitHub.

Masking

Before upload, squally-reporter 0.10.2 and newer replaces secrets with [redacted]; Squally applies the same rules again when results arrive, so results from older reporters are masked as well.

• What is masked: Bearer and Basic authorization values, keys starting with sk_, GitHub tokens of every type, Squally keys (sqly_), AWS access key IDs, and the password in a scheme://user:password@host URL.
• Where: error messages, stacks and snippets, the page snapshot, the test output, the commit message, step titles and annotation descriptions and locations - and, in the reporter, the browser-console file. Squally removes typed values from step titles again on arrival.
• Not masked: screenshots and traces - they are uploaded as Playwright wrote them. Names, file paths and branches are not free text and are sent as they are. A secret in any other format, such as password=… in plain text, is not recognised.

Sending less

• Turn screenshots or traces off, or record them less often, in playwright.config.ts: screenshot: "off", trace: "off" or trace: "on-first-retry". Without a trace there is no browser console either.
• Keep secrets out of test output and error messages - the masking catches the formats above, nothing else.
• SQUALLY_SEND_REPOSITORY=false keeps the repository’s name in CI. The base URL is not sent unless you set SQUALLY_SEND_BASE_URL=true.