Fix with Squally

Where it is

On the page of a test in a run, a failed attempt has one Fix button next to its Error heading. It opens a panel on the right with two sections:

• Fix in your agent - a prompt for your own coding agent, see Fix in your agent.
• Fix with Squally - Squally asks Claude for a fix and shows the answer in the panel.

Nothing is sent when the panel opens. A suggestion starts only when someone clicks Suggest a fix. Any member of the organization can click it.

The answer

• Cause - what the evidence shows, and whether the test looks flaky or failed the same way on every attempt.
• Fix - numbered steps. A step shows a code change as a diff only when a piece of evidence points at a line, with the evidence named under it (“Based on the code frame.”). Squally does not see your app code. When no evidence points at a line, the answer says so instead of guessing.
• The last step is a command to check the fix locally, for example npx playwright test tests/cart.spec.ts -g "multi-buy".

One answer per error

An answer is reused by every attempt with the same error in the project, in this run or a later one. The panel then says where it came from (“Suggested for #101 · same error”) and makes no new call. Suggest again asks for a new answer, for example after the code changed.

Turning it on

It needs both:

• a Standard or Pro plan, and
• the project’s Suggest fixes with Claude switch, off by default. Only organization admins can turn it on, in the project’s settings.

While the switch is off, the panel says so: an admin gets a link to the project’s AI settings, a member reads “Ask an admin to enable AI analysis.”

An answer already stored stays visible after a downgrade or with the switch off; the panel only stops offering a new one.

Fix in your agent needs neither.

What is sent

For the attempt it is asked about, Squally sends:

• the test’s name and file path, the browser, the run’s branch, the attempt number and its outcome;
• how the test’s other attempts in this run ended, and the local check command;
• the error messages, the top lines of the stack, the code snippet - the lines around the failure - and its location;
• an excerpt of the page snapshot Playwright recorded at the failure;
• durations, the timeout and whether it was hit;
• a derived error category;
• how often the test failed and was flaky in recent CI runs.

Not sent: the commit message, the author’s name, the test output and browser console, screenshots, traces, the recorded steps, and any account data. Error text is stored with secrets masked, so masked text is what is sent.

The suggestion is processed by Anthropic (Claude). The answer is stored with the attempt and deleted with its run.