Skip to content
squallyDocs
CI setup

Tested revision

The two commits a run can carry - the one it is reported under, and the application revision it tested, as your CI job reports it.

A run can carry two commits, and they answer different questions:

FieldWhat it is
commitShaThe commit the run is reported under: the pushed commit, or for a pull request its head commit. Squally’s GitHub check and the commit links use it.
testedRevisionThe application revision the run tested, as your CI job reports it. Opt-in: null unless the CI job sets TESTED_REVISION to that revision, or sets SAME_REPO_AS_APP=true, which makes the commit the job checked out the tested revision - on a pull request usually the merge commit, not its head (below).

Squally does not know the tested revision by itself. Your tests may run against an application deployed from another repository, or from another commit - only you can say which.

The tested revision is shown in Squally - on the run’s Metadata tab, in the read API, and by commit - and that is all it does. It does not move the GitHub check or the pull-request comment to the application’s repository: those always go on commitSha, in the repository the tests ran in. With the tests in a separate repository, link that one - see Which repository to link.

Setting it

Section titled “Setting it”
# The tests live in the same repo as the application: the commit this CI job
# checked out is the tested revision.
SAME_REPO_AS_APP=true


# The tests live in a separate repo (e.g. a dedicated E2E repo pointed at a
# deployed app): pass the tested revision itself - a commit SHA, tag, or
# branch name/ref.
TESTED_REVISION=v1.2.0
  • SAME_REPO_AS_APP=true sends the commit your job checked out: your CI provider’s own variable for it (GITHUB_SHA, CI_COMMIT_SHA, CIRCLE_SHA1, GIT_COMMIT, BUILD_SOURCEVERSION, BITBUCKET_COMMIT, BUILDKITE_COMMIT, BUILD_VCS_NUMBER, CODEBUILD_RESOLVED_SOURCE_VERSION, TRAVIS_COMMIT, DRONE_COMMIT_SHA), or git rev-parse HEAD where there is none - local runs included. Only a commit SHA is sent; if neither yields one, the run has no tested revision.
  • TESTED_REVISION wins when both are set.
  • Neither is required. Without them, results are sent and stored as before, and testedRevision is null.

The value is normalised before it is stored, so the same revision written two ways compares equal: it is lowercased, and a leading v or a refs/heads/ prefix is removed - v1.2.0 is stored as 1.2.0. The reporter says so once per run when it happens.

On a pull request

Section titled “On a pull request”

The two commits differ, on purpose. The run is reported under the pull request’s head commit - that is what the GitHub check and pull-request comment need (see Pull requests). The tested revision is the merge commit CI checked out, because that is what the tests ran against. A plain GitLab merge-request pipeline checks out the source branch rather than a merge, so there the tested revision is that branch’s commit - the same as the head; a merged-results pipeline checks out the merge. On a Bitbucket pull request and a GitHub pull_request_target run, it comes from git rev-parse HEAD.

Finding a run by commit

Section titled “Finding a run by commit”

?sha= on List runs takes a commit SHA or a prefix of one and matches either field: the commit a run is reported under, or its tested revision.

These meanings were corrected in read API 1.0.0-beta.4: commitSha is described for the first time, and testedRevision is no longer described as a pull request’s head behind a merge commit - which no reporter ever sent. See the API changelog.