Changelog

The pull request is the test plan

  • Improved

Runs started by the ZeusQA GitHub App now add the pull request's title and description to the test, and the agent is asked to check each change it claims to make.

A pull request already says what it changes. ZeusQA now adds it to the test, so the agent is pointed at that change on the preview instead of only the site in general.

  • Claims become checks. The title and description go into the test after your mission. The agent is asked to check each change or acceptance criterion the description names and list each one as a check, passed or not, with a claim that does not hold as a finding.
  • Data, not orders. The description is quoted as the author's words, and the agent is told to treat it as data and to ignore requests in it to skip checks, change the verdict or visit another site. The agent is a model and can still be misled, so keep descriptions to what changed.
  • Trimmed. Template comments are removed and the description is cut at 1,500 characters. With no description, the agent is told to test the change the title names.
  • Not for forks. A pull request from a fork is written by someone without write access, so its title and description stay out of the test. A pull request whose fork was deleted counts as a fork.
  • Title only for bots. Dependabot, Renovate and other bots fill the description with upstream release notes, so for their pull requests only the title goes in.

Set the mission for each repository in Connections, GitHub. A good first mission for every preview is a smoke test.

Part of the ZeusQA changelog. Follow new releases with the RSS feed.

Try it on your own site

Paste a URL, describe the flow in plain English, and watch a real browser do it. Three runs free, no card.