Run QA journeys and gate on them
This guide runs a project’s Agentic QA journeys from the terminal or the cloud, and makes a script fail when a step fails. For a first walk through the commands, start with the QA tutorial.
Create the journeys
Section titled “Create the journeys”Write a JSON file that lists your journeys:
[ { "name": "Checkout as a guest", "startPath": "/cart", "locale": "en-US", "viewport": { "width": 1280, "height": 800 }, "steps": [ { "instruction": "Click \"Checkout\"" }, { "instruction": "Choose \"Continue as guest\"", "expectedResult": "The shipping address form is shown" } ] }]Each journey needs a name, one start page and at least one step. Give the start page as
startPath (a path on the site, such as /cart) or as startUrl (a full URL on one of the
project’s web targets), never both. locale and viewport are optional, and so is a step’s
expectedResult.
Import it:
b8e qa journeys import journeys.jsonOne invalid journey refuses the whole file, naming the journey, step and field.
Write steps the agent can check
Section titled “Write steps the agent can check”- Put one action in each step.
- Name each control the way the page labels it:
Click "Continue as guest", not “continue”. - When the route matters, name it:
Open "Returns" from the footer. - Write the expected result as something on the page: a heading, a value, a message, a control that is enabled or disabled. “Looks good” or “is easy to use” is never judged a pass.
- Start each journey on a page reachable signed out, or set it to run as a persona.
About Agentic QA covers why each rule is there.
Update journeys by importing again
Section titled “Update journeys by importing again”Importing a journey with no sourceKey always creates a new one. To keep one file as the source of
your suite, give each journey a sourceKey that is unique in the file, such as your test case’s key:
{ "sourceKey": "SHOP-101", "name": "Checkout as a guest", "startPath": "/cart", "steps": [ … ] }Importing again then updates the project’s journey with that key: its name, start page and steps
are replaced, and its locale and viewport are kept unless the file gives new ones. A journey with a
new key is added. Journeys imported without a key stay duplicates; delete the extra copies with
b8e qa journeys delete <id>.
To start a file from a journey that already exists, print it in the import shape:
b8e qa journeys show "Checkout as a guest" --format json | jq '[.]' > journeys.jsonThat output carries no sourceKey. Add one, import the file, and delete the original journey: from
then on, importing the file updates the new one.
To bring in test cases from a test-management tool’s CSV or Excel export, use the dashboard instead: see Import QA test cases from a spreadsheet.
Run on your machine
Section titled “Run on your machine”b8e qa audit --journeys "Checkout as a guest"--journeys takes ids or exact names, comma-separated. A name that contains a comma has to be
given in its own --journeys:
b8e qa audit --journeys "Checkout: card, then refund" --journeys qaj_456Journeys run at once, up to your plan’s concurrent QA runs. If the project has a web target in more than one environment, say which one the journeys start on:
b8e qa audit --journeys qaj_123 --environment stagingTo watch the browser, add --no-headless.
Size the timeout
Section titled “Size the timeout”--timeout is a budget for each journey, 30 minutes by default. Allow about 30 seconds per step on
heavy pages:
b8e qa audit --journeys qaj_123 --timeout 15mRun in the cloud
Section titled “Run in the cloud”To run without a browser on your machine, add --cloud:
b8e qa audit --journeys qaj_123,qaj_456 --cloudThe journeys run one after another on one Binclusive-hosted VM, up to 5 per run. The CLI prints each
run id as the VM starts it, follows the runs, and exits the same way a local run does.
--timeout and --headless do not apply. Cloud runs are available where the project offers them.
When the cloud run starts, the CLI prints Cloud QA run <id> started. Pass that id to stop it:
b8e qa audit stop 0b6f2c1e-6d0a-4c8e-9a51-3f2d7f1c9e42It prints the server’s answer: OUTCOME cancelled, already_ended, or not_found (exit 3). A
run’s page in the dashboard has the same Stop cloud run button.
From the dashboard, open the project’s Agentic QA → Journeys, choose Run on a journey, then Run in the cloud. The same dialog shows the CLI command for a run from your machine.
Fail a script on a Finding
Section titled “Fail a script on a Finding”b8e qa audit exits with the run’s verdict:
| Exit | Meaning |
|---|---|
0 | every step passed or completed |
1 | at least one Finding: a step failed |
3 | a journey failed to run, or its results are unavailable |
8 | the journeys ran, but some steps could not be checked |
In a script, pass --ci so the command refuses every prompt. It then needs --journeys and a
selected project:
b8e qa audit --ci --project prj_your_project_id --journeys qaj_123,qaj_456Decide whether 8 should fail your job. It means a step was inconclusive, blocked or hit an error,
not that the site is wrong.
b8e qa audit runs as a person, so the machine needs a b8e auth login session. A hosted CI runner
has none; use --cloud from a machine that is signed in, or start the run from the dashboard.
Read a run later
Section titled “Read a run later”b8e qa audit listb8e qa audit report qar_123report prints a stored run in the same shape as a live one and exits with the same codes. A run
that is still going exits 8, or 1 once it has a Finding. A run that never started exits 3. Ctrl-C during a local run stops the browser but not the run, so its
report still shows what it recorded.
For machine-readable output, add --format json to audit, list or report:
b8e qa audit --journeys qaj_123 --format json > qa-report.jsonFor every flag, see the qa audit reference and the
qa journeys import reference.