Skip to content

Run QA journeys and gate on them

This guide runs a project’s Agentic QA journeys from the terminal or the cloud, and makes a script fail when a step fails. For a first walk through the commands, start with the QA tutorial.

Write a JSON file that lists your journeys:

[
{
"name": "Checkout as a guest",
"startPath": "/cart",
"locale": "en-US",
"viewport": { "width": 1280, "height": 800 },
"steps": [
{ "instruction": "Click \"Checkout\"" },
{
"instruction": "Choose \"Continue as guest\"",
"expectedResult": "The shipping address form is shown"
}
]
}
]

Each journey needs a name, one start page and at least one step. Give the start page as startPath (a path on the site, such as /cart) or as startUrl (a full URL on one of the project’s web targets), never both. locale and viewport are optional, and so is a step’s expectedResult.

Import it:

Terminal window
b8e qa journeys import journeys.json

One invalid journey refuses the whole file, naming the journey, step and field.

  • Put one action in each step.
  • Name each control the way the page labels it: Click "Continue as guest", not “continue”.
  • When the route matters, name it: Open "Returns" from the footer.
  • Write the expected result as something on the page: a heading, a value, a message, a control that is enabled or disabled. “Looks good” or “is easy to use” is never judged a pass.
  • Start each journey on a page reachable signed out, or set it to run as a persona.

About Agentic QA covers why each rule is there.

Importing a journey with no sourceKey always creates a new one. To keep one file as the source of your suite, give each journey a sourceKey that is unique in the file, such as your test case’s key:

{ "sourceKey": "SHOP-101", "name": "Checkout as a guest", "startPath": "/cart", "steps": [ … ] }

Importing again then updates the project’s journey with that key: its name, start page and steps are replaced, and its locale and viewport are kept unless the file gives new ones. A journey with a new key is added. Journeys imported without a key stay duplicates; delete the extra copies with b8e qa journeys delete <id>.

To start a file from a journey that already exists, print it in the import shape:

Terminal window
b8e qa journeys show "Checkout as a guest" --format json | jq '[.]' > journeys.json

That output carries no sourceKey. Add one, import the file, and delete the original journey: from then on, importing the file updates the new one.

To bring in test cases from a test-management tool’s CSV or Excel export, use the dashboard instead: see Import QA test cases from a spreadsheet.

Terminal window
b8e qa audit --journeys "Checkout as a guest"

--journeys takes ids or exact names, comma-separated. A name that contains a comma has to be given in its own --journeys:

Terminal window
b8e qa audit --journeys "Checkout: card, then refund" --journeys qaj_456

Journeys run at once, up to your plan’s concurrent QA runs. If the project has a web target in more than one environment, say which one the journeys start on:

Terminal window
b8e qa audit --journeys qaj_123 --environment staging

To watch the browser, add --no-headless.

--timeout is a budget for each journey, 30 minutes by default. Allow about 30 seconds per step on heavy pages:

Terminal window
b8e qa audit --journeys qaj_123 --timeout 15m

To run without a browser on your machine, add --cloud:

Terminal window
b8e qa audit --journeys qaj_123,qaj_456 --cloud

The journeys run one after another on one Binclusive-hosted VM, up to 5 per run. The CLI prints each run id as the VM starts it, follows the runs, and exits the same way a local run does. --timeout and --headless do not apply. Cloud runs are available where the project offers them.

When the cloud run starts, the CLI prints Cloud QA run <id> started. Pass that id to stop it:

Terminal window
b8e qa audit stop 0b6f2c1e-6d0a-4c8e-9a51-3f2d7f1c9e42

It prints the server’s answer: OUTCOME cancelled, already_ended, or not_found (exit 3). A run’s page in the dashboard has the same Stop cloud run button.

From the dashboard, open the project’s Agentic QA → Journeys, choose Run on a journey, then Run in the cloud. The same dialog shows the CLI command for a run from your machine.

b8e qa audit exits with the run’s verdict:

ExitMeaning
0every step passed or completed
1at least one Finding: a step failed
3a journey failed to run, or its results are unavailable
8the journeys ran, but some steps could not be checked

In a script, pass --ci so the command refuses every prompt. It then needs --journeys and a selected project:

Terminal window
b8e qa audit --ci --project prj_your_project_id --journeys qaj_123,qaj_456

Decide whether 8 should fail your job. It means a step was inconclusive, blocked or hit an error, not that the site is wrong.

b8e qa audit runs as a person, so the machine needs a b8e auth login session. A hosted CI runner has none; use --cloud from a machine that is signed in, or start the run from the dashboard.

Terminal window
b8e qa audit list
b8e qa audit report qar_123

report prints a stored run in the same shape as a live one and exits with the same codes. A run that is still going exits 8, or 1 once it has a Finding. A run that never started exits 3. Ctrl-C during a local run stops the browser but not the run, so its report still shows what it recorded.

For machine-readable output, add --format json to audit, list or report:

Terminal window
b8e qa audit --journeys qaj_123 --format json > qa-report.json

For every flag, see the qa audit reference and the qa journeys import reference.