Gemini failure summaries, GA
One-sentence AI summaries of every failing run now ship to all Pro teams. Backed by Gemini 2.5 Flash.
Docs
QAClan runs on your machine. You install the agent, start the local web UI at localhost:7823, and record, edit, and run Playwright regression suites from your browser. Results sync to qaclan.com for team reporting — always in the background, never blocking.
The entire authoring and execution experience lives on your laptop. Your scripts, your Playwright install, your browsers. The cloud is a read replica for the rest of your team — never a hard dependency.
Run qaclan serve and manage projects, features, scripts, suites, runs and environments at localhost:7823.
Click Record in the Scripts page — the agent launches Playwright codegen and captures your flow as editable Python.
Scripts in a suite share QACLAN_STORAGE_STATE. The agent rewrites your script at runtime so login carries across the whole run.
Every local run is queued for push in the background. No cloud? The queue drains when the network returns.
One data model, three ways to touch it:
Head to Quick Start — about four minutes from nothing to a green run in your browser.
Three steps: get your auth key, install the agent, log in and launch the web UI. The browser opens at localhost:7823 and you're in.
Create an account at qaclan.com and copy your auth key from Settings → Auth Key.
$ curl -fsSL https://raw.githubusercontent.com/qaclan/agent/master/install.sh | sh
Supports Linux (amd64) and macOS (arm64).
QAClan ships a standalone .exe — no Python required. Pick the build for your CPU:
qaclan-windows-amd64.exeqaclan-windows-arm64.exeOr grab the latest from the releases page. After downloading, open PowerShell in the folder containing the .exe and run the commands below from there.
> npm install -g [email protected]
> playwright install chromium
$ qaclan login --key <your_auth_key> && qaclan serve
> .\qaclan-windows-amd64.exe login --key <your_auth_key>
> .\qaclan-windows-amd64.exe serve
Use qaclan-windows-arm64.exe on ARM64 devices.
The browser opens at http://localhost:7823 — start managing your Playwright tests locally.
QAClan's vocabulary is small. Learn these seven entities and you've learned the whole tool.
The top-level workspace, usually one per product. Every other entity belongs to a project. The agent always has exactly one active project — switch it from the sidebar dropdown.
A group of scripts by app area — "Authentication", "Checkout". Each feature has a channel: web (Playwright browser flows) or api (HTTP contract checks). These docs cover web.
A single Python file executing one logical user flow. Its source is either RECORDED (captured via Playwright codegen) or WEB_CREATED (written in the UI editor). Scripts may declare a start_url and a list of var_keys like {{EMAIL}} to be substituted at run time.
A suite is an ordered list of suite items, each pointing at a script. Same script can appear in many suites — it's always one file on disk. Reorder items by dragging in the UI.
A suite run is one execution of a suite. Each script inside gets a script run with status (PASSED, FAILED, SKIPPED), duration, console errors, network failures, screenshots, and a step-by-step breakdown.
A named bag of KEY=value pairs (some marked is_secret). Applied at run time — injected into the script subprocess as env vars, and used to resolve {{KEY}} template placeholders inside script files.
One pip install, one Playwright browser download. The agent ships the CLI, the web UI, and the runner.
This is the default, recommended path. No Python required.
$ curl -fsSL https://raw.githubusercontent.com/qaclan/agent/master/install.sh | sh
The installer drops the qaclan binary into a directory on your PATH and bundles the Playwright browsers it needs. Supports Linux (amd64) and macOS (arm64).
Download the standalone .exe that matches your CPU:
qaclan-windows-amd64.exeqaclan-windows-arm64.exeOr grab the latest from the releases page.
The Windows binary uses your system-installed Playwright instead of bundling it. Install Node.js (LTS) from nodejs.org, then in PowerShell:
> npm install -g [email protected]
> playwright install chromium
$ qaclan status
✓ signed out
✓ data dir ~/.qaclan
✓ no active project
Only needed if you're hacking on the agent itself.
$ git clone https://github.com/qaclan/agent.git && cd agent
$ python3 -m venv env && source env/bin/activate
$ pip install -r requirements.txt
$ playwright install chromium firefox webkit
$ python qaclan.py serve --port 7823
bash build.sh for a release single-file binary in dist/, or bash build.sh --dev for a faster standalone directory build. Both use Nuitka.Everything the agent needs sits under ~/.qaclan/:
| Path | Holds |
|---|---|
~/.qaclan/qaclan.db | SQLite — projects, features, scripts, suites, suite items, runs, environments, sync queue |
~/.qaclan/scripts/ | Script .py files on disk (one per script) |
~/.qaclan/runs/ | Per-run artifacts — screenshots, console logs, network logs |
~/.qaclan/state/ | Storage state files per suite (shared Playwright sessions) |
~/.qaclan/config.json | Auth key, server URL, active project id |
~/.qaclan/scripts/ into git if you want to share them.Linux / macOS — re-run the installer; it replaces the existing qaclan binary in place:
$ curl -fsSL https://raw.githubusercontent.com/qaclan/agent/master/install.sh | sh
Windows — download the latest .exe from the releases page and overwrite the existing file.
The agent ships a cleanup command — it removes ~/.qaclan/ entirely. Pass -y to skip the prompt.
$ qaclan uninstall
? Delete ~/.qaclan (14 projects, 482 scripts)? (y/N)
One auth key per user, tied to your qaclan.com account. Grab it from the dashboard, pass it to qaclan login, you're done.
Go to qaclan.com and Sign in with Google — Google OAuth is the only sign-in method. An auth key is generated automatically on first sign-up.
Open Settings → Auth Key to view and copy it. The key looks like qc_… (a qc_ prefix followed by a 32-byte url-safe random token).
$ qaclan login --key <your_auth_key>
On Windows, substitute the binary you downloaded:
> .\qaclan-windows-amd64.exe login --key <your_auth_key>
The agent validates the key against the server, then stores it in ~/.qaclan/config.json (or %USERPROFILE%\.qaclan\config.json on Windows).
If you start qaclan serve without being logged in, the first screen prompts for your auth key. Paste it, click Save, and you're in.
If a key leaks, open Settings → Auth Key → Regenerate Key on qaclan.com. This invalidates the old key immediately — any agent still using it will need to qaclan login --key <new_key> before its next call succeeds.
$ qaclan logout
Removes the key from config.json. Subsequent commands (except login, logout, serve) will error until you log in again.
--key.Everything you do day-to-day happens here. Start it with qaclan serve, open localhost:7823, and you land on the Scripts page of the active project.
The sidebar shows six items, in this order:
localhost:7823 you land here, filtered to the active project.The top bar carries four global controls:
localStorage under qaclan-theme.| Page | What it's for |
|---|---|
| Environment | Named bags of env vars applied at run time. Mark individual vars as secret. |
| Features | Group scripts by app area. A feature has a channel: web or api. |
| Scripts | Create, record, edit, delete scripts. Opens the in-browser editor (CodeMirror 6, Python). |
| Suites | Assemble ordered lists of scripts. Drag to reorder. Run, or open the run dialog. |
| Runs | History of suite runs with per-script breakdown, screenshots, console/network errors, step timeline. |
| Settings | Manage your auth key — display the connected account, update the key, or disconnect. |
A project is your workspace — usually one per product. The agent always operates on the active project.
Open the project switcher in the top-right of the top bar → + New project. Enter a name. The new project is created and immediately made active.
Click the project switcher in the top bar → pick a project. Everything in the UI — scripts, features, suites, runs, environment — filters to that project.
Open the project in the switcher, then use the Delete project option. Confirm the count of features, scripts, suites, and runs you're about to remove — or use the CLI:
$ qaclan project delete <project_id>
cloud_id references.Every project has a local ID prefixed proj_ and, once synced, a cloud_id (UUID). You'll see the local ID in the Settings page and in run URLs.
Features group scripts by app area. A feature has a channel — web for Playwright browser flows, api for HTTP contract checks.
Sidebar → Features → + New feature. Choose a name and a channel (web / api). That's it.
Click a feature's row to rename it. Channel is locked after creation — you can't flip a web feature to api without recreating.
Hover a row → trash icon → confirm. The agent tells you how many scripts you're about to remove with it.
A script is a single Python file that drives one logical flow. Record it with Playwright codegen, or write it from scratch in the in-browser editor.
Sidebar → Scripts → + Record. In the dialog:
{{BASE_URL}} that will resolve at run time.Hit Start recording. A Chromium window opens to your start URL and Playwright codegen captures every click, fill, and navigation. Close the window when you're done — the agent saves the generated .py and tags the script source: RECORDED.
{{BASE_URL}} so the same script runs against dev, staging, and prod. The agent substitutes from the active environment at run time.Sidebar → Scripts → + New script. Name, feature, confirm. The in-browser editor opens with a template — same editor used for edits.
QAClan ships CodeMirror 6 with the Python language pack and the oneDark theme. Shortcut keys match what you'd expect:
Anywhere in the script — or in the start URL — you can write {{KEY}}. At run time the agent substitutes from the active environment. Every key you reference is recorded on the script's var_keys list so the UI can warn you if an environment is missing one.
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch()
context = browser.new_context() # agent rewrites this at runtime
page = context.new_page()
page.goto("{{BASE_URL}}/login")
page.get_by_label("Email").fill("{{EMAIL}}")
page.get_by_label("Password").fill("{{PASSWORD}}")
page.get_by_role("button", name="Sign in").click()
page.wait_for_url("**/dashboard")
browser.close()
Before each run the agent patches your script in memory — you don't write this boilerplate yourself:
browser.new_context() becomes browser.new_context(storage_state=QACLAN_STORAGE_STATE if set and exists else None). Before the script exits, context state is saved back to the same path. That's how scripts in a suite share a browser session.page.goto(...) and .click(...) the agent inserts page.wait_for_load_state("networkidle"). No more flaky "element not attached" failures.{{KEY}} placeholders become literal strings from the active environment.Click any script to open the editor. Rename inline. Delete from the row menu. Every change is written immediately to ~/.qaclan/scripts/<name>.py.
A suite is an ordered list of scripts you run as a unit. Scripts share browser session state through the suite.
Sidebar → Suites → + New suite. Name it. Pick a channel (web / api — matches the scripts you'll add).
Open the suite → + Add scripts. Pick from the list of scripts in the active project. Add as many as you want; they appear in the order you added them.
Drag a row by its handle to reorder. Order matters — scripts share session state in sequence, so "sign-in" must come before "view-dashboard".
Every script in a suite is given the same QACLAN_STORAGE_STATE file path. Thanks to runtime patching (see Scripts), the first script's cookies, localStorage, and auth headers are picked up by the second, the second by the third, and so on.
The same script can live in any number of suites — add it to a "Smoke" suite and a "Full Regression" suite simultaneously. The script exists in one place on disk and each suite just holds a reference (si_… suite item) with its own order index.
Row menu → Remove from suite. The script itself is untouched.
Open any suite, hit Run. The run dialog asks for browser, resolution, headless mode, stop-on-fail, and environment.
| Option | Default | Description |
|---|---|---|
| Environment | none | Which environment to apply — its vars are injected into os.environ for every script subprocess and used to resolve {{KEY}} placeholders. |
| Browser | chromium | One of chromium, firefox, webkit. |
| Resolution | 1280x800 | Viewport WxH. Stored on the suite run for reference. |
| Headless | on | Toggle the visible browser window. Uncheck to watch the run happen. |
| Stop on fail | off | Abort the suite the moment any script fails. |
Once started, the Runs page streams status in real time. Each script row flips from QUEUED → RUNNING → PASSED or FAILED, and the header rollup updates: total / passed / failed / skipped.
For each script run the agent stores:
PASSED, FAILED, or SKIPPED (when stop_on_fail ended the suite early)console.error() from the pagestep_… entries) with per-step duration and errorPASSED only if every child script run is PASSED. One FAILED → the suite run is FAILED.Clicking into a script run shows the console log, network log, step timeline, and screenshot. All artifacts live under ~/.qaclan/runs/<run_id>/ — bring-your-own-viewer if you prefer.
Named bags of variables. Apply one at run time and its values are injected into the script subprocess and used to resolve {{KEY}} placeholders. The sidebar item is labelled Environment (singular).
Sidebar → Environment → + New environment. Name it (dev, staging, prod — whatever maps to your deployment).
Click into an environment → + Add variable. Each variable has three fields:
{{KEY}} in scripts and start URLs, and available as os.environ["KEY"] at runtime.is_secret. Secret values are masked in the UI and redacted from logs.Row menu → Duplicate (hits POST /api/envs/:name/copy). Convenient for "make a staging copy of dev and change two values".
When you open the run dialog on any suite, pick the environment from the dropdown. Every variable is injected; every {{KEY}} is resolved. If a referenced key is missing, the run fails fast with a clear error before any browser launches.
Sync is best-effort. Changes are queued locally and a background worker pushes them to the server. Failures never block the agent — they retry with backoff.
Two buttons in the top bar control sync manually:
You don't normally need either — the background worker keeps things in sync as you work. The buttons are for "I want this to happen now".
Sync is full-fidelity. Everything the agent tracks is pushed, including script file contents:
.py file contents (sent as plain-text file_content)delete ops that supersede any pending upsert for the same entityEvery change the agent makes (create project, record script, edit a file, complete a run, delete an env var) lands in a local sync_queue table with three fields: entity_type, entity_id, op (upsert or delete). A unique constraint deduplicates repeated edits — only the latest state is pushed.
A background worker drains the queue in a strict order: projects → features → suites → scripts → environments → env vars → suite items → runs. Upserts go before deletes. On failure, each row records last_error, increments attempts, and is retried with exponential backoff.
You can work offline indefinitely. The queue grows; nothing is lost. On the next successful network call the worker drains through every pending op. No feature of the agent requires the server to be reachable.
Onboarding a new machine? Click Pull (↓) in the top bar — or run qaclan pull. The agent fetches:
~/.qaclan/scripts/<local_id>.pyEntities are matched by cloud_id: existing records are upserted, new ones are created.
Every cloud surface on qaclan.com — dashboard, reports, AI script analysis — is gated to the Team plan. The free Community plan is local-only: CLI and agent web UI, no cloud views.
The top strip of the Reports page rolls up the whole workspace into four numbers: average suite health, flaky script count, average pass rate, and average time to green. Quick glance, first thing in the morning.
A composite score per suite, graded A–F. The score combines pass rate, flakiness penalty, and trend direction into a single number so you can compare across suites. Rendered as gauges plus a ranked list.
| Input | Effect on score |
|---|---|
| Pass rate | Base — pass_rate × 100 |
| Flakiness | Penalty — flakiness × 30 |
| Trend direction | Bonus/penalty for improving / declining |
Line chart of pass percentage per suite over a 7-, 14-, or 30-day window, plus a direction label: improving, declining, or stable. Click through a suite to see every run that contributed.
A script is flagged flaky when it flips between PASSED and FAILED across recent runs. The formula is transitions / (total_runs − 1); anything over 0.3 lands on the flaky list. Rendered as a horizontal bar chart ranked by flakiness %.
Per-script runtime across the last N runs, shown as a table: script name, average duration, trend badge, sparkline, percentage change. Trend classification:
For every failure incident, the number of runs it took until the suite was green again. Plotted as a bar chart, one bar per incident. Low bars = fast recovery. Long tails = the team left it broken.
When a suite fails, which script fails first? This report ranks scripts by how often they're the earliest failure in a failed suite run — the most common "domino" at the top of a regression.
On any script's detail page, Analyze with AI sends the script to Gemini (gemini-2.0-flash) which returns a structured report:
The result is cached on the server — open the page later and you see the same report without re-running the model.
Invite teammates, share a common view of the workspace and run history, pay per seat. Any teammate can pull the shared workspace and run the same suites locally.
From qaclan.com: Team → Invite. Enter an email; we generate an invite token (SHA-256 hashed, 48-hour expiry) and email the link. They accept by signing in with Google using that email.
.py file contentsOnce a teammate accepts their invite and logs in to qaclan.com, they get their own auth key from Settings → Auth Key. From there, setup is three commands:
# install the agent (Linux / macOS)
$ curl -fsSL https://raw.githubusercontent.com/qaclan/agent/master/install.sh | sh
# authenticate
$ qaclan login --key <their_auth_key>
# pull the entire team workspace locally
$ qaclan pull
✓ 6 projects · 84 features · 482 scripts · 12 environments
qaclan pull mirrors everything the team has synced — projects, features, suites, suite items, scripts (with their .py file contents recreated under ~/.qaclan/scripts/), and environments with their values. The new teammate can start qaclan serve immediately and run the same regression the rest of the team runs.
qaclan serve, click Pull (↓) in the top bar and the workspace lands in their web UI.Each active member is a seat at $30/month. Billing runs through Paddle. Add or remove seats any time — proration handled on the next invoice.
The CLI is for CI, automation, and headless workflows. Everything you can do in the web UI, you can do from the terminal. Run qaclan <command> --help for the exhaustive flag list.
Five commands bypass authentication — login, logout, uninstall, serve, and the hidden _pw-install. Every other command requires a valid stored auth key; unauthenticated calls exit non-zero with a hint to run qaclan login.
| Command | Description |
|---|---|
qaclan login [--key KEY] [--server URL] | Validate the auth key and store it in ~/.qaclan/config.json. Prompts interactively if --key is omitted. --server overrides the cloud endpoint. |
qaclan logout | Clear the stored auth key. |
qaclan status | Print active project, counts of features / scripts / suites / runs, and last sync time. |
qaclan serve [--port 7823] [--host 127.0.0.1] [--no-browser] | Start the local Flask web UI. --host 0.0.0.0 for Docker. --no-browser skips auto-opening a browser tab. |
qaclan push [--all] | Force a full resync — enqueues every entity and drains the sync queue. --all covers every project; otherwise only the active one. |
qaclan pull | Download the team workspace from the cloud and merge into the local DB by cloud_id. Recreates script files on disk. |
qaclan uninstall [-y, --yes] | Delete ~/.qaclan/ entirely. -y skips the confirmation prompt. |
sync commandSync happens automatically in a background worker; use push to force it and pull to fetch.| Command | Description |
|---|---|
qaclan project create <name> | Create a project and make it active. |
qaclan project list | List all local projects; marks the active one. |
qaclan project use <project_id> | Switch the active project by id (proj_…). |
qaclan project show | Print details for the active project. |
qaclan project delete <project_id> | Cascade-delete a project and everything under it. Interactive confirmation. |
| Command | Description |
|---|---|
qaclan env create <name> | Create an environment under the active project. |
qaclan env list [<env_name>] | List environments and their variable counts. Pass a name to show that environment's variables. |
qaclan env set <env> <KEY> <value> [--secret] | Set or update a variable. --secret masks the value in env list. |
qaclan env delete <env_name> | Remove an environment. Interactive confirmation. |
| Command | Description |
|---|---|
qaclan web feature create <name> | Create a web-channel feature under the active project. |
qaclan web feature list | List features with script counts. |
qaclan web feature delete <feature_id> | Delete the feature. Shows the cascade impact before confirming. |
Records a new script via Playwright codegen. Launches a visible browser; when you close it, the captured Python is written to ~/.qaclan/scripts/, runtime-patched, and scanned for {{KEY}} placeholders.
| Flag | Description |
|---|---|
--feature <feature_id> | Required. Parent feature. |
--name <name> | Required. Script name. |
--url <url> | Optional. Start URL for codegen. |
| Command | Description |
|---|---|
qaclan web script list [--feature <feature_id>] | List scripts, optionally filtered by feature. |
qaclan web script show <script_id> | Print the script's raw Python source. |
qaclan web script import <file_path> --name <name> --feature <feature_id> | Import an existing .py file. Injects storage-state handling and scans for {{KEY}} placeholders. |
qaclan web script patch | Re-patch every script to current runtime-patch standards. Idempotent — skips already-patched files. |
qaclan web script delete <script_id> | Delete a script and its file. Shows suite memberships before confirming; cascades to suite items. |
~/.qaclan/scripts/ in your editor of choice, or use the web UI.| Command | Description |
|---|---|
qaclan web suite create <name> | Create an empty suite. |
qaclan web suite list | List suites with script counts, last-run timestamp and status. |
qaclan web suite show --suite <suite_id> | Print suite details and its ordered scripts. |
qaclan web suite add --suite <suite_id> --script <script_id> | Append a script. Channel must match. |
qaclan web suite reorder --suite <suite_id> --scripts id1,id2,id3 | Replace the suite's order with a comma-separated list of script ids. |
qaclan web suite remove --suite <suite_id> --script <script_id> | Remove a script from the suite. |
qaclan web suite delete <suite_id> | Delete the suite. Interactive confirmation. |
Execute a suite. --suite accepts a local id, cloud id, or exact suite name within the active project.
| Flag | Type / default | Description |
|---|---|---|
--suite <suite_ref> | string — required | Suite to run. |
--env <env_name> | string | Apply an environment — injects vars into each script subprocess and resolves {{KEY}} placeholders. |
--browser | choice — chromium | One of chromium, firefox, webkit. |
--resolution <WxH> | string | Viewport, e.g. 1920x1080. |
--headless | bool — off | Hide the browser window. |
--stop-on-fail | bool — off | Abort the suite on the first failing script. |
# staging smoke on tablet viewport, headless, stop on first failure
$ qaclan web run --suite Smoke --env staging \
--browser firefox --resolution 1024x768 --headless --stop-on-fail
# all three engines in a loop
for b in chromium firefox webkit; do
qaclan web run --suite "Full Regression" --browser $b --headless
done
qaclan runs is a hybrid — invoked without a subcommand it prints the list; the detail view lives under the sibling run group.
| Command | Description |
|---|---|
qaclan runs [--suite <suite_id>] | Print a table of recent suite runs. Optional --suite filter. |
qaclan run show <run_id> | Detailed breakdown of a single suite run — per-script results, error messages, counts. |
The api group is scaffolded but not implemented. Subcommands currently print a "coming soon" notice:
qaclan api feature create|list|deleteqaclan api suite create|listqaclan api run --suite <id> [--env <name>]qaclan serve and you'll see every change live.Free forever for solo QA — no limit on what you can test locally. Team dashboards and analytics on the Pro plan.
| Capability | Solo | Pro |
|---|---|---|
| Local agent, web UI, full recorder | ✓ | ✓ |
| All three browsers (Chromium / Firefox / Webkit) | ✓ | ✓ |
| Unlimited scripts & suites | ✓ | ✓ |
| Offline runs, local history | ✓ | ✓ |
| Cloud run history on qaclan.com | ✓ | ✓ |
| Team workspace & invitations | — | ✓ |
| Pass rate & duration trends | — | ✓ |
| Flaky test detection | — | ✓ |
| Gemini failure summaries | — | ✓ |
| Time-to-green tracking | — | ✓ |
| Priority support | — | ✓ |
Recent additions, newest first. Patch releases are folded into their nearest minor entry.
One-sentence AI summaries of every failing run now ship to all Pro teams. Backed by Gemini 2.5 Flash.
The in-browser script editor switched from a plain textarea to CodeMirror 6 with the oneDark theme and Python language support.
The agent now queues unsent items in a local sync_queue table when the server is unreachable and drains on the next successful network call.
qaclan.com now ranks the top N scripts by flip count over the chosen window. Click to see the sequence of runs where it flipped.
--browser webkit works on Linux without extra setup. Playwright bumped to 1.48.
Scripts in a suite now automatically share QACLAN_STORAGE_STATE. Log in once, reuse across the suite — runtime patching handles it transparently.