QAce — Agentic E2E Testing

Run QAce natural-language end-to-end tests without leaving your editor. Pick a suite or the test in the current file, choose an environment, and watch results stream into the QAce output channel — a failing run offers to open the dashboard.

What's new in 3.4

• The QAce Browser uses VS Code's built-in browser. QAce: Open test browser lets you pick an environment or type a URL, then Open it or Run a suite / test / plan — the app opens in the native Simple Browser (the browser that renders reliably), and a run opens it at #qace-run-<id> so the QAce overlay drives the test there. No custom webview, no proxy, no headless browser. The agent drives when your app carries the QAce overlay (Settings → Instrument, or the browser extension).

What's new in 2.2

• Code tab — manage codebase awareness from the sidebar: index / re-index the whole workspace, see status, and remove the index anytime. The Envs tab shows whether an index is active. The agent reads your indexed code when testing (sharper plans and failure→file tracing); it still runs without one, just DOM-only and less precise.

What's new in 2.1

• Tabbed sidebar — the stacked sections are gone. The QAce panel is now one view with tabs: Home · Tests · Suites · Envs · Runs · Plan · Code · Chat. Lists keep their inline ▶ run actions; runs open their report on click; the Build chat lives in the Chat tab (history survives tab switches).
• In-editor test browser — QAce: Open test browser (♠ globe icon, Home tab, or an environment's 🌐 action) opens your app in a panel inside the editor. Pick an environment or type a URL, then:
  • ▶ Suite / ▶ Test — creates an overlay run on the server and the QAce agent drives the tests visibly in the panel (gold cursor, HUD, self-healing steps), reporting results back like any other run;
  • ⚡ Plan — auto-runs the whole test plan on the current page.

What's new in 2.0

• Code index upload — QAce: Upload code index (or the status-bar item) uploads a lightweight index of your workspace (hashes, languages, routes, component names, small excerpts) so the server can plan and target tests against your actual code. Turn on qace.autoIndex to re-upload after saves; add qace.indexExcludes patterns on top of the built-in denylist.
• Plan view + Build chat — a plan browser (pages → steps with status) and a chat that drafts and refines the plan with you (now the Plan and Chat tabs).
• Run impacted tests — QAce: Run impacted tests (from git diff) runs only the tests affected by your uncommitted changes.
• Run watcher — long runs keep being polled in the background (status-bar progress) and are reported when they finish.
• Note: the code index requires a token with the new code:write scope — if you signed in before 2.0, run QAce: Sign in once to refresh your token.

Sidebar

Click the QAce icon in the activity bar for a single tabbed panel:

• Home — connection status, sign-in, quick actions (open the test browser, run a suite, run the current test, impacted tests, dashboard) and inline settings (server URL + executor).
• Tests / Suites — inline ▶ runs any row (you pick the environment).
• Envs — your environments; 🌐 opens one in the in-editor test browser.
• Runs — recent runs with live status dots; click one to print its summary and open the markdown report.
• Plan — the AI test plan grouped by page with per-page progress.
• Code — index / re-index / remove the workspace code index, with status and the environments it powers.
• Chat — the QAce Build assistant; it drafts and refines the plan, and the Plan tab refreshes automatically.

The title bar has 🌐 (test browser) and ⟳ (refresh); every tab auto-refreshes after a run. When you're not signed in, each tab offers a sign-in link.

Requirements

• A QAce server (SaaS https://qace.io, or self-hosted).
• A scoped API key (personal access token) with the runs:write scope. Create one in the dashboard under Settings → API keys. runs:write lets the extension start runs and read tests/suites/environments/runs.
• VS Code 1.84+ (the extension uses the built-in fetch, available on the Node 18 runtime that ships with 1.84 and later).

Setup

1. Install the extension and open the QAce panel from the activity bar.
2. Click Sign in with QAce (or run QAce: Sign in). Your browser opens qace.io, you approve, and the editor is signed in automatically through a secure vscode:// callback — no token to copy. The editor receives a scoped, revocable qace_pat_ (manage it under Settings → API keys on qace.io). Prefer a token? QAce: Set API key still works — paste a qace_pat_… with the runs:write scope; it's stored in the OS keychain, not settings.json.

The extension always talks to the hosted platform at https://qace.io — there is nothing to configure.

How browser sign-in works

The extension opens https://qace.io/auth/extension with a one-time state and a vscode://drinktea.qace/… callback. After you authorize on qace.io, the page mints a scoped PAT and redirects back to the editor, which verifies state and stores the token. The callback target is restricted to editor URI schemes, so a token can never be redirected elsewhere. (Requires a QAce server with this endpoint deployed.)

Commands

Settings

The QAce server is always https://qace.io (not configurable).

How it works

The extension calls the QAce REST API with your PAT as a bearer token:

• GET /api/tests, GET /api/suites, GET /api/environments to populate pickers.
• POST /api/runs ({ suiteId | testIds, environmentId, executor }) to start a run. Server-driven runs execute to completion, so results are reported as soon as the call returns; the summary and any failures are written to the output channel.

Notes

• Deep links open the dashboard home (/dashboard); navigate to the run from there.
• auto executor is recommended — it respects each environment's preference and falls back correctly for API vs. browser tests.

License

MIT © QAce