Contica Test Tool

Native VS Code workbench for Contica integration negative test packs.

What it does

The Contica test workflow has four phases — Fetch → Analyse → Generate → Test. This extension covers Analyse and Generate: it reads a pre-structured bundle of integration artifacts and produces a zip of negative test files. Bundle-repo Environments ship alongside them.

The extension does not fetch or assemble the bundle itself. It expects the input to already exist as structured data and folders — a manifest.json plus schema / mapping / example files laid out as described in Preparing a bundle. Preparing that bundle is the Fetch phase: manual prerequisite work you do by hand before using the extension. There is no automated fetcher; the integration owner follows the input-data requirements to produce a conforming bundle folder. The Test phase — running the generated packs — is future work. The Analyse and Generate engine runs natively in TypeScript; it does not shell out to any external tool.

How it works

┌──────────────────── FETCH  (manual prerequisite) ─────────────────────┐
│                                                                        │
│  You prepare a bundle folder before using this extension:              │
│                                                                        │
│    INT0001/                                                            │
│    ├── manifest.json   ←  declares schemas, mappings, examples         │
│    ├── schemas/                                                        │
│    ├── mappings/                                                       │
│    └── examples/                                                       │
│                                                                        │
└───────────────────────────────┬────────────────────────────────────────┘
                                │  bundle on disk
                                ▼
┌──────────────────── ANALYSE ───────────────────────────────────────────┐
│                                                                        │
│  1. bundleLoader       reads manifest + schemas / mappings / examples  │
│  2. entityAnalyser     counts constraints deterministically            │
│  3. lmClient  ─────────────────────────────────────►  Claude (LLM)    │
│               ◄──── proposed test plan + gaps ──────────────────────  │
│  4. groundingValidator validates every AI claim against the bundle     │
│  5. AnalysisStoreService  persists analysis files to the env clone     │
│                                                                        │
└───────────────────────────────┬────────────────────────────────────────┘
                                │
                                ▼
                  ┌─────────────────────────────┐
                  │       TESTER REVIEW          │
                  │                             │
                  │  Summary │ Plan │ Gaps tabs  │
                  │                             │
                  │  • toggle / add scenarios   │
                  │  • resolve each gap         │
                  │  • click Approve Plan       │
                  └──────────────┬──────────────┘
                                 │  approved plan
                                 ▼
┌──────────────────── GENERATE ──────────────────────────────────────────┐
│                                                                        │
│  1. Read approved plan                                                 │
│  2. mutator            applies 6 mutation operations to message files  │
│  3. lmClient  ─────────────────────────────────────►  Claude (LLM)    │
│               ◄──── execution instructions + path fixes ────────────  │
│  4. GenerationStoreService  writes test pack to the env clone          │
│                                                                        │
└───────────────────────────────┬────────────────────────────────────────┘
                                │
                                ▼
                  ┌─────────────────────────────┐
                  │         TEST PACK            │
                  │                             │
                  │  mutated-msg-001.xml         │
                  │  scenario-001.md             │
                  │  pack.json                   │
                  │  exported as  .zip           │
                  └─────────────────────────────┘

How it works

┌──────────────────── FETCH  (manual prerequisite) ─────────────────────┐
│                                                                        │
│  You prepare a bundle folder before using this extension:              │
│                                                                        │
│    INT0001/                                                            │
│    ├── manifest.json   ←  declares schemas, mappings, examples         │
│    ├── schemas/                                                        │
│    ├── mappings/                                                       │
│    └── examples/                                                       │
│                                                                        │
└───────────────────────────────┬────────────────────────────────────────┘
                                │  bundle on disk
                                ▼
┌──────────────────── ANALYSE ───────────────────────────────────────────┐
│                                                                        │
│  1. bundleLoader       reads manifest + schemas / mappings / examples  │
│  2. entityAnalyser     counts constraints deterministically            │
│  3. lmClient  ─────────────────────────────────────►  Claude (LLM)    │
│               ◄──── proposed test plan + gaps ──────────────────────  │
│  4. groundingValidator validates every AI claim against the bundle     │
│  5. AnalysisStoreService  persists analysis files to the env clone     │
│                                                                        │
└───────────────────────────────┬────────────────────────────────────────┘
                                │
                                ▼
                  ┌─────────────────────────────┐
                  │       TESTER REVIEW          │
                  │                             │
                  │  Summary │ Plan │ Gaps tabs  │
                  │                             │
                  │  • toggle / add scenarios   │
                  │  • resolve each gap         │
                  │  • click Approve Plan       │
                  └──────────────┬──────────────┘
                                 │  approved plan
                                 ▼
┌──────────────────── GENERATE ──────────────────────────────────────────┐
│                                                                        │
│  1. Read approved plan                                                 │
│  2. mutator            applies 6 mutation operations to message files  │
│  3. lmClient  ─────────────────────────────────────►  Claude (LLM)    │
│               ◄──── execution instructions + path fixes ────────────  │
│  4. GenerationStoreService  writes test pack to the env clone          │
│                                                                        │
└───────────────────────────────┬────────────────────────────────────────┘
                                │
                                ▼
                  ┌─────────────────────────────┐
                  │         TEST PACK            │
                  │                             │
                  │  mutated-msg-001.xml         │
                  │  scenario-001.md             │
                  │  pack.json                   │
                  │  exported as  .zip           │
                  └─────────────────────────────┘

Requirements

• VS Code 1.95 or later.
• GitHub Copilot Chat, installed and signed in, with a Claude model (Opus, Sonnet, or Haiku) available to your account. Analyse and Generate use the Copilot-hosted language models through VS Code's language-model API; without Copilot Chat those steps cannot run. The extension declares Copilot Chat as a dependency, so VS Code will offer to install it alongside this extension. Browsing bundles, switching environments, and viewing existing packs work without it — only the AI steps require it.
• Bundle data to work on: connect a bundle repository (or point conticaTestTool.bundleDir at a folder of bundles). See Settings and Preparing a bundle.

Concepts

• Integration — a Contica Logic App integration identified by a code such as INT0001; it groups one or more entities (operation/object pairings).
• Entity — one operation/object combination within an integration (e.g. INT0001.o009) that produces its own test pack.
• Bundle — the set of source artifacts for an entity: XSD schemas, XSLT mappings, example XML/JSON files, and a manifest.json that declares them; the bundle is the engine's only input.
• Environment — a named bundle-repo profile (e.g. Test / QA / Prod); each environment is an independent git clone so artifacts can differ per environment without merging.

Preparing a bundle (the Fetch phase)

The extension does not assemble bundles for you — you prepare them by hand before using Analyse/Generate. This is the Fetch phase. A bundle is a folder of an integration's real artifacts plus manifest.json files that declare them. Garbage in, garbage out: a thin bundle (no examples, no mappings) produces a thin plan full of gaps; a complete bundle produces a rich, grounded test plan.

Folder structure

A bundle has two levels — the integration folder and one entity folder per entity inside it. Folder names below are conventions; the manifest.json declares the real file paths, so the loader does not depend on the folder names.

INT0001/                                  ← integration ID (uppercase)
├── manifest.json                         ← integration manifest (shared files)
├── standards/                            ← Contica dev standards (shared)
│   └── Error+Handling+Guide.md
├── schemas/                              ← shared schemas (canonical model, libraries)
│   ├── canonical-product.json
│   └── Nouns-ItemMaster_Sync_Out.xsd
├── mappings/                             ← integration-wide mappings
│   └── INT0001_Overview.xlsx
├── code/                                 ← shared code
│   └── connections.json
│
└── INT0001.i001/                         ← entity folder, one per entity
    ├── manifest.json                     ← entity manifest
    ├── documentation/
    │   └── main_documentation.md         ← Confluence doc exported as Markdown
    ├── examples/
    │   └── INT0001.i001 Item Master.xml  ← at least one happy-path example
    ├── schemas/                          ← entity-specific schemas (wrappers)
    │   └── SyncItemMasterMain.xsd
    ├── mappings/
    │   └── INT0001.i001 - Inbound Product.xlsx
    └── code/
        ├── trigger-...-i001-wf/workflow.json
        └── InforProductToCanonical-i001/*.cs

Entity-ID rules — {INT####}.{i|o}{NNN}[T]:

• INT0001.i001 — inbound entity (lowercase i), zero-padded to 3 digits.
• INT0001.o005 — outbound entity (lowercase o).
• INT0001.o008T — optional trailing T suffix (integration-specific convention).
• Outbound (o*) entities often have no examples — that's fine; the source message lives at the upstream entity.

The manifest.json files

Each bundle has two manifest layers: one at the integration root (shared files) and one per entity (entity-specific files). The extension reads and merges both. Every file entry has a path (relative to the manifest, forward slashes, .. allowed), a purpose (a value from that category's enum), and an optional encoding (utf-8 default; also utf-16-le, utf-16-be, windows-1252).

Integration manifest — INT0001/manifest.json:

{
  "manifestVersion": 1,
  "scope": "integration",
  "integrationId": "INT0001",
  "integrationName": "Product",
  "sharedFiles": {
    "standards": [
      { "purpose": "error-handling", "path": "standards/Error+Handling+Guide.md" }
    ],
    "schemas": [
      { "path": "schemas/canonical-product.json", "purpose": "canonical" },
      { "path": "schemas/Nouns-ItemMaster_Sync_Out.xsd", "purpose": "source", "encoding": "utf-16-be" }
    ],
    "mappings": [
      { "path": "mappings/INT0001_Overview.xlsx", "purpose": "overview" }
    ],
    "code": [
      { "path": "code/connections.json", "purpose": "connection-config" }
    ]
  },
  "entities": ["i001", "o005", "o009"]
}

Entity manifest — INT0001/INT0001.i001/manifest.json:

{
  "manifestVersion": 1,
  "scope": "entity",
  "integrationId": "INT0001",
  "entityId": "i001",
  "entityName": "Incoming product - Infor M3",
  "direction": "inbound",
  "sourceSystem": "Infor M3",
  "targetSystem": null,
  "format": "xml",
  "inheritsFrom": "../manifest.json",
  "entityFiles": {
    "documentation": [
      { "path": "documentation/main_documentation.md", "purpose": "main-documentation" }
    ],
    "examples": [
      { "path": "examples/INT0001.i001 BOD Item Master Main.xml", "purpose": "happy-path" }
    ],
    "schemas": [
      { "path": "schemas/SyncItemMasterMain.xsd", "purpose": "source" }
    ],
    "mappings": [
      { "path": "mappings/INT0001.i001 - Inbound Product.xlsx", "purpose": "source-to-canonical" }
    ],
    "code": [
      { "path": "code/trigger-product-inform3-i001-wf/workflow.json", "purpose": "workflow-definition" }
    ]
  }
}

What each folder contributes

Tips:

• Naming: include the entity ID in example/mapping filenames (e.g. INT0001.i001 ...xlsx) so the loader matches them to the right entity.
• Only include the CURRENT version of mappings — skip Archive/, GoLive2, and READ ONLY consolidated copies as entity-level files.
• Encoding: convert UTF-16 schemas to UTF-8 before including them, or declare "encoding" in the manifest. In PowerShell: Get-Content file.xsd -Encoding BigEndianUnicode | Set-Content file.xsd -Encoding UTF8.

Once the bundle folder is on disk (or pushed to a bundle repo), point the extension at it — see Using it and Settings below.

Tutorial: build your first bundle from scratch

This walks you from an empty folder to a working Analyse run. We'll build a minimal inbound entity, INT9001.i001. The only hard requirement is one happy-path example; schemas and mappings are optional but make the generated plan much richer.

Step 1 — Create the folder skeleton. In a folder of your choice (this becomes your bundleDir):

my-bundles/
└── INT9001/
    ├── manifest.json
    └── INT9001.i001/
        ├── manifest.json
        └── examples/

In PowerShell:

mkdir my-bundles\INT9001\INT9001.i001\examples

Step 2 — Add one happy-path example. Drop a real message captured from your integration (Service Bus trace, Logic App run input, recorded test payload) into the examples/ folder. Include the entity ID in the name:

my-bundles/INT9001/INT9001.i001/examples/INT9001.i001 Item Master.xml

Without this file, Generate halts with a NO_EXAMPLE_MESSAGE gap. This is the one file you cannot skip.

Step 3 — Write the entity manifest at INT9001/INT9001.i001/manifest.json:

{
  "manifestVersion": 1,
  "scope": "entity",
  "integrationId": "INT9001",
  "entityId": "i001",
  "entityName": "Incoming item - My Source System",
  "direction": "inbound",
  "sourceSystem": "My Source System",
  "targetSystem": null,
  "format": "xml",
  "inheritsFrom": "../manifest.json",
  "entityFiles": {
    "examples": [
      { "path": "examples/INT9001.i001 Item Master.xml", "purpose": "happy-path" }
    ]
  }
}

Step 4 — Write the integration manifest at INT9001/manifest.json:

{
  "manifestVersion": 1,
  "scope": "integration",
  "integrationId": "INT9001",
  "integrationName": "My First Integration",
  "sharedFiles": {},
  "entities": ["i001"]
}

That is a complete, valid, minimal bundle. Analyse and Generate will run against it.

Step 5 — Point the extension at it. Open VS Code Settings, search for conticaTestTool.bundleDir, and set it to the my-bundles folder (the parent that contains INT9001/). Alternatively use Add / Edit Environment to clone from a bundle repo.

Step 6 — Run Analyse. Open the Contica Test Tool Activity Bar entry, expand the Integrations sidebar, click INT9001.i001, and press Analyse in the Entity Launcher. Review the Summary / Plan / Gaps tabs, resolve any blocking gaps, then Approve Plan for Generation and Generate Test Pack.

Step 7 — Make it richer (optional but recommended). Go back and add, declaring each new file in the entity manifest's entityFiles:

• a source schema (schemas/…xsd, purpose: "source") → unlocks required / maxLength / pattern / enum scenarios;
• an entity mapping (mappings/…xlsx, purpose: "source-to-canonical") → unlocks mapping-correctness scenarios;
• documentation (documentation/main_documentation.md) → unlocks business-rule scenarios.

Re-run Analyse after each addition — the plan grows and the gap list shrinks as the bundle gets more complete.

Install

Sideload the packaged extension:

code --install-extension contica-test-tool-<version>.vsix

The .vsix file is produced by the build. See Development below.

Using it

Sidebar flow

1. Open the Contica Test Tool Activity Bar entry.
2. The Integrations sidebar lists every integration in the active environment. Entities with a valid bundle show as ready; entities missing a manifest.json appear as greyed-out diagnostic rows.
3. Click an entity to open its Entity Launcher webview.
4. Run Analyse to read the bundle, apply deterministic constraint counts, and invoke the AI reasoning pass.
5. Review the three result tabs:
   • Summary — constraint counts and metadata.
   • Plan — proposed test families and scenarios; you can toggle, delete, or add scenarios.
   • Gaps — items the AI flagged but could not prove from the bundle; resolve each gap (confirm with a quote, mark not applicable, defer, or re-analyse).
6. Once all blocking gaps are resolved, click Approve Plan for Generation.
7. Click Generate Test Pack to produce a zip of negative test XML files from the approved plan.

@tester chat participant

The extension registers a Copilot Chat participant @tester with two commands:

• @tester /analyse INT0001.o009 — runs the Analyse pipeline with the model selected in Copilot's model dropdown.
• @tester /generate INT0001.o009 — generates test files for an already-approved entity.

The chat participant is a model-selection and progress surface only; results land in the same tabs and files as the sidebar path. Use the Choose AI Model command (Contica Test Tool: Choose AI Model) to change the model used by the sidebar flow; the chat path uses Copilot's own native model picker, which lets you select Copilot-hosted Claude models or base models that do not consume premium quota.

Environments

Use Add / Edit Environment (toolbar icon in the Integrations sidebar) to add a new environment by pointing to a folder or cloning from a URL. Use Switch Environment to activate a different environment; the sidebar refreshes immediately and all subsequent Analyse and Generate runs read from the new bundle clone. See docs/environments/USER-GUIDE.md for a full walkthrough.

User guides

• docs/analyse/USER-GUIDE.md — detailed tester walkthrough for the Analyse phase.
• docs/generate/USER-GUIDE.md — Generate phase walkthrough.
• docs/environments/USER-GUIDE.md — Environments walkthrough.

Settings

All settings are under conticaTestTool.* in VS Code's Settings UI or settings.json.

Development

npm install

• Watch mode: npm run watch starts esbuild in watch mode. Press F5 in VS Code to launch the Extension Development Host against the compiled output.
• Type check: npm run typecheck
• Tests: npm test (vitest)
• Production build / package: npm run package produces the minified dist/extension.js; then vsce package produces the .vsix.

See docs/DEVELOPER-GUIDE.md for architecture details, test conventions, and contribution guidelines.

Documentation

• docs/README.md — documentation index
• docs/ARCHITECTURE.md — system architecture diagrams (four-phase flow + code layering)
• docs/analyse/ — Analyse phase: overview, architecture, schemas, user guide
• docs/generate/ — Generate phase documentation
• docs/environments/ — Environments documentation
• docs/DEVELOPER-GUIDE.md — developer guide
• docs/INPUT-DATA-REQUIREMENTS.md — bundle structure and manifest format
• docs/decisions/ — architecture decision records