Agentic testing

Overview

Plain-English or auto-generated API tests, compiled once into deterministic artifacts, replayed without a model, and self-healed when your API changes.

Most agentic testing tools either drive a browser with a model at run time (expensive, nondeterministic) or work from a spec with no view of the code behind it. Goosy already clones and parses your repository for security scanning, so its testing module grounds tests in the same source — specs are extracted or generated from your code and versioned by commit SHA, and when a test fails, Goosy can tell “the API intentionally changed” apart from “the API broke” using a real diff between spec versions, not a guess.

The pipeline

Stage	What happens	Uses a model?
Spec	An OpenAPI spec is found in the repo, or generated from route/handler code if none exists.	Only if generating
Compile	A plain-English intent (or a spec operation, for smoke tests) becomes a versioned, validated test artifact.	Yes — draft + verify
Run	The compiled artifact is replayed against your environment with a plain HTTP client.	No
Repair	A failure is classified as drift, regression, flake, or inconclusive, and drift is auto-repaired.	Yes, only on failure

The run stage never calls a model — a compiled artifact is a fixed sequence of HTTP requests, extractions, and assertions, so the same artifact against the same environment behaves the same way every time. Models only run at authoring time (compile) and when something actually fails (repair), which is what keeps a passing suite's cost at zero tokens.

The two suite types

Smoke suites — one test per non-destructive operation in your spec, generated automatically with zero authoring. Regenerating a smoke suite after a spec change is idempotent: it updates in place rather than duplicating tests.
Custom suites — tests you write as a plain-English intent, e.g. “create an order, then fetch it, and expect the totals to match.” Compiled the same way, but authored by you.

Destructive operations are opt-in

A `DELETE`, `POST`, `PUT`, or `PATCH` is never run automatically. It has to be explicitly opted in for the target environment, or the environment has to be flagged disposable — and this is checked twice: once when a smoke suite is generated, and again at the moment a test actually runs, so a change in opt-ins between compile and run time still gets caught. See Environments, specs & suites.

← Deep ScanEnvironments, specs & suites →