The traffic you already have is the contract test
Andriy Mykulyak · · 8 min read
A mock proves the frontend works against the contract. It proves nothing about the backend. The backend has integration tests, and they send real requests to the real service, but nobody checks what comes back against the document, so the document and the service drift apart one field at a time, and the first consumer to notice is a customer.
The binary that serves the mock, apigenie-server, can sit in front of the real backend instead,
check each request and response against the document, and tell you where the implementation and
the contract disagree. There is no test to write: the traffic you already send is the test. This
post runs it on Carpet Express with 0.10.0. The backend here is a second apigenie-server serving
the same document as a mock, so you can run it with nothing else installed; the part that matters
is the proxy in between.
The validating proxy
$ apigenie-server carpet-express.openapi.yaml --port 8080 & # stands in for the backend
$ apigenie-server carpet-express.openapi.yaml --proxy-to http://127.0.0.1:8080 \
--record sessions/checkout --report contract.json --report-junit contract.xml --fail-on errors
18:58:41.048 server recording the session checkout in sessions/checkout
Listening on http://127.0.0.1:4010/v2 (spec: carpet-express.openapi.yaml, 38 operations)
Proxy: http://127.0.0.1:8080 (requests: log, responses: log, unknown routes: forward, mode: always)
Contract report: contract.json, contract.xml
Recording the session to sessions/checkout
Admin API: http://127.0.0.1:4010/__apigenie
Dashboard: http://127.0.0.1:4010/__apigenie/ui
Point whatever sends requests at port 4010: your integration tests, the frontend in development, a load test, a QA session. Each exchange goes to the real server unchanged and comes back checked. A response with a status the document doesn’t declare, a body that doesn’t match its schema, a missing required header, a media type nobody promised: each is a violation with a check name, a severity and the location. The Contract screen of the dashboard shows them live, with coverage: which operations, request bodies and declared responses the traffic has exercised and which it never touched.
When the server stops, the report is written, and the exit status is 5 if there were errors or if
--min-coverage 80 wasn’t reached. The JUnit file means your CI shows each violation as a failed
test with no plugin.
By default the proxy reports and forwards, because test suites send bad requests on purpose.
--strict makes it enforce: requests that don’t match the document are answered 422 without
reaching the upstream, and upstream responses that don’t match are replaced by a 502, so a
contract violation can’t hide behind a lenient server.
Sending the requests yourself
When there is no traffic yet, apigenie test sends it. Requests live in .http files, the format
the JetBrains HTTP client and VS Code’s REST Client read, so they double as documentation. The two
requests of tests/shipments.http:
@host = http://127.0.0.1:4010/v2
@token = sandbox-token
### Quote rates for a parcel from Kraków to Berlin
# @name quote
POST {{host}}/rates
Content-Type: application/json
Authorization: Bearer {{token}}
{
"origin": {"line1": "ul. Długa 1", "postalCode": "30-001", "city": "Kraków", "countryCode": "PL"},
"destination": {"line1": "Invalidenstraße 117", "postalCode": "10115", "city": "Berlin", "countryCode": "DE"},
"parcels": [{"weight": {"value": 2.5, "unit": "kg"}}]
}
?? status == 200
### Book a parcel shipment
POST {{host}}/shipments
Content-Type: application/json
Authorization: Bearer {{token}}
Idempotency-Key: {{$guid}}
{ "mode": "parcel", "service": "dhl.paket", "reference": "ORD-1001", ... }
?? status == 201
The first time I ran it, the two Authorization lines were missing. The backend answered 401,
and the proxy reported two things at once: the requests broke the security requirement, and the
document declares no 401 for POST /rates:
$ apigenie test --spec carpet-express.openapi.yaml tests/shipments.http --header "X-Apigenie-Test: checkout"
FAIL POST http://127.0.0.1:4010/v2/rates -> 401 (5 ms) quoteRates
status == 200: found 401
error security: a bearer token (oauth2) are missing (unauthorized; request.authorization.missing)
FAIL POST http://127.0.0.1:4010/v2/shipments -> 401 (8 ms) createShipment
status == 201: found 401
error security: a bearer token (oauth2) are missing (unauthorized; request.authorization.missing)
Requests: 0 passed, 2 failed, 0 skipped
Contract: 2 requests (0 undocumented), 2 errors, 1 warnings
Coverage: 2.3% of 133 branches, 2 of 38 operations
error POST /rates: security: a bearer token (oauth2) are missing (unauthorized; request.authorization.missing)
warn POST /rates: response.status: POST /rates declares no response for status 401; it has 200, 422 (unknown-status; response.status.unknown)
error POST /shipments: security: a bearer token (oauth2) are missing (unauthorized; request.authorization.missing)
Test run failed: 2 failed requests; 2 error violations
$ echo $?
5
The image at the top of this post is the Contract screen after that run. The warning is the kind of finding this setup exists for: the service answers 401, as it should, and the document never said so. With the token, both requests pass, and the proxy’s own report says the same:
PASS POST http://127.0.0.1:4010/v2/rates -> 200 (7 ms) quoteRates
PASS POST http://127.0.0.1:4010/v2/shipments -> 201 (27 ms) createShipment
Requests: 2 passed, 0 failed, 0 skipped
Contract: 2 requests (0 undocumented), 0 errors, 0 warnings
Coverage: 4.5% of 133 branches, 2 of 38 operations
not tested: GET /shipments (listShipments), POST /shipments/search (searchShipmentsPost), QUERY /shipments/search (searchShipments), POST /shipments/imports (importShipments), POST /shipments/cancellations (cancelShipments), GET /shipments/{shipmentId} (getShipment), DELETE /shipments/{shipmentId} (cancelShipment), PATCH /shipments/{shipmentId} (updateShipment), GET /shipments/{shipmentId}/label (getShipmentLabel), GET /labels/{labelId}.{extension} (downloadLabelFile) and 26 more
Each request is bound to its operation and checked with the same checks as the proxy, so the
assertions you write are about behaviour and the contract checks come for free. --generate sends
a request per operation from the document’s examples when you have nothing yet, and test import
converts a Postman collection.
Beyond HTTP
This is where most contract-testing tools stop, and where Carpet Express gets interesting: its
events are an AsyncAPI document with Kafka, WebSocket, MQTT, HTTP and AMQP channels. A KAFKA
region of an .http file produces a record and reads what arrives on the channel’s topic; an
AMQP region publishes to an exchange and reads a queue; a WS region speaks WebSocket; an
MQTT region speaks MQTT 5, with its properties checked against the document’s MQTT bindings; a
GRPC region calls a method, client and bidirectional streams included; a GRAPHQL subscription
runs over graphql-ws. Each message is checked against the channel’s message schema, headers and
bindings, and counts in the coverage.
The proxy does the same for traffic you don’t send. --proxy-to kafka://localhost:9092 turns the
server into a validating tap: it reads the topics of the document’s Kafka channels, outside any
consumer group, and checks each record whoever produced it. --proxy-to amqp:// and
--proxy-to mqtt:// do the same for those brokers, and a .proto file makes it a gRPC proxy.
This post has no broker running, so that part is the reference’s to show, not mine.
Record once, replay each run
Integration tests against real backends are slow and flaky. Tests against hand-written stubs are
fast and drift. Session recording is the third option: real answers, replayed. The proxy above ran
with --record sessions/checkout, and when it stopped it wrote the session:
18:58:43.153 server recorded 2 exchanges of the session checkout in sessions/checkout/session.yaml
$ find sessions -type f | sort
sessions/checkout/0001-quoteRates.yaml
sessions/checkout/0002-createShipment.yaml
sessions/checkout/session.yaml
Each exchange is tagged with the test that sent it, read from the X-Apigenie-Test header, and
the secrets are gone from the files:
session:
sequence: 1
id: 2
test: checkout
kind: http
time: 2026-10-29T18:58:43.101Z
durationMs: 5
source: proxy
summary: POST /v2/rates 200
stubs:
- name: quoteRates-1
operationId: quoteRates
request:
method: POST
path: /v2/rates
headers:
Content-Type: application/json
Authorization: '[redacted]'
Then, each run, without the backend:
$ apigenie-server carpet-express.openapi.yaml --replay sessions/checkout --replay-ignore Idempotency-Key
Listening on http://127.0.0.1:4010/v2 (spec: carpet-express.openapi.yaml, 38 operations)
Replaying the session in sessions/checkout (2 exchanges, missing: fail)
$ apigenie test --spec carpet-express.openapi.yaml tests/shipments.http --header "X-Apigenie-Test: checkout"
PASS POST http://127.0.0.1:4010/v2/rates -> 200 (0 ms) quoteRates
PASS POST http://127.0.0.1:4010/v2/shipments -> 201 (1 ms) createShipment
Requests: 2 passed, 0 failed, 0 skipped
Each request gets the next unused recorded exchange of its test that matches it: method and path,
query in any order, the headers the recording kept, and the body as JSON regardless of key order,
with dates, readOnly fields and whatever --replay-ignore names left out. The option is there
because the first replay failed: Idempotency-Key is a new GUID on each run, so the booking
matched nothing. A request that matches nothing gets a 502 that names the closest recording and
how the request differs:
$ curl -H "X-Apigenie-Test: checkout" http://127.0.0.1:4010/v2/shipments/shp_8Yb2Kk1qP0wZcT3uXr5sN7aD
{
"type": "https://apigenie.pl/mock/errors#replay-missing",
"title": "No recorded exchange matches",
"status": 502,
"detail": "no recorded exchange of test 'checkout' matches GET /v2/shipments/shp_8Yb2Kk1qP0wZcT3uXr5sN7aD; the closest, 0001-quoteRates.yaml (replayed already), differs at method, path, header Content-Type and 1 more",
And when the server stops, the run exits 5 if an exchange was never replayed, with the list. The suite can’t drift silently. Tests replay side by side with their own counters, so parallel workers don’t steal each other’s exchanges; a Playwright suite needs three lines:
test.beforeEach(async ({ page }, testInfo) => {
const name = testInfo.titlePath.join(" > ");
await page.setExtraHTTPHeaders({ "X-Apigenie-Test": name });
await fetch("http://127.0.0.1:4010/__apigenie/replay/reset", {
method: "POST", body: JSON.stringify({ test: name }),
});
});
When the backend changes, --replay-missing passthrough --replay-update re-records what is new
and keeps the rest. The dashboard has Record and Save session buttons for the same thing, and
session export --har turns a session into a HAR file for anything else that reads one.
What it does not do
The contract is the document, not a consumer’s expectations: a team that works Pact-style, where
consumers write the contract and providers verify it, keeps Pact. The proxy checks what passes
through it, so an operation no traffic reaches is listed as never called, not tested; that is what
--min-coverage is for. And a recorded session is as good as the backend was on the day: the
rates in 0001-quoteRates.yaml are what a mock made up, which is why the first thing to record
is a real one.
Try it
curl -sSL https://api-genie.dev/examples/carpet-express.tar.gz | tar xz
cd carpet-express
apigenie-server carpet-express.openapi.yaml --port 8080 &
apigenie-server carpet-express.openapi.yaml --proxy-to http://127.0.0.1:8080 --record sessions/demo &
apigenie test --spec carpet-express.openapi.yaml tests/shipments.http --header "X-Apigenie-Test: demo"
Then open the dashboard and delete one Authorization
line. Next week:
Lint every API format with the rules you already have.