How to generate pytest tests from an OpenAPI spec

Your OpenAPI spec already says what every endpoint takes and returns. So in theory, writing the tests is data entry. In practice nobody does it, because typing out request/assert pairs for forty endpoints is miserable and there's always a ticket that matters more.

So people either skip API tests or write the version that looks like this:

def test_get_user():
    r = requests.get(f"{BASE}/users/1")
    assert r.status_code == 200

This test barely earns its keep. It stays green when the endpoint drops half the fields, returns id as a string, or hands back an empty object — as long as the status is 200. I've watched suites full of these sit at "all passing" while the API quietly broke its own contract.

What's actually worth asserting

Three things, roughly in order of how often they catch something:

• The response body matches the schema. Fields present, types right. This is the check that finds real regressions, and it's the one people skip because writing it by hand is annoying.
• Error paths return the documented code. No token, missing query param, a string where an int belongs. A lot of APIs answer these with a 500 and a stack trace instead of the 400/401/422 the spec promises.
• The status code. Necessary, nearly worthless on its own.

Doing it by hand

Say the spec describes GET /users/{id} returning a user with an integer id and a string email. Pull the schema straight out of the spec and validate against it with jsonschema:

import requests
from jsonschema import validate

BASE = "https://api.example.com"

USER = {
    "type": "object",
    "required": ["id", "email"],
    "properties": {
        "id": {"type": "integer"},
        "email": {"type": "string"},
    },
    "additionalProperties": False,   # catch fields the spec never promised
}

def test_get_user(token):
    r = requests.get(f"{BASE}/users/1", headers={"Authorization": f"Bearer {token}"})
    assert r.status_code == 200
    validate(r.json(), USER)

def test_get_user_without_auth():
    assert requests.get(f"{BASE}/users/1").status_code == 401

The additionalProperties: False line is worth calling out. Without it, an endpoint can leak an internal field into the response and your test won't care. With it, you find out. Whether you want that strictness depends on the API, but decide on purpose instead of by accident.

Now do that for every endpoint. That's the wall people hit around test three.

Generating it from the spec instead

Since the spec already carries the paths, params and schemas, you can emit the suite instead of typing it. If you build the generator yourself, budget for the parts that are fiddlier than they look:

• $ref resolution. Real specs point $ref at shared components, sometimes in other files, sometimes in a loop. You have to inline them and stop cleanly on circular references or the generator spins.
• Which negatives to emit. Missing required param, wrong type, no auth. Anything past that and you drown in low-value tests.
• 422 vs 400. FastAPI validates into 422, plenty of frameworks use 400. Read it off the spec, don't hardcode one.

That's what apitestgen does — you paste an OpenAPI, Swagger or Postman file and get back runnable pytest (and Jest, Playwright, Postman, Schemathesis), schema assertions and $ref handling included. The free tier runs on your own spec, so the honest way to judge it is to feed it something real and read the output before you trust it with anything.

Keep the URL and token out of the code

Read them from the environment so one suite runs against local, staging and prod without edits:

# conftest.py
import os, pytest

@pytest.fixture(scope="session")
def token():
    return os.environ["API_TOKEN"]

Then run pytest in CI on every push, with the token supplied as a per-environment secret. A contract break shows up as a red check on the PR instead of a 2am page.

None of this is clever. It's just the part of testing that's tedious enough to keep getting deferred, which is exactly why generating it is worth the ten minutes it takes to try.