# Linter performance

Source: https://api-genie.dev/docs/reference/cli/lint-performance
Part of the reference of [lint](https://api-genie.dev/docs/reference/cli/lint.md).

This page gives how long `apigenie lint`, Spectral and vacuum take to check three OpenAPI documents
with four rulesets, and their memory use. The measurements were made on 2026-11-08 with a Python
script. A kit, a Node.js script with the same rulesets and command lines, now replaces it. The kit
is in API Genie's source repository, which is not public:
[Time the linters on your documents](#time-the-linters-on-your-documents) gives the command lines
it runs.

## What was measured

The three linters, at these versions:

| Linter | Version | Runs as |
| --- | --- | --- |
| `apigenie lint` | the development version after 0.12.0 (it reports 0.12.0), built from source at commit f854324b9 with the `macos-release` preset | a native arm64 binary |
| [Spectral](https://github.com/stoplightio/spectral) | 6.17.1 | `spectral` on Node.js 24.11.1 |
| [vacuum](https://github.com/daveshanley/vacuum) | 0.39.3 | the native arm64 binary of its [release](https://github.com/daveshanley/vacuum/releases/tag/v0.39.3) |

vacuum's npm package starts the binary from a Node.js script, which would add the start of
Node.js to each run.

The machine: a MacBook Pro with an Apple M1 (8 cores) and 16 GiB of memory, on macOS 27.0.1, on AC
power.

The documents:

| Document | Source | Size |
| --- | --- | ---: |
| small | [`petstore-with-fake-endpoints-models-for-testing.yaml`](https://github.com/OpenAPITools/openapi-generator/blob/10e6c21c97abb098cc67f14dfdd02171ce56b115/modules/openapi-generator/src/test/resources/3_0/spring/petstore-with-fake-endpoints-models-for-testing.yaml), a test document of OpenAPI Generator | 52 KiB |
| medium | the Spotify Web API, from [APIs.guru](https://github.com/APIs-guru/openapi-directory/tree/main/APIs/spotify.com/1.0.0) | 282 KiB |
| large | the GitHub REST API: [`api.github.com.yaml`](https://github.com/github/rest-api-description/blob/4dcc32eb422d590253c1d29ce469b84d6d776def/descriptions/api.github.com/api.github.com.yaml) of 2023-12-22, from GitHub's rest-api-description | 8.2 MiB |

The measured copy of the large document has six more lines: an `x-apigenie` extension that names
two properties, `+1` and `-1`, for generated code.

The rulesets:

- **default**: each linter's default rules: `spectral:oas`, vacuum's recommended ruleset, and the
  `recommended` ruleset of `apigenie lint`. Spectral runs no rules without a ruleset file, so the
  kit passes it one that extends `spectral:oas`. They are different rules, so this row shows each
  linter with its own default, not the same work.
- **builtin**: 26 rules of `spectral:oas` that the three linters implement natively.
  `apigenie lint` runs them as 24 rules under its own names, as
  [Migrate from Spectral](/docs/guides/migrate-from-spectral) maps them.
- **declarative**: 10 custom rules written with the core functions `truthy`, `falsy`, `defined`,
  `casing`, `pattern`, `length` and `enumeration`, including `$..` paths that walk the whole
  document. The three linters read the same ruleset file.
- **javascript**: the same 10 rules, with each check written as a custom JavaScript function.
  Spectral and `apigenie lint` load the same ES modules; vacuum loads the same functions written
  for its function API.

Each case ran 7 times after 1 untimed run; the large document ran 3 times, with no untimed run.
The wall time table gives the median.

## Wall time

Median wall time in seconds:

| Document | Ruleset | Spectral | vacuum | `apigenie lint` |
| --- | --- | ---: | ---: | ---: |
| small | default | 0.688 | 0.053 | 0.020 |
| small | builtin | 0.727 | 0.050 | 0.023 |
| small | declarative | 0.635 | 0.051 | 0.021 |
| small | javascript | 0.670 | 0.098 | 0.030 |
| medium | default | 1.148 | 0.115 | 0.026 |
| medium | builtin | 0.795 | 0.118 | 0.028 |
| medium | declarative | 0.730 | 0.101 | 0.059 |
| medium | javascript | 0.728 | 0.285 | 0.158 |
| large | default | 23.951 | 3.562 | 0.244 |
| large | builtin | 3.897 | 1.993 | 0.149 |
| large | declarative | 3.479 | 4.457 | 1.123 |
| large | javascript | 3.820 | 5.203 | 2.234 |

In the large builtin row, `apigenie lint` checks 243 operations that Spectral and vacuum skip:
[Issues reported](#issues-reported) says which.

Spectral's median on the small document is 0.635 s or more with each ruleset. The start of
Node.js and the loading of the ruleset were not timed apart.

With its default ruleset, Spectral takes 23.951 s on the large document, against 3.897 s with the
builtin rules. Among the rules the default ruleset adds are `oas3-valid-media-example` and
`oas3-valid-schema-example`, which validate the examples. That run was not profiled.

`apigenie lint` takes longer with custom rules than with its own. On the large document, it takes
1.123 s with the declarative rules and 2.234 s with the JavaScript functions, against 0.149 s with
the builtin rules. Spectral takes about the same time with the declarative rules and with the
JavaScript functions.

vacuum takes 2 to 3 times longer with the JavaScript functions than with the declarative rules on
the small and medium documents.
It runs a function's script once for each node the rule selects
([`js_plugin.go`](https://github.com/daveshanley/vacuum/blob/v0.39.3/plugin/javascript/js_plugin.go)).
Its batch mode (`batch: true` in `functionOptions`) runs it once for all nodes; the kit doesn't use
it.

## Memory and CPU

Peak memory on the large document, over the four rulesets:

| Linter | Peak memory |
| --- | ---: |
| `apigenie lint` | 87 to 115 MiB |
| Spectral | 570 to 1,860 MiB |
| vacuum | 1.3 to 2.2 GiB |

Spectral and `apigenie lint` use about one core. vacuum runs on several. On the large document,
its CPU time is 2 to 2.7 times its wall time, up to 13.3 s of CPU with the JavaScript functions.

## Issues reported

The issue counts show whether the linters did comparable work. With the builtin rules, the totals
of Spectral, vacuum and `apigenie lint` are close on the small (87, 81, 96) and medium (282, 304,
273) documents. On the large one, they are not:

| Builtin rules, large document | Spectral | vacuum | `apigenie lint` |
| --- | ---: | ---: | ---: |
| Issues | 100 | 556 | 2,064 |
| Of them, in the 243 webhooks under `x-webhooks` | 0 | 0 | 1,955 |
| Of them, components only the webhooks use, reported as unused | 0 | 261 | 0 |
| Of them, request bodies without a description | 0 | 241 | 0 |
| Of them, enums with a value not of the schema's type | 0 | 0 | 4 |

The GitHub REST API describes its webhooks under `x-webhooks`, an extension of OpenAPI 3.0.
`apigenie lint` checks the webhooks as operations. It reports 1,701 header parameters without a
description, 241 webhooks without tags and 13 without a description. Spectral and vacuum don't
check the webhooks. vacuum reports the 249 schemas and 12 examples that only the webhooks use as
unused. vacuum's `operation-description` also reports 241 request bodies without a description.
`schema-enum-types` reports 4 enums that `typed-enum` passes in Spectral and vacuum. They list
strings in a schema without `type`. Before 0.13.0, it also reported 84 enums that list `null` in a
schema with `nullable: true`, which OpenAPI 3.0 allows.

With the custom rules, the totals of Spectral, vacuum and `apigenie lint` on the small document
are 63, 60 and 61 (declarative) and 63, 48 and 61 (javascript). Spectral's 63 include one parser
error, on a response code written as a number. vacuum's `casing` accepts the property `_class`,
which the other two report. With the JavaScript functions, vacuum reports the
casing of property names once per `properties` object: 23 issues, against 37 in Spectral and 36 in `apigenie lint`. The kit's
function for vacuum reports at the `properties` object, with a message that does not name the
property, and vacuum reports such results once.

## Notes on the runs

- Spectral 6.17.1 stops on the large document with the path `$..[?(@.properties)]`, at a `null`
  in the GitHub document: `Cannot read properties of null (reading 'properties')`. The kit uses
  `$..[?(@ && @.properties)]`, and the large declarative and javascript rows come from a second
  run with that path. In the first run, with the declarative rules, vacuum took 4.575 s and
  `apigenie lint` 1.150 s. With the JavaScript functions, they took 4.988 s and 2.263 s.
- In the second run, one of vacuum's 3 runs of the large document with the JavaScript functions
  exited with status 2. Its cause wasn't recorded. Its time is one of the 3 samples of the median.
- VS Code and a Safari tab each used about one core during the runs. The 1-minute load average was
  about 6 at the start. The runs of a case are close together, but the shortest times can carry a
  few milliseconds of noise.
- The counts per rule under [Issues reported](#issues-reported), and the custom-rule totals on the
  small document, come from a rerun with the kit on 2026-11-09, except the `apigenie lint`
  counts of the large document with the builtin rules (next note). It ran in a Linux cloud
  container with the npm packages of Spectral, vacuum and API Genie 0.12.0. It ran each small and
  medium case 2 times, and the large document with the builtin rules once. Its builtin totals
  match those of the timed runs on the Mac on the three documents.
- The `apigenie lint` counts of the large document with the builtin rules come from a run on the
  Mac on 2026-11-09. It used a build at commit 063c2ff96, with the `schema-enum-types` fix that
  0.13.0 brings. Its counts are 84 lower than the 2,148 of the timed runs. The small and medium
  counts didn't change. The times weren't measured again.
- A run on Linux in a cloud container on 2026-11-08 agreed on the small and medium documents. Its
  large runs of Spectral and vacuum failed, and it is not used here.

## Time the linters on your documents

These are the command lines the kit runs. With the three documents above, the first group runs
the default rows again. The other rows need the kit's builtin, declarative and javascript
rulesets, which this page doesn't give. Use the same command lines to time the three linters on a
document of yours.

The first group runs each linter's default rules. Spectral runs no rules without a ruleset file,
so `default.yaml` holds the one line `extends: spectral:oas`. vacuum runs its recommended ruleset,
and `apigenie lint` its `recommended` ruleset:

```sh
spectral lint -q -f json -o spectral.json -r default.yaml openapi.yaml
vacuum lint -b -q --no-update-check --format json -o vacuum.json openapi.yaml
apigenie lint --formatter json --no-config openapi.yaml > apigenie.json
```

The second group runs a Spectral ruleset of yours, `ruleset.yaml`:

```sh
spectral lint -q -f json -o spectral.json -r ruleset.yaml openapi.yaml
vacuum lint -b -q --no-update-check --format json -o vacuum.json -r ruleset.yaml openapi.yaml
apigenie lint --formatter json --no-config --ruleset ruleset.yaml openapi.yaml > apigenie.json
```

If the ruleset calls JavaScript functions, Spectral and `apigenie lint` load them from its
`functionsDir`. vacuum doesn't load Spectral's functions: the kit gives it the same checks written
for vacuum's function API, with `-f` and their directory. For the builtin rows, the kit passes
`apigenie lint` a configuration file with `--config`. The file turns the same rules on under their
API Genie names.

The kit takes the wall time of each run itself. On macOS, and on Linux with GNU `time` installed,
it starts the linter with `/usr/bin/time` for the CPU time and the peak memory. The option is `-l`
on macOS and `-v` on Linux. Put the same prefix in front of a command line to read them for a run
of yours:

```sh
/usr/bin/time -v apigenie lint --formatter json --no-config openapi.yaml > apigenie.json
```

On the large document, in a 4-core Linux cloud container with API Genie 0.12.0, it prints the
report below. The tables come from the Mac, not from this run:

```text
Command exited with non-zero status 1
	Command being timed: "apigenie lint --formatter json --no-config openapi.yaml"
	User time (seconds): 0.70
	System time (seconds): 0.13
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.84
	...
	Maximum resident set size (kbytes): 99792
	...
	Exit status: 1
```

GNU `time` prints its report to standard error. `User time` and `System time` add up to the CPU
time. `Maximum resident set size` is the peak memory, in kbytes; macOS `time -l` gives it in bytes.
Each linter exits with 1 when it reports an error. GNU `time` then starts its report with
`Command exited with non-zero status 1`.

`apigenie` from Homebrew or a release archive is the native executable. The npm package installs
a Node.js launcher instead. Time the executable of its platform package:
`@mykulyak/api-genie-<platform>/bin/apigenie` (`apigenie.exe` on Windows) in the `node_modules` it
was installed to, where `<platform>` is `darwin-arm64`, `linux-x64` or `win32-x64`. Time vacuum's
release binary too, not its npm package, which starts the binary from a Node.js script.
