# Catching breaking changes with apigenie diff

Source: https://apigenie.pl/blog/catching-breaking-changes-with-apigenie-diff

## Why I care about breaking changes

When you work API-first, the description of the API is the contract. Clients
are generated from it, mocks answer from it, and other teams plan their work
around it. That is the whole point, and it is also the risk: a small edit to
the description can break code you have never seen.

The edits that do the damage rarely look dangerous. Someone removes an
operation nobody seems to use. Someone makes a query parameter required, or
renames a Protobuf field, or tightens a GraphQL argument type. In review, these
are one-line changes in a long YAML or `.proto` file, and they are easy to
approve.

I wanted a tool that reads both versions of the description, tells me exactly
what changed, and says which of those changes will hurt consumers. That tool
is `apigenie diff`.

## What it compares

The command takes two files, the old one and the new one:

```sh
apigenie diff [OPTIONS] SOURCE_FILE DEST_FILE
```

Both files must be of the same kind: both OpenAPI 3.x documents, both Swagger
2.0 documents, both AsyncAPI 3.x documents, both Protobuf files or both
GraphQL files. The differ picks the format from the file itself: a `swagger`
field at the root means Swagger, an `asyncapi` field means AsyncAPI, a `.proto`
extension means Protobuf, and `.graphql`, `.graphqls` or `.gql` means GraphQL.

Each format is compared the way its consumers see it:

- **OpenAPI and Swagger.** Paths, operations, parameters, request bodies,
  responses, schemas, security and so on. The severity of a schema change
  depends on the direction of the data: a change in a request is judged by
  what it does to callers, a change in a response by what it does to readers.
  Read-only properties don't matter in requests.
- **AsyncAPI.** Servers, channels, operations, messages with their payloads
  and headers, security schemes, bindings and components. Here the direction
  is decided by who sends the message. A new required payload property breaks
  the senders of a message the application receives, while the receivers of a
  message the application sends can ignore it.
- **Protobuf.** Messages, enums, services and extend blocks, down to fields,
  enum values and methods. Fields and enum values are matched by their
  numbers, because the numbers identify them in the encoded data. So a renamed
  field is a modified field, and a field that got another number is a removed
  one plus an added one. Removing a field is non-breaking when the new file
  reserves its number, which is how Protobuf retires fields.
- **GraphQL.** The type system: root operation types, types of every kind with
  their fields, arguments and enum values, directive definitions and applied
  directives. Type extensions are merged into what they extend, so moving a
  field into an `extend type` block is not a change. Breaking changes follow
  what graphql-js's `findBreakingChanges()` decides.

The [diff reference](/docs/reference/cli/diff) describes the matching
rules of each format in detail.

## Severities

Every change gets one of three severities:

- **breaking**: existing consumers stop working, for example a removed
  operation or a field whose type changed incompatibly;
- **non-breaking**: the API changes, but existing consumers keep working, for
  example an added operation or an added optional parameter;
- **non-functional**: nothing changes for the programs involved, for example
  descriptions in GraphQL or comments in Protobuf.

The defaults come from the `recommended` rules of each format. They are
reasonable, but every team has its own contract with its consumers, so the
rules are configurable in the `commands.diff` block of the
[configuration file](/docs/reference/cli/configuration). A property can be
set to `breaking`, `non_breaking`, `non_functional`, `none` to ignore the
change, or `nested` to take the highest severity of the changes inside. Rules
can depend on conditions and can be overridden for specific locations. For
example, this lets deprecated operations be removed without failing the build:

```yaml
version: '1.0'
commands:
  diff:
    extends: [recommended]
    rules:
      pathItem:
        operations:
          remove_item:
            if: deprecated
            then: non_breaking
            else: breaking
```

The `swagger`, `asyncapi`, `protobuf` and `graphql` sections of the same block
configure the other formats.

## Reading the output

By default, `apigenie diff` prints the `changelog` report with the `stylish`
formatter. Here is the beginning of what it prints for two versions of the
Petstore example:

```text
$ apigenie diff petstore.source.yaml petstore.dest.yaml
petstore.dest.yaml
   1  1      breaking  Metadata changed
  98  7  non-breaking  Added operation PUT /pets/{petId} (updatePet)
  87  7      breaking  Removed operation DELETE /pets/{petId} (deletePet)
  25  7      breaking  Modified operation GET /pets (listPets)
  25  7      breaking    Added operation.parameters['query:status']
  ...
```

Each line has a position in the destination file, a severity and a
description. Changes inside a modified operation are indented under it. For
Protobuf and GraphQL, what has been removed is reported at the closest
enclosing declaration that is still there, as in this Protobuf example from
the reference:

```text
shop.proto
  19  1      breaking  Modified message 'Customer'
  19  1  non-breaking    Removed field 'fax'
  25  3      breaking    Modified field 'full_name'
  25  3      breaking      Modified field name
```

Two options control the output. `--report` chooses what to print: `status`,
`changelog` (the default) or `detailed`. `--formatter` chooses how: `html`,
`json`, `markdown`, `stylish` (the default), `unix` or `visualstudio`. A
Markdown changelog is handy for release notes, JSON for further processing:

```sh
apigenie diff --report detailed --formatter markdown v1/openapi.yaml v2/openapi.yaml
```

## Failing CI on breaking changes

The exit status is what makes the command useful in a pipeline. It exits with
status 1 when there are breaking changes, and 0 otherwise. Any CI system can
use that directly:

```sh
git show main:api/openapi.yaml > /tmp/openapi.base.yaml
apigenie diff --config apigenie.yaml /tmp/openapi.base.yaml api/openapi.yaml
```

The exit status considers all changes, whatever the report shows, so choosing
a shorter report doesn't hide a breaking change.

There is also `--update-version`. Instead of printing a report, it updates
`info.version` of the destination file to the smallest version compatible with
the changes: a new major version for breaking changes, minor for non-breaking
and patch for non-functional ones. It keeps the rest of the file intact,
prints nothing and exits with 0. Protobuf and GraphQL files have no version,
so the option is rejected for them.

## In the GitHub Action

If your API lives on GitHub, the
[API Genie GitHub Action](/docs/ci/github-action) runs the
differ for you:

```yaml
on: pull_request
jobs:
  api:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: mykulyak/api-genie-action@v0
        with:
          command: diff
          files: api/openapi.yaml
```

With `command: diff`, each file is compared with its version at the base
commit of the pull request, or the commit before a push, and the step fails on
breaking changes. Set `base` to compare with another commit, branch or tag, or
`base-file` to compare two files of the repository.

The changes show up as annotations on the files of the pull request, so the
reviewer sees "breaking" next to the line that caused it, and a report is added
to the job summary. Annotations come from the default `unix` formatter, so
leave `formatter` alone if you want them. The `report` input picks the report,
as `--report` does on the command line. On pull requests, the action can also
post the report as a comment and update it in place on later runs; that needs
the `pull-requests: write` permission.

If you would rather decide yourself what a breaking change means for the
build, set `fail-on: never` and read the `status` output, which is `none`,
`non-breaking` or `breaking`.

## The history view: `apigenie changelog`

`apigenie diff` compares two files you give it. Often the question is
different: what happened to this API over the last releases? The
[changelog command](/docs/reference/cli/changelog) answers it from git
history. It compares the consecutive committed versions of an OpenAPI, Swagger,
AsyncAPI, Protobuf or GraphQL file and rates each change with the same rules
as `diff`, including your `commands.diff` configuration.

```sh
apigenie changelog api/openapi.yaml --revisions tags --tag-pattern 'v*' --unreleased --formatter markdown
```

This prints one entry per release tag, plus the changes made since the last
release, in Markdown that can be published as is. With `--by operation`, the
changes are grouped by operation instead, so you can see the history of a
single endpoint. Unlike `diff`, `changelog` exits with 0 even when there are
breaking changes: it describes history, it doesn't gate it.

## Where to start

Take the last released version of your API description and the current one,
and run `apigenie diff` on them. If the output surprises you, that is exactly
the change that would have surprised your consumers. Then put the same check
in CI, so the next one is caught in review instead of in production.
