apigenie-generator server go (AsyncAPI)

Tier 2 · Release 0.12.0: Level 1 ✓ · Level 2 ✓ · Level 3 ✓ · Verification

Generate a Go Kafka application of the Kafka channels of an AsyncAPI document.

Usage

apigenie-generator server go [...OPTIONS] INPUT_FILE

Description

apigenie-generator server go generates from OpenAPI documents (see apigenie-generator server go) and from AsyncAPI 3.x documents, telling them apart by their openapi or asyncapi field. Documents referencing other files are bundled first.

From an AsyncAPI document it generates a Go module (Go 1.22) using the Kafka client franz-go (github.com/twmb/franz-go/pkg/kgo, pinned at v1.18.1 in go.mod, with a complete go.sum). By default the application is the one the document describes: it sends the messages of send operations and receives the messages of receive operations. Set the perspective option to counterpart to swap them.

The generator writes the Kafka channels: the channels of servers with the kafka or kafka-secure protocol, and channels with a Kafka binding in documents without servers. Channels of other protocols are skipped and reported as info.

type labelFarm struct{}

func (labelFarm) OnRenderLabel(ctx context.Context, message events.RenderLabel) (events.RequestLabelRenderReply, error) {
	var reply events.LabelRendered
	reply.Key = message.Key
	reply.Payload = render(message.Payload)
	return reply, nil // sent to TopicLabelRenderResults with the request's correlation id
}

func (labelFarm) OnUnknown(ctx context.Context, record *kgo.Record) error { return nil }

func main() {
	var handlers events.Handlers
	handlers.RequestLabelRender = labelFarm{}
	// ...
	app, err := events.NewApp(events.Options{Brokers: []string{"localhost:9092"}}, handlers)
	if err != nil {
		log.Fatal(err)
	}
	defer app.Close()
	err = app.PublishShipmentEvents().SendShipmentDelivered(ctx, delivered)
	// ...
	_ = app.Run(ctx) // consumes until ctx is done
}

The module has one package at its root, named by package_name, --package-name or the document’s title (letters and digits, lower case):

File Contents
types.gen.go the types of the schemas the payloads use: structs with json tags, enums, and unions as a struct whose Value holds one variant of a sealed interface (switch v := event.Value.(type)); each with a Validate method
messages.gen.go a struct per message (Key, Headers, Payload), its headers struct and header constants, and the code writing, reading and validating records
kafka.gen.go the server hosts, topics, group and client ids, the handler interfaces, Handlers, App and the senders
avro.gen.go the structs of the Avro records of the payloads, with their schemas and codecs
security.gen.go the security schemes of the servers and the franz-go options of each
runtime_*.gen.go the Kafka runtime (consumers, requesters, headers), validation, Nullable[T], Date
go.mod, go.sum the module, requiring franz-go
cmd/<package>/main.go a starting point, written once and never overwritten
  • Messages. Headers are the message’s headers merged with those of its traits as AsyncAPI does (JSON Merge Patch: a later required list replaces an earlier one). Header values are text on Kafka, so headers are string, int64, float64 or bool fields, pointers when optional. Headers with a constant value, e.g. the ce_specversion and ce_type of CloudEvents in binary mode, are constants (ShipmentStatusChangedCeType), which senders write and receivers compare. Messages are validated as a whole before they are sent and after they are received.
  • Received operations. NewApp(options, handlers) takes a handler per received operation (PublishShipmentEventsHandler) with an On<Message> method per message and OnUnknown. Several messages of a channel are told apart by a payload property or a header with a distinct constant value (e.g. type or ce_type); otherwise the first message whose payload decodes strictly and is valid is chosen. Records matching no message go to OnUnknown. Run consumes the operations’ topics, each in its consumer group, until its context is done; Close commits the offsets of the handled records and leaves the groups. Records that aren’t valid messages, and handler errors, go to Options.OnError (logged by default); the records are committed either way.
  • Sent operations. app.PublishShipmentEvents() returns the sender of the operation, with a Send<Message>(ctx, message) method per message that validates the message and returns once the brokers have it.
  • Request/reply with a reply channel: the requester’s Send<Message> returns the reply (one of the reply messages, RequestLabelRenderReply), matched by the request’s correlation id, which is generated when the request has none; Options.ReplyTimeout bounds the wait when the context has no deadline. A replier’s handler returns the reply, which is sent to the reply topic with the request’s correlation id copied (to a header, or a payload property, as the correlationId locations say).

Group and client ids come from the groupId and clientId schemas of the Kafka operation bindings, of the operation or its traits: their const, single enum value, default or first example. Without a value, <package name in kebab case>-<operation> is used, and the doc comment of the constant names the binding’s pattern. Options.GroupIDs replaces them by operation id. The partitions, replicas and topic configuration of channel bindings are documented on the topic constants; the generated code creates no topics.

main.go connects to the brokers of the -brokers flag, else of the KAFKA_BROKERS environment variable (comma-separated), else localhost:9092, wires stub handlers that return api.ErrNotImplemented, and stops cleanly on SIGINT or SIGTERM. With the -security flag, else the KAFKA_SECURITY environment variable, it connects with that security scheme (see Security).

Security

Each SASL or client certificate scheme of the Kafka servers is a constant of security.gen.go (SecurityKafkaScram), with its key as value, and a function returning the franz-go options of the scheme: KafkaScramSecurity(username, password) for SASL, KafkaMtlsSecurity(certFile, keyFile, caFile) for a client certificate. SecurityFromEnvironment(scheme) takes the credentials from environment variables named after the scheme key, e.g. KAFKA_SCRAM_USERNAME and KAFKA_SCRAM_PASSWORD; add what it returns to Options.ClientOptions:

security, err := events.SecurityFromEnvironment(events.SecurityKafkaScram)
if err != nil {
	log.Fatal(err)
}
app, err := events.NewApp(events.Options{Brokers: brokers, ClientOptions: security}, handlers)
Scheme Options of franz-go
scramSha256, scramSha512 kgo.SASL with the SCRAM mechanism of <KEY>_USERNAME and <KEY>_PASSWORD, and kgo.DialTLSConfig on kafka-secure servers
plain, userPassword the same with SASL PLAIN
X509 kgo.DialTLSConfig with the PEM files named by <KEY>_CERT (the client certificate), <KEY>_KEY (its private key) and <KEY>_CA (the CA certificates)

Other schemes (gssapi, oauth2, …) are left out with a warning.

Avro payloads

A message whose schemaFormat is an Avro one (application/vnd.apache.avro;version=1.9.0, with or without +json or +yaml) carries a struct of its record schema as its payload, in avro.gen.go, with json tags of the field names. Enums are string types with a constant per symbol (CarrierScanStatusAtHub), fixed types byte arrays. The logical types timestamp-millis, timestamp-micros and date are time.Time, time-millis and time-micros time.Duration; a union with null is a pointer. A root schema that isn’t a record, records with decimal fields and unions of other types are skipped with a warning. The message struct is <Message>Message when the record’s name is the message’s (CarrierScanMessage).

The generated codec writes and reads the payloads in Avro’s binary encoding, with the id of the writer’s schema where the Kafka binding of the message says: before the payload (schemaIdLocation: payload, Confluent’s wire format) or in the apicurio.value.globalId header (schemaIdLocation: header, SchemaIDHeader). Senders get the id from Options.SchemaRegistry, by the subject of schemaLookupStrategy (TopicIdStrategy, {topic}-value, by default), once per schema; without one, from the Confluent-compatible registry at the schemaRegistryUrl of the Kafka binding of the first Kafka server that has one, which registers the schema with POST /subjects/{subject}/versions. ConfluentSchemaRegistry takes a URL and basic authentication:

registry := events.ConfluentSchemaRegistry{URL: "https://schemas.events.carpetexpress.example", Username: "ingest", Password: os.Getenv("REGISTRY_PASSWORD")}
app, err := events.NewApp(events.Options{Brokers: brokers, SchemaRegistry: registry}, handlers)

Receivers decode records with the schema of the generated struct, so the writer’s schema must be the same; a record that isn’t a payload of it is rejected. Request/reply operations with Avro messages are skipped with a warning.

Not supported, and reported as warnings: payloads of other non-JSON schema formats (the messages are skipped, and operations left without a message too), reply addresses given by runtime expressions, and Kafka topics with parameters.

Since 0.12.0, Validate checks not, the schemas of draft-07 dependencies and if/then/else when their schemas use only required and the const and enum of properties. contentEncoding: base64 strings are []byte. prefixItems tuples whose positions are references or scalars of one type are slices of that type, checked position by position, as in server go. With a correlation id in the payload at a place that isn’t a string field of the payload’s structs, the request must carry the id and the replier’s handler must copy it.

Extensions

Extension Effect
x-go-name name of the type of a schema, or of the field of a property
x-go-type Go type of a schema, e.g. decimal.Decimal; with x-go-type-import: {path: ...} for its package
x-go-type-skip-optional-pointer the optional property isn’t a pointer
x-go-json-ignore the field gets the tag json:"-"
x-omitempty true or false overrides the omitempty of the field
x-enum-varnames, x-enumNames names of the enum constants
x-enum-descriptions comments of the enum constants
x-apigenie.codegen.name name of the type of a component schema, when it has no x-go-name
x-apigenie.codegen.disabled true on a channel or an operation leaves it out (with the operations of a channel left out)

The extensions can live in an overlay for this target instead of in the document, e.g. go.overlay.yaml:

overlay: 1.0.0
info:
  title: Go types
  version: 1.0.0
actions:
  - target: $.components.schemas.Order
    update:
      x-go-name: PurchaseOrder
  - target: $.components.schemas.Order.properties.total
    update:
      x-go-type: decimal.Decimal
      x-go-type-import:
        path: github.com/shopspring/decimal
apigenie-generator server go asyncapi.yaml --overlay go.overlay.yaml -o out

Options

-c, –config

Path to the configuration file to use. See the “Configuration” section for more information.

–overlay OVERLAY

Path to an OpenAPI Overlay (1.0 to 1.2) applied to the input before generating, e.g. one setting the extensions of the “Extensions” section. Can be repeated: the overlays apply in order, after those listed under commands.generate.server_go.overlay in the configuration file (with paths relative to it). Warnings and errors about a value an overlay set point to the overlay. The overlays apply to the AsyncAPI document bundled with the files it references.

–strict

Makes an overlay action whose target selects nothing an error (exit status 2, nothing is generated) instead of a warning (OVERLAY:LINE:COL: action #N (TARGET) selects no nodes.).

–check

Writes nothing: compares what the generator would write with the output directory, prints the files that would be created or changed, and exits with 1 when there are any (since 0.12.0). See Check that generated code is up to date.

-o, –output

Directory where the module is generated.

–package-name

Name of the Go package, as letters and digits in lower case (carpet-express-events gives carpetexpressevents); defaults to the title of the API (events when it gives no valid name). Its kebab case is the base of default group and client ids.

–package-version

Ignored: Go modules have no version in go.mod.

-p, –property

Applies to OpenAPI documents only; using it with an AsyncAPI document is an error.

Configuration

The options are read from the asyncapi section of the configuration of apigenie-generator server go; its other keys configure the generation from OpenAPI documents.

version: '1.0'
commands:
  generate:
    server_go:
      asyncapi:
        module_path: github.com/acme/events
        package_name: events
        initialisms: [SKU]
        main: true
        perspective: application
Option Description
module_path Module path of go.mod (default: example.com/<package>)
package_name Name of the package, a lower-case Go identifier (default: from --package-name or the title)
initialisms Words written in capitals in identifiers, besides the common ones (ID, URL, …)
main true (default): write cmd/<package>/main.go when it doesn’t exist
perspective application (default): the described application; counterpart: its counterpart, receiving what it sends

Verification

The codegen conformance suite runs this target on the Carpet Express AsyncAPI fixture. Level 1 compares the output with its snapshots and checks each feature, Level 2 runs the language's type checker and linter, and Level 3 runs the generated code in contract tests against the fixture. All targets.

This target's support tier is Tier 2. It goes through all three levels before every release, like Tier 1. It doesn't aim to support every feature: a feature of the fixture can stay unsupported, with the reason its manifest gives.

ResultsLevel 1Level 2Level 3
Release 0.12.0, server go✓ snapshots of 3.0; 31 supported, 0 not yet✓ pass (go version go1.22.5 linux/amd64)✓ 7 of 7 operations pass (go version go1.22.5 linux/amd64)
Release 0.12.0, server go (counterpart)✓ snapshots of 3.0; 31 supported, 0 not yet✓ pass (go version go1.22.5 linux/amd64)✓ 7 of 7 operations pass (go version go1.22.5 linux/amd64)
Development version, server go✓ snapshots of 3.0; 31 supported, 0 not yetruns before a releaseruns before a release
Development version, server go (counterpart)✓ snapshots of 3.0; 31 supported, 0 not yetruns before a releaseruns before a release
Was this helpful?