apigenie-generator server go (AsyncAPI)
Tier 2 · Release 0.12.0: Level 1 ✓ · Level 2 ✓ · Level 3 ✓ · Verification
Generate a Go Kafka application of the Kafka channels of an AsyncAPI document.
Usage
apigenie-generator server go [...OPTIONS] INPUT_FILE
Description
apigenie-generator server go generates from OpenAPI documents (see
apigenie-generator server go) and from AsyncAPI 3.x documents,
telling them apart by their openapi or asyncapi field. Documents referencing other files are
bundled first.
From an AsyncAPI document it generates a Go module (Go 1.22) using the Kafka client
franz-go (github.com/twmb/franz-go/pkg/kgo, pinned at
v1.18.1 in go.mod, with a complete go.sum). By default the application is the one the
document describes: it sends the messages of send operations and receives the messages of
receive operations. Set the perspective option to counterpart to swap them.
The generator writes the Kafka channels: the channels of servers with the kafka or
kafka-secure protocol, and channels with a Kafka binding in documents without servers. Channels
of other protocols are skipped and reported as info.
type labelFarm struct{}
func (labelFarm) OnRenderLabel(ctx context.Context, message events.RenderLabel) (events.RequestLabelRenderReply, error) {
var reply events.LabelRendered
reply.Key = message.Key
reply.Payload = render(message.Payload)
return reply, nil // sent to TopicLabelRenderResults with the request's correlation id
}
func (labelFarm) OnUnknown(ctx context.Context, record *kgo.Record) error { return nil }
func main() {
var handlers events.Handlers
handlers.RequestLabelRender = labelFarm{}
// ...
app, err := events.NewApp(events.Options{Brokers: []string{"localhost:9092"}}, handlers)
if err != nil {
log.Fatal(err)
}
defer app.Close()
err = app.PublishShipmentEvents().SendShipmentDelivered(ctx, delivered)
// ...
_ = app.Run(ctx) // consumes until ctx is done
}
The module has one package at its root, named by package_name, --package-name or the
document’s title (letters and digits, lower case):
| File | Contents |
|---|---|
types.gen.go |
the types of the schemas the payloads use: structs with json tags, enums, and unions as a struct whose Value holds one variant of a sealed interface (switch v := event.Value.(type)); each with a Validate method |
messages.gen.go |
a struct per message (Key, Headers, Payload), its headers struct and header constants, and the code writing, reading and validating records |
kafka.gen.go |
the server hosts, topics, group and client ids, the handler interfaces, Handlers, App and the senders |
avro.gen.go |
the structs of the Avro records of the payloads, with their schemas and codecs |
security.gen.go |
the security schemes of the servers and the franz-go options of each |
runtime_*.gen.go |
the Kafka runtime (consumers, requesters, headers), validation, Nullable[T], Date |
go.mod, go.sum |
the module, requiring franz-go |
cmd/<package>/main.go |
a starting point, written once and never overwritten |
- Messages. Headers are the message’s headers merged with those of its traits as AsyncAPI
does (JSON Merge Patch: a later
requiredlist replaces an earlier one). Header values are text on Kafka, so headers arestring,int64,float64orboolfields, pointers when optional. Headers with a constant value, e.g. thece_specversionandce_typeof CloudEvents in binary mode, are constants (ShipmentStatusChangedCeType), which senders write and receivers compare. Messages are validated as a whole before they are sent and after they are received. - Received operations.
NewApp(options, handlers)takes a handler per received operation (PublishShipmentEventsHandler) with anOn<Message>method per message andOnUnknown. Several messages of a channel are told apart by a payload property or a header with a distinct constant value (e.g.typeorce_type); otherwise the first message whose payload decodes strictly and is valid is chosen. Records matching no message go toOnUnknown.Runconsumes the operations’ topics, each in its consumer group, until its context is done;Closecommits the offsets of the handled records and leaves the groups. Records that aren’t valid messages, and handler errors, go toOptions.OnError(logged by default); the records are committed either way. - Sent operations.
app.PublishShipmentEvents()returns the sender of the operation, with aSend<Message>(ctx, message)method per message that validates the message and returns once the brokers have it. - Request/reply with a reply channel: the requester’s
Send<Message>returns the reply (one of the reply messages,RequestLabelRenderReply), matched by the request’s correlation id, which is generated when the request has none;Options.ReplyTimeoutbounds the wait when the context has no deadline. A replier’s handler returns the reply, which is sent to the reply topic with the request’s correlation id copied (to a header, or a payload property, as thecorrelationIdlocations say).
Group and client ids come from the groupId and clientId schemas of the Kafka operation
bindings, of the operation or its traits: their const, single enum value, default or first
example. Without a value, <package name in kebab case>-<operation> is used, and the doc comment
of the constant names the binding’s pattern. Options.GroupIDs replaces them by operation id.
The partitions, replicas and topic configuration of channel bindings are documented on the topic
constants; the generated code creates no topics.
main.go connects to the brokers of the -brokers flag, else of the KAFKA_BROKERS environment
variable (comma-separated), else localhost:9092, wires stub handlers that return
api.ErrNotImplemented, and stops cleanly on SIGINT or SIGTERM. With the -security flag, else
the KAFKA_SECURITY environment variable, it connects with that security scheme (see
Security).
Security
Each SASL or client certificate scheme of the Kafka servers is a constant of security.gen.go
(SecurityKafkaScram), with its key as value, and a function returning the franz-go options of
the scheme: KafkaScramSecurity(username, password) for SASL, KafkaMtlsSecurity(certFile, keyFile, caFile) for a client certificate. SecurityFromEnvironment(scheme) takes the credentials
from environment variables named after the scheme key, e.g. KAFKA_SCRAM_USERNAME and
KAFKA_SCRAM_PASSWORD; add what it returns to Options.ClientOptions:
security, err := events.SecurityFromEnvironment(events.SecurityKafkaScram)
if err != nil {
log.Fatal(err)
}
app, err := events.NewApp(events.Options{Brokers: brokers, ClientOptions: security}, handlers)
| Scheme | Options of franz-go |
|---|---|
scramSha256, scramSha512 |
kgo.SASL with the SCRAM mechanism of <KEY>_USERNAME and <KEY>_PASSWORD, and kgo.DialTLSConfig on kafka-secure servers |
plain, userPassword |
the same with SASL PLAIN |
X509 |
kgo.DialTLSConfig with the PEM files named by <KEY>_CERT (the client certificate), <KEY>_KEY (its private key) and <KEY>_CA (the CA certificates) |
Other schemes (gssapi, oauth2, …) are left out with a warning.
Avro payloads
A message whose schemaFormat is an Avro one (application/vnd.apache.avro;version=1.9.0, with
or without +json or +yaml) carries a struct of its record schema as its payload, in
avro.gen.go, with json tags of the field names. Enums are string types with a constant per
symbol (CarrierScanStatusAtHub), fixed types byte arrays. The logical types timestamp-millis,
timestamp-micros and date are time.Time, time-millis and time-micros time.Duration; a
union with null is a pointer. A root schema that isn’t a record, records with decimal fields
and unions of other types are skipped with a warning. The message struct is
<Message>Message when the record’s name is the message’s (CarrierScanMessage).
The generated codec writes and reads the payloads in Avro’s binary encoding, with the id of the
writer’s schema where the Kafka binding of the message says: before the payload
(schemaIdLocation: payload, Confluent’s wire format) or in the apicurio.value.globalId header
(schemaIdLocation: header, SchemaIDHeader). Senders get the id from Options.SchemaRegistry,
by the subject of schemaLookupStrategy (TopicIdStrategy, {topic}-value, by default), once
per schema; without one, from the Confluent-compatible registry at the schemaRegistryUrl of the
Kafka binding of the first Kafka server that has one, which registers the schema with POST /subjects/{subject}/versions. ConfluentSchemaRegistry takes a URL and basic authentication:
registry := events.ConfluentSchemaRegistry{URL: "https://schemas.events.carpetexpress.example", Username: "ingest", Password: os.Getenv("REGISTRY_PASSWORD")}
app, err := events.NewApp(events.Options{Brokers: brokers, SchemaRegistry: registry}, handlers)
Receivers decode records with the schema of the generated struct, so the writer’s schema must be the same; a record that isn’t a payload of it is rejected. Request/reply operations with Avro messages are skipped with a warning.
Not supported, and reported as warnings: payloads of other non-JSON schema formats (the messages are skipped, and operations left without a message too), reply addresses given by runtime expressions, and Kafka topics with parameters.
Since 0.12.0, Validate checks not, the schemas of draft-07
dependencies and if/then/else when their
schemas use only required and the const and enum of properties. contentEncoding: base64
strings are []byte. prefixItems tuples whose positions are references or scalars of one type
are slices of that type, checked position by position, as in server go. With a correlation
id in the payload at a place that isn’t a string field of the payload’s structs, the request must
carry the id and the replier’s handler must copy it.
Extensions
| Extension | Effect |
|---|---|
x-go-name |
name of the type of a schema, or of the field of a property |
x-go-type |
Go type of a schema, e.g. decimal.Decimal; with x-go-type-import: {path: ...} for its package |
x-go-type-skip-optional-pointer |
the optional property isn’t a pointer |
x-go-json-ignore |
the field gets the tag json:"-" |
x-omitempty |
true or false overrides the omitempty of the field |
x-enum-varnames, x-enumNames |
names of the enum constants |
x-enum-descriptions |
comments of the enum constants |
x-apigenie.codegen.name |
name of the type of a component schema, when it has no x-go-name |
x-apigenie.codegen.disabled |
true on a channel or an operation leaves it out (with the operations of a channel left out) |
The extensions can live in an overlay for this target instead of in the document, e.g.
go.overlay.yaml:
overlay: 1.0.0
info:
title: Go types
version: 1.0.0
actions:
- target: $.components.schemas.Order
update:
x-go-name: PurchaseOrder
- target: $.components.schemas.Order.properties.total
update:
x-go-type: decimal.Decimal
x-go-type-import:
path: github.com/shopspring/decimal
apigenie-generator server go asyncapi.yaml --overlay go.overlay.yaml -o out
Options
-c, –config
Path to the configuration file to use. See the “Configuration” section for more information.
–overlay OVERLAY
Path to an OpenAPI Overlay (1.0 to 1.2) applied to the input before generating, e.g. one setting the
extensions of the “Extensions” section. Can be repeated: the overlays apply in order, after those
listed under commands.generate.server_go.overlay in the configuration file (with paths relative to
it). Warnings and errors about a value an overlay set point to the overlay. The overlays apply to
the AsyncAPI document bundled with the files it references.
–strict
Makes an overlay action whose target selects nothing an error (exit status 2, nothing is generated)
instead of a warning (OVERLAY:LINE:COL: action #N (TARGET) selects no nodes.).
–check
Writes nothing: compares what the generator would write with the output directory, prints the files that would be created or changed, and exits with 1 when there are any (since 0.12.0). See Check that generated code is up to date.
-o, –output
Directory where the module is generated.
–package-name
Name of the Go package, as letters and digits in lower case (carpet-express-events gives
carpetexpressevents); defaults to the title of the API (events when it gives no valid
name). Its kebab case is the base of default group and client ids.
–package-version
Ignored: Go modules have no version in go.mod.
-p, –property
Applies to OpenAPI documents only; using it with an AsyncAPI document is an error.
Configuration
The options are read from the asyncapi section of the configuration of apigenie-generator server go;
its other keys configure the generation from OpenAPI documents.
version: '1.0'
commands:
generate:
server_go:
asyncapi:
module_path: github.com/acme/events
package_name: events
initialisms: [SKU]
main: true
perspective: application
| Option | Description |
|---|---|
module_path |
Module path of go.mod (default: example.com/<package>) |
package_name |
Name of the package, a lower-case Go identifier (default: from --package-name or the title) |
initialisms |
Words written in capitals in identifiers, besides the common ones (ID, URL, …) |
main |
true (default): write cmd/<package>/main.go when it doesn’t exist |
perspective |
application (default): the described application; counterpart: its counterpart, receiving what it sends |
Verification
The codegen conformance suite runs this target on the Carpet Express AsyncAPI fixture. Level 1 compares the output with its snapshots and checks each feature, Level 2 runs the language's type checker and linter, and Level 3 runs the generated code in contract tests against the fixture. All targets.
This target's support tier is Tier 2. It goes through all three levels before every release, like Tier 1. It doesn't aim to support every feature: a feature of the fixture can stay unsupported, with the reason its manifest gives.
| Results | Level 1 | Level 2 | Level 3 |
|---|---|---|---|
Release 0.12.0, server go | ✓ snapshots of 3.0; 31 supported, 0 not yet | ✓ pass (go version go1.22.5 linux/amd64) | ✓ 7 of 7 operations pass (go version go1.22.5 linux/amd64) |
Release 0.12.0, server go (counterpart) | ✓ snapshots of 3.0; 31 supported, 0 not yet | ✓ pass (go version go1.22.5 linux/amd64) | ✓ 7 of 7 operations pass (go version go1.22.5 linux/amd64) |
Development version, server go | ✓ snapshots of 3.0; 31 supported, 0 not yet | runs before a release | runs before a release |
Development version, server go (counterpart) | ✓ snapshots of 3.0; 31 supported, 0 not yet | runs before a release | runs before a release |