MSK

The MSK source allows Tangent to consume logs directly from an Amazon MSK (Kafka) cluster.
It supports plaintext and SCRAM-SHA authentication, as well as optional TLS configuration.

Schema

string

required

A comma-separated list of Kafka broker addresses in the form host:port.
Example:"b-1.example.com:9092,b-2.example.com:9092"

string

required

Kafka topic to subscribe to.

string

Consumer group ID for this Tangent node.
Defaults to tangent-node.

string

Kafka security protocol to use.
Defaults to PLAINTEXT.
Typical values: PLAINTEXT, SASL_SSL, SSL.

string

Path to CA certificate file if using SSL/TLS.

string

Path to the client certificate file if required by your MSK cluster.

string

Path to the client private key file if required by your MSK cluster.

string

Specifies how incoming Kafka message payloads are decoded before being sent to your plugin.
See Decoding Options for details.

`auth`

Authentication settings for connecting to MSK.

string

required

Authentication mode.
Currently only scram is supported.

string

SASL mechanism to use for SCRAM authentication.
Defaults to SCRAM-SHA-512.

string

required

Username for SASL authentication.

string

required

Password for SASL authentication.
This is stored securely using SecretString.

Example

tangent.yaml

sources:
  kafka_in:
    type: msk
    bootstrap_servers: "b-1.example.com:9092,b-2.example.com:9092"
    topic: logs
    group_id: tangent-node
    security_protocol: SASL_SSL
    ssl_ca_location: /etc/ssl/certs/ca.pem
    auth:
      mode: scram
      sasl_mechanism: SCRAM-SHA-512
      username: tangent
      password: supersecret
    decoding: json

🧠 Notes

Tangent uses Kafka’s standard consumer group semantics.
If auth.mode is set to scram, ensure the brokers support SASL/SCRAM.
TLS parameters are optional but recommended in production.
The decoding field determines how message bytes are parsed (e.g. json, ndjson, raw).

tangent.yaml

Runtime

Plugins

DAG

Sources

Sinks

Schema

`auth`

Example

🧠 Notes

​Schema

​auth

​Example

​🧠 Notes

Schema

`auth`

Example

🧠 Notes