Skip to main content
The MSK source allows Tangent to consume logs directly from an Amazon MSK (Kafka) cluster.
It supports plaintext and SCRAM-SHA authentication, as well as optional TLS configuration.

​
Schema

​
bootstrap_servers
string
required
A comma-separated list of Kafka broker addresses in the form host:port.
Example:"b-1.example.com:9092,b-2.example.com:9092"
​
topic
string
required
Kafka topic to subscribe to.
​
group_id
string
Consumer group ID for this Tangent node.
Defaults to tangent-node.
​
security_protocol
string
Kafka security protocol to use.
Defaults to PLAINTEXT.
Typical values: PLAINTEXT, SASL_SSL, SSL.
​
ssl_ca_location
string
Path to CA certificate file if using SSL/TLS.
​
ssl_certificate_location
string
Path to the client certificate file if required by your MSK cluster.
​
ssl_key_location
string
Path to the client private key file if required by your MSK cluster.
​
decoding
string
Specifies how incoming Kafka message payloads are decoded before being sent to your plugin.
See Decoding Options for details.

​
auth

Authentication settings for connecting to MSK.
​
auth.mode
string
required
Authentication mode.
Currently only scram is supported.
​
auth.sasl_mechanism
string
SASL mechanism to use for SCRAM authentication.
Defaults to SCRAM-SHA-512.
​
auth.username
string
required
Username for SASL authentication.
​
auth.password
string
required
Password for SASL authentication.
This is stored securely using SecretString.

​
Example

tangent.yaml
sources:
  kafka_in:
    type: msk
    bootstrap_servers: "b-1.example.com:9092,b-2.example.com:9092"
    topic: logs
    group_id: tangent-node
    security_protocol: SASL_SSL
    ssl_ca_location: /etc/ssl/certs/ca.pem
    auth:
      mode: scram
      sasl_mechanism: SCRAM-SHA-512
      username: tangent
      password: supersecret
    decoding: json

​
🧠 Notes

  • Tangent uses Kafka’s standard consumer group semantics.
  • If auth.mode is set to scram, ensure the brokers support SASL/SCRAM.
  • TLS parameters are optional but recommended in production.
  • The decoding field determines how message bytes are parsed (e.g. json, ndjson, raw).