Skip to main content
The SQS source allows Tangent to consume messages directly from an Amazon Simple Queue Service (SQS) queue.
This is useful for event-driven pipelines, asynchronous log forwarding, or integrating with other AWS services that publish to SQS.

​
Schema

​
queue_url
string
required
The full URL of the SQS queue to poll.
Example: https://sqs.us-east-1.amazonaws.com/123456789012/my-queue
​
wait_time_seconds
int
Long polling duration in seconds.
Controls how long Tangent waits for messages before returning an empty response.
Defaults to 20.
​
visibility_timeout
int
The duration (in seconds) that a message remains invisible to other consumers after Tangent retrieves it.
Defaults to 60.
​
decoding
object
Controls how Tangent parses message payloads after retrieval.
See Decoding for supported formats (json, ndjson, msgpack, etc.).

​
Example

tangent.yaml
sources:
  sqs_logs:
    type: sqs
    queue_url: "https://sqs.us-east-1.amazonaws.com/123456789012/app-logs"
    wait_time_seconds: 20
    visibility_timeout: 60
    decoding:
      format: json
      compression: auto

​
Behavior

  • Tangent uses the AWS SDK to long-poll the specified queue for messages.
  • When a message arrives:
    • The payload is decoded according to the decoding configuration.
    • Each decoded record is passed into the pipeline.
  • Messages are automatically deleted after successful processing.
  • Visibility timeouts prevent duplicate processing in concurrent consumers.

​
Best Practices

  • Keep wait_time_seconds high (e.g., 20s) to reduce API calls and costs.
  • Ensure the IAM role or credentials used by Tangent have sqs:ReceiveMessage and sqs:DeleteMessage permissions.
  • Tune visibility_timeout to be slightly longer than your typical plugin or sink processing time to avoid message re-delivery.
  • Use decoding.format: json or ndjson for structured logs, or text for plain message bodies.

​
See also