Skip to main content
Sinks define where Tangent writes data to — for example, S3 buckets, local files, or testing targets like blackhole.
They are the terminal nodes of the DAG: after records are processed by plugins, they are serialized and written by sinks.

​
Sink Types

TypeDescriptionExample
s3Write to Amazon S3 buckets with WAL durabilitymy-bucket
fileWrite to local or mounted filesystem paths/var/lib/tangent/output.json
blackholeDiscard output (useful for testing)—

​
Common Configuration

​
type
string
required
Selects the sink type.
One of: s3, file, or blackhole.
​
encoding
object
Controls how Tangent serializes records before writing.
See Encoding for supported formats (ndjson, json, avro, parquet).
​
compression
object
Configures compression for sink objects.
See Compression for algorithm details and levels.

​
Example (S3 Sink)

tangent.yaml
sinks:
  s3_main:
    type: s3
    bucket_name: my-bucket
    region: us-west-2
    encoding:
      type: parquet
      schema: ./schemas/arrow_schema.json
    compression:
      type: gzip
      level: 6

​
Behavior

  • Each sink manages its own WAL and upload queue.
  • Data rotation happens automatically based on time or size thresholds.
  • Tangent guarantees at-least-once delivery — WALs ensure safe recovery on restart.
  • Encoding and compression are handled transparently before upload.

​
See also