OTEL Telemetry Data Compression

Not all telemetry compresses equally. The structural characteristics of traces, metrics, and logs produce different compression profiles.

Traces

Traces are the most compressible OTEL signal. A typical span carries fixed-width identifiers (trace_id, span_id), repetitive resource attributes (service.name, host.name, sdk.version), repetitive span attributes (http.method, http.status_code, db.system), and timestamps as nanosecond-precision integers with small deltas.

Uncompressed size: ~500 bytes/span is a common planning estimate (Jaeger + Elasticsearch), though ClickHouse achieves ~80 bytes/span after column-oriented compression.

Metrics

Metrics payloads consist of monotonically increasing timestamps (perfect for delta/DoubleDelta encoding), floating-point gauge values with small inter-sample deltas (ideal for Gorilla/XOR encoding), and repeated label sets.

Logs

Logs are the least predictable signal. Freeform message bodies introduce entropy that resists compression, but structured fields (severity, resource attributes, scope) still compress well.

Tested on representative OTLP protobuf payloads (mixed traces, metrics, logs; ~2 MB uncompressed batch):

The zstd Advantage for Telemetry

zstd wins on the ratio-per-CPU-cycle metric that matters most in telemetry pipelines:

1. zstd -1 compresses 5x faster than gzip -6 while achieving comparable or better ratios
2. Decompression is ~4x faster (1550 vs 390 MB/s) — critical for query-time backends like ClickHouse
3. Decompression speed is constant across all zstd levels — you can crank up the level at write time without penalizing read performance
4. Dictionary mode is available for small payloads

When NOT to Use zstd

• Prometheus Remote Write v1: Protocol mandates snappy. Prometheus 3.x Remote Write 2.0 adds zstd support with ~30% bandwidth improvement.
• Universal OTLP interop: The OTLP spec requires servers to support gzip. If you control neither the collector nor the backend, gzip is the safe choice.
• Extremely CPU-constrained edge collectors: snappy or lz4 may be preferable.

Specification Requirements

Per the OTLP Specification 1.9.0:

• All OTLP servers must support none and gzip
• Additional algorithms (zstd, snappy) are optional — negotiated via headers or gRPC encoding
• The environment variable OTEL_EXPORTER_OTLP_COMPRESSION accepts values gzip or none

Collector Configuration

OTLP gRPC exporter:

exporters:
  otlp:
    endpoint: backend.example.com:4317
    compression: zstd        # gzip | zstd | snappy | none
    tls:
      insecure: false

OTLP HTTP exporter:

exporters:
  otlphttp:
    endpoint: https://ingest.example.com:4318
    compression: gzip        # gzip | zstd | snappy | none
    headers:
      authorization: "Bearer ${OTEL_TOKEN}"

OTel Arrow: The Columnar Protocol

OTel Arrow replaces protobuf row-encoding with Apache Arrow columnar encoding inside gRPC streams. This is the single most impactful compression improvement available:

• How it works: Resource attributes, span names, status codes, and repeated fields are stored as dictionary-encoded Arrow columns. Timestamps become contiguous integer arrays. The Arrow IPC format is then compressed with zstd.
• Bandwidth reduction: 30–70% less than OTLP/gRPC + zstd (same batch size). ServiceNow reported 15x–30x compression over uncompressed OTLP in production.
• Status: otelarrowexporter and otelarrowreceiver are included in opentelemetry-collector-contrib releases. Phase 2 (Rust-based pipeline) announced in 2025.
• Best for: High-volume internal hops (agent → gateway, gateway → backend). Not yet widely supported by SaaS backends.

exporters:
  otelarrow:
    endpoint: gateway.internal:4317
    arrow:
      num_streams: 4
    compression: zstd

Layer Model

SDK (app) ──[OTLP/gRPC+gzip]──> Agent Collector
  ──[OTel Arrow+zstd]──> Gateway Collector
    ──[batch processor]──> Exporter ──[OTLP/gRPC+zstd]──> Backend
      ──[ClickHouse ZSTD(1) + DoubleDelta codecs]──> Disk

Collector-Level Optimizations

Before data even hits compression, reduce what you send:

1. Batch processor — Larger batches compress better:

processors:
  batch:
    send_batch_size: 8192
    timeout: 200ms

2. Filter processor — Drop low-value spans (health checks, readiness probes):

processors:
  filter:
    error_mode: ignore
    traces:
      span:
        - 'attributes["http.route"] == "/healthz"'

3. Attributes processor — Remove high-cardinality attributes that bloat payloads without aiding queries.

ClickHouse (SigNoz, ClickStack)

ClickHouse offers the most granular compression control. Recommended codec assignments for OTEL trace schema:

CREATE TABLE otel_traces (
    Timestamp           DateTime64(9)   CODEC(DoubleDelta, ZSTD(1)),
    TraceId             FixedString(32) CODEC(ZSTD(1)),
    SpanId              String          CODEC(ZSTD(1)),
    ParentSpanId        String          CODEC(ZSTD(1)),
    SpanName            LowCardinality(String) CODEC(ZSTD(1)),
    SpanKind            Int8            CODEC(T64, ZSTD(1)),
    ServiceName         LowCardinality(String) CODEC(ZSTD(1)),
    Duration            UInt64          CODEC(T64, ZSTD(1)),
    StatusCode          Int16           CODEC(T64, ZSTD(1)),
    HttpStatusCode      Int16           CODEC(T64, ZSTD(1)),
    SpanAttributes      Map(String, String) CODEC(ZSTD(1)),
    ResourceAttributes  Map(String, String) CODEC(ZSTD(1)),
    Events              String          CODEC(ZSTD(1))
) ENGINE = MergeTree()
ORDER BY (ServiceName, SpanName, toDateTime(Timestamp))

Grafana Tempo

Tempo compresses trace blocks before pushing them to object storage (S3, GCS, Azure Blob). Recommended configuration:

storage:
  trace:
    backend: s3
    block:
      encoding: zstd

zstd reduces storage to ~15% of uncompressed (~6.7:1).

Prometheus / VictoriaMetrics (Metrics)

• Prometheus Remote Write 1.0: Snappy-only. OTel prometheusremotewrite exporter enforces this.
• Prometheus Remote Write 2.0 (Prometheus 3.x): Adds zstd option; ~30% bandwidth reduction over RW 1.0.
• VictoriaMetrics: Uses zstd compression for its own remote write protocol; 40–60% additional bandwidth savings over Prometheus RW with snappy.

Resource Attributes

Resource attributes (service.name, service.version, host.name, cloud.region) are identical across every span/metric/log from a single process. In OTLP protobuf, they are sent once per ResourceSpans per batch — already efficient.

Span Attributes from Semantic Conventions

Semantic conventions produce highly repetitive keys (http.method, http.route, http.status_code, db.system, rpc.method) and often repetitive values (GET, POST, 200, 500, mysql, grpc). A batch of 1000 spans from the same HTTP service might have only 5–10 unique attribute key sets and 20–50 unique value combinations.

zstd's larger default window (128 KB at level 1 vs gzip's 32 KB) captures longer-range repetitions.

Columnar Advantage (OTel Arrow / ClickHouse)

When data is pivoted from row-oriented to column-oriented:

• String columns become arrays of repeated values → dictionary encoding reduces to integer indexes
• Numeric columns become sorted/semi-sorted integer sequences → delta/DoubleDelta encoding
• Timestamp columns become monotonically increasing nanosecond values → DoubleDelta yields near-zero residuals

This is why OTel Arrow + zstd achieves 7:1 to 12:1 while plain protobuf + zstd achieves 4:1 to 6:1.

Zstandard supports dictionary compression — a pre-trained dictionary of common byte patterns that seeds the compressor before processing each payload. For small data (<64 KB), a dictionary can improve ratios by 2x–5x.

# Collect 10K sample OTLP protobuf payloads
zstd --train samples/*.pb -o otel-traces.dict

# Compress with dictionary
zstd -D otel-traces.dict payload.pb

# Decompress with dictionary
zstd -D otel-traces.dict -d payload.pb.zst

Decision Matrix

Cost Impact Estimates

For a mid-size deployment generating 50 GB/day of raw (uncompressed) telemetry:

Quick-Start Collector Config

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 512
    spike_limit_mib: 128

  filter:
    error_mode: ignore
    traces:
      span:
        - 'attributes["http.route"] == "/healthz"'
        - 'attributes["http.route"] == "/readyz"'

  attributes:
    actions:
      - key: http.request.header.authorization
        action: delete
      - key: http.request.header.cookie
        action: delete

  batch:
    send_batch_size: 8192
    timeout: 200ms

exporters:
  otlp/backend:
    endpoint: signoz.internal:4317
    compression: zstd
    retry_on_failure:
      enabled: true
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 5000

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, filter, attributes, batch]
      exporters: [otlp/backend]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp/backend]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, filter, attributes, batch]
      exporters: [otlp/backend]

OTLP Protocol & Specification

• OTLP Specification 1.9.0
• OpenTelemetry Protocol Exporter
• OTel Collector gRPC Config

OTel Arrow

• Achieve a 10x Reduction in Telemetry Traffic Using OTel Arrow (2023)
• OTel Arrow in Production (2024)
• OTel Arrow GitHub Repository

ClickHouse Compression

• Optimizing ClickHouse with Schemas and Codecs
• Compression in ClickHouse (Docs)
• Building an Observability Solution with ClickHouse — Part 2: Traces

SigNoz

• SigNoz Technical Architecture
• Optimising OTel Pipelines to Cut Observability Costs

Prometheus & VictoriaMetrics

• Prometheus Remote Write 2.0 Specification
• VictoriaMetrics Remote Write Protocol

A.1 The "Just Ship It" Default

• Wire: compression: gzip on all OTLP exporters
• Backend: ClickHouse with default ZSTD(1) codec
• Expected ratio: 8:1 — 15:1 end-to-end
• Effort: Minimal; gzip is universally supported, ZSTD(1) is ClickHouse Cloud default

A.2 The Optimized Mid-Tier

• Wire: compression: zstd on OTLP/gRPC exporters
• Collectors: batch size 8192, filter processor for health checks, attributes processor to strip PII
• Backend: ClickHouse with per-column codecs: DoubleDelta, ZSTD(1) on timestamps, T64, ZSTD(1) on integers, LowCardinality + ZSTD(1) on enum-like strings
• Expected ratio: 15:1 — 25:1 end-to-end

A.3 The Maximum Compression Frontier

• Wire: OTel Arrow + zstd between gateway collectors
• Collectors: Aggressive filtering (tail-based sampling, log dedup processor, attribute trimming)
• Backend: ClickHouse with full codec optimization + tiered storage (hot SSD / cold S3)
• Cold archival: Export historical data to Parquet, compress with zstd -19 or LZMA2
• Expected ratio: 25:1 — 40:1 end-to-end (hot), 50:1+ (cold archive)

A.5 Key Numbers to Remember

Document prepared 2026-02-24. Compression ratios are approximate and vary with workload characteristics, attribute cardinality, and batch sizes. Always benchmark with representative production data before committing to a compression strategy.