Streaming data pipelines

Streaming pipelines move events sub-minute, with event-time semantics, ordered delivery guarantees, and windowed processing. At the same row volume, streaming typically costs 2x to 3x a 15-minute micro-batch. Pick streaming only when latency is a real product requirement, not a nice-to-have. Estuary Flow is the cheapest managed option at published $0.75/GB.source - pulled 2026-06-21

When streaming is actually required

› Fraud detection, payment authorisation, abuse signals - latency is product-critical.
› Operational dashboards where staleness has a measurable revenue impact.
› CDC where the warehouse must reflect a master DB within seconds.

When 15-minute micro-batch is enough

Most analytics workloads do not need streaming. A 15-minute Fivetran or Airbyte sync covers BI dashboards, attribution, customer-360 builds, and most ML feature stores. The cheaper path is usually right; verify the latency requirement first.

Streaming-native finalists

Estuary Flow

Cheapest managed, CDC + streaming in one bill

Estuary vs Fivetran

Decision aid: streaming-native vs batch with premium streaming

Warehouse compute tax

Streaming multiplies the warehouse bill too

Written by Oliver Wakefield-Smith, Founder of Digital Signet. Independent reference, no vendor sponsorship.

Sources logged at /sources. Pricing-change history at /changelog. Last reviewed 2026-06-21.