EtLT: The Middle Ground Your Data Pipeline Has Been Missing | Apache SeaTunnel

Skip to main content<br>⭐️ If you like Apache SeaTunnel, give it a star on GitHub<br>⭐️

Ask a data engineer whether their pipeline is ETL or ELT and you'll get an instant answer. Old-school engineers say ETL. dbt users say ELT.<br>Both answers are incomplete. There's a third pattern that more accurately describes what modern data pipelines actually do — and it's been hiding in plain sight: EtLT .<br>The Three Paradigms​<br>ETL: Transform Before You Land​<br>Raw data is extracted, passed through a dedicated transformation tier (Spark, DataStage, Informatica), and only then written to the destination warehouse.<br>Source ──► [Transform tier] ──► Destination

Pros: The warehouse always holds clean, business-ready data. Compliance controls are enforced before data ever lands.

Cons: The transform tier becomes a bottleneck. Schema changes require coordinated updates across multiple layers. Running a dedicated compute cluster for transforms is expensive.<br>ELT: Land First, Transform In-Place​<br>Popularized by dbt. Raw data lands directly in the warehouse (BigQuery, Snowflake, ClickHouse), and SQL does the transformations in place. The transform "tier" is just the warehouse itself.<br>Source ──► Destination (raw layer) ──► [SQL inside warehouse] ──► Business layer

Pros: Raw data is preserved for auditing. You reuse warehouse compute. Iteration is fast.

Cons: Sensitive fields — SSNs, email addresses, phone numbers — land in plaintext. There's no second chance to mask them once they're in the warehouse. Data quality issues only surface after Load, at which point downstream models may already be contaminated.<br>EtLT: A Lightweight Transform In-Transit​<br>Source ──► [tiny t] ──► Destination (sanitized raw layer) ──► [T inside warehouse]

The tiny t is a small set of row-level transformations that happen while data is in flight:<br>tiny t operationPurposeField projection / column pruningDrop unused columns before transfer — save bandwidthPII maskingPhone numbers, SSNs, emails are anonymized before landing — compliance enforced at the pipeline layer, not as an afterthoughtType normalizationSource VARCHAR "2023-01-01" becomes DATE on arrival — no type-casting SQL needed in the warehouseRow filteringUnwanted events can be discarded pre-Load; stateful CDC transitions require changelog-aware handlingField renamingAlign to destination naming conventions without an alias layerNULL backfillReduce COALESCE calls in downstream aggregation SQLThe big T (post-Load Transform) is where actual business logic lives: multi-table JOINs, metric calculations, aggregations, ML feature engineering.<br>Why Engineers Keep Overlooking EtLT​<br>The reason is tooling — not concept.<br>Legacy ETL tools (Informatica, DataStage) made transformations expensive and complex. Engineers overcorrected by pushing all logic into the pipeline, which made pipelines brittle.<br>Modern ELT tools (Airbyte, Fivetran) swung to the opposite extreme: move data from source to destination with almost no in-transit processing, then let dbt handle everything.<br>The gap neither camp fills: when you need both in-transit operations (masking, filtering) and complex analytical SQL in the destination, you end up with awkward workarounds in both tool families.<br>EtLT fills exactly that gap.<br>SeaTunnel Is Built for EtLT​<br>Apache SeaTunnel describes itself in its official documentation as an "EL(T) data integration platform" — the parentheses around T are intentional. The Transform step is lightweight and optional. This is not a marketing choice; it's an architectural constraint.<br>The Three-Layer Model Maps Directly to EtLT​<br>SeaTunnel's execution model has three stages: Source → Transform → Sink .<br>Source (E)<br>└──► Transform (tiny t) ← optional, lightweight processing<br>└──► Sink (L)<br>└──► [Warehouse SQL / dbt] (T) ← big T lives here

SeaTunnel draws a clear line around what Transform can do. The official docs state:<br>"Transform can only be used for some simple transformations of data, such as converting a column to uppercase/lowercase, modifying column names, or splitting one column into multiple columns."

This describes the intended scope of the built-in transforms. In the current Zeta SQL Transform, JOIN and GROUP BY are not supported, so cross-table joins and aggregations belong in the destination system or another dedicated processing layer.<br>Built-in Transforms Cover the Typical tiny t Operations​<br>SeaTunnel Transformtiny t operationFieldMapperColumn renaming, field projectionFilterColumn projection with include/exclude listsReplaceField value substitution (masking/redaction)SplitSplit one column into multiple (e.g., address parsing)SQL TransformRow filtering and lightweight SQL expressions; current Zeta implementation does not support JOIN or GROUP BYCopyField duplicationSeaTunnel provides both map and flat-map transform interfaces. Its current built-in transforms focus on processing individual records and schemas rather than cross-row...