| title | titleSuffix | description | author | ms.author | ms.subservice | ms.custom | ms.topic | ms.date |
|---|---|---|---|---|---|---|---|---|
Mapping data flow transformation overview |
Azure Data Factory & Azure Synapse |
An overview of the different transformations available in mapping data flow |
kromerm |
makromer |
data-flows |
synapse |
conceptual |
05/15/2024 |
[!INCLUDEappliesto-adf-asa-md]
[!INCLUDEdata-flow-preamble]
Below is a list of the transformations currently supported in mapping data flow. Click on each transformations to learn its configuration details.
| Name | Category | Description |
|---|---|---|
| Aggregate | Schema modifier | Define different types of aggregations such as SUM, MIN, MAX, and COUNT grouped by existing or computed columns. |
| Alter row | Row modifier | Set insert, delete, update, and upsert policies on rows. |
| Assert | Row modifier | Set assert rules for each row. |
| Cast | Schema modifier | Change column data types with type checking. |
| Conditional split | Multiple inputs/outputs | Route rows of data to different streams based on matching conditions. |
| Derived column | Schema modifier | Generate new columns or modify existing fields using the data flow expression language. |
| External call | Schema modifier | Call external endpoints inline row-by-row. |
| Exists | Multiple inputs/outputs | Check whether your data exists in another source or stream. |
| Filter | Row modifier | Filter a row based upon a condition. |
| Flatten | Formatters | Take array values inside hierarchical structures such as JSON and unroll them into individual rows. |
| Flowlet | Flowlets | Build and include custom re-usable transformation logic. |
| Join | Multiple inputs/outputs | Combine data from two sources or streams. |
| Lookup | Multiple inputs/outputs | Reference data from another source. |
| New branch | Multiple inputs/outputs | Apply multiple sets of operations and transformations against the same data stream. |
| Parse | Formatters | Parse text columns in your data stream that are strings of JSON, delimited text, or XML formatted text. |
| Pivot | Schema modifier | An aggregation where one or more grouping columns has its distinct row values transformed into individual columns. |
| Rank | Schema modifier | Generate an ordered ranking based upon sort conditions |
| Select | Schema modifier | Alias columns and stream names, and drop or reorder columns |
| Sink | - | A final destination for your data |
| Sort | Row modifier | Sort incoming rows on the current data stream |
| Source | - | A data source for the data flow |
| Stringify | Formatters | Turn complex types into plain strings |
| Surrogate key | Schema modifier | Add an incrementing non-business arbitrary key value |
| Union | Multiple inputs/outputs | Combine multiple data streams vertically |
| Unpivot | Schema modifier | Pivot columns into row values |
| Window | Schema modifier | Define window-based aggregations of columns in your data streams. |