Azure Data Factory Data Flows Training v005
Download as PPTX, PDF1 like2,100 views
Mapping Data Flow is a new feature of Azure Data Factory that allows building data transformations in a visual interface without code. It provides a serverless, scale-out transformation engine for processing big data with unstructured requirements. Mapping Data Flows can be authored and designed visually, with transformations, expressions, and results previews, and then operationalized with Data Factory scheduling, monitoring, and control flow.
Azure Data Factory Data Flows Training v005
- 1. Azure Data Factory Mapping Data Flows https://aka.ms/dflinks MicrosoftTrainingv005
- 3. Mapping Data Flow is a new feature of Azure Data Factory that allows you to build data transformations code-free in a visual user interface • Transform at scale, in the cloud • Code-free pipelines do NOT require understanding of Spark / Scala / Python / Java • Serverless scale-out transformation execution engine • Resilient data transformation Flows built for big data scenarios with unstructured data requirements • Operationalized with Data Factory scheduling, control flow and monitoring
- 7. Authoring and designing Mapping Data Flows
- 11. Mapping Data Flow Transformations
- 32. Mapping Data Flows Expressions
- 34. List of columns being modified All available functions, fields, parameters … Build expressions here with full auto-complete and syntax checking View results of your expression in the data preview pane with live, interactive results
- 35. here
- 48. Operationalizing Mapping Data Flows
- 53. Continuous Integration and Continuous Development
- 55. Mapping Data Flow common scenarios
- 62. Mapping Data Flows Best Practices
- 73. Data Flow Script
- 74. https://docs.microsoft.com/en-us/azure/data-factory/data-flow- script
- 75. Source projection (1) Source properties Unpivot transformation (3) Aggregate Transformation (2) Sort (4) Sink (5) 1 2 3 4 5 • Syntax: input_name transform_type(properties) ~> stream_name
- 76. 1 2 3 4 5 6 Source projection (1) Source properties Distinct Aggregate (3) Row Count Agg (5) Select transformation mappings (2) Select properties (2) Row Count Agg (4) Sink transformation (6) • ~> name_of_transform • New branch does not require any script element