ADF Samples
Description: Copy one Text/CSV file from one folder to another folder in BLOB
Components: Copy data
Description: Copy all Text/CSV files from one folder to another folder in BLOB
Components: Get Metadata, ForEach, Copy data
Background: https://www.youtube.com/watch?v=duyr9tPh_Yk&list=PLm7Nm9btqfe3QNbrLY-Vyu8-wh3EanG6t
Description: Load data from ADLS Gen2 for Game project
Components: Stored Procedure & Copy data
Note: For more info about Game project, please see https://github.com/Afsarsoft/SQL-Game
Description: Export data from Azure SQL DB to ADLS Gen2
Components: Copy data
Note: For more info about Game project, please see https://github.com/Afsarsoft/SQL-Game
Description: Dynamically Export data from Azure SQL DB to ADLS Gen2
Components: Lookup, ForEach, Copy data
Process flow:
- Get few tables, using Lookup
SELECT TABLE_SCHEMA, TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE' AND TABLE_SCHEMA = 'game' AND TABLE_NAME IN ('Discount', 'Game', 'GameTeam', 'Order', 'Partner', 'PartnerInfo', 'Retailer', 'Team', 'Type'); - For each table, bring tables
- Copy data to ADLS Gen2 using Copy data
Note: For more info about Game project, please see https://github.com/Afsarsoft/SQL-Game
Background: https://www.youtube.com/watch?v=KsO2FHQdILs It is documented in https://microsoft-bitools.blogspot.com/2019/06/staging-with-azure-data-factory-foreach.html
Description: Dynamically Export data from Azure SQL DB to ADLS Gen2
Components: Stored procedure, Lookup, ForEach, if Condition, Copy data
Process flow:
- Truncate Table History, using Stored procedure
- Get few tables, using Lookup
SELECT TABLE_SCHEMA, TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE' AND TABLE_SCHEMA = 'game' AND TABLE_NAME IN ('Discount', 'Game', 'GameTeam', '[Order]', 'Partner', 'PartnerInfo', 'Retailer', 'Team', 'Type', 'History'); - For each table, bring tables which are not empty by using Lookup and If Condition
@greater(activity('LookupTable').output.firstRow.count,0) - Copy data to ADLS Gen2 using Copy data
Note: For more info about Game project, please see https://github.com/Afsarsoft/SQL-Game
Description: Delete all files from ADLS Gen2 folder.
Components: Get Metadata, ForEach, Delete