Split the Text files using ADF Pipelines

Requirements: If the file size of an Excel file exceeds the specified criteria, we need to address the following: The Excel file contains multiple sheets, and these sheets contain inter-related data. We utilize SQL Views to retrieve data from these sheets using the “Get Data” activity in Excel. Although we can split the Excel file… Continue reading Split the Text files using ADF Pipelines

Copying Excel files from source to destination in Excel format using ADF Pipeline

As Azure Data Factory (ADF) does not offer Excel as a sink option, an alternative method is required to copy Excel files using ADF pipelines. Implementation Step 1: Utilize the “Get Metadata” activity to retrieve the list of files stored in the blob storage. Select the “Child Items” option from the field list to include… Continue reading Copying Excel files from source to destination in Excel format using ADF Pipeline

Dynamically copying all types of files, including Excels, JSON, CSV, and TXT, using a single copy data activity in ADF

The objective is to copy various types of files, including JSON, CSV, Text, and Excel files, using a single Copy data activity in Azure Data Factory (ADF) pipeline. Implementation: Step 1: Utilize the “Get Metadata” activity to retrieve the list of files stored in the blob storage. Select the “Child Items” option from the field… Continue reading Dynamically copying all types of files, including Excels, JSON, CSV, and TXT, using a single copy data activity in ADF

Implementing ADF Branches and Filters for Employee Training

In one organization, there is one analysis of employee’s performance on different technologies based on the technologies they know and ratings from their manager. If they don’t meet the required criteria, then that employee will get training for those technologies. Input: emp_id MSBI PowerBI ratings from the manager 101 null x 3 101 null null… Continue reading Implementing ADF Branches and Filters for Employee Training