Competencies ➢ Proven experience as a Data Engineer, preferably with a focus on Azure Data Factory and Azure Synapse Analytics. ➢ Strong understanding of data integration concepts, ETL/ELT processes, and data warehousing principles. ➢ In-depth knowledge of Azure Data Factory and its various components, including pipelines, activities, data flows, and triggers. ➢ Hands-on experience with Azure Synapse Analytics, including data ingestion, data transformation, and data loading techniques. ➢ Proficient in SQL and scripting languages (e.g., PowerShell, Python) for data manipulation and automation tasks. ➢ Familiarity with data modeling, dimensional modeling, and schema design for data warehousing. ➢ Solid understanding of cloud computing concepts and experience working in a cloud environment (preferably Azure). ➢ Experience with Azure Data Lake Storage, Azure SQL Database, and other Azure data-related services. ➢ Strong problem-solving skills and the ability to analyze complex data requirements and translate them into practical solutions. ➢ Excellent communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams. ➢ Azure certifications related to data engineering (e.g., Azure Data Engineer Associate) will be a plus.
Responsibilities: ➢ Design, develop, and implement end-to-end data integration solutions using Azure Data Factory (ADF) and Azure Synapse Analytics. ➢ Collaborate with cross-functional teams to understand data requirements, design data models, and develop data pipelines for efficient data processing and analysis. ➢ Develop ETL/ELT workflows, data ingestion processes, and data transformation activities using ADF pipelines and activities. ➢ Optimize data processing and storage in Azure Synapse Analytics, ensuring high performance and scalability. ➢ Perform data quality checks, data validation, and data profiling to ensure data accuracy and integrity. ➢ Implement data security and access controls to protect sensitive data in compliance with organizational and industry standards. ➢ Monitor and troubleshoot data integration and data warehousing solutions, identifying and resolving performance issues and bottlenecks. ➢ Collaborate with data architects and data scientists to design and implement data lakes, data marts, and data warehouses to support analytics and reporting requirements. ➢ Stay updated with the latest Azure Data Engineering trends, best practices, and technologies, and provide recommendations for process improvements and tool enhancements. ➢ Document technical specifications, data flows, and solution designs to ensure effective knowledge transfer and maintainable solutions. ➢ Provide technical guidance and mentorship to junior team members, sharing your expertise in Azure Data Factory and Azure Synapse Analytics.