Azure Data Engineer (ADB+ADF)(RARR Job 5129)

We are seeking an experienced Data Engineer with strong expertise in Python, PySpark, Azure Databricks (ADB), and Azure Data Factory (ADF). The ideal candidate will be responsible for building scalable data pipelines, transforming large datasets, and orchestrating data workflows using modern Azure data services.

Key Responsibilities:

• Develop and maintain data pipelines and ETL workflows using ADF and Databricks.

• Perform large-scale data processing using PySpark within Azure Databricks.

• Write clean, efficient, and production-grade Python code for data transformation and business logic.

• Design and implement batch and real-time data ingestion processes.

• Work closely with data architects, analysts, and business teams to ensure data quality and integrity.

• Monitor and optimize pipeline performance and troubleshoot data issues.

Required Skills:

• Strong programming skills in Python.

• Expertise in PySpark for distributed data processing.

• Hands-on experience with Azure Databricks (ADB).

• Proficiency in Azure Data Factory (ADF) for pipeline orchestration.

• Solid understanding of ETL/ELT concepts and data modeling fundamentals.

• Ability to work in an Agile environment and collaborate with cross-functional teams.