Senior Azure Data Engineer(RARR Job 5374)

Key Responsibilities:

• Design and develop scalable data pipelines\Notebooks using Microsoft Fabric or Synapse Analytics
• Should be a research-oriented person in Data Projects and have a future focus mindset (Think outside of the box)
• Build and manage Lakehouses and Data Warehouses using Fabric’s OneLake architecture.
• Integrate data from diverse sources into Fabric using Dataflows Gen2, Notebooks, and Pipelines.
• Collaborate with BI developers to enable seamless integration with Power BI and other reporting tools.
• Implement data governance, security, and compliance within the Fabric ecosystem.
• Optimize data storage and processing for performance and cost-efficiency.
• Monitor and troubleshoot data workflows and ensure high data quality and reliability.
• Document architecture, data models, and processes.
• Should have experience on automated functional Testing along with development

Skills required: 

1. Pyspark
2. Data Modelling
3. Spark SQL
4. Microsoft Fabric (Understanding of Shortcuts, Mirroring, Data flows and all the feature)
5. Data Ingestion Design Patterns