Data Engineer (NCS/Job/ 3514)
Job Skills
Job Description
Role Overview:
We are looking for a highly skilled Senior Data Engineer who will be responsible for designing, building, and optimizing scalable data pipelines on Azure, supporting sales, inventory, and business analytics use cases. The ideal candidate should have strong expertise in Azure data services, Python/PySpark, SQL, and a solid understanding of sales and inventory domain processes.
Primary Responsibilities:
Design and develop scalable data pipelines using Azure Data Factory, Azure Synapse Analytics, and related services
Architect and optimize solutions using Synapse (Dedicated SQL Pools, Serverless SQL Pools, Spark Pools)
Manage Synapse workspaces, pipelines, linked services, and integration runtimes
Develop high-performance data transformation workflows using Synapse Spark
Build and optimize data transformations using Python and PySpark
Develop and manage SQL-based data models for reporting and analytics
Integrate multiple data sources (ERP, Sales, External feeds) into unified models
Implement data quality checks, validation frameworks, and monitoring
Support Power BI dashboards with optimized backend data models
Work with large-scale sales and inventory datasets with performance optimization
Collaborate with stakeholders on KPIs and business requirements
Secondary Responsibilities:
Implement data engineering pipelines and workflows
Configure workload management in Synapse Dedicated SQL Pools
Support data governance, access control, and security best practices
Optimize cost and performance across Azure services
Provide technical documentation and knowledge transfer
Collaborate with BI and ML teams for advanced analytics
Support UAT, production deployment, and monitoring
Required Technical Skills:
Strong experience with Azure Fabric, Azure Synapse Analytics, and Azure Data Factory
Expertise in Synapse (Spark Pools, Dedicated & Serverless SQL Pools)
Knowledge of Azure Data Lake (ADLS Gen2)
Proficiency in Python and PySpark
Strong SQL skills (query optimization & performance tuning)
Experience with large sales and inventory datasets
Strong understanding of data modeling (Star schema, Fact-Dimension)
Experience supporting Power BI backend optimization
Soft Skills:
Strong communication and stakeholder management
Ability to work in a fast-paced, client-facing environment
Strong analytical and problem-solving skills
Ability to manage timelines independently
