Azure Data Engineer (NCS/Job/ 2516)

We are seeking an experienced and self-driven Data Engineer to design, build, and maintain the modern data infrastructure that supports our analytics, data science, and product teams. In this role, you will develop scalable data pipelines and manage large, complex datasets using Azure Databricks and the broader Azure data ecosystem. Working closely with engineers, analysts, and data scientists, you will ensure that data is reliable, accessible, and optimized for insight and decision-making across the organization. This is an exciting opportunity to shape and scale our data platform and contribute to the next generation of data-driven

Accountabilities:

• Curate structured, semi-structured, and unstructured data by creating efficient, cost effective and scalable pipelines leveraging databricks and Azure Analytics stack
• Create and maintain robust data pipeline architecture, ensuring data quality, reliability, and scalability.
• Assemble and manage large, complex data sets to meet functional and non-functional business requirements.
• Identify, design, and implement process improvements, including automation of manual processes and optimization of data delivery.
• Work closely with Product Owners, Scrum Masters, and Enterprise Data Management leadership to ensure timely and high-quality delivery of data solutions.
• Align with Data Engineering chapter standards and processes.
• Mentor and coach the team members to achieve engineering excellence
• Work with architects to design robust, scalable and cost-effective data solutions

Education & Experience:

• 6 years in a Data Engineer role, with a degree in Computer Science, Statistics, Informatics, Information Systems, or a related field.
• Solid software engineering and DataSecOps background, including CICD tools expertise
• Strong ability to effectively translate complex technical concepts into simple terms.
• Experience with Databricks, including recent features available in the tool.
• Experience on Microsoft Azure analytics stack, as well as real-time analytics tools
• Advanced proficiency in SQL and familiarity with various databases.
• Understanding of data modelling techniques for analytics.
• Ability to perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Experience with data security operations, preferably on the Azure stack.
• Ability to troubleshoot and solve complex data-related issues.
• Awareness of data management frameworks (e.g., DAMA) and exposure to data quality and catalogue tools such as Collibra or Alation will be highly rewarded

API development and optimised strategies to serve real-time data will be highly regarded