Data Engineer (RARR Job 6479)

Job Summary: Data Engineer role is centered on strong ELT, data pipelines, and SQL capabilities, with exposure to Starburst-based virtual data layer as an added advantage. Design and build scalable data pipelines and lightweight ELT frameworks to support analytics, reporting, and data products. Work across structured and semi-structured data sources, ensuring high data quality, performance, and governance. Exposure to data virtualization (e.g., Starburst/Trino) will be an added advantage.

Experience: 6–10 years. Strong experience in data engineering and pipeline development.

Key Responsibilities:

• Design and implement robust ELT/ETL pipelines for batch and near real-time processing
• Build and manage data ingestion frameworks from diverse sources (databases, APIs, files, spreadsheets/EUCs)
• Develop and optimize data pipelines and transformations using SQL and distributed processing frameworks
• Ensure data quality, validation, and lineage tracking across pipelines
• Support data product development by delivering curated, reusable datasets
• Work with Data Modelers to align pipelines with enterprise data models
• Implement data governance, access controls, and compliance standards
• Leverage virtual data layers (Starburst/Trino) for federated access where applicable

Must-Have Skills:

• Strong experience in ELT/ETL, data pipelines, and data transformations
• Advanced SQL (query optimization, complex joins, performance tuning)
• Hands-on experience in Python / PySpark or equivalent
• Experience working with data warehouses / Lakehouse architectures
• Strong understanding of data ingestion patterns (batch + streaming basics)
• Knowledge of data quality, governance, and lineage concepts

Good-to-Have Skills:

• Exposure to Starburst / Trino / Presto or any data virtualization platform
• Understanding of data products / Data Mesh concepts
• Experience with cloud data platforms (AWS/Azure/GCP)