Data Engineer (NCS/Job/ 2397)

Data Engineer

We are looking for a Data Engineer with strong hands-on experience in SQL, Python, and PySpark to design, build, and optimize scalable data pipelines. The ideal candidate should be comfortable working on big data platforms and cloud-based data solutions using Databricks and AWS.

Primary Skills (Mandatory)

• SQL – Advanced querying, performance tuning, data transformation
• Python – Data processing, scripting, ETL development
• PySpark – Large-scale data processing and optimization
• Databricks – Notebook development, Spark job orchestration, Delta Lake

Secondary Skills (Good to Have)

• AWS – S3, EC2, EMR, Glue, Lambda (AWS exposure is a plus)
• Experience with data warehousing concepts and ETL frameworks

🚫 Note: Azure experience is not required

Key Responsibilities

• Develop and maintain scalable ETL/ELT pipelines using PySpark and SQL
• Build high-performance data solutions on Databricks
• Optimize data processing workflows for large datasets
• Collaborate with analysts and business teams to deliver reliable datasets
• Ensure data quality, consistency, and performance
• Work with AWS services for data storage and computation (if applicable)

Required Experience

• Strong hands-on experience in SQL, Python, PySpark
• Proven experience working with Databricks
• Experience in big data processing frameworks
• Understanding of distributed systems and data engineering best practices

Nice to Have

• Experience with AWS-based data architectures
• Knowledge of Delta Lake, data modeling, and performance tuning
• Exposure to CI/CD pipelines for data workflows