Big Data Engineer with Spark Scala(NCS/Job/ 493)

Responsibilities:

• Design, develop, test, and deploy big data solutions using Spark Streaming.

• Collaborate with cross-functional teams to gather requirements and deliver high-quality results.

• Develop scalable and efficient algorithms for processing large datasets using Scala programming language.

• Troubleshoot issues related to data processing pipelines and optimize performance for better efficiency.

Qualifications:

• 5+  years of experience in Big Data development with expertise in Spark Streaming.

• Strong knowledge of Scala and Spark.

• Experience with Python, SparkSQL, and Kubernetes for container orchestration and management.

• Familiarity with Apache Hudi and Apache Iceberg for efficient data storage and retrieval.

• Strong problem-solving skills to troubleshoot and resolve complex data processing issues.