Spark Developer + AWS EMR(RARR Job 4677)

We are seeking a skilled Spark Developer with extensive experience in Apache Spark and AWS EMR (Elastic MapReduce). The ideal candidate will be responsible for designing, developing, and implementing big data solutions using Spark on AWS, ensuring the performance, scalability, and reliability of data processing pipelines.

Key Responsibilities:

• Design and develop big data solutions using Apache Spark on AWS EMR for large-scale data processing.
• Create and manage data pipelines to efficiently move and transform large datasets.
• Implement data ingestion processes using Spark for various data formats such as CSV, JSON, Parquet, and ORC.
• Collaborate with data engineers, data scientists, and stakeholders to understand requirements and deliver high-performance solutions.
• Optimize existing Spark jobs for better performance and scalability on AWS EMR.
• Leverage other AWS services such as S3, Lambda, Glue, and Redshift for seamless integration with the data pipeline.
• Troubleshoot, debug, and resolve issues related to Spark processing and AWS EMR cluster management.
• Monitor and maintain AWS EMR clusters to ensure reliability and cost efficiency.
• Ensure adherence to best practices for security and data governance in AWS.