Senior Data Engineers (NCS/Job/ 3773)

We are hiring Senior Data Engineers to lead large-scale AWS migration and data platform initiatives. This is a senior individual contributor role focused on migrating legacy file storage and SQL Server databases into AWS-native environments while ensuring data integrity, scalability, and operational reliability.

You will drive architecture decisions, build migration and reporting pipelines, and define data quality and cutover strategies in a Kubernetes and AWS-based ecosystem.

Key ResponsibilitiesData Migration

• Lead migration of:
• NAS/NFS/SMB file storage to Amazon S3
• SQL Server databases to Aurora PostgreSQL / RDS
• Handle large-scale file migrations (100+ GB files)
• Design incremental sync, reconciliation, checksum validation, rollback, and cutover strategies
• Build multi-hop data flows between S3, Kubernetes processing clusters, and analytics platforms

Data Engineering & Reporting

• Design and build AWS-based ETL/ELT pipelines
• Develop ingestion, transformation, and reporting workflows
• Enable analytics using Redshift, Athena, and Aurora
• Optimize pipeline performance, scalability, and cost

Data Quality & Reliability

• Build validation, reconciliation, and quarantine frameworks
• Implement monitoring, retry, and escalation workflows
• Support DR testing, performance tuning, and migration validation

Infrastructure & CI/CD

• Work within Terraform-managed AWS environments
• Contribute to GitLab CI/CD pipelines
• Support security and compliance initiatives across IAM, KMS, encryption, and secrets management

Required Skills

• 7+ years of Data Engineering experience
• Strong experience with:
• NAS/NFS/SMB → S3 migrations
• SQL Server → Aurora/PostgreSQL migrations
• Strong Python and SQL expertise
• Hands-on experience with:
• boto3
• SQLAlchemy
• ETL scripting
• Strong AWS knowledge:
• S3
• Aurora/RDS
• IAM
• KMS
• CloudWatch
• ECS/EKS
• Experience with Docker, Kubernetes, GitLab CI/CD, and Terraform
• Ability to independently drive technical and architectural decisions

Preferred Skills

• Spark, Airflow, or Kubernetes-based data pipelines
• Exposure to Go or Java microservices
• Pytest, Playwright, or K6 testing frameworks
• Blue/green cutover and rollback planning experience
• Experience supporting ML/data science workloads