Python Developer-F2F (RARR Job 5892)
Job Skills
Job Description
• Design and develop ETL/ELT pipelines to ingest data from various sources (APIs, flat files, databases) into PostgreSQL, Delta Lake, or other storage formats.
• Write modular and efficient Python code using libraries like pandas, numpy, sqlalchemy, and requests.
• Transform and model data using frameworks such as dbt (Data Build Tool), ensuring modularity, reusability, and maintainability.
• Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
• Write comprehensive data test cases (e.g., row counts, data type checks, uniqueness, foreign key integrity) to ensure data quality and consistency.
• Collaborate with analysts and stakeholders to understand data requirements and translate them into technical solutions.
• Maintain documentation for data pipelines, models, and transformation logic.
Required Skills
• Python Programming: Strong experience in writing production-grade Python scripts for data ingestion and transformation.
• Data Libraries: Proficiency in pandas, numpy, sqlalchemy, pyarrow, fastparquet, openpyxl, and requests.
• SQL: Solid understanding of SQL for querying, transforming, and optimizing datasets.
• Data Transformation: Hands-on experience with dbt or similar tools for transforming raw data into analytics-ready datasets.
• Pipeline Monitoring: Familiarity with monitoring and alerting tools (e.g., logging, custom alerts, Airflow/Dagster/Prefect dashboard monitoring).
• Data Testing: Experience writing test cases using dbt tests, pytest, great_expectations, or custom validation scripts.
• Database Knowledge: Good working knowledge of PostgreSQL and Delta Lake.
Preferred Skills (Bonus)
• Experience with workflow orchestrators like Airflow, Dagster, or Prefect.
• Exposure to cloud platforms (AWS/GCP/Azure) and storage services like S3, ADLS.
• Familiarity with version control systems like Git and CI/CD pipelines.
• Knowledge of performance optimization techniques in Python and SQL.
