Senior Data Engineer

Job Summary

The Senior Data Engineer at Devsinc is responsible for designing, building, and maintaining scalable data pipelines and processing systems that enable analytics, data products, and AI/ML capabilities. The role involves collaboration with multiple teams to deliver production-ready datasets and high-performance data solutions.

Responsibilities

• Design and maintain scalable ETL/ELT pipelines for data ingestion, transformation, validation, and delivery.
• Build, schedule, and manage production-grade data workflows using Apache Airflow.
• Create distributed data-processing jobs with Apache Spark.
• Develop efficient, reusable Python code for data processing and pipeline automation.
• Optimize complex SQL queries for data transformation, analysis, and quality checks.
• Design pipelines to handle large-scale structured, semi-structured, and unstructured data.
• Implement Redis for caching and high-performance data access.
• Integrate data from APIs, databases, files, third-party providers, and other sources.

Requirements

• Bachelor’s degree in Computer Science, Software Engineering, Data Science, or related field.
• 4-6 years of professional experience in Data Engineering or a related role.
• Strong experience designing and implementing production-grade ETL/ELT pipelines.
• Proficiency in Python for data processing and automation.
• Advanced SQL skills and understanding of relational databases.
• Experience with Apache Airflow for workflow orchestration.
• Hands-on experience with Apache Spark and distributed data processing.
• Knowledge of data modelling, transformation patterns, and pipeline architecture.
• Experience with Redis, caching strategies, and high-performance data access.
• Ability to process large datasets and optimize pipeline and query performance.
• Understanding of data quality, validation, monitoring, and pipeline reliability.
• Familiarity with Linux, Git, Docker/containers, and modern software engineering practices.
• Strong analytical, troubleshooting, communication, and collaboration skills.

Nice to haves

• Experience with cloud data platforms and object storage such as AWS, Azure, or GCP.
• Experience with PostgreSQL, data warehouses, or analytical databases.
• Experience processing geospatial or large-scale location-based data.
• Familiarity with DuckDB, Apache Sedona, Trino, Presto, or similar technologies.
• Experience handling high-volume event, mobility, transactional, or geospatial data.
• Knowledge of CI/CD pipelines and infrastructure-as-code practices.
• Experience supporting data products, analytics platforms, or AI/ML pipelines.

We refresh listings regularly, but some roles close early on the source platform.