Senior Data Engineer

Job Description

We are looking for a highly skilled Senior Data Engineer with 10 years of experience in designing developing and maintaining scalable data platforms and enterprise-grade data pipelines. The ideal candidate should possess strong expertise in big data technologies cloud platforms distributed processing systems and modern data engineering practices.

As a Senior Data Engineer you will be responsible for building robust ETL/ELT frameworks optimizing large-scale data workflows enabling real-time and batch processing and supporting analytics and AI-driven initiatives across the organization.

You will collaborate with Data Architects Data Scientists Analysts Product Teams and DevOps teams to deliver scalable reliable and secure data solutions.

Roles & Responsibilities

• Design and develop scalable ETL/ELT pipelines for structured and unstructured data
• Build and maintain enterprise data lakes data warehouses and lakehouse architectures
• Develop high-performance batch and streaming data processing systems
• Optimize Spark jobs SQL queries and cloud infrastructure performance
• Implement data quality monitoring observability and governance solutions
• Integrate data from multiple internal and external sources
• Collaborate with cross-functional teams to understand business and analytics requirements
• Ensure platform scalability reliability security and fault tolerance
• Lead technical discussions code reviews and mentoring of junior engineers
• Support CI/CD automation and DevOps best practices
• Troubleshoot production issues and perform root cause analysis
• Contribute to architectural decisions and technology evaluations

Required Skills

• Strong expertise in Python and SQL
• Hands-on experience with Apache Spark / PySpark
• Experience with Apache Kafka or streaming technologies
• Strong understanding of data modeling and database design
• Experience with ETL/ELT frameworks and orchestration tools
• Expertise in GCP Snowflake Airflow Hadoop ecosystem
• Knowledge of distributed systems and scalable architectures

Preferred Skills

• Experience with Delta Lake Apache Iceberg or Hudi
• Exposure to real-time analytics and event-driven architecture
• Docker and Kubernetes experience
• Infrastructure as Code (Terraform)
• Experience supporting AI/ML or LLM data pipelines
• Knowledge of data governance and observability tool

Qualifications

• Bachelors or Masters degree in Computer Science Engineering Information Technology or related field