Lead Data Engineer

• Ability to work in a fastpaced startup mindset.
• Should be able to manage all aspects of data extraction transfer and load activities.
• Develop data pipelines that make data available across platforms.
• Should be comfortable in executing ETL (Extract Transform and Load) processes which include data ingestion data cleaning and curation into a data warehouse database or data platform.
• Work on various aspects of the AI/ML ecosystem data modeling data and ML pipelines.
• Work closely with DevOps and senior Architect to come up with scalable system and model architectures for enabling realtime and batch services.

• 5 years of experience as a data engineer or data scientist with a focus on data engineering and ETL jobs.
• Wellversed with the concept of Data warehousing Data Modelling and/or Data Analysis.
• Experience using & building pipelines and performing ETL with industrystandard best practices on Redshift (more than 2 years).
• Ability to troubleshoot and solve performance issues with data ingestion data processing & query execution on Redshift.
• Good understanding of orchestration tools like Airflow.
• Strong Python and SQL coding skills.
• Strong Experience in distributed systems like spark.
• Experience with AWS Data and ML Technologies (AWS Glue MWAA Data Pipeline EMR Athena Redshift Lambda etc.).
• Solid handson with various data extraction techniques like CDC or Time/batch based and the related tools (Debezium AWS DMS Kafka Connect etc.) for near realtime and batch data extraction.

data engineering,data analytics,data extraction,data loading,etl,data cleaning,data warehouse,database,data science,machine learning,data modeling,data analysis,amazon redshift,data processing,orchestration,airflow,python,sql,distributed systems,aws,spark,glue,athena,aws lambda,cdc,kafka