Sr Data Engineer

Overview

Key responsibilities

• Design construct and maintain largescale data processing systems.
• Develop and optimize ETL processes for data extraction transformation and loading.
• Collaborate with data scientists and analysts to deploy scalable and efficient data models and algorithms.
• Implement data governance and security best practices to ensure data integrity and privacy.
• Monitor and troubleshoot data infrastructure and performance issues.
• Conduct code reviews and provide technical guidance to junior team members.
• Lead efforts in data architecture and database design to support business needs.
• Work with stakeholders to understand and address datarelated requirements and challenges.
• Identify opportunities for automation and optimization within the data ecosystem.
• Stay abreast of industry trends and emerging technologies in data engineering.
• Participate in the evaluation and selection of new tools and technologies.
• Create and maintain documentation for data processes systems and standards.
• Collaborate with DevOps and IT teams to ensure smooth deployment and operation of data systems.
• Support data governance initiatives and compliance with regulatory standards.
• Contribute to the continuous improvement of data engineering practices within the organization.

• Create the data objects as per the data model specifications.
• Build data pipelines for data ingestion curation and transformation as per specifications.
• Data ingestion (batch & real time) and transformation
• Ensures that all code configuration data and system documentation is accurate complete understandable and adheres to company standards
• Leverage CI/CD for code development and deployment.

aws,amazon glue,aws transfer family,s3,data engineering,transformation,glue,aws lambda,amazon redshift,amazon mwaa (airflow),redshift