Lead Data Engineer
Lead Data Engineer
Posted on :
16-05-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Responsibilities
- Data Architecture & Design:
- Lead the design and implementation of robust data architectures that support data warehousing (DWH) data integration and analytics platforms.
- Develop and maintain ETL (Extract Transform Load) pipelines to ensure the efficient processing of large datasets.
- ETL Development:
- Design develop and optimize ETL processes using tools like Informatica Power Center Intelligent Data Management Cloud (IDMC) or custom Python scripts.
- Implement data transformation and cleansing processes to ensure data quality and consistency across the enterprise.
- Data Warehouse Development:
- Build and maintain scalable data warehouse solutions using Snowflake Databricks Redshift or similar technologies.
- Ensure efficient storage retrieval and processing of structured and semistructured data.
- Big Data & Cloud Technologies:
- Utilize AWS Glue and PySpark for largescale data processing and transformation.
- Implement and manage data pipelines using Apache Airflow for orchestration and scheduling.
- Leverage cloud platforms (AWS GCP) for data storage processing and analytics.
- Data Management & Governance:
- Establish and enforce data governance and security best practices.
- Ensure data integrity accuracy and availability across all data platforms.
- Implement monitoring and alerting systems to ensure data pipeline reliability.
- Collaboration & Leadership:
- Work closely with data Stewards analysts and business stakeholders to understand data requirements and deliver solutions that meet business needs.
- Mentor and guide junior data engineers fostering a culture of continuous learning and development within the team.
- Lead datarelated projects from inception to delivery ensuring alignment with business objectives and timelines.
- Database Management:
- Design and manage relational databases (RDBMS) to support transactional and analytical workloads.
- Optimize SQL queries for performance and scalability across various database platforms.
Required Skills & Qualifications:
- Education: Bachelor s or Master s degree in Computer Science Information Systems Engineering or a related field.
- Experience:
- Minimum of 7 years of experience in data engineering ETL and data warehouse development.
- Proven experience with ETL tools like Informatica Power Center or IDMC.
- Experience in DBT is mandatory
- Strong proficiency in Python and PySpark for data processing.
- Experience with cloudbased data platforms such as AWS Glue Snowflake Databricks or Redshift.
- Handson experience with SQL and RDBMS platforms (e.g. Oracle MySQL PostgreSQL).
- Familiarity with data orchestration tools like Apache Airflow.
- Technical Skills:
- Advanced knowledge of data warehousing concepts and best practices.
- Working experience on DBT is must.
- Strong understanding of data modeling schema design and data governance.
- Proficiency in designing and implementing scalable ETL pipelines.
- Experience with cloud infrastructure (AWS GCP) for data storage and processing.
- Soft Skills:
- Excellent communication and collaboration skills.
- Ability to lead and mentor a team of engineers.
- Strong problemsolving and analytical thinking abilities.
Ability to manage multiple projects and prioritize tasks effectivelyBe specific when describing each of the responsibilities. Use genderneutral inclusive language.
Example: Determine and develop user requirements for systems in production to ensure maximum usability
Qualifications
Preferred Qualifications:
- Experience with machine learning workflows and data science tools.
- Certification in AWS Snowflake Databricks or relevant data engineering technologies.
- Experience with Agile methodologies and DevOps practices.
Some qualifications you may want to include are Skills Education Experience or Certifications.
Example: Excellent verbal and written communication skills
data governance,pyspark,informatica power center,python,databricks,data warehouse,rdbms,data integration,snowflake,aws,etl,apache airflow,idmc,data warehousing,data architecture,sql,dbt
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
