Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Responsibilities:
- Design develop and maintain scalable data pipelines for the efficient collection storage and processing of large datasets.
- Develops and design best practices around data collection and maintain orchestration of data flows for batch near-real time data.
- Develop and design best practices of CDC and SCD for existing systems to maintain data consistency
- Participates in evaluation and deployment of big data tools relevant to the development.
- Ability to identify different platforms techniques and methodology used in big data platforms and demonstrate for various system in the organization.
- Optimize and tune data systems for performance and reliability.
- Develop and implement data models and schemas to support business intelligence and analytics needs.
- Monitor and troubleshoot data pipelines to ensure smooth and efficient operation.
- Stay up-to-date with the latest advancements in data engineering and incorporate them into our data infrastructure.
- Document data engineering architectures and follow best practices.
Qualifications:
- Bachelors or Masters degree in Computer Science Engineering Data Science or a related field.
- 5 years of experience as a Data Engineer Data Architect Big Data Administrator or similar role.
- Strong programming skills in Python.
- Experience with UNIX system administration using command line.
- Experience with data processing frameworks such as Airflow Airbyte DBT Hadoop Spark Hive and Kafka
- Proficiency in SQL and experience with relational databases.
- Knowledge of data modeling ETL processes and data warehousing solutions.
- Knowledge of Structuring data in various schemas suitable for a data lake warehouse or marts.
- Experience in implementing scalable reliable and efficient data storage solutions with PostgreSQL MinIO.
- Experience in enhancing the performance of big data applications including optimizing system operations.
- Excellent problem-solving skills and the ability to work independently or as part of a team.
- Strong communication skills and the ability to explain complex technical concepts to non-technical audiences.
Employment Type
Full Time
