This is a remote position.
We are seeking a Data Engineer (ETL and Spark) to join our team. You are responsible for designing building and maintaining the systems that collect store and process data. You have to ensure data is accessible reliable and secure for analysis by data scientists and analysts. This includes building data pipelines managing databases and implementing data quality checks.
Requirements
- Experience developing ETL and ELT pipelines.
- Experience with Spark GraphDB Azure Databricks.
Expertise in Data Partitioning. - Experience with Data Conflation.
- Experience developing Python Scripts.
- Experience training LLMs with structured and unstructured data sets.
Benefits
- Work Location: Remote
- 5 days working
Experience developing ETL and ELT pipelines. Experience with Spark, GraphDB, Azure Databricks. Expertise in Data Partitioning. Experience with Data Conflation. Experience developing Python Scripts. Experience training LLMs with structured and unstructured data sets.