Lead Data Quality & Automation Engineer

San Francisco, CA - USA

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

Who We Are

Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to customers.

Job Description

Job Title : Lead Data Quality & Automation Engineer

Job Type : W2/C2C

Experience : 8-14 years

Location : San Francisco California

Responsibilities :

5 years of professional experience in software development with at least 2 years in a lead or senior engineering role.
Strong proficiency in Python programming and experience building data processing applications.
Hands-on expertise with Apache Spark and Hadoop for big data processing.
Solid experience with workflow orchestration tools like Apache Airflow.
Experience designing and managing data ingestion and integration pipelines with Apache NiFi.
Understanding on Data Quality automation CI/CD Jenkins Oracle Power BI Splunk
Deep understanding of data quality concepts data validation techniques and distributed data systems.
Experience with cloud platforms (AWS GCP or Azure) is a plus.
Lead the design and implementation of data quality frameworks and automated validation pipelines using Python Apache Spark and Hadoop ecosystem tools.
Develop deploy and maintain scalable ETL/ELT workflows using Apache Airflow and Apache NiFi to ensure seamless data ingestion transformation and quality checks.
Collaborate with cross-functional teams to understand data quality requirements and translate them into technical solutions.
Define and enforce data quality standards rules and monitoring processes.
Perform root cause analysis on data quality issues and implement effective fixes and enhancements.
Mentor and guide junior engineers conducting code reviews and fostering best practices.
Continuously evaluate and integrate new tools and technologies to enhance data quality capabilities.
Ensure high code quality performance and reliability in all data processing pipelines.
Create comprehensive documentation and reports on data quality metrics and system architecture.

Qualification:

Bachelors degree or equivalent combination of education and experience.

Who We Are Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to customers. Job Description Job Title : Lead Data Quality & Automation Engineer Job Type : W2/C2C Experience : 8-14 years Location : San F...

Who We Are

Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to customers.

Job Description

Job Title : Lead Data Quality & Automation Engineer

Job Type : W2/C2C

Experience : 8-14 years

Location : San Francisco California

Responsibilities :

5 years of professional experience in software development with at least 2 years in a lead or senior engineering role.
Strong proficiency in Python programming and experience building data processing applications.
Hands-on expertise with Apache Spark and Hadoop for big data processing.
Solid experience with workflow orchestration tools like Apache Airflow.
Experience designing and managing data ingestion and integration pipelines with Apache NiFi.
Understanding on Data Quality automation CI/CD Jenkins Oracle Power BI Splunk
Deep understanding of data quality concepts data validation techniques and distributed data systems.
Experience with cloud platforms (AWS GCP or Azure) is a plus.
Lead the design and implementation of data quality frameworks and automated validation pipelines using Python Apache Spark and Hadoop ecosystem tools.
Develop deploy and maintain scalable ETL/ELT workflows using Apache Airflow and Apache NiFi to ensure seamless data ingestion transformation and quality checks.
Collaborate with cross-functional teams to understand data quality requirements and translate them into technical solutions.
Define and enforce data quality standards rules and monitoring processes.
Perform root cause analysis on data quality issues and implement effective fixes and enhancements.
Mentor and guide junior engineers conducting code reviews and fostering best practices.
Continuously evaluate and integrate new tools and technologies to enhance data quality capabilities.
Ensure high code quality performance and reliability in all data processing pipelines.
Create comprehensive documentation and reports on data quality metrics and system architecture.