Machine Learning Software Engineer - Contractor
Machine Learning Software Engineer - Contractor
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Project overview: Supporting AML migrate their pricing airflow data jobs from hive on emr to spark on kubernetes
- Onboard onto AML systems and gain access to: AWS Datadog Windfarm Airflow AML Airflow GitHub Google Docs Slack
Assist with the conversion refactoring and migration of AML Pricing Airflow DAGs and tasks from Hive running on AWS EMR to Spark running on AWS EKS
This would include performing data validation to ensure the new DAG produces an output equivalent to the original one
Collaborate with David Shu from the DIGIT team as well as Robert Gay towards the overall objective of retiring the AML-hosted Pricing Airflow cluster
As we learn more about the nature of modernizing Hive to Spark as well as organizing our jobs to efficiently perform data validation a change in direction may be necessary; this will be confirmed by Robert Gay.
Redfin provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination based on race color religion age sex national origin disability status genetics protected veteran status sexual orientation gender identity or expression and any other characteristic protected by applicable federal state or local law. If you need accommodation in the application or recruitment process because of a disability or special need please contact.
Redfin accepts applications on an ongoing basis.
Required Experience:
Contract
Employment Type
Full-Time
