Bigdata Tech Lead
Job Location:
Chicago, IL - USA
Monthly Salary:
Posted:
Application Deadline:
Vacancies:
Job Summary
10 years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws.
The ideal candidate will possess deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS.
Hands-on experience with Cloudera Hadoop stack (HDFS Hive Spark Oozie Python)
Proficiency in writing maintaining and troubleshooting shell scripts (Bash or similar) for automation and operational support.
Migration to AWS will need -
Analyze existing Cloudera Hadoop architecture workflows and data pipelines.
Develop a comprehensive migration strategy to transfer data processing logic and applications to AWS services.
Design target architecture using AWS big data services such as Amazon EMR Airflow Glue Lambda.
Evaluate and select appropriate AWS tools for data ingestion storage processing and analytics.
Oversee the migration process including data transfer application re-platforming and validation.
Ensure the new architecture meets performance scalability reliability and security requirements.
Collaborate with developers data engineers DevOps and stakeholders to ensure a seamless migration.
Create detailed technical documentation and migration runbooks.
Mentor team members on AWS big data best practices.
Strong understanding of data security compliance and governance in the cloud.
Proven track record of migrating or modernizing big data workloads to AWS.
Experience with cloud automation and Infrastructure as Code (e.g. Terraform).
AWS certifications (e.g. AWS Certified Solutions Architect AWS Certified Big Data - Specialty) are highly desirable.
10 years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws.
The ideal candidate will possess deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS.
Hands-on experience with Cloudera Hadoop stack (HDFS Hive Spark Oozie Python)
Proficiency in writing maintaining and troubleshooting shell scripts (Bash or similar) for automation and operational support.
Migration to AWS will need -
Analyze existing Cloudera Hadoop architecture workflows and data pipelines.
Develop a comprehensive migration strategy to transfer data processing logic and applications to AWS services.
Design target architecture using AWS big data services such as Amazon EMR Airflow Glue Lambda.
Evaluate and select appropriate AWS tools for data ingestion storage processing and analytics.
Oversee the migration process including data transfer application re-platforming and validation.
Ensure the new architecture meets performance scalability reliability and security requirements.
Collaborate with developers data engineers DevOps and stakeholders to ensure a seamless migration.
Create detailed technical documentation and migration runbooks.
Mentor team members on AWS big data best practices.
Strong understanding of data security compliance and governance in the cloud.
Proven track record of migrating or modernizing big data workloads to AWS.
Experience with cloud automation and Infrastructure as Code (e.g. Terraform).
AWS certifications (e.g. AWS Certified Solutions Architect AWS Certified Big Data - Specialty) are highly desirable.
Required Skills:
BigdataHadoopClouderaHDFSAmazon EMRHiveLamda
