Bigdata Developer

Mandatory Skills: Spark Scala EMR and AWS

Requirements: Bachelors degree preferably Computer Science Engineering or other quantitative fields

6 years of related experience in designing and implementing enterprise applications using big data

5 years of experience in a senior level engineering role mentoring other engineers which includes engineering best practices unblocking code reviews unit testing managing deployments technical guidance system design etc.

5 years of experience working with largescale data and developing SQL queries

Advanced experience with scripting languages (e.g. Python Bash node.js) and programming languages (e.g. SQL Java Scala) to design build and maintain complex data processing ETL (Extract Transform Load) tasks and AWS automation.

5 years of handson experience with AWS cloud services such as Apache Spark with Scala AWS EMR Airflow RedShift

4 years of experience with Restful APIs and web services

Excellent communication skills including discussions of technical concepts soft skills conducting peerprogramming sessions and explaining development concepts Indepth understanding of Spark framework scripting languages (e.g. Python Bash node.js) and programming languages (e.g. SQL Java Scala) to design build and maintain complex data processing ETL (Extract Transform Load) tasks and AWS automation. A firm understanding of unit testing. Possess indepth knowledge of AWS services and data engineering tools to diagnose and solve complex issues efficiently specifically AWS EMR for big data processing.

Indepth understanding of GIT or other distributed version control systems.

Excellent communication. Essential to performing at maximum efficiency within the team.

Collaborative attitude. This role is part of a larger more dynamic team that nurtures collaboration. Strong technical process and problemsolving proficiency.

Thorough understanding of complex data structures and transformations such as nested JSON XML Avro or Parquet into structured formats suitable for analysis and large datasets (100 gigs or more). Advance skills in data cleansing deduplication and quality validation to maintain highquality data in the transformed datasets. Experience in the healthcare industry or another highly regulated field is a plus

Scala,AWS,EMR,PYSPARK,SPARK