Senior AWS PySpark Developer
Share
Job Location:
South San Francisco, CA - USA
Monthly Salary:
Not Disclosed
Posted on:
12 hours ago
Vacancies:
1 Vacancy
Job Summary
Role Senior AWS PySpark Developer
Location Hybrid South San Francisco CA
Location Hybrid South San Francisco CA
We are seeking an experienced Sr. AWS PySpark Developer with 8 -10 years of experience to design build and optimize our data pipelines and analytics architecture. The ideal candidate will have a strong background in data wrangling and analysis with a deep understanding of AWS data services.
Key Responsibilities:
Design build and optimize robust data pipelines and data architecture on the AWS cloud platform.
Wrangle explore and analyze large datasets to identify trends answer business questions and pinpoint areas for improvement.
Develop and maintain a next-generation analytics environment providing a self-service centralized platform for all data-centric activities.
Formulate and implement distributed algorithms for effective data processing and trend identification.
Configure and manage Identity and Access Management (IAM) on the AWS platform.
Collaborate with stakeholders to understand data requirements and deliver effective solutions.
Design build and optimize robust data pipelines and data architecture on the AWS cloud platform.
Wrangle explore and analyze large datasets to identify trends answer business questions and pinpoint areas for improvement.
Develop and maintain a next-generation analytics environment providing a self-service centralized platform for all data-centric activities.
Formulate and implement distributed algorithms for effective data processing and trend identification.
Configure and manage Identity and Access Management (IAM) on the AWS platform.
Collaborate with stakeholders to understand data requirements and deliver effective solutions.
Required Skills & Experience:
8-10 years of experience as a Data Engineer or Developer.
Proven experience building and optimizing data pipelines on AWS.
Proficiency in scripting with Python.
Strong working knowledge of:
o Big Data Tools: AWS Athena.
o Relational & NoSQL Databases: AWS Redshift and PostgreSQL.
o Data Pipeline Tools: AWS Glue AWS Data Pipeline or AWS Lake Formation.
o Container Orchestration: Kubernetes Docker Amazon ECR/ECS/EKS.
Experience with wrangling exploring and analyzing data.
Strong organizational and problem-solving skills.
8-10 years of experience as a Data Engineer or Developer.
Proven experience building and optimizing data pipelines on AWS.
Proficiency in scripting with Python.
Strong working knowledge of:
o Big Data Tools: AWS Athena.
o Relational & NoSQL Databases: AWS Redshift and PostgreSQL.
o Data Pipeline Tools: AWS Glue AWS Data Pipeline or AWS Lake Formation.
o Container Orchestration: Kubernetes Docker Amazon ECR/ECS/EKS.
Experience with wrangling exploring and analyzing data.
Strong organizational and problem-solving skills.
Preferred Skills:
Experience with machine learning tools (SageMaker TensorFlow).
Working knowledge of stream processing (Kinesis Spark-Streaming).
Experience with analytics and visualization tools (Tableau Power BI).
Knowledge of optimizing AWS Redshift performance.
Education
Bachelors or Masters Degree in Information Technology Computer Science or relevant field.
Experience with machine learning tools (SageMaker TensorFlow).
Working knowledge of stream processing (Kinesis Spark-Streaming).
Experience with analytics and visualization tools (Tableau Power BI).
Knowledge of optimizing AWS Redshift performance.
Education
Bachelors or Masters Degree in Information Technology Computer Science or relevant field.
