Senior Data Engineer
Senior Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
Senior Data Engineer
data Engineer that is good with Pyspark and SQL. On prem work not cloud but GCP is a nice to have
Hybrid twice a week in Hoboken NJ
Needs:
12 IT Experience
Pyspark
Python
SQL
GCP Nice to have
Good CLients
Any work authorization
6 month contract
You can subvend
Job Description:
About Team:
Everyone has data but the sheer volume of data at Walmart can be limitless. In the Data Engineering team we help Walmart manage this data by building pipelines and data lakes to prepare big data for analysis and unlocking actionable insights in realtime. We also use crossdepartmental data and machine learning to build a holistic view of true profitability saving millions of dollars across item categories and geographies while assisting our leadership in making better decisions faster.
What youll do:
Collaborate with crossfunctional teams to understand data requirements and design data solutions that meet business needs
Develop and maintain data pipelines and ETL processes using Spark and Scala
Design build and optimize data models and data architecture for efficient data processing and storage
Implement data integration and data transformation workflows to ensure data quality and consistency
Monitor and troubleshoot data pipelines to ensure data availability and reliability
Conduct performance tuning and optimization of data processing systems for improved efficiency and scalability
Work closely with data scientists and analysts to provide them with the necessary data sets and tools for analysis and reporting
Stay uptodate with the latest industry trends and technologies in data engineering and apply them to enhance the data infrastructure
What youll bring:
Proven working experience as a Data Engineer with a minimum of 3 years in the field.
Strong programming skills in Scala and experience with Spark for data processing and analytics
Familiarity with Google Cloud Platform (GCP) services such as BigQuery GCS Dataproc Pub/Sub etc.
Experience with data modeling data integration and ETL processes
Strong knowledge of SQL and database systems
Understanding of data warehousing concepts and best practices
Proficiency in working with largescale data sets and distributed computing frameworks
Strong problemsolving and analytical skills
Excellent communication and teamwork abilities
Pyspark,SQL,Python,Spark
Employment Type
Full Time
