Data Engineer (Python & PySpark Focus)
Share
Job Location:
Westlake, OH - USA
Monthly Salary:
Not Disclosed
Posted on:
22 hours ago
Vacancies:
1 Vacancy
Job Summary
Data Engineer (Python & PySpark Focus)
We are seeking a highly motivated and experienced Data Engineer to join our team focusing on building optimizing and deploying robust scalable data solutions. The ideal candidate will possess deep expertise in Python and PySpark to drive complex data transformations and support high-volume performance-critical simulation initiatives.
Key Responsibilities
* Design build and maintain high-performance ETL/ELT data pipelines using Python and PySpark.
* Apply expertise in Pythons data analysis libraries including Pandas and NumPy to perform complex data manipulation cleansing and transformation.
* Develop and manage data processing jobs leveraging PySpark for distributed computing across large-scale datasets.
* Implement DevOps practices and tooling (e.g. Docker Jenkins Terraform CloudFormation) for the automated deployment and orchestration of Python applications and data pipelines.
* Collaborate with data scientists and analysts to ensure data quality availability and consistency for advanced modeling and reporting.
* Utilize AWS or other cloud services (e.g. S3 Glue EMR Snowflake) to architect and maintain cloud-based data ecosystems.
* Write and optimize complex SQL queries for data extraction integrity checks and performance tuning.
Required Technical Skills
* 5 years of experience in Data Engineering or a related technical field.
* Expert-level proficiency in Python including a strong command of core concepts and specialized data libraries (Pandas NumPy).
* Solid hands-on experience with PySpark for building scalable data workflows.
* Strong background in DevOps principles and tools for deploying Python-based data applications (e.g. containerization CI/CD).
* Experience with cloud platforms (AWS strongly preferred) and associated data services (e.g. AWS Glue S3 Lambda Snowflake).
* Advanced knowledge of SQL and experience with modern data warehousing and database performance tuning.
* Familiarity with distributed data processing technologies (e.g. Apache Spark Hadoop).
Required Skills :
Basic Qualification :
Additional Skills :
Background Check : No
Drug Screen : No
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
