Python, Data Engineer, AWS
Share
Job Location:
Columbus, NE - USA
Monthly Salary:
Not Disclosed
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
Python Data Engineer AWS
Key responsibilities
- Develop and maintain data pipelines Build test and maintain ETL (Extract Transform Load) processes to move and transform data.
- Design and manage data architecture Construct and manage data infrastructure including data warehouses data lakes and databases.
- Automate data processes Use Python to automate tasks and build data processing applications.
- Integrate data sources Connect internal and external data sources and APIs and ensure system compatibility.
- Ensure data quality and security Implement data quality checks security controls and access management policies.
- Collaborate with stakeholders Work with data scientists analysts and other teams to understand requirements and deliver solutions.
- Optimize systems Monitor system performance troubleshoot issues and implement optimizations for reliability and efficiency.
Required skills and qualifications
- Programming Proficiency in Python is a must. SQL and other languages like Java or Scala are also valuable.
- Databases Strong knowledge of database management systems (relational and NoSQL) and data modeling.
- Big Data Technologies Experience with big data frameworks such as Apache Spark and Hadoop.
- Cloud Platforms Familiarity with cloud services is often required such as AWS GCP or Azure.
- Version Control Experience with version control systems like Git.
- Problem-Solving Strong analytical and problem-solving skills.
- Communication Excellent communication skills to collaborate with technical and non-technical teams.
- Education A bachelors degree in Computer Science Engineering or a related field is typically required.
Key responsibilities
- Develop and maintain data pipelines Build test and maintain ETL (Extract Transform Load) processes to move and transform data.
- Design and manage data architecture Construct and manage data infrastructure including data warehouses data lakes and databases.
- Automate data processes Use Python to automate tasks and build data processing applications.
- Integrate data sources Connect internal and external data sources and APIs and ensure system compatibility.
- Ensure data quality and security Implement data quality checks security controls and access management policies.
- Collaborate with stakeholders Work with data scientists analysts and other teams to understand requirements and deliver solutions.
- Optimize systems Monitor system performance troubleshoot issues and implement optimizations for reliability and efficiency.
Required skills and qualifications
- Programming Proficiency in Python is a must. SQL and other languages like Java or Scala are also valuable.
- Databases Strong knowledge of database management systems (relational and NoSQL) and data modeling.
- Big Data Technologies Experience with big data frameworks such as Apache Spark and Hadoop.
- Cloud Platforms Familiarity with cloud services is often required such as AWS GCP or Azure.
- Version Control Experience with version control systems like Git.
- Problem-Solving Strong analytical and problem-solving skills.
- Communication Excellent communication skills to collaborate with technical and non-technical teams.
- Education A bachelors degree in Computer Science Engineering or a related field is typically required.
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
