Job Summary:
This role involves designing building and maintaining scalable data pipelines and infrastructure using cloud platforms and modern data tools. The Senior Data Engineer will collaborate with cross-functional teams to understand data needs and integration requirements optimize SQL queries manage data orchestration and validation and ensure data quality. This position requires extensive experience with AWS services cloud data warehouses (like Snowflake) and data integration tools. The ideal candidate will be proficient in Python SQL and various data engineering best practices.
Location: Houston Texas United States
Responsibilities:
- Design build and maintain scalable ETL/ELT pipelines and infrastructure using cloud platforms and modern data tools.
- Collaborate with cross-functional teams (Data Architects Analysts Data Scientists) to understand data needs and integration requirements.
- Optimize and troubleshoot complex SQL queries for performance and reliability.
- Manage data orchestration validation and automation processes.
- Leverage AWS services (EC2 S3 Lambda SQS etc.) and tools like Snowflake Matillion Informatica Cloud Talend and AWS Glue.
- Ensure data quality and consistency through rigorous validation frameworks.
- Utilize GitHub for version control and continuous integration with ETL pipelines.
- Support issue resolution and continuous improvement of data operations and architecture.
Required Skills & Certifications:
- Bachelors degree in Computer Science Engineering or related field.
- Minimum of 8 years experience in data engineering ETL and data warehouse/lake development.
- Expertise in Python SQL Numpy Pandas and dynamic programming.
- Hands-on experience with AWS and cloud data warehouses (Snowflake BigQuery Redshift).
- Experience with data integration tools (Matillion Informatica Cloud Talend AWS Glue).
- Proficient in GitHub for code management and collaboration.
- Familiarity with data platforms and technologies such as Spark Hadoop Hive NoSQL streaming and APIs.
- Knowledge of Agile/Scrum development practices.
Preferred Skills & Certifications:
- None specified
Special Considerations:
- None specified
Scheduling:
- Not specified