Data Engineer

Brighton, MI - USA

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

Data Engineer

Brighton MA (Onsite)

Long Term

Key Responsibilities:

Design develop and implement end-to-end big data solutions across the enterprise.
Develop and maintain big data applications using Apache Spark Scala AWS Glue AWS Lambda SNS/SQS and CloudWatch.
Build and optimize data pipelines ensuring high performance scalability and reliability.
Collaborate with cross-functional teams to design and document integration and application technical designs.
Conduct peer reviews of functional and design documentation to maintain high-quality standards.
Develop configure and perform unit testing and code reviews to ensure coding best practices.
Troubleshoot and resolve complex issues during testing phases and identify root causes efficiently.
Perform performance testing and optimize system performance.
Manage and maintain SQL-based databases preferably Amazon Redshift.
Utilize Snowflake for data warehousing (experience in Snowflake is an added advantage).
Implement ETL/ELT processes and ensure data quality across various systems.
Work with Git repositories and manage CI/CD deployment pipelines for continuous integration and delivery.
Provide production support including troubleshooting and environment tuning.
Ensure adherence to best practices and technical standards throughout the project lifecycle.

Required Skills & Experience:

6 years of relevant experience in big data design and development.
Proficiency in Scala and/or Python for application development.
Strong expertise in Spark AWS Glue Lambda SNS/SQS and CloudWatch.
Hands-on experience with ETL/ELT frameworks and data integration.
Advanced knowledge of SQL with experience in Redshift preferred.
Familiarity with Snowflake and cloud-based data solutions is advantageous.
Experience with CI/CD processes Git and production support in large-scale environments.

Data Engineer Brighton MA (Onsite) Long Term Key Responsibilities: Design develop and implement end-to-end big data solutions across the enterprise. Develop and maintain big data applications using Apache Spark Scala AWS Glue AWS Lambda SNS/SQS and CloudWatch. Build and optimize data pipelines ...