Job Summary Data Engineer: We are seeking a skilled midlevel Data Engineer to join our team and focus on quality assurance quality checking and ETL processes. The successful candidate will be responsible for ensuring the integrity and accuracy of data transferred from a shared file transfer service to an S3 bucket and subsequently into and through our Snowflake data platform. This data will be utilized by downstream applications and reporting systems. These applications and the corresponding consumed data are critical to business process .
Key Responsibilities:
Quality Assurance & Quality Checking: Implement and maintain data quality to ensure the accuracy and reliability of data throughout the ETL process.
ETL Processes: Design develop and optimize ETL workflows to efficiently transfer data from file transfer services to S3 buckets and Snowflake.
Data Integration: Ensure seamless data integration into data platform enabling efficient consumption by downstream applications and reporting tools.
Data Quality Management: Address data quality challenges including inconsistencies in source data that do not meet ingestion requirements which can lead to load failures or data backouts.
Collaboration: Work closely with business owners data analysts business intelligence teams and other stakeholders to understand data requirements and deliver highquality data solutions.
Monitoring & Troubleshooting: To preserve data flow and integrity monitor pipelines identify issues and implement solutions.
Qualifications (Knowledge/Skills/Abilities):
Expert analytical and problemsolving skills.
Demonstrated excellence in communication and collaboration abilities.
Proven capacity to define success deliver and operate in an agile setting.
Skills:
Skill
Required/Desired
Amount
of Experience
Demonstrated midlevel experience in data engineering with an emphasis on data quality assurance and ETL processes.
5
Years
Expertise in Python PyPI and SQL
5
Years
Demonstrate a strong understanding of cybersecurity principles related to code development DevOps data access and fundamental cybersecurity.
5
Years
Understanding of fundamental publiccloud capabilities.
5
Years
Proven capacity to comprehend business needs and convert them into technical requirements.
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.