AWS Data Engineer

Dear Consultant

I hope you are doing well.

My name is Omkar Nath Sharma and I represent Synchrony Systems Inc. We are currently supporting an implementation partner on a full-time opportunity. Based on your background your experience may align well with this role.

Position Details:

Note:

• This role is not open for C2C/C2H/1099 or any contract arrangements
• This opportunity is available for candidates who can work on a full-time W2 basis without sponsorship.

Job Description

Must Have Technical/Functional Skills:

• We are seeking a AWS Data Engineer
• Strong expertise in AWS services including:
• Lambda
• S3
• EventBridge
• Kinesis (Data Streams & Firehose)
• Glue (ETL Crawlers)
• Step Functions
• Amazon Connect
• Athena
• Macie
• Proficient in:
• Python
• PySpark
• Experience in:
• Glue Crawlers for schema discovery and cataloging
• Parquet-based data storage
• Building scalable data pipelines
• Strong understanding of event-driven architectures (Pub/Sub model)
• Hands-on experience with Terraform (Infrastructure as Code)
• Familiarity with CI/CD tools and automation pipelines
• Messaging platforms like Kafka and Amazon EventBridge
• Designing real-time data processing systems
• Using Glue Crawlers Athena for data lake architectures

Roles & Responsibilities:

• Develop a comprehensive plan for migrating near real-time fraud detection campaigns from on-premises systems to AWS.
• Design and implement event-driven architectures to process inbound dialer data (fraud events) using services such as Amazon EventBridge Kafka Kinesis Data Streams and Kinesis Firehose.
• Build and manage scalable data pipelines using AWS Glue (ETL jobs Crawlers) PySpark and Python for data ingestion transformation and processing.
• Configure and manage Glue Crawlers to automatically discover schemas and update the Data Catalog.
• Store and optimize data using Parquet format and enable analytics through Amazon Athena for efficient querying.
• Develop integrations between Customer Profiles and messaging platforms to automatically trigger profile updates and downstream processes.
• Implement automation to trigger fraud-related outbound calls based on updates in customer profiles.
• Design and orchestrate workflows using AWS Step Functions to manage complex processing pipelines.
• Provision and manage cloud infrastructure using Terraform (Infrastructure as Code).
• Optimize system architecture for scalability reliability cost-efficiency and ensure data integrity and security.
• Conduct end-to-end testing o f the entire framework to validate functionality performance and reliability.
• Deploy automate and manage resources using CI/CD pipelines.
• Continuously monitor system performance and implement optimizations post-deployment.
• Maintain detailed documentation of architecture workflows and operational processes.