Amazon S3 Engineer

Charlotte, VT - USA

Monthly Salary: Not Disclosed

Posted on: 13 hours ago

Vacancies: 1 Vacancy

Job Summary

Location : CharlotteNC;PlanoTX;

Must Have Technical/Functional Skills

Primary Skil: Amazon Data Engineer

Secondary: AWS Data Engineer Amazon S3 Shell Scripting Autosys

Experience: Minimum 10 years

Roles & Responsibilities

Key Responsibilities:

Design develop and execute Data Pipelines and test cases to ensure data integrity and quality.

Develop implement and optimize data pipelines that integrate Amazon S3 for scalable data storage retrieval and processing within ETL workflows.

Leverage Amazon S3 for data storage retrieval and management within ETL workflows including the ability to write scripts for data transfer between S3 and other systems.

Utilize Amazon S3s advanced features such as versioning lifecycle policies access controls and server-side encryption to ensure secure and efficient data management.

Write maintain and troubleshoot scripts or code (using PySpark Shell or similar languages) to automate data movement between Amazon S3 and other platforms ensuring high performance and reliability.

Collaborate with cross-functional teams to troubleshoot and resolve data-related issues utilizing Amazon S3 features such as versioning lifecycle policies and access management.

Document ETL processes maintain technical documentation and ensure best practices are followed for data stored in Amazon S3 environments.

Familiarity with Hadoop or Spark is often preferred.

Validate HiveQL HDFS file structures and data processing within the Hadoop cluster.

Strong analytical and troubleshooting skills.

Excellent communication for collaborating with developers and stakeholders.

Knowledge in Metadata dependent ETL process and batch/job framework

Tools & Skills:

Amazon S3: Data storage retrieval and management; scripting for ETL data transfer; advanced features including versioning lifecycle policies access controls and server-side encryption; automation of data movement using Python Shell or similar languages; troubleshooting and collaboration for data-related issues; documentation and best practices for ETL processes.

PySpark

SQL

Oracle

Domain: Banking knowledge Payments knowledge preferred.

Environment: Cloudera Platform

Concept: Cloud Storage Amazon S3 AWS Data Warehousing Data Transformation ETL/ELT Data Quality.

Salary :00/Per Annum

Location : CharlotteNC;PlanoTX; Must Have Technical/Functional Skills Primary Skil: Amazon Data Engineer Secondary: AWS Data Engineer Amazon S3 Shell Scripting Autosys Experience: Minimum 10 years Roles & Responsibilities Key Responsibilities: Design develop and execute Data Pipelines...