ETL Developer No C2C Onsite Role No Relo
Share
Job Location:
Jersey, NJ - USA
Monthly Salary:
Not Disclosed
Posted on:
27 days ago
Vacancies:
1 Vacancy
Job Summary
Position Summary
We are seeking a highly skilled Senior Data Engineer to design develop and manage robust Extract Transform Load (ETL) and Extract Load Transform (ELT) pipelines entirely within the Amazon Web Services (AWS) ecosystem. This role is crucial for integrating diverse financial data sources into performant data warehouses and data lakes ensuring scalability security and data quality. Deep domain experience within the Banking/Financial Services industry is mandatory.
Core Responsibilities
- Pipeline Architecture: Architect and implement robust scalable ETL/ELT pipelines leveraging native AWS services for optimal data ingestion and processing.
- Data Integration: Integrate data from a variety of sources including external APIs transactional databases and flat files into centralized AWS-based data platforms.
- Transformation Development: Develop complex data transformation logic utilizing PySpark Python and SQL primarily executed within AWS Glue and AWS Lambda environments.
- Orchestration & Monitoring: Establish monitor and maintain workflow orchestration using tools such as AWS Step Functions Glue Workflows or Apache Airflow on Amazon MWAA.
- Data Governance & Quality: Ensure stringent data quality consistency and lineage tracking utilizing services like the AWS Glue Data Catalog and AWS Lake Formation.
- Performance Optimization: Proactively identify and execute optimizations for ETL performance and cost-efficiency through techniques like partitioning parallelism and resource tuning.
- Security & Compliance: Implement and enforce security best practices including data encryption management of IAM roles and precise VPC configurations.
- Collaboration & Documentation: Partner closely with data engineers analysts and DevOps teams to support critical analytics and reporting needs. Maintain comprehensive documentation of ETL processes data flows and architectural diagrams.
Required Technical Stack
The candidate must demonstrate expert proficiency in developing scalable data solutions using the following AWS services and technologies:
- Serverless ETL: AWS Glue for serverless data integration and AWS Lambda for lightweight real-time transformations.
- Data Storage: Amazon S3 for building and managing the core Data Lake.
- Data Warehousing: Experience with Amazon Redshift or Amazon RDS for relational data warehousing needs.
- Orchestration: Experience with AWS Step Functions AWS Glue Workflows or Apache Airflow on Amazon MWAA.
- Transformation Languages: Expert level skills in PySpark Python and SQL.
Key Skills
- SQL
- Pentaho
- PL/SQL
- Microsoft SQL Server
- SSIS
- Informatica
- Shell Scripting
- T Sql
- Teradata
- Data Modeling
- Data Warehouse
- Oracle
