AWS Big Data Platform Engineer (Netezza to AWS Migration)

Role: AWS Big Data Platform Engineer (Netezza to AWS Migration)

Hybrid: 2 days a week in office

Job Summary: We are seeking a highly skilled Senior Platform Engineer with extensive experience in Big Data technologies AWS Cloud and data platform engineering. The ideal candidate will have strong expertise in designing developing and supporting scalable data pipelines and distributed systems along with hands-on experience in Big Data ecosystem tools AWS services and real-time data processing. This role involves working on data platform modernization cloud migrations Data Warehousing and ETL in a fast-paced enterprise environment.

Key Responsibilities

Design develop maintain and support scalable data pipelines using Big Data and AWS technologies.

Lead and support data platform migration initiatives (On-Prem to AWS Cloud) ideally with Netezza background.

Develop and manage ETL/ELT processes using tools like SSIS Pentaho or similar.

Implement and manage AWS services such as EMR S3 EC2 Redshift Glue and Airflow.

Build and optimize data workflows and orchestration pipelines using Airflow.

Work with real-time streaming technologies such as Kafka and Spark Streaming.

Perform data ingestion transformation and validation from multiple data sources.

Optimize SQL queries for performance and scalability.

Monitor system performance troubleshoot issues and ensure system reliability.

Collaborate with cross-functional teams including developers architects and infrastructure teams.

Required Technical Skills

Cloud Technologies

Strong experience in AWS Cloud services:

EMR EC2 S3 VPC RDS Redshift

AWS Glue IAM CloudWatch CloudFormation

Airflow (or AWS Managed Workflows)

Databases

Experience working with:

Netezza is mandatory

SQL-based systems: SQL Server and PostgreSQL

Data warehouses: Teradata Redshift Netezza

ETL Tools

Hands-on experience with:

SSIS

Programming & Scripting

Strong proficiency in:

SQL

Shell scripting

Good to have:

Python

Operating Systems

Strong experience in Unix/Linux environments

Key Qualifications

10 years of experience in Data Engineering / Big Data / Platform Engineering.

Hands-on experience in AWS and Hadoop ecosystems.

Experience in data migration projects (e.g. Hadoop to AWS DB migrations).

Strong understanding of data warehousing concepts and ETL design patterns.

Expertise in query optimization and performance tuning.

Experience supporting production data platforms.

Bachelors degree in Computer Science Engineering or related field.

Preferred Qualifications

Experience in Banking / Financial Services domain

Experience with data lake architecture and governance

Knowledge of DevOps tools and CI/CD pipelines

Experience in platform engineering and framework development

Soft Skills

Excellent problem-solving and analytical skills

Strong communication and collaboration abilities

Ability to work independently and lead initiatives

Experience mentoring team members

Nice-to-Have

Experience with tools like Ansible Git JIRA

Exposure to NoSQL databases (HBase MongoDB)

Experience in data masking and security frame