Data Engineer

Stamford, CT - USA

Monthly Salary: Not Disclosed

Posted on: 1 hour ago

Vacancies: 1 Vacancy

Job Summary

NO CONSULTANT OR CONTRACTORS. THEY WANT STABLE WORK HISTORY. NO JOB HOPPERS

Develop our comprehensive data processing pipeline transforming on-premises Kafka streams into both actionable business insights and regulatory compliance reports through AWS cloud services (S3 Glue Athena EMR).

Design robust ETL processes and build automated scalable data solutions aligned with our zero-maintenance vision delivering high-quality outputs for both business decision-making and regulatory requirements. About your team:

We are the Realtime Order Analytics and Reporting team a dynamic group focused on transforming financial transaction data into valuable business intelligence and regulatory reporting.

Our team:

Works with cutting-edge technologies including AWS cloud services and realtime data processing
Operates in a collaborative environment where innovation and ideas are encouraged
Maintains a balance between technical excellence and business impact
Values automation and efficiency in all our solutions
Fosters continuous learning and professional development
Plays a critical role in supporting business decision-making and ensuring regulatory compliance
Embraces agile methodologies to deliver high-quality solutions efficiently

Were looking for someone who shares our passion for data engineering and wants to make a significant impact by turning complex financial data into actionable insights.

What will be your responsibilities:

Designing developing and maintaining ETL workflows using AWS services
Processing data from Kafka streams and S3 storage to generate insights
Implementing data transformation logic using Python PySpark and PyAthena
Creating and optimizing data models for both analytical and regulatory reporting needs
Building automated data quality checks and monitoring systems
Developing and maintaining documentation for data pipelines and processes
Troubleshooting and resolving data pipeline issues
Contributing to architectural decisions for data infrastructure
Ensuring data solutions meet performance security and compliance requirements
Continuously improving our data systems for scalability and reduced maintenance

Qualifications

Bachelors or masters degree in Computer Science or a related field
3 years of professional software engineering experience in Python PySpark and PyAthena
3 years of professional experience in Python as a primary language (non-scripting)
Extensive experience in Pandas or NumPy
Experience with ETL processes and data warehousing concepts
Familiarity with cloud technologies particularly AWS (S3 Glue Athena EMR)
Experience using ELK Stack (Elasticsearch Logstash Kibana)
Thorough understanding of databases and SQL
1 years of professional experience with Linux operating systems
An analytical mind and business acumen
Strong communication skills

Good to have:

Experience with financial markets or the brokerage industry
Experience with business intelligence tools especially Tableau
Experience with version control systems (e.g. Git BitBucket)
Experience with CI/CD Practices and Tools
To be successful in this position you will have the following:
Self-motivated and able to handle tasks with minimal supervision.
Superb analytical and problem-solving skills.
Excellent collaboration and communication (Verbal and written) skills.
Outstanding organizational and time management skills

NO CONSULTANT OR CONTRACTORS. THEY WANT STABLE WORK HISTORY. NO JOB HOPPERS Develop our comprehensive data processing pipeline transforming on-premises Kafka streams into both actionable business insights and regulatory compliance reports through AWS cloud services (S3 Glue Athena EMR). Design ro...