Senior Data Engineer

Herndon, VA - USA

Monthly Salary: Not Disclosed

Posted on: 30+ days ago

Vacancies: 1 Vacancy

Job Summary

As a Senior Data Engineer you will be responsible for building and optimizing data pipelines managing data storage and processing systems and ensuring the availability scalability and reliability of our data platform. You will collaborate with cross-functional teams including data scientists software engineers and business stakeholders to understand data requirements and deliver efficient high-quality data solutions. We are seeking an accomplished individual to join our esteemed team of ATPCO data engineers dedicated to tackling intricate data processing challenges within the Airline industry. This role offers an opportunity to work on cutting-edge solutions that involve handling hundreds of terabytes of data daily leveraging the latest AWS services.

You Will:

Collaborate with data scientists analysts and other stakeholders to understand data requirements and design data models and schemas that facilitate data analysis and reporting

Design develop and maintain scalable and efficient data pipelines and ETL processes to ingest process and transform large volumes of data from various sources into usable formats

Build and optimize data storage and processing systems including data warehouses data lakes and big data platforms using AWS services such as Amazon Redshift AWS Glue AWS EMR AWS S3 and AWS Lambda to enable efficient data retrieval and analysis

Implement and manage real-time data streaming architectures using AWS services like Amazon Kinesis or Apache Kafka to enable real-time data processing and analytics

Perform data profiling data cleansing and data transformation tasks to prepare data for analysis and reporting

Implement data security and privacy measures to protect sensitive and confidential data using AWS security services and features

Design and implement data architectures following Data Mesh principles within the AWS environment including domain-oriented data ownership self-serve data infrastructure and federated data governance

Provide technical guidance and mentorship to junior data engineers reviewing their work and ensuring adherence to best practices and standards

Qualifications:

Strong programming skills in languages like Python Java or Scala with experience in data manipulation and transformation frameworks

Proven experience as a data engineer with experience in designing and building large-scale data processing systems

Strong understanding of data modeling concepts and data management principles

In-depth knowledge of SQL and experience working with relational and non-relational databases

Knowledge of Data Mesh principles and experience designing and implementing data architectures following Data Mesh concepts within the AWS ecosystem

Experience with real-time data streaming architectures using AWS services like Amazon Kinesis or Apache Kafka

Familiarity with AWS cloud services such as AWS Glue AWS Lambda AWS EMR AWS S3 Amazon Redshift and their data-related features and functionalities

Familiarity with AWS security services and features for data security and privacy

Bachelors or Masters degree in Computer Science Information Systems or a related field

Salary Range: $111890 $131735

*The disclosed range estimate has not been adjusted for applicable geographic differential associated with the location*

Additional Information :

At ATPCO we are deeply committed to diversity equity and inclusion. Our supportive policies promote work-life balance through flexible work arrangements and we cultivate a workplace where every employee feels valued respected and a true sense of belonging.

We consider qualified applicants for employment without regard to race gender age color religion national origin citizenship status marital status disability sexual orientation protected military/veteran status gender identity or expression genetic information marital status medical condition or any other legally protected factor

All your information will be kept confidential according to EEO guidelines.

Remote Work :

Yes

Employment Type :

Full-time