Data Engineer, Amazon Security Experience

The Amazon Security Experiences (ASE) organization is the data intelligence backbone of Amazon Security. We build and operate the data platform that powers security insights builder experience analytics and tooling decisions across Amazons security ecosystem. Every insight surfaced to security leadership from builder productivity trends to security issue quality metrics flows through systems our team designs builds and maintains.

We operate at scale processing large volumes of security and operational data daily through hundreds of pipelines consisting of transformations reporting queries ingestions and infrastructure management scripts. Our engineers work directly with source systems across AWS and SDO (Security Developer and Operations) data domains to procure data convert it into structured formats build large-scale processing pipelines design analytical data models and maintain infrastructure with the highest security and compliance standards.

We are seeking a Data Engineer I to join our Security Data Intelligence (SDI) this role you will build and maintain data pipelines that power security analytics across Amazon. Working with well-defined requirements and guidance from senior engineers you will create physical data models optimize data pipelines for defined datasets and deliver reliable well-documented solutions on schedule. You will partner with teams across ASE BizOps MapSec and other security organizations to implement data ingestion flows and support analytical models that serve the broader Amazon Security organization.

Key job responsibilities
Learn and apply team-established data processing guidelines and best practices when building and testing pipelines. Measure and improve dataset quality across owned pipelines identifying issues and proposing solutions to the team. Flag limitations or opportunities in data processing tools to senior engineers for evaluation.

Learn and understand the teams existing data architecture including how security data flows through ingestion transformation and serving layers. Build data solutions at the component level within the established team data architecture following defined logical data models. Create physical data models for assigned datasets applying appropriate persistence and integration techniques. Understand the implications of implementation decisions (e.g. normalization distribution) and seek guidance from senior engineers on trade-offs.

Write secure stable testable and maintainable code with minimal defects following team coding standards. Clearly document solutions for ease of use and maintainability by teammates and data consumers. Get all data models pipeline designs and code reviewed by senior engineers before deployment. Test code thoroughly and understand the maintenance characteristics runtime properties and dependencies of delivered solutions.

Build and maintain data pipeline infrastructure for assigned datasets using team-established patterns and tools. Apply basic database optimization techniques such as partitioning distribution and indexing as directed. Understand the underlying system infrastructure that data solutions depend on including differences between data persistence solutions. Follow team guidelines for infrastructure management security compliance and automation practices.

Solve straightforward well-defined problems with guidance from senior engineers for example building ingestion pipelines for a new security data source where the schema and requirements are defined or troubleshooting data quality issues in existing datasets by researching root causes and thoroughly resolving defects. Handle problems even outside your own domain by either resolving them or ensuring a clear hand-off to the right owner.

Take well-defined requirements and deliver solutions on schedule within assigned scope. Participate in team design scoping and prioritization discussions to understand project context. Manage your own task execution effectively seeking guidance when blocked or uncertain. Communicate progress and dependencies clearly to your team lead and senior engineers.

Build good working relationships with teammates peer engineers and data consumers across ASE and SDO. Partner effectively with Business Analysts Data Scientists and peer SDEs to understand data requirements. Help train and mentor DE interns when applicable. Participate in the interview process. Recognize discordant views and participate in constructive dialogue with peers.

Follow team-established best practices for data discovery naming conventions and data security. Classify store and handle data per Amazon security and compliance policies. Ensure proper access control data integrity and auditability for all produced datasets. Track security risks in data handling mitigate issues and escalate in a timely manner.

Identify data quality issues and pipeline deficiencies in owned components and propose solutions to the team. Participate in code reviews design discussions and team planning sessions. Make small improvements to team development processes when directed. Actively seek knowledge of team data architecture and apply learnings to improve your data solutions.

Participate in on-call rotation and respond to pipeline incidents following established runbooks and escalation procedures. Understand the maintenance characteristics runtime properties and dependencies of team data systems. Contribute to monitoring and alarming setup for owned pipelines with guidance from senior engineers. Document incident resolutions and contribute to runbook updates for owned components.

- 2 years of data engineering experience
- Experience with data modeling warehousing and building ETL pipelines
- Experience with one or more query language (e.g. SQL PL/SQL DDL MDX HiveQL SparkSQL Scala)
- Experience with one or more scripting language (e.g. Python KornShell)

- Experience with big data technologies such as: Hadoop Hive Spark EMR
- Experience with any ETL tool like Informatica ODI SSIS BODI Datastage etc.

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status disability or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process including support for the interview or onboarding process please visit for more information. If the country/region youre applying in isnt listed please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience qualifications and location. Amazon also offers comprehensive benefits including health insurance (medical dental vision prescription Basic Life & AD&D insurance and option for Supplemental life plans EAP Mental Health Support Medical Advice Line Flexible Spending Accounts Adoption and Surrogacy Reimbursement coverage) 401(k) matching paid time off and parental leave. Learn more about our benefits at WA Seattle - 101300.00 - 160000.00 USD annually