Data Engineer II

Job Posting Title:

Req ID:

Job Description:

Disney Entertainment and ESPN Product & Technology

Technology is at the heart of Disneys past present and future. Disney Entertainment and ESPN Product & Technology is a global organization of engineers product developers designers technologists data scientists and more all working to build and advance the technological backbone for Disneys media business globally.

The team marries technology with creativity to build world-class products enhance storytelling and drive velocity innovation and scalability for our businesses. We are Storytellers and Innovators. Creators and Builders. Entertainers and Engineers. We work with every part of The Walt Disney Companys media portfolio to advance the technological foundation and consumer media touch points serving millions of people around the world.

Here are a few reasons why we think youd love working here:

1. Building the future of Disneys media: Our Technologists are designing and building the products and platforms that will power our media advertising and distribution businesses for years to come.

2. Reach Scale & Impact: More than ever Disneys technology and products serve as a signature doorway for fans connections with the companys brands and stories. Disney. Hulu. ESPN. ABC. ABC Newsand many more. These products and brands and the unmatched stories storytellers and events they carry matter to millions of people globally.

3. Innovation: We develop and implement groundbreaking products and techniques that shape industry norms and solve complex and distinctive technical problems.

Ad Platforms is responsible for Disneys industry-leading ad technology and products driving advertising performance innovation and value in Disneys sports news and entertainment content across all media platforms.

Job Summary

The Product Engineering team is part of Ad Platforms with a mission to help advertisers reach the right audience and deliver personalized optimized experiences tailored to customer needs and interests. Our Audience Platform offers a unified solution for audience data exchange and enrichment as well as high-concurrency low-latency audience targeting services powered by advanced data technologies. This is a dynamic cross-domain team delivering end-to-end solutions across diverse technical areas including machine learning big data microservices and data visualization.

The Principal Product Dev Engineer role will focus on providing engineering and architectural direction for Disneys Advertising Data environment. The right person should possess a deep understanding of data technology features functions and services coupled with a positive solution-oriented attitude to drive the team forward with high-quality technical deliveries and improved processes and best practices. This person will be an important leader and influencer with strong communication skills that will work closely with a cross-functional team to deliver performant solutions for our world-class Ad Sales business partners.

The right candidate must be fluent in all aspects of data development (including data pipelines datasets observability lineage data quality logging and alerting) which are a vital key to success. The right person should also blend their engineering abilities with a strong sense of craftsmanship.

Were looking for a passionate hands-on technical engineering expert who wants to come here to do their very best work and make their mark add their chapter to the long and storied history of The Walt Disney Company. Someone who holds themselves and their teammates accountable in a professional collaborative manner. A collaborative technologist who seeks to bring the best out in themselves and those around them. Someone who can provide a fresh perspective and innovative insight to our initiatives.

Responsibilities

Lead the team to design develop maintain operate and support resilient data pipelines data services in Scala Python / Spark and Java while maintaining strict uptime SLAs

Collaborate with the teams to extend functionality of current Operational Data platform data services and build new integrations with APIs both internal and external to the Data organization

Drive and provide guidance to teams on architecture design development integrations and technical work breakdown for complex applications and data flows and work to align SLAs and data delivery to the business strategies principles and practices

Review both the architecture and architectural standards with an eye to propose changes to update and improve both in order to optimize and automate where possible

Lead the team to implement quality coding practices across all development efforts related to our batch and streaming data pipelines using Scala Python Databricks and other key technologies.

Able to mentor coach and educate a team of skilled Data Engineers and Software Engineers on the best use of data capabilities with development best practices (fault tolerance reliability reuse observability etc) while cultivating a focus on continuous improvement

Work to implement shared libraries in Scala and Python that abstract complex business logic to allow consistent functionality across all data pipelines across the data organization

Lead the resolution of critical incidents and provide leadership in proactively addressing data/pipeline issues; continuously assess technology to build more stable scalable and resilient data flows; promote and expand on the use of the CI/CD pipelines to improve the deployment and build process

Lead and facilitate engineering solutioning sessions to provide technical guidance to solve any technical roadblocks for the team; contribute to development planning and implementation; help team to understand and breakdown work

Tech stack includes Airflow Spark Flink Databricks Delta Lake Apache Iceberg Snowflake Scala Python

Collaborate and lead the development and documentation of both internal and external standards and best practices for pipeline configurations naming conventions partitioning strategies and more

Ensure high operational efficiency and quality of the data platform datasets to ensure our solutions meet SLAs and project reliability and accuracy to all our stakeholders (Engineering Data Science Operations BI and Analytics teams)

Engage with and understand our customers forming relationships that allow us to understand and prioritize both innovative new offerings and incremental platform improvements Drive standards for detailed documentation of the work and changes to support data quality and data governance requirements

Leadership & Collaboration

Mentor engineers conduct design/code reviews and enforce data engineering best practices. Collaborate with infra product and governance teams to align on priorities and safe adoption of AI.

Drive delivery of high-profile AI applications by balancing execution speed with system reliability

Basic Qualifications

Hold a Bachelors degree in Computer Science Information Systems Electrical Engineering or another comparable field of study

Have 10 years of experience in software development or software engineering in a large enterprise environment using modern languages frameworks and data platforms/tools with at least 5 years of data engineering experience developing large data pipelines

Strong algorithmic problem-solving expertise

Have demonstrable expertise with relevant data technologies such as MSK Kenesis S3 Flink Spark nifi SingleStore Snowflake AWS Glue Terraform K8s EKS Lambda Airflow DataDog and Grafana for observability with advanced Scala and Python programming skills Have a deep understanding of data architecture data management data governance security and automation tools

Have a track record of championing quality engineering; be adamant that all code is written in a way that can be tested with unit tests

Have hands-on production environment experience with distributed processing systems such as Spark data pipeline orchestration systems such as Airflow for creating and maintaining data pipelines

Have experience working with high-performing teams using Agile and Lean methodologies and frameworks

Care about your craft and have opinions about the right way to do things with technology

Preferred Qualifications

Hold a Masters degree in Computer Science Software Engineering or related technical discipline is highly desirable

Previous work experience in Ad Serving or Ad Technology platforms Excellent leadership and communication skills

Experience with processing large amount of data at petabyte level with experience with machine learning and AI

Strong curiosity about how Disney delivers the Magic and a desire to be a part of it Experience with:

Python Java Databricks LangChain Vector Stores (Pinecone Weaviate FAISS pgvector) SQL AWS big data tech stack (like S3 Glue MWAA)

Required Education

Bachelor or above in computer science or a related quantitative field; or equivalent practi

Job Posting Segment:

Job Posting Primary Business:

Primary Job Posting Category:

Employment Type:

Primary City State Region Postal Code:

Alternate City State Region Postal Code:

Date Posted: