As a Data Engineer on the Services Data Science & Analytics team you will be responsible for designing developing and maintaining robust data pipelines to support Apple Services analytics initiatives. Our teams task is to build a comprehensive aggregate data layer that enables efficient and flexible executive reporting highly customized data applications and powerful ML inference and this role you will work closely with data scientists BI engineers and business and product teams to build scalable data pipelines and solutions. As a data engineer you must effectively collaborate to bridge the gap between business needs analytical solutions and engineering requirements. Additionally proactive collaboration with other data engineering teams is essential for scaling solutions across teams and Apple role requires expertise in data engineering tools and patterns including developing PySpark jobs using orchestration tools for scheduling and monitoring understanding CI/CD processes and managing a dynamic data lake. The team maintains pipelines that require on-call support and monitoring. A successful engineer will have a strong intuition for quickly identifying and resolving bugs and efficiently solving technical :- Independently design technical solutions to process massive datasets (billions of daily records) and unify analytics across Apple Services- Build data pipelines using Python PySpark and SQL- Manage an evolving data schema ensuring that data structures can extend over time while also maintaining backward compatibility- Collaborate with Business Intelligence and Data Science teams to design aggregate tables used across multiple teams and workflows- Integrate new tools and packages into the teams workflow and codebase- Provide on-call support and monitoring- Mentor and provide data engineering best practices across the organization
5 years of experience as a Data Engineer or similar role
5 years of experience with data schema design and data modeling
5 years of experience in Python and SQL
2 years of Spark/PySpark experience
Business Intelligence experience - aggregated data sources metric definition data visualization with tools like Tableau
Strong critical thinking debugging and technical design skills
Bachelors degree in Computer Science Statistics Mathematics Engineering Information Systems Economics or related field
Deep experience in Spark
Experience in digital subscriptions media and/or app-based engagement data
Experience in Data Governance & User Privacy
Experience with container orchestration and deployment frameworks (eg. Kubernetes Docker)
Data Platform engineering experience to support deployment of pipelines across many diverse teams
Masters degree in Computer Science Statistics Mathematics Engineering Information Systems Economics or related field
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.