Staff Data Engineer
Staff Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
$ 145600 - 276400
1 Vacancy
Job Description
Our Company
Changing the world through digital experiences is what Adobes all about. We give everyonefrom emerging artists to global brandseverything they need to design and deliver exceptional digital experiences! Were passionate about empowering people to create beautiful and powerful images videos and apps and transform how companies interact with customers across every screen.
Were on a mission to hire the very best and are committed to creating exceptional employee experiences where everyone is respected and has access to equal opportunity. We realize that new ideas can come from everywhere in the organization and we know the next big idea could be yours!
About the Role
We are looking for a highly skilled staff level Data Engineer to join our team as a technical lead. This role involves designing and handling scalable data platforms and machine learning solutions using Databricks and cloud technologies. Your tasks will include coordinating Databricks environments optimizing cloud infrastructure and developing productiongrade data pipelines and ML models. A key aspect of the role is to ensure strong data governance security and compliance while seamlessly integrating cloudnative services.
You will collaborate with data scientists product teams and external engineering partners to advance standard methodologies in MLOps infrastructure automation and data processing. This position is ideal for an individual skilled in managing data platforms and deploying ML models while effectively working with multifunctional teams and senior collaborators. It presents chances for creativity leadership and significant impact through datadriven product decisions and largescale data solutions.
Key Responsibilities
Databricks Platform Administration
Coordinate and manage Databricks workspaces clusters and workflows.
Configure rolebased access control (RBAC) manage user groups entitlements and integrate with identity providers like Okta.
Optimize cluster configurations autoscaling and cost management.
Monitor performance debug Spark job failures and fix performance bottlenecks in notebooks and SQL queries.
Cloud Platform Expertise
Proficient in managing cloud infrastructure on AWS and Azure including IAM networking (VPCs VNets) and storage integrations (S3 ADLS Blob Storage).
Experienced in securing cloud environments with identity management rolebased access control and policy configurations.
Integrate Databricks with cloudnative services and manage storage credentials/mount points.
Security & Networking
Manage secure VPCs networking and cloud storage (S3 ADLS).
Implement access controls encryption and ensure compliance with data governance policies (GDPR HIPAA).
Automation & DevOps
Automate Databricks tasks using REST APIs Python or Bash.
Implement job orchestration with Databricks Workflows Apache Airflow.
Develop CI/CD pipelines using Jenkins Azure DevOps or GitHub Actions.
Machine Learning & Data Science
Apply Databricks to build deploy and manage scalable machine learning models in production environments.
Monitor and maintain ML models in production ensuring accuracy performance and business relevance.
Implement anomaly detection systems to monitor key business metrics and ensure data quality and integrity.
Collaboration & Leadership
Collaborate with data scientists product and external data engineering teams.
Develop documentation runbooks and training sessions to upskill data scientists.
Establish and advocate for standard methodologies in areas like notebook development database design scalable data processing and MLOps.
MustHave Skills
10 years of professional experience in software engineering data engineering and machine learning.
Experience with Databricks administration and database management including building deploying and managing ML models and data pipelines.
Proficiency in PySpark and Python (including Pandas Scikitlearn) for scalable data processing.
Ability to write and maintain scalable distributed data processing code optimized for performance and reliability.
Strong expertise in cloud platforms: AWS or Azure.
Experience with CI/CD pipelines and infrastructure automation.
Proficient in data governance compliance and security standard methodologies.
Experience with container orchestration and deployment systems like Docker.
Familiarity with networking concepts including VPC setup cloud storage integrations and security configurations.
Strong communication collaboration and guidance skills demonstrated through successful teamwork with engineering product and business colleagues
Preferred Qualifications
Experience with designing building and supporting machine learning pipelines and collaborating with ML teams.
Understanding of generative AI models (e.g. GPT Claude) and experience working with custom models tailored to specific product needs.
Expertise in anomaly detection systems and related monitoring solutions.
Knowledge of retrieval systems including vector search and embeddingbased search to extract relevant information from large unstructured datasets (e.g. documents knowledge bases).
Advanced proficiency in SQL and Apache Spark tuning.
Application Window Notice
If this role is open to hiring in Colorado (as listed on the job posting) the application window will remain open until at least 04/02/2025 12:01 AM Pacific Time in compliance with Colorado pay transparency regulations. If this role does not have Colorado listed as a hiring location no specific application window applies and the posting may close at any time based on hiring needs.Our compensation reflects the cost of labor across several U.S. geographic markets and we pay differently based on those defined markets. The U.S. pay range for this positionis $145600 $276400 annually. Paywithin this range varies by work locationand may also depend on jobrelated knowledge skillsand experience. Your recruiter can share more about the specific salary range for the job location during the hiring process.At Adobe for sales roles starting salaries are expressed as total target compensation (TTC base commission) and shortterm incentives are in the form of sales commission plans. Nonsales roles starting salaries are expressed as base salary and shortterm incentives are in the form of the Annual Incentive Plan (AIP).
In addition certain roles may be eligible for longterm incentives in the form of a new hire equity award.
Adobe will consider qualified applicants with arrest or conviction records for employment in accordance with state and local laws and fair chance ordinances.Adobe is proud to be anEqual Employment Opportunityand affirmative action employer. We do not discriminate based on gender race or color ethnicity or national origin age disability religion sexual orientation gender identity or expression veteran status or any other applicable characteristics protected by law.Learn more.
Adobe aims to make accessible to any and all users. If you have a disability or special need that requires accommodation to navigate our website or complete the application process emailor call.
Adobe values a free and open marketplace for all employees and has policies in place to ensure that we do not enter into illegal agreements with other companies to not recruit or hire each others employees.
Required Experience:
Staff IC
Employment Type
Full-Time
