Job Location: San Francisco Bay Area CA office with 50% travel to client location in San Francisco
As a Sr. Staff Technical Solutions Engineer and tech subject matter expert you will partner closely with our Field and Engineering teams to deliver high-touch specialized support and tailored technical solutions for Databricks largest and most strategic customers in the Digital Native Business (DNB) segment. In this customer-facing role you will leverage your technical expertise in Apache Spark and other data technologies to triage and resolve complex product issues and unblock our customers most critical technical challenges.
The Impact You Will Have
- Perform advanced Troubleshooting and Root Cause Analysis to resolve performance and reliability issues in Spark SQL Delta Streaming and Databricks runtime features using tools like Spark UI metrics Mosaic AI Model Service DAGs and event logs.
- Discover requirements for continuous monitoring to detect early performance issues working with R&D and NOC teams to optimize the DNB customer environments.
- Build Rapid POCs Test/Deploy/Monitor the solutions built by Databricks Engineering to address customer challenges and showcase advanced Spark/ML/AI runtime capabilities aligned with their business goals.
- Develop comprehensive playbooks and maintain a knowledge base of common issues and solutions for Spark ML and AI workflows.
- Train customer engineering and business teams on best practices in performance tuning debugging and effectively leveraging Databricks Features.
- Pilot new best practices processes/ programs champion process improvements and collaborate with cross-functional teams to enhance the customer experience.
- Advocate for customers in business review meetings and maintain close relationships as a trusted advisor and primary technical point of contact.
- Collaborate onsite with Field Engineering Sales and Product teams during customer engagements and technical presentations to provide rapid solutions to production-impacting issues demonstrating deep technical expertise and building strong customer trust.
What We Look For
- Technical Expertise in Big Data and Spark: 812 years of experience designing building and troubleshooting distributed computing applications with 4 years delivering production-scale Spark/ML/AI solutions using Python Java or Scala.
- Data Engineering Specialization: Hands-on expertise with Data Lakes SQL-based databases and Cloud-based Data Warehousing/ETL tools like Snowflake Redshift Bigquery etc
- Advanced Tech Skills: Deep knowledge of Spark core internals Delta/Iceberg JVM optimization and memory management with additional proficiency in AI ecosystems like Machine Learning Deep Learning and Generative AI.
- Cloud and CI/CD Skills: Practical experience with AWS Azure or GCP coupled with expertise in building and managing CI/CD pipelines monitoring and alerting systems.
- Customer-Facing Experience: 35 years in customer-facing roles such as Technical Account Manager or Solutions Architect demonstrating strong communication relationship-building and problem-solving skills.
- Advanced Proactive Problem Solving Skills: Proven ability to anticipate identify and mitigate risks while planning solutions for production challenges. Effectively use sound business judgment risk avoidance and subject matter expert resources to coordinate team efforts to solve problems.
- Collaboration and Leadership: Proven ability to work with cross-functional teams and senior leadership to address roadblocks mitigate risks and drive customer success while creating impactful documentation for self-service solutions.
Required Experience:
Staff IC