Manager Big Data Engineering Databricks Lead Hybrid
Job Location:
Atlanta, GA - USA
Monthly Salary:
Posted:
Application Deadline:
Vacancies:
Job Summary
Publicis Sapient is a digital transformation partner helping established organizations get to their future digitally enabled state both in the way they work and the way they serve their customers. We help unlock value through a start-up mindset and modern methods fusing strategy consulting and customer experience with agile engineering and problem-solving creativity. United by our core values and our purpose of helping people thrive in the brave pursuit of next our 20000 people in 53 offices around the world combine experience across technology data sciences consulting and customer obsession to accelerate our clients businesses through designing the products and services their customers truly value.
- Combine your technical expertise leadership skills and problem-solving passion to work closely with clients translating complex business challenges into modern data platform solutions that deliver measurable business value.
- Lead the architecture design and delivery of enterprise-scale data engineering solutions built on Databricks and cloud-native data platforms.
- Drive data modernization initiatives helping clients migrate from traditional data architectures to modern lakehouse and cloud-based the development and optimization of batch and streaming data pipelines using Databricks Spark and cloud-native data services.
- Design scalable data foundations that support analytics machine learning Generative AI and AI-enabled experiences through high-quality data products and with stakeholders to define data platform roadmaps architecture standards governance practices and delivery approaches.
- Establish best practices for engineering excellence performance optimization data quality observability reliability security and operational support.
- Support AI-enabled engineering use cases by designing scalable retrieval patterns context engineering approaches and modern data services that power machine learning and agentic solutions.
- Conduct technical feasibility assessments project estimation architecture reviews and solution planning activities for large-scale client engagements.
- Mentor and develop engineers while providing technical leadership delivery oversight and career guidance across multiple project teams.
- Contribute to practice growth through client engagement solution development capability building hiring and thought leadership.
Your Skills and Experience
- 10 years of demonstrated experience leading the implementation of enterprise-scale data platforms and end-to-end data engineering solutions in production environments.
- Hands-on experience with Databricks as a primary data engineering platform including Delta Lake Databricks Workflows Databricks SQL notebooks jobs and modern Lakehouse architecture patterns.
- Strong experience designing and implementing scalable data platforms on one or more public cloud platforms including Amazon Web Services (AWS) Microsoft Azure or Google Cloud Platform (GCP).
- Advanced proficiency in Python and practical experience using Python-based frameworks for data engineering platform automation and AI-enabled engineering workflows.
- Strong expertise in Apache Spark PySpark Spark SQL and distributed data processing technologies.
- Experience implementing both batch and real-time data pipelines using technologies such as Spark Streaming Glue ETL Lambda Dataflow Azure Data Factory Databricks or similar frameworks.
- Experience with data modeling dimensional modeling data warehousing and modern architectural patterns including Lakehouse and data mesh approaches.
- Experience with columnar data platforms such as Snowflake BigQuery Redshift Vertica or similar technologies.
- Experience with NoSQL technologies such as DynamoDB Bigtable Cosmos DB or equivalent distributed databases.
- Experience implementing software engineering best practices including source control CI/CD automated testing release management infrastructure automation and production support processes.
- Familiarity with MLOps concepts and supporting data engineering responsibilities related to model deployment validation monitoring rollback governance and operational reliability.
- Experience leading engineering teams managing delivery workstreams and collaborating effectively across cross-functional and client-facing environments.
- Strong communication stakeholder management and consulting skills.
- Experience supporting AI-enabled solutions through the design and implementation of scalable production-grade data platforms.
- Exposure to AI engineering concepts including context engineering retrieval-augmented generation (RAG) agentic architectures semantic search and production data services supporting AI-powered experiences.
- Experience building and maintaining the data pipelines that support retrieval systems including document ingestion parsing chunking metadata extraction embedding generation and incremental indexing processes.
- Familiarity with vector databases semantic search platforms graph-based knowledge stores and modern retrieval architectures.
- Exposure to cloud AI services such as Vertex AI Azure AI Services AWS AI Services or similar AI platforms.
- Experience supporting AI and machine learning lifecycle requirements including evaluation datasets monitoring operational telemetry validation workflows release management and platform observability.
- Understanding of platform requirements for managing agent state conversation history session context memory stores and durable retrieval structures that support AI-enabled applications.
- Ability to apply enterprise data engineering principles such as lineage governance provenance observability and data contracts to AI-enabled platforms and retrieval systems.
- Experience supporting agentic frameworks orchestration platforms or emerging AI engineering technologies is a plus.
- Experience with Snowflake and zero-copy architecture patterns is a plus particularly within retail financial services energy logistics manufacturing or CPG industries.
- Databricks Data Engineer Associate Professional or Machine Learning certifications.
- Certifications in AWS Microsoft Azure Google Cloud Snowflake or related cloud and data technologies.
- Experience leading Databricks-based modernization initiatives and enterprise-scale Lakehouse implementations.
- Demonstrated experience applying AI engineering concepts and Generative AI technologies in production business environments.
- Hands-on experience supporting AI/ML and LLM lifecycle requirements including deployment support monitoring validation evaluation infrastructure and operational governance.
- Experience in retail financial services energy manufacturing logistics healthcare CPG or other data-intensive industries.
- Experience working in consulting digital transformation or client-facing delivery environments.
- Understanding of Agile product and modern delivery methodologies.
Salary Range: $130000-$180000
The range shown represents a grouping of relevant ranges currently in use at Publicis Sapient. Actual range for this position may differ depending on location and specific skillset required for the work.
- An inclusive workplace that promotes diversity and collaboration.
- Access to ongoing learning and development opportunities.
- Competitive compensation and benefits package.
- Flexibility to support work-life balance.
- Comprehensive health benefits for you and your family.
- Generous paid leave and holidays.
- Wellness program and employee assistance.
As part of our dedication to an inclusive and diverse workforce Publicis Sapient is committed to Equal Employment Opportunity without regard for race color national origin ethnicity gender protected veteran status disability sexual orientation gender identity or religion. We are also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability you may contact us at
Required Experience:
Manager
About Company
Publicis Media is one of the four solutions hubs of Publicis Groupe ([Euronext Paris FR0000130577, CAC 40], alongside Publicis Communications, Publicis.Sapient and Publicis Healthcare. Led by Steve King, CEO, Publicis Media is powered by its five global brands, Starcom, Zenith, Spark ... View more
