Role: Solutions Engineer - Data
Location: Deerfield IL (Hybrid - 3 days/week onsite)
- Act as the lead engineer for each domain
- Collaborate with Solution Architects to ensure understanding needed for engineering level guidance to nearshore teams
- Translate architecture design to engineering level guidance
- Own specifying architectural designs into sprint-ready engineering tasks for assigned domain
- Ensure delivery to the solution architecture within domain
- Answer design questions for sprint teams
- Participate in backlog refinement sessions to align team with the architecture
- Act as the internal technical expert supporting Product design and Engineering teams on demonstrations and technical discussions.
- Create/Update documentation such as solution design patterns and engineering standards.
- Lead technical discussions around APIs integrations data flow compliance and security.
- Serve as a trusted advisor on best practices technical implementation and scalability.
-
Required:
- Experience with Python Spark Databricks Azure Event Hub ADF Delta Lake Tables
- Able to write optimized ETL pipelines manage distributed compute jobs and handle large-scale data transformations.
- Experience with Azure Data Factory (orchestration) and Azure Event Hub (ingestion/event streaming)
- Should know how to manage data movement ingestion triggers and pipeline tuning in a production Azure environment.
- Able to code in Spark and understand execution models partitioning strategies caching broadcast joins and how these affect performance in Azures ecosystem.
- Able to collaborate with Solution Architects and lead technical discussions
- Lead developer experience working with a team of at least 12
-
Desired:
- Scala expertise
- Pharmacy domain knowledge (or retail healthcare or regulated data domains)
- Engineering standards and documentation authorship
- Understanding of API design and integration fluency