Lead Data Engineer with Databricks

Chicago, IL - USA

Monthly Salary: Not Disclosed

Posted on: 8 hours ago

Vacancies: 1 Vacancy

Job Summary

Position: Lead Data Engineer with Databricks

Location: Chicago IL*** HYBRID***

Duration: 1 Years

Job Description:-

Priorities (in order):
1. SQL Proficiency: Advanced SQL skills for querying and analyzing data optimizing query performance and troubleshooting data quality issues looks for starburst Trino experience here as an additional plus
1. Kafka: Experience supporting Kafka streaming pipelines including troubleshooting consumer lag partition rebalancing connector issues and broker performance
2. Kubernetes: Strong experience troubleshooting containerized applications in Kubernetes environments including pod debugging resource management network policies and storage issues
3. Splunk: Proficiency with Splunk for log analysis creating queries (SPL) building dashboards and setting up alerts for proactive monitoring dont need Splunk admin experience- just be able to look into logs effectively
3. HashiCorp Vault: Hands-on experience with Vault for secrets management troubleshooting authentication issues token expiration and secret rotation problems

Experience with Apache Iceberg or other table formats (Delta Lake Hudi) and understanding of their operational characteristics
Knowledge of Protocol Buffers (protobuf) serialization format and schema evolution
Familiarity with Starburst/Trino or other distributed SQL engines (Presto Athena) and their query optimization

Position: Lead Data Engineer with Databricks Location: Chicago IL*** HYBRID*** Duration: 1 Years Job Description:- Priorities (in order): 1. SQL Proficiency: Advanced SQL skills for querying and analyzing data optimizing query performance and troubleshooting data quality issues looks for starbur...