Data Engineer, Active Grid Response
San Francisco, CA - USA
Job Summary
- Building ETL/ELT pipelines that ingest transformer pole and sensor telemetry into Gridwares Data Lake and Lakehouse
- Developing and maintaining real-time and batch ingestion processes using Python SQL Databricks and Spark
- Implementing data quality checks validation rules and automated testing for stable operations
- Collaborating with Software Firmware and Data Science teams to define ingestion schemas and transformations
- Working with cloud-native tools to optimize pipeline throughput and cost efficiency
- Monitoring pipelines for reliability troubleshooting issues and contributing to on-call rotations
- Writing documentation for data processes models and metadata
- 24 years of experience as a Data Engineer (or Backend Engineer with heavy data exposure)
- Strong proficiency inPythonandSQL
- Familiarity with data warehouses Lakehouse platforms or big data tools (Databricks Spark or equivalent)
- Experience with pipeline orchestration tools (Airflow Dagster Prefect etc.)
- Understanding of event-driven systems or streaming platforms (Kafka Kinesis Pub/Sub)
- Solid foundation in data modeling testing and version control
- Ability to work collaboratively in a high-autonomy fast-paced environment
- Experience with IoT telemetry ingestion or time-series data
- Exposure to Unity Catalog governance or schema enforcement
- Understanding of Protobuf Avro Parquet or serialization formats
- Hands-on experience with observability tools (Grafana OpenTelemetry)
Required Experience:
IC
About Company
This describes the ideal candidate; many of us have picked up this expertise along the way. Even if you meet only part of this list, we encourage you to apply! Benefits Health, Dental & Vision (Gold and Platinum with some providers plans fully covered) Paid parental leave Alternating ... View more