Senior Data Engineer

CONVO is seeking a Senior Data Engineer to build reliable data pipelines and connectors powering its CPG-focused agentic platform. Youll develop scalable batch and streaming pipelines using Python Spark Databricks Kafka dbt and Delta Lake transforming source data into governed RAW and CUBE layers while ensuring data quality performance observability and reliability. Youll also build data-product and feature-serving interfaces that enable ML models agents and downstream applications to consume trusted data efficiently.

• Build reliable connectors and RAW-to-CUBE pipelines across batch and streaming paths and expose governed data and ML features to downstream services.

• Develop Python/Spark ingestion and transformation pipelines from source systems into RAW and Enriched CUBE layers.
• Implement streaming paths alongside scheduled workloads.
• Create dbt/SQL transformations data-quality controls and contract-validation checks.
• Optimize Delta Lake layout SQL performance partitioning and incremental processing.
• Build feature-serving and data-product interfaces for models agents and applications.
• Instrument pipelines for lineage freshness throughput failure recovery and cost.

• Minimum experience: 5 years in data engineering including 3 years delivering production Spark or python pipelines.
• Advanced Python and production Apache Spark/Databricks development.
• Kafka or comparable event-streaming technology.
• Strong SQL performance tuning and dimensional/data-product implementation.
• dbt and Delta Lake including incremental patterns and table optimization.
• Batch and streaming reliability patterns: idempotency checkpointing replay and late-arriving data.
• Automated data quality observability and schema-contract testing.

• ML feature stores or online/offline feature consistency.
• CPG retail ERP POS or syndicated-data pipelines.
• Kubernetes-based data workloads and cloud cost optimization.

• Production connectors and RAW-to-CUBE pipelines with automated tests.
• Batch/streaming operational dashboards and recovery procedures.
• Documented data products and feature-serving interfaces.
• Performance and cost baselines for the implemented workloads.

• Works under the Data Architect with source-system owners ML/optimization teams platform infrastructure and downstream application teams.