Senior Data Engineer

About the role

CONVO is looking for a Senior Data Engineer to design and operate reliable data pipelines that power its CPG‑focused agentic platform. You will build both batch and streaming workflows, transform raw data into governed layers, and create interfaces for ML models and downstream applications.

Key responsibilities

• Develop Python/Spark ingestion and transformation pipelines that move data from source systems into RAW and enriched CUBE layers.
• Implement streaming pipelines alongside scheduled batch workloads using Kafka and Databricks.
• Create dbt/SQL transformations, data‑quality controls, and contract‑validation checks.
• Optimize Delta Lake layout, partitioning, and incremental processing for performance and cost.
• Build feature‑serving and data‑product interfaces for models, agents, and applications.
• Instrument pipelines with lineage, freshness, throughput monitoring, and failure‑recovery mechanisms.

Required profile

• 5+ years of data‑engineering experience, including 3+ years delivering production Spark or Python pipelines.
• Strong expertise in Apache Spark/Databricks and advanced Python programming.
• Hands‑on experience with Kafka or comparable event‑streaming platforms.
• Deep knowledge of SQL performance tuning, dimensional modeling, and data‑product implementation.
• Proficiency with dbt, Delta Lake, and batch/streaming reliability patterns (idempotency, checkpointing, replay).

Required skills

• Python
• Apache Spark
• Databricks
• Kafka
• dbt
• Delta Lake
• SQL
• Kubernetes