Senior Data Bricks Architect

About the role

We are seeking a hands‑on Senior Data Bricks Architect to design, build, and govern a Lakehouse data platform from the ground up. You will own the end‑to‑end architecture, from raw ingestion through Medallion layers to serving, and set engineering standards for the data organization.

Key responsibilities

• Design and implement a production‑grade Medallion Architecture (Bronze/Silver/Gold) across all pipelines.
• Establish best practices for Delta Lake table design, partitioning, Z‑ordering, and optimization.
• Lead the setup and rollout of Unity Catalog for centralized data governance, including fine‑grained access control and audit logging.
• Build and maintain production‑grade pipelines using PySpark, Delta Live Tables, and Databricks Workflows.
• Implement CI/CD with Databricks Repos, manage code promotion across environments.
• Optimize Spark execution plans, right‑size cluster configurations, and manage compute cost governance.
• Define Python coding standards, notebook best practices, and reusable library patterns.

Required profile

• 5+ years of experience with Databricks and Lakehouse architectures.
• Proven engineering leadership and ability to drive technical standards.
• Hands‑on expertise in designing end‑to‑end data flows, both streaming and batch.

Required skills

• Databricks platform (Unity Catalog, Delta Lake, Delta Live Tables, Workflows, Repos).
• PySpark and Spark performance tuning.
• Python programming and Git integration.
• CI/CD pipelines for data engineering.
• Serverless and photon‑enabled SQL warehouses.
• Data governance concepts: row‑level/column‑level security, data masking, lineage tracking.