Data Engineer

About the role

We are looking for a Data Engineer to own and evolve our CADS data platform built on Microsoft Fabric. The role will ensure the enterprise data platform remains scalable, secure and performant, supporting a range of internal services and external reporting needs.

Key responsibilities

• Design, build and maintain the CADS data flow on a Fabric lakehouse architecture, including ingestion, transformation and synchronization with supporting databases.
• Develop and operate data pipelines using Azure Data Factory and SQL MERGE/COALESCE patterns, leveraging queue tables and API gateways for efficient bulk processing.
• Implement data migration strategies and ensure sensitive fields such as SSNs are hashed or encrypted in compliance with security standards.
• Diagnose and resolve complex data‑flow and transformation issues, including recurring ADF/SQL internal errors, and provide architecture‑level fixes.
• Act as the senior escalation point for CADS‑related pipeline incidents and support the Tech 2 – Fabric & Integrations specialist.
• Write performant SQL queries, troubleshoot poorly performing statements and optimise data access.
• Participate in CADS team architecture and refactor sessions to continuously improve the platform.

Required profile

• Proven experience in data engineering with a focus on Microsoft Fabric and Azure Data Factory.
• Strong problem‑solving skills and ability to act as an escalation point for complex pipeline issues.
• Understanding of data security best practices, including encryption and hashing of sensitive data.

Required skills

• Microsoft Fabric
• Azure Data Factory (ADF)
• SQL (MERGE, COALESCE, query optimisation)
• Python (including SPLINK deduplication scripts)
• Data lakehouse architecture
• Data encryption / hashing
• Data migration techniques
• SPLINK