Data Engineer - AWS, Databricks & Lakehouse

About the role

A growing Data & AI team is looking for a hands-on Data Engineer to modernize and scale its data platform. The role focuses on migrating legacy C#/.NET ingestion processes to a cloud-native AWS architecture and building an analytical platform with Databricks/Lakehouse technologies.

Key responsibilities

• Design and build scalable ingestion pipelines using AWS S3, Lambda, Glue, Step Functions and API Gateway.
• Develop Python and/or C#/.NET data-processing and integration solutions.
• Build APIs and file-based ingestion mechanisms for client data.
• Develop data mapping and transformation workflows into SQL Server.
• Help design and implement a Databricks Lakehouse platform.
• Optimize ETL/ELT pipelines for performance, reliability and scalability.
• Support and modernize existing C#/.NET ingestion processes and troubleshoot production pipeline issues.
• Implement data governance, security and compliance for a multi-tenant SaaS environment.
• Collaborate with Software Engineering, Architecture, Product, Analytics and Data Science teams.

Required profile

• Bachelor's degree in Computer Science, IT or a related field.
• 3+ years of experience in Data Engineering.

Required skills

• Strong AWS data engineering.
• Python and/or C#/.NET.
• SQL Server.
• ETL/ELT, API and file-based ingestion.
• Familiarity with Databricks, Spark and Lakehouse architecture.
• Nice to have: Azure DevOps/CI/CD, Amazon Bedrock/AgentCore, data governance and security, real-time processing, ML/AI pipelines and multi-tenant SaaS experience.