Data Engineer in Test

About the role

We are looking for a Data Engineer in Test to support our data transformation initiative. You will ensure that data pipelines, transformations, and integrations deliver accurate and consistent data exactly when and where it is needed.

Key responsibilities

• Write PySpark and SQL scripts to validate data pipelines, transformations, and integrations.
• Design and execute automated tests for data validation, storage, and retrieval using Azure Data Lake, Synapse, and Data Factory.
• Maintain and enhance test suites (smoke, performance, functional, regression) as new features are released.
• Participate in data reconciliation and verify data‑quality frameworks across the platform.
• Collaborate with development teams to assess workflow changes and update test cases.
• Document testing processes, share best practices, and communicate progress to stakeholders.

Required profile

• Bachelor’s degree in Computer Science or a related field.
• 3‑6 years of hands‑on experience in data engineering, preferably in Azure Data Lake environments.
• Proven experience with high‑traffic, large‑scale data products and ensuring data quality under demanding conditions.
• Strong understanding of Agile/Scrum ceremonies and ability to work in an iterative development environment.

Required skills

• Azure Data Factory
• Azure Synapse Analytics
• Azure Databricks
• Azure Data Lake
• PySpark
• SQL (query writing and optimization)
• Python
• Functional and system integration testing in big‑data contexts
• Behaviour‑driven development (BDD) test design
• Agile Scrum methodology
• Jira with X‑Ray or Zephyr for test case management