Data Engineer – RAG & LLM Integration

About the role

We are seeking a highly skilled Data Engineer to join our AI and data team. You will design, build, and maintain large‑scale data infrastructure with a focus on Retrieval‑Augmented Generation (RAG) pipelines and Large Language Model (LLM) integration, turning raw, unstructured data into production‑ready systems.

Key responsibilities

• Design, develop, and operate scalable ETL/ELT pipelines for structured and unstructured data.
• Architect end‑to‑end RAG pipelines, including document ingestion, chunking, embedding, indexing, retrieval, and reranking.
• Integrate LLMs (OpenAI, Anthropic, Llama, etc.) into production workflows.
• Manage and optimise vector databases such as Pinecone, Weaviate, FAISS, or Chroma.
• Perform prompt engineering and context optimisation for domain‑specific use cases.
• Build and maintain cloud‑based data infrastructure on AWS, GCP, or Azure.
• Monitor pipeline performance, data quality, and model output reliability.
• Collaborate with ML engineers, backend developers, and product teams.
• Document systems, pipelines, and architectural decisions.

Required profile

• 2+ years of experience as a Data Engineer or in a similar role.
• Strong problem‑solving skills and ability to work independently.
• Hands‑on production experience with RAG architectures and LLM integration.

Required skills

• Python
• SQL
• Vector databases (Pinecone, Weaviate, FAISS, Chroma)
• RAG frameworks (LangChain, LlamaIndex, Haystack)
• Data orchestration tools (Airflow, Prefect, Dagster)
• Cloud platforms (AWS, GCP, Azure)
• Big data tools (Spark, Kafka, dbt)