AI Data Engineer (Generative AI & Scalable Data Pipelines)

About the role

We are looking for an AI Data Engineer to design, build and optimise large‑scale data pipelines that power generative AI applications. The role combines data engineering, cloud infrastructure and AI model deployment to deliver real‑time, retrieval‑augmented insights.

Key responsibilities

• Build and optimise ETL pipelines for large‑scale AI applications, including API integration, web scraping and real‑time data processing.
• Develop and maintain scalable AI data infrastructure using PySpark, Pandas, SQL and cloud services (Azure, AWS, GCP).
• Implement retrieval‑augmented generation (RAG) pipelines with vector databases such as FAISS, ChromaDB and Pinecone.
• Deploy and monitor AI applications using FastAPI, Streamlit and Docker to ensure real‑time performance.
• Collaborate with cross‑functional teams to guarantee data security, compliance and model reliability in production.

Required profile

• Bachelor's or Master's degree in Computer Science, Data Engineering, Artificial Intelligence or a related field.
• 2‑4 years of experience in data engineering, AI/ML or cloud‑based AI infrastructure.
• Strong analytical, problem‑solving and communication skills.

Required skills

• Python
• PySpark
• Pandas
• SQL
• Azure, AWS, GCP
• FAISS, ChromaDB, Pinecone
• FastAPI
• Streamlit
• Docker
• Kubernetes