Hands-on AI Technical Lead - Production LLM Systems

About the role

AllysAI builds AI-native enterprise systems on its AllysOS platform for banks, pharmaceutical companies, government and energy organisations. As Hands-on AI Technical Lead you take technical ownership of production LLM systems, from architecture and implementation through evaluation, deployment and ongoing reliability, and lead the engineers building them. Most of your time goes into designing, coding, testing and shipping, working directly with client technical teams.

Key responsibilities

• Design the architecture: choose between deterministic workflows, RAG, single or multiple agents, and explain the trade-offs of models, frameworks and infrastructure.
• Build Python services, APIs, retrieval pipelines, tool integrations and agent workflows, including MCP where appropriate.
• Develop document AI pipelines (OCR, extraction, classification, cross-document validation) with clear handling of missing, conflicting or uncertain information.
• Create test datasets, measure retrieval and output quality separately, investigate failures and set up regression tests and release criteria.
• Select deployment approaches for client-controlled cloud or on-premises environments and manage monitoring, latency, cost, reliability and recovery.
• Enforce access controls and data boundaries, protect sensitive information and control what agents can access and do.
• Divide work across engineers, review code, set standards and work with client teams through integration, testing and production handover.

Required profile

• 5+ years of professional software engineering, including at least 2 years building and operating production LLM or generative AI systems used by real users.
• Experience evaluating AI systems beyond demos: test-set design, error analysis, measurable quality and production safeguards.
• Experience deploying systems with Docker, CI/CD, monitoring and secure deployment practices.
• Evidence of leading engineers while staying directly involved in implementation.
• Clear English and the ability to discuss architecture, delivery risks and technical trade-offs with clients.
• A disciplined approach to AI-assisted development, able to explain the tools used and how code is verified.

Required skills

• Strong Python and backend engineering, including APIs, databases, integrations and automated testing.
• RAG, vector search, re-ranking, structured outputs and agent/tool orchestration.
• Relevant: Azure (Azure OpenAI, AI Search, Document Intelligence, API Management); strong AWS or GCP experience also relevant.
• Relevant: regulated environments, private networking, locally hosted models, restricted-connectivity or on-premises deployments, and production document workflows with complex layouts or multilingual content.

What we offer

• Competitive compensation based on experience and role scope.
• Flexible time off.
• Hands-on technical ownership and direct exposure to enterprise clients.
• The chance to build reusable capabilities across multiple production deployments.