AI Engineer, Inference Platform

About the role

Intelligent Inference is seeking an AI Engineer to own the end‑to‑end request path of its sovereign inference platform. You will work on the Go‑based API gateway and the high‑performance serving engines that power open‑weight models for customers across Pakistan and the broader AMEA region.

Key responsibilities

• Extend and operate the Go API gateway, handling routing, authentication, rate limiting, quota enforcement, per‑model metering, streaming, request shaping and failure handling.
• Deploy and tune open‑weight models on Huawei Ascend 910B clusters using the CANN/MindIE stack, and on Nvidia GPU clusters with vLLM and SGLang where applicable.
• Own serving performance, including continuous batching, KV‑cache configuration, tensor and expert parallel layouts, pre‑fill and decode behaviour, and context‑length trade‑offs.
• Run model quantisation pipelines from BF16 to INT8 and INT4, evaluate quality‑cost impact and report findings.
• Maintain the end‑to‑end serving stack ensuring data residency and compliance with local regulations.

Required profile

• Strong background in building high‑performance backend services, preferably with Go.
• Hands‑on experience with GPU (Nvidia) and NPU (Huawei Ascend) hardware for AI inference.
• Familiarity with model serving frameworks such as vLLM, SGLang and the CANN/MindIE ecosystem.
• Understanding of quantisation techniques and ability to assess trade‑offs.
• Ability to work in a fast‑moving startup environment and take ownership of the full request lifecycle.

Required skills

• Go programming language
• Nvidia GPU acceleration
• Huawei Ascend NPU and CANN/MindIE stack
• vLLM and SGLang serving frameworks
• Model quantisation (BF16, INT8, INT4)