Lead Confluent Kafka Engineer

About the role

We are looking for a Lead Confluent Kafka Engineer to design, implement and operate Apache Kafka and Confluent Platform solutions across development, UAT and production environments. The role requires deep technical expertise and the ability to guide cross‑functional teams in delivering secure, scalable, and highly available streaming platforms.

Key responsibilities

• Install, configure, administer and support Kafka/Confluent clusters, including broker setup, topic management, partitioning, replication, retention policies and capacity planning.
• Execute upgrades, patch deployments, rolling restarts and scaling activities while maintaining production stability.
• Deploy and maintain ecosystem components such as Schema Registry, Kafka Connect, Control Center and related services.
• Investigate and resolve production issues related to broker performance, ISR, consumer lag, latency, throughput and storage utilization.
• Configure and maintain security controls (TLS/mTLS, SASL, ACLs, certificates, keystores and truststores).
• Support Java/Spring Boot applications that produce or consume Kafka messages.
• Collaborate with application, infrastructure, DevOps and security teams to onboard new workloads and drive environment promotions.
• Contribute to CI/CD pipelines, deployment automation and release management for Kafka platforms.
• Perform health monitoring, performance optimisation, capacity forecasting, incident response and root‑cause analysis.
• Create and maintain operational documentation, runbooks, disaster‑recovery procedures and backup/recovery processes.

Required profile

• 5‑10 years of hands‑on experience supporting Apache Kafka/Confluent Platform in production.
• Strong background in Java development and Spring Boot integration with Kafka.
• Solid Linux administration skills and understanding of networking and JVM performance.
• Experience implementing Kafka security (TLS/mTLS, SASL, ACLs, certificate management).
• Proven ability to troubleshoot complex streaming issues and optimise performance.

Required skills

• Apache Kafka / Confluent Platform
• Java, Spring Boot
• Linux system administration
• TLS/mTLS, SASL, ACLs, certificate management
• Kubernetes, OpenShift, Helm (containerised Kafka deployments)
• Kafka Connect, Schema Registry, Confluent Control Center
• CI/CD tools (Jenkins, GitLab CI, Azure DevOps)
• Monitoring tools (Splunk, AppDynamics, Prometheus, Grafana)