Kafka Developer

Job Description

Role Summary

The Senior Kafka Administrator / Developer will design, build, operate and improve enterprise Kafka and event-streaming platforms across AWS and Kubernetes, including Amazon EKS. The role requires strong hands-on Kafka administration, integration development, cloud-native operations and production support experience.

The role works closely with application, integration, cloud, security and operations teams to deliver reliable event-driven solutions, resolve complex issues, automate repeatable tasks and support platform lifecycle, governance and service improvement activities.

Key Responsibilities

· Administer and support Kafka platforms, including brokers, topics, partitions, replication, consumer groups, Kafka Connect, Schema Registry and related services.

· Design and develop Kafka producers, consumers, stream-processing components and reusable integration patterns using Java, Spring Boot, Python or similar technologies.

· Deploy, operate and improve Kafka and supporting components on Kubernetes and Amazon EKS using Helm, CI/CD, GitOps and approved cloud-native patterns.

· Manage platform reliability, resilience, capacity, performance, patching, upgrades, backup, recovery and lifecycle activities across production and non-production environments.

· Troubleshoot complex issues across applications, brokers, connectors, schemas, networking, storage, certificates and cloud infrastructure.

· Implement monitoring, alerting and dashboards for broker health, throughput, latency, replication, disk usage, consumer lag and connector health.

· Apply security and governance controls, including TLS, authentication, authorisation, ACLs, secrets, certificates, least-privilege access and vulnerability remediation.

· Support incidents, problems, changes and releases, including root-cause analysis, runbook improvement and automation of recurring operational tasks.

· Guide application teams on topic design, partitioning, message contracts, schema evolution, retries, dead-letter handling, idempotency and supportability.

· Maintain architecture, configuration, operational, recovery and support documentation.

Technical Leadership

· Act as the senior technical point of contact for Kafka platform design, engineering and operational matters.

· Mentor engineers, review technical work and promote consistent engineering and operational standards.

· Lead planning for upgrades, migrations, resilience improvements and platform modernisation.

· Communicate risks, dependencies, options and recommendations clearly to technical and business stakeholders.

Key Skills & Experience

Technical Experience

· Strong hands-on experience administering Apache Kafka or Confluent Platform in complex enterprise environments.

· Solid understanding of Kafka brokers, topics, partitions, replication, consumer groups, Kafka Connect, Schema Registry and MirrorMaker.

· Experience developing Kafka producers, consumers and integration services using Java, Spring Boot, Python or equivalent technologies.

· Hands-on experience with Kubernetes and Amazon EKS, including Docker, Helm, storage, networking, ingress, secrets and identity integration.

· Working knowledge of AWS services such as EKS, EC2, S3, IAM, CloudWatch, Route 53 and core networking services.

· Experience with CI/CD, GitOps and Infrastructure as Code tools such as Jenkins, Argo CD, Terraform or CloudFormation.

· Strong Linux, networking, TLS, DNS, certificate and distributed-systems troubleshooting capability.

· Experience with observability tools such as Prometheus, Grafana, CloudWatch, Dynatrace, Splunk or equivalent.

Preferred Qualifications

• Degree in Information Technology, Engineering, Computer Science, or related discipline.
  
• AWS, Azure, ITIL, Kubernetes, Database, or relevant technology certifications.
  
• Experience managing large-scale enterprise or government technology environments.
  
• Experience with hybrid cloud, platform modernization, and operational transformation programs.
  

Requirements
Role Summary: The Senior Kafka Administrator / Developer will design, build, operate and improve enterprise Kafka and event-streaming platforms across AWS and Kubernetes, including Amazon EKS. The role requires strong hands-on Kafka administration, integration development, cloud-native operations and production support experience. The role works closely with application, integration, cloud, security and operations teams to deliver reliable event-driven solutions, resolve complex issues, automate repeatable tasks and support platform lifecycle, governance and service improvement activities. Key Responsibilities • Administer and support Kafka platforms, including brokers, topics, partitions, replication, consumer groups, Kafka Connect, Schema Registry and related services. • Design and develop Kafka producers, consumers, stream-processing components and reusable integration patterns using Java, Spring Boot, Python or similar technologies. • Deploy, operate and improve Kafka and supporting components on Kubernetes and Amazon EKS using Helm, CI/CD, GitOps and approved cloud-native patterns. • Manage platform reliability, resilience, capacity, performance, patching, upgrades, backup, recovery and lifecycle activities across production and non-production environments. • Troubleshoot complex issues across applications, brokers, connectors, schemas, networking, storage, certificates and cloud infrastructure. • Implement monitoring, alerting and dashboards for broker health, throughput, latency, replication, disk usage, consumer lag and connector health. • Apply security and governance controls, including TLS, authentication, authorisation, ACLs, secrets, certificates, least-privilege access and vulnerability remediation. • Support incidents, problems, changes and releases, including root-cause analysis, runbook improvement and automation of recurring operational tasks. • Guide application teams on topic design, partitioning, message contracts, schema evolution, retries, dead-letter handling, idempotency and supportability. • Maintain architecture, configuration, operational, recovery and support documentation. Technical Leadership • Act as the senior technical point of contact for Kafka platform design, engineering and operational matters. • Mentor engineers, review technical work and promote consistent engineering and operational standards. • Lead planning for upgrades, migrations, resilience improvements and platform modernisation. • Communicate risks, dependencies, options and recommendations clearly to technical and business stakeholders. ________________________________________ Key Skills & Experience Technical Experience • Strong hands-on experience administering Apache Kafka or Confluent Platform in complex enterprise environments. • Solid understanding of Kafka brokers, topics, partitions, replication, consumer groups, Kafka Connect, Schema Registry and MirrorMaker. • Experience developing Kafka producers, consumers and integration services using Java, Spring Boot, Python or equivalent technologies. • Hands-on experience with Kubernetes and Amazon EKS, including Docker, Helm, storage, networking, ingress, secrets and identity integration. • Working knowledge of AWS services such as EKS, EC2, S3, IAM, CloudWatch, Route 53 and core networking services. • Experience with CI/CD, GitOps and Infrastructure as Code tools such as Jenkins, Argo CD, Terraform or CloudFormation. • Strong Linux, networking, TLS, DNS, certificate and distributed-systems troubleshooting capability. • Experience with observability tools such as Prometheus, Grafana, CloudWatch, Dynatrace, Splunk or equivalent. ________________________________________ Preferred Qualifications • Degree in Information Technology, Engineering, Computer Science, or related discipline. • AWS, Azure, ITIL, Kubernetes, Database, or relevant technology certifications. • Experience managing large-scale enterprise or government technology environments. • Experience with hybrid cloud, platform modernization, and operational transformation programs.