Kafka Admin

Job Title: Senior Big Data / Hadoop / Kafka / DevOps Administrator

Location: Denver, CO | St. Louis, MO also accepted.In-person interview is mandatory at the selected work location.

The key skills we should focus on are:

• Strong Terraform experience


• CI/CD expertise (any major toolset/platform is fine)


• Kafka experience any flavor, preferably with Confluent and Amazon MSK

Key Responsibilities:

Kafka Administration

• Deploy, configure, and manage Apache Kafka clusters and AWS MSK


• environments.


• Monitor broker health, partitions, replication factors, and consumer lag.


• Perform capacity planning and cluster scaling activities.


• Manage Kafka security using SSL, SASL, ACLs, and encryption standards.


• Troubleshoot producer, consumer, and broker performance issues.


• Support Kafka Connect, Schema Registry, Cruise Control, and MirrorMaker


• implementations.

AWS Cloud Administration

• Manage cloud infrastructure services including EC2, S3, IAM, VPC, EBS,


• CloudWatch, CloudTrail and AWS Glue.


• Support AWS Managed Streaming for Kafka (MSK), EMR, Lambda, and Airflow


• environments.


• Implement cloud security best practices and governance controls.


• Perform infrastructure provisioning and automation using Infrastructure as Code


• (IaC).


• Monitor cloud resource utilization and optimize operational costs.


• DevOps & Automation


• Design and maintain CI/CD pipelines using Jenkins, GitHub Actions, GitLab CI,


• or similar tools.


• Automate infrastructure deployment using Terraform, CloudFormation, and


• Ansible.


• Manage source control repositories and release processes.


• Implement monitoring and alerting solutions using Prometheus, Grafana, Splunk,


• ELK, or CloudWatch.


• Support containerization technologies such as Docker and Kubernetes.


• Develop automation scripts using Python, Shell, or Bash.

Operations & Support

• Provide Level 2 and Level 3 production support.


• Participate in on-call support rotations and incident management activities.


• Perform root cause analysis (RCA) and implement preventive measures.


• Create and maintain operational documentation and standard operating


• procedures.


• Ensure compliance with security, audit, and regulatory requirements.

Required Skills

• Apache Kafka Administration


• AWS Cloud Services


• Linux (RHEL/Rocky Linux)


• Shell Scripting and Python


• Jenkins, Git, Ansible, Terraform


• Docker and Kubernetes


• Monitoring Tools (Grafana, Prometheus, Splunk)


• Networking, Security, and High Availability Concepts


• Performance Tuning and Capacity Planning

Preferred Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or related field.


• Experience with Cloudera CDP, AWS MSK, Airflow, and Spark.


• AWS, GCP, Kafka, or Kubernetes certifications.


• Experience supporting large-scale production environments handling petabyte-


• scale data workloads.

Key Achievements Expected

• Maintain platform availability above 99.9%.


• Adopt AI-assisted engineering practices to improve operational efficiency,


• reduce manual effort, and accelerate troubleshooting and documentation.


• Automate repetitive operational tasks.


• Improve cluster performance and resource utilization.


• Ensure secure, scalable, and reliable data platform operations.


• Support enterprise data engineering, analytics, and AI/ML workloads efficiently.