Case: event-driven automation with Kafka and Ansible
Starting point
The client wanted to provide a new self-service capability for their own clients. Under the hood this new feature required creating, modifying and deleting infrastructure on demand. So far such changes had been done in a static way with infrastructure as code tools which was not dynamic enough.
Project
We started the project by creating a development environment for the Ansible code that was required in the project. Once the code was mostly functional we spun up a staging environment in AWS EC2 for testing automation in a production-like environment where we could process messages from Kafka and do integrations with AWS and Keycloak. As the final step infrastructure automation code was deployed to production.
End result
At the end of the project the steps that were done manually earlier were automatically handled by infrastructure automation. The tooling created a Git commit for each change so that changes could not be introduced without leaving a trace. Ansible did not manage just the individual changes but all of the affected infrastructure. This ensured that configuration divergences got fixed automatically.
Follow-up
The original event listener was home-grown. Some years later we upgraded the whole systems and migrated to ansible-rulebook (also known as “event-driven Ansible” or EDA for short).
Main technologies
- Ansible
- Ansible Rulebook
- Kafka
- Keycloak
- AWS