Organic isn't always good for you
After the nth time on an hours long call with multiple engineers trying to sort out what should have been a simple issue, we realized something needed to be done. As a fast growing startup, infrastructure was built by those without expertise or knowledge of best practices. Environments were not architected, but developed organically. Items were built as quickly as possible, not accounting for the -ilities - scalability, reliability, maintainability (and security). Trying to retrofit those concepts into our existing ball of yarn would result in outages, tears, and extremely slow progress on resolving any tech debt.
In this talk I will go through our research process to to determine that moving to a greenfield IaC solution was probably the shorter and cheaper path forward. Greenfield is a bad word in a lot of companies, but that's usually because tech orgs will get to about 60% and are now stuck maintaining two environments.
As this is currently a journey in progress, I will share some of the techniques we used to get developer support of our initiative, and hopefully drive us to a state where we'll be able to entirely sunset the legacy environment. Make the new environment so appealing they won't want to work in the old.
I will also go into the design of our new infrastucture to create this appealing landing zone, including making use of open source tools and Kubernetes to do things as inexpensively as possible and using industry standard methods. As well, giving consideration for on demand resources to minimize costs. The end result should be infrastructure that can support a company through technical maturity and future growth, while taking into account security, speed of delivery, and resilience.
This presentation should leave you with some insight on alternatives that you might be able to chase in your own company when you're drowning in a pile of bad networking, resources that would have been retired if they were tagged, questionble data flows, and clickops from someone you've never met.
