SlackOps - Automation Framework

At Slack, the operations team relies on a suite of tools and libraries named SlackOps. This framework has substantially increased our velocity by providing consistent tooling and enhanced our incident response toolkit. This presentation will give an overview of the development lifecycle, and walk through the procedure to build, test, and deploy operational tools.

These tools provide us with the benefit of repeatability and testability, while also maintaining the flexibility given by a scripting language like Python. Also, we have found them simple enough to be used by new team members, yet critical enough to be continually improved over time. We will also share our success applying this pattern to essential Python services designed to run on each of our hosts in production.

This talk will provide a hard-learned set of best practices for structuring your DevOps toolkit. Although focusing on Open Source Python tools like flask, nose, and boto, the approach will also apply more broadly to your language. Thanks to our use of pex, we can build self-contained python binaries which do not require complex dependency management- we will detail our build/deploy pipeline utilizing Jenkins and Chef as well.