Building the Agile Enterprise: Combining Kanban and tooling to scale Agile beyond your team
By adopting Kanban for our teams, employing an organization-wide release process and utilizing tooling for continuous integration of software, we have a process and system in place that allows us to effectively scale our Agile methodology across an enterprise and beyond a single team.
Like many organizations, Constant Contact adopted Scrum, hoping to see productivity gains versus the existing waterfall-based methodology. We did see improved team output and responsiveness to changing market conditions at first adoption, when only one or two teams were using Scrum. However, over time, we fell victim to our success. More teams, looking for similar gains, began to adopt Scrum. Coordination between Scrum teams became a challenge, especially as the number of teams grew to 20 and some of those teams moved on to adopt Kanban. Following Scrum, we employed Scrum of Scrums and Meta-Scrum to coordinate between teams and implement organization-level planning. Over time, we found that these practices did not scale and were not able to provide the performance gains we sought.
We addressed the challenge of organization-level planning in a variety of ways:
- We moved all teams to Kanban. This allows us to move beyond working in sprints and have teams deliver value more continuously.
- We modified Scrum of Scrums to be inter-team planning and dependency management
- We established organization-level backlogs and hold regular prioritization and backlog grooming exercises, in addition to doing so at the team level.
- We hold short, frequent releases and allows teams to integrate with a release or wait for the next release. We have implemented the concept of release trains.
- Importantly, we have created an IT and operations infrastructure to support these activities, based in part on these tools:
- vagrant for building and deploying virtual development environments to developers
- git for source code control, allowing flexible code branching strategies. Flexible branching is a key to achieving release trains.
- puppet for automated application deployments
- jenkins for automated software builds
- maven for managing build dependencies
Application developers can deploy to a vagrant virtual machine for local testing. Once the developer is satisfied with the code, it can be checked it into the local github. The change will be picked up by jenkins, which will perform the build and CI tests and will then push the build to the puppetmaster. Puppet will then push the change down to the test or production cluster. The infrastructure and tooling we have put together allows us to achieve continuous integration of software and moves us closer to continuous deployments.
On the process front, we collect a variety of metrics to gauge performance of teams and the quality of the software release. Cycle times for work items are averaged and tracked by team to give a measure of each team's throughput. Variance of cycle times are tracked as a measure of teams' consistency of estimation. Operational variables are tracked to measure the quality of deployments and releases. These includes: production down time, live defects, process defects, number of software patches needed, among others. We adopt cumulative flow diagrams from Kanban and use them to measure progress of releases and employ them across teams in addition to within a team.
We believe that our process and tooling work together to achieve better organizational agility than the basic Scrum methodology. Between our major and minor software releases, we are deploying production-ready software nearly weekly. Many organizations would be able to benefit from process and tooling enhancements that we have implemented.
This proposal is for the Agile ALM Connect portion of EclipseCon. We have enough material for a one-hour session. We could produce two 35-minute sessions, do a one-hour session or compress everything into 35 minutes. If possible, we would like one of the 1-hour spots.
High-level Bullet Points
- Principles of team Kanban have been applied to the organization to achieve more continuous delivery of value
- Cumulative flow diagrams are used to track the progress of an entire software release
- Tooling is necessary to achieve short release cycles. We use tooling to implement continuous deployments.
- Scrum of Scrums is modified to be inter-team planning and dependency management
- Organizational backlog is groomed and prioritized, feeding into team backlogs
Gil Irizarry is a certified Scrum master, a trained kanban coach and was PMP-certified from 2005 to 2009. He is currently a program manager at Constant Contact. He has presented at a variety of conferences and user groups, including:
- organized and delivered the session "Beyond a Scrum of Scrums: Scaling Up Agile with Kanban" at the Agile Development Practices West 2011 conference. Delivered a similar session for Agile New England.
- organized and delivered the class "Transitioning Your Team to Kanban: Theory and Practice" for Agile New England and Project Summit Boston 2011.
- led the "Agile in Production" meeting at ACM SIGGRAPH 2010
- led the "Managing Creative Projects" meeting at ACM SIGGRAPH 2011
- organized the Motion Graphics Birds-of-a-Feather meetings at ACM SIGGRAPH 2008, 2009 and 2010
- presented at the Boston Final Cut Pro and Motion Graphics Los Angeles user groups
- organized and led the course "Know Your Rights: A Legal Primer for Software Developers, Artists, and Content Creators" at ACM SIGGRAPH 2008
Gil has a BS in Computer Science from Cornell University, a Master of Liberal Arts in Visual Studies from Harvard University and a certificate in Management from MIT Sloan School of Management. He has been actively practicing Scrum and kanban for the last five years.
Karen Hannon has a BS in Computer and Information Science from University of Massachusetts at Amherst. She is a certified Scrum master. She has been both an engineering manager and product owner, leading both Scrum and kanban teams. She is currently leading the effort to improve continuous deployments at Constant Contact.