Gene Kim was bitten by the entrepreneurial bug early. In 1992, while still a student at Purdue, Kim co-authored an open source tool called Tripwire, which would become a free software security and data integrity tool useful for monitoring and alerting on specific file changes on a range of systems. Kim would become the Chief Technology Officer of Tripwire, a role he would have until mid-2010.
In 1999, while still at Tripwire, Kim began to formally study IT organizations, noting the methods used by high performing organizations. One observation was that these organizations often had IT operations, security, audit, management, and governance working together to solve common business objectives. This research would eventually lead to a number of books that Kim would co-author. Visible Ops Security was released in 2004, and The Visible Ops Handbook was released in 2009. When Kim left Tripwire, it was to dedicate himself to this research, to speaking, and to consulting with companies around the world.
In 2013, Kim (together with Kevin Behr and George Spafford) authored The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win. As the title suggests, the book is told as a novel. The hero of the story is Bill, the IT manager of a company called Parts Unlimited. The company is pursuing a critical IT initiative that is over budget and late. The CEO tells Bill that he has 90 days to fix the mess, or IT will be outsourced. The notion of DevOps is born, referring to a development method that pushes communication, collaboration, integration, automation and measurement of cooperation between software developers and other IT team members. DevOps also highlights the interdependence of software development and IT operations. It aims to help an organization rapidly produce software products and services and to improve operations performance.
Though Kim and his co-authors did not create the idea of DevOps, their novel has helped to popularize it.
As Kim notes from his study of over 14,000 IT professionals worldwide, high-performing organizations are two and a half times more likely than their peers to exceed profitability, market share, and productivity goals.
(To listen to an unabridged podcast version of this interview, please visit this link. To read future interviews like this one, please click the "Follow" link above.)
Peter High: Gene, what is driving demand for DevOps?
Gene Kim: I have been studying high performing technology organizations since 1999. I think the reason everybody is so interested in DevOps is that it solves a problem that we have all experienced in our careers, which is how do we simultaneously enable the fast flow of features from delivery through test and operations while preserving world class reliability, stability, and security. I think most of us felt these were mutually opposed; you could either get fast flow or great reliability. What DevOps really represents to me are the cultural norms and the technical practices that enable us to finally get both. We know that organizations can do this because companies like Amazon, Google, Etsy, and Netflix have been able to replicate this. We can get incredibly high rates of flow from Dev to Ops, as measured by high frequencies of deployments and short lead times, while also preserving great reliability. We phrase it like this because it is part cultural and part technical. My area of passion is really codifying what are those necessary steps to get from here to there.
High: How does one get “from here to there?” What are some of the people and process changes that an organization has to think about in order to explore DevOps more fully?
Kim: Part of it is cultural, part of it is incentive structures between Dev, Test, and Operations, and part of it are the technical practices like continuous testing and delivery. These are all the preconditions that get one to compress dev-test cycle from months to, ideally, minutes, as the high performers do.
High: From a staffing perspective, do you find that there is a certain type of person or set of skills that organizations should recruit in order to achieve this?
Kim: There are certain characters that we see in every one of these high performers. In the Phoenix Project we framed it in three ways, which tended to be a set of principles from which one can derive all of the observed DevOps patterns. The first is all about accelerating flow as you go from left to right, Dev to Ops, in the value stream. The second is the flow of feedback, how do you create effective feedback from Ops back to Dev so that when something goes wrong you can either prevent it or detect it more quickly. The third is about creating a culture of continual learning and experimentation using the notions of high trust culture. The most competitive organizations are ones that can learn, those who can turn local improvement into global solutions.
Broadly speaking, that means we need managers and practitioners who can work not only in their Dev, Testing, or Ops siloes, but can optimize outcomes for the entire value stream. The goal is the fast flow of idea to production, as well as fast feedback when something goes wrong. That sets the stage for doing things like AB testing and creating organizational learning not just through dev, test, and operations but through the entire business value stream. Ultimately, I think that is how we win in the marketplace. Create a learning environment. That has been in the literature for decades but has never been more relevant than in IT, specifically DevOps.
Gene Kim
High: Is there a common denominator—size, scale, or speed—in organizations that you have found have been able to leverage DevOps? It goes across industries, but are there certain kinds of companies that are best suited for this?
Kim: That question is what set us out to run the DevOps Enterprise Summit back in October. The goal was to not have the “unicorn” companies (Amazon, Netflix) present, but the horses. What we were interested in was how large, complex organizations that have been around for decades—maybe even centuries—are adopting DevOps practices and replicating outcomes that we have seen with the unicorns.
The stories that were told were, in my mind, breathtaking. It was organizations like Disney, GE Capital, Nationwide Insurance, the Department of Homeland Security. All of these organizations that have the legacy of success, yet they are all realizing that winning in the marketplace will require them to focus not on control costs but optimize for speed.
There were three takeaways for me. The principles are very much the same, but level of savviness and sophistication required to actually mobilize the subversive and innovative effort like DevOps is breathtaking. Essentially, it takes a courageous leader to mobilize an organization. Often they have to take on the naysayers; it takes a very special person to be able to drive a transformation like that.
I think the paths taken had a lot of similarities. They were often driven by architecture, or a Director of Operations or Development. Some of these leaders would show up in the most surprising of places.
There were fifty speakers, and I asked all of them to end with a slide that says “here is what I am looking for help on.” The three themes that showed up the most often were; one, how to do automated testing with legacy apps that have been around for decades; two, how to overcome the obstacle of DevOps when Security and Compliance is not in favor; and three, how to get upper management to allow us to mash the pedal to the floor. Those three represent the grandest challenges in acceleration at large, complex organizations.
High: It seems to be the confluence of factors from a technology, process, and people perspective that are fueling this need for speed. I would love your opinion on that.
Kim: I couldn’t agree with you more. I don’t say this lightly: I think we are living in a time of miracles. There are things that are becoming commonplace that were just not possible five years ago. In fact, the notion of doing tens, hundreds, let alone thousands of deployments per day is something that we probably would have thought immoral and hopelessly reckless even five years ago.
In order to get fast flow we need to create feedback a quickly, cheaply, and as rampantly as possible. It used to be the fact that development was responsible for, at best, writing unit tests and checking that in. These days, it is now considered completely commonplace that every developer has an entire integration-test environment running in containers on their laptop. That is something that used to take hours, or probably more likely, days or weeks to do in the old days. I think this so dramatically increases development productivity and our ability to innovate and test ideas in the marketplace. It is almost as if we finally figured out what technology is for.
High: Containers seem to be coming up more and more in my conversations with executives. Can you give an overview of the concept of containers?
Kim: I think the best way is to give a concrete example that caused my jaw to hit the floor. This story came from Elizabeth Sullivan, she helped pioneer the concept of automated testing ten years ago and is now the Director of Quality Engineering at Pivotal Labs. She described how one of their biggest problems when she joined two and a half years ago was that for any change put into production, it would take developers thirty minutes to two hours to have to deploy the changes. Her conclusion was that that was far too long. In other words, if developers had to wait four hours to discover if the one change they made was good or bad, you essentially won’t know until it blows up in front of the customer after the big deployment.
The resolution was to take the entire integration-test environment, which is very complex, and get it all running on dev laptops so that when a developer makes a change they can now spin-off an entire new surface area in integration-test environments in less than five minutes. By giving developers fast feedback on their work, it dramatically increased the quality of their work as well as the customer outcomes.
High: I know you’re working on your next book right now. Can you talk a bit about some of the concepts you are putting together in your next book?
Kim: The Phoenix Project, we put that book out almost two years ago now. That’s been the most fun adventure I have ever been on. It was really put into the form of a novel. Much like many of us that read The Goal, a novel about plant management, which causes many ah-ha moments; our goal was to see if we could replicate those ah-ha moments and use that for the current generation of technology.
So the DevOps Handbook is really meant to be the nonfiction companion guide that describes in detail how the management team of Parts Unlimited (Phoenix Project) achieved their transformation. We break it down in steps from a planning and organization perspective to the, say, four or five projects to reduce lead times, increase deployment frequencies, etc. That is based on studying how high performing organizations work and the learnings we have had from benchmarking over 14,000 organizations.
High: I am curious, how did you elect to do The Phoenix Project in novel form?
Kim: When I read The Goal, it really just changed my life. And that was probably fifteen years ago, and even though I had no experience in plant management there was no doubt in my mind that the lessons being imparted in that book were relevant to the work we do in IT.
I think that the surface area of some of the problems that DevOps solves goes against some of our biggest preconceived notions, and I think the form of a novel is very well suited towards that just to show that there is a common journey that we have. So many of the problems we commonly see in our careers can be reflected in how dev, test, and operations work—or don’t work—together. Spending half of the book portraying those problems to a fictitious company gave us the right to be able to describe what the solution is. I think only once we show that we understand the problem do we earn the right to talk about the solution.
High: You were a cofounder of Tripwire and the CTO of the organization for roughly thirteen years. How did you decide to go from being a successful entrepreneur to writing and researching as a primary occupation?
Kim: My journey with Tripwire was one of the most rewarding and satisfying points of my career. One of the key parts of that was being in a position where I could study these high performing organizations. That journey started in 1999, and by the time 2006 rolled around it became a study not just of how operations and security were working together, but how they were achieving all IT goals involving dev, test, operations, and information security. That has been the passion, for me, for the majority of my career. I’ve never been having as much fun as I am right now.
_____________________
To receive updates about The DevOps Handbook release and receive an advance copy, please sign up here.
Peter High is President of Metis Strategy, a business and IT advisory firm. His latest book, Implementing World Class IT Strategy, has just been released by Wiley Press/Jossey-Bass. Peter will provide a free video or teleconference lecture on the book for any team that purchases copies of it for the IT leadership team. He is also the author of World Class IT: Why Businesses Succeed When IT Triumphs. Peter moderates the Forum on World Class IT podcast series. Follow him on Twitter @WorldClassIT.
