A 100-year-old organization's journey from mainframe to open source to DevOps

623 readers like this.
Devops gears laptop

The Church Pension Group provides insurance, annuities, and pension products to 35,000 members, including Episcopalian clergy and spouses, lay employees and spouses, and administrators. From an IT perspective, we are a shared-service organization with eight executive vice presidents and a CEO. Our 75-person IT department is based at our headquarters in Manhattan, and we also support an investment office in Hong Kong, a denominational publisher, a call center in each U.S. time zone, and field reps that sell our products throughout the country.
 
When I started here 16 years ago, we were a highly siloed group of business units, each with its own IT staff and consultants. While we had faithfully serviced our clients for over eighty years, it was time to bring our core technology up to date and have it match our service levels. The mainframe that never missed cranking out a pension check hadn’t been coded since 1969, and the staff used a Lotus 1-2-3 spreadsheet to load the pension check numbers into it and calculate pension benefits based on the highest seven of eight years of earnings – since the mainframe only held five years of data. In fact, when I walked into the building in 1999 our server room consisted of a picnic table with six machines someone had bought at CompUSA. There was no TCP/IP in the building, just a single DSL line that helped a NetWare machine talk to the Internet. After founding a startup in San Francisco’s Silicon Valley, this was all a bit mystifying.
 
Fast-forward a decade. After bringing IT closer to present-day reality, I helped lead the charge out of siloed, old-school behaviors and towards remaking the enterprise in the way our clients saw us, which is as one entity. Shared service organizations started springing up to replace the highly people-intensive processes sitting in their own siloes. Two of these people-intensive, hands-on groups were the network services and system administration groups in IT, and part of the change has been to hasten the evolution of these groups into a DevOps organization. 

From mainframe to open to DevOps

A first step out of this impasse was to look at virtual servers as a way of saying, “How can we get more bang for our buck?” Our server room represented a lot of very pricey real estate, but it only seemed to keep growing. We asked ourselves, why did additional growth have to mean a new electrical circuit and adding new cooling? Within about three years we were 90 percent virtualized, and we got very comfortable with running virtual machines.   
 
As we have evolved to an organization that is more about letting developers work on code versus plugging machines in, we adopted Puppet, an open-source configuration management utility. This has very much simplified our ability to get a machine into a developer’s hands, and what used to take a couple of hours now takes seven minutes. The result has been contagious in a good way, because once we started Puppetizing everything developers said, ”It’s nice for you to get me a machine, but I need my entire stack.” Now we have started to deliver stacks of development environments rather than a single machine and we are looking seriously at containers.
 
As a 100-year-old organization, we do tend toward waterfall in our methodology. But I believe that the flexibility of DevOps made moving to agile a little more comfortable for our developers.  This year we moved to a hybrid approach between waterfall and agile. That meant getting the business comfortable with agile from beginning to end. Think of it as waterfall specification and definition, then iterating stories inside of that so the business people could say, “Okay, you know where we’re going, so we’re going to do this piece now. How does that piece work? All right. Great. Now we’re going to do this piece.”
 
As long as the business knows that we have their goals in mind, they are willing to bear with the agile approach. It’s been highly effective to put their anxiety aside by showing them that we’ve got a clear map ahead of us. At the same time, a DevOps approach has helped our developers be comfortable with, “Oh, we need to quickly react. And we need four new environments because we’ve got to get this story delivered, and we need to be ready for our planning meeting tomorrow morning.” So it continues to be a win-win for us, and the way our clients see us is finally becoming the way we really are.

The Rev. Clayton D. Crawley is Executive Vice President, Chief Information Officer of the Church Pension Group. Ordained to the priesthood in 1994, Clayton was a founding partner at an Internet start-up company in San Francisco until taking the position as Associate Rector at All Saints' Episcopal Church in Palo Alto in 1997.