Je suis DBA —

It’s the data, stupid: Why database admins are more important than ever

Specialized databases, cloud, and DevOps expand, not eliminate, role of the DBA.

It may not be all about the tables anymore, but the DBA role is still essential—even if the person doing it doesn't have the title.
It may not be all about the tables anymore, but the DBA role is still essential—even if the person doing it doesn't have the title.

For those of us who have been in the information technology realm for too long, the title "database administrator" conjures up very specific images. We picture someone pulling hair out over issues with backups or snapshots not happening, schemas growing out of control, capacity plans blown up by new application demands, sluggish queries, and eternal performance tuning.

That old-school role of the DBA still exists in some places, particularly large enterprises where giant database clusters still rule the data center. But virtualization, cloud data storage, micro-services, the "DevOps" approach to building and running applications, and a number of other factors have significantly changed how organizations store and manage their data. Many of the traditional roles of the DBA seem to be moot in the shiny, happy world promised by the new generation of databases.

"NoSQL" databases don't require a pre-defined schema, and many have replication built in by default. Provisioning new servers can be reduced to clicking a few radio buttons and check boxes on a webpage. Development teams just point at a cloud data store such as Amazon Web Services' Simple Storage Service (S3) and roll. And even relational database vendors such as Oracle, Microsoft, and IBM are pushing customers toward data-as-a-service (DaaS) models that drastically simplify considerations about hardware and availability.

You might expect this to mean that DBAs' jobs are getting easier. If so, your expectations would be wrong.

"I think [DBAs'] roles have become much more complex," said Chris Lalonde, vice president and GM of Data at Rackspace. "While there is definitely more automation and tooling, the counter to that is that many of the newer technologies are less mature and require more care and feeding. I would say that many of the traditional tasks of DBAs still exist today or need to exist."

In fact, all these great new database technologies highlight the data professional, whether that person is called a DBA, data architect, data engineer, or, in some cases, data scientist. "Data is even more important today," said Kenny Gorman, a database veteran and co-founder of the real-time data service company Eventador. "Businesses used to rely on databases to be sound, run smoothly, and give good reporting. But now, data actually makes you more competitive, and there are more job titles working with data and more technologies that use it. And the database professional is at the core of that."

One step forward...

Non-relational platforms offered a promise to reduce the workload of DBAs, and in some ways they do. Ravi Mayuram, senior vice president of products and engineering at CouchBase, compared the shift in what DBAs have to do to how driving a car has changed over the years: once upon a time, "to drive one you had to essentially be an engineer, and when something went wrong you needed to pull to the side of the road and get under the hood." Now most things take care of themselves, he said, "but I can't do anything myself to fix it."

Databases such as MongoDB and CouchBase, while not relational, support SQL queries, and they have other aspects that make them approachable to experienced DBAs. But they also allow for "dynamic deployment decisions, which you couldn't do with relational systems," Mayuram claimed. "Adding new data structures used to require a schema change and downtime."

Data as a Service has been embraced for "a fraction of what companies do," Mayuram said. "Most companies don't have mission critical information in the cloud."

While a big relational database system requires an understanding of everything about the hardware and software stack, "the next generation of DBAs will be less involved in that," Mayuram explained. "There will be a requirement for a DBA—someone who is more database intensive, but not exclusively," to focus on tasks like capacity planning. The DBA of the future will need to know when to provision more servers and when to retire them.

That sort of dynamic scalability is what has driven the adoption of cloud-based data services based on specialized databases and “Data-as-a-Service” schemes (either built in-house or hosted with a vendor). In either case, provisioning services can take care of setting up hardware, network, and storage. In theory, the DBA focuses on figuring out when applications will need more database capacity. "This is a DevOps sort of role, dealing with dynamic provisioning—it's a slightly different profile," Mayuram said. "With more efficiency, they'll need a fraction less of DBA skills, but that requires them to be more capacity planners and understand the development side better."

For those unfamiliar with the term, DevOps is a practice now used widely in Web and service development. It describes application development teams working in collaboration with IT operations staff to continually improve performance, automation, and scalability of software and systems. The DevOps approach has been a major driver of the adoption of NoSQL databases and other non-traditional data storage and query technologies. DevOps has driven the development of Data-as-a-Service—largely because of the need to automate the scaling up and down of database capacity. But even in the purely relational world, the shift toward turning databases into a cloud service is reducing the need (and the ability) for DBAs to have fine-grain control over hardware configuration.

So far, Data as a Service has been embraced for "a fraction of what companies do," Mayuram said. "Most companies don't have mission critical information in the cloud." Early adopters, he noted, are taking a hybrid approach, with some creating internally hosted DaaS platforms based on cloud computing platforms within their own data centers. But other companies are largely keeping their critical relational systems as they are and using cloud approaches for new projects. "They still have DBAs taking care of existing apps and have DevOps teams handling database deployment in a micro services environment—services that don't need to be in a relational system," Mayuram explained.

With companies preserving their relational databases and increasingly needing to bridge the gap between the old and the new, things got more complex instead of simpler. And even when organizations completely outsource the applications that have typically placed the biggest demands on DBAs, they're still left with the need for some sort of data professional to make sense of what they've gotten themselves into.

Channel Ars Technica