[Originally published in TechCrunch on August 10, 2021]
I worked at Google for six years. Internally, you have no choice — you must use Kubernetes if you are deploying microservices and containers (it’s actually not called Kubernetes inside of Google, it’s called Borg). But what was once solely an internal project at Google has since been open-sourced, and has become one of the most talked-about technologies in software development and operations.
And for good reason. What used to take a large team of engineers can now be accomplished by a single developer from their laptop. At times, Kubernetes can feel like a superpower. But with all of the benefits of scalability and agility comes immense complexity. The truth is, very few software developers truly understand how Kubernetes works under the hood.
I like to use the analogy of a watch. From the user’s perspective, it’s very straightforward until it breaks. To actually fix a broken watch requires expertise most people simply do not have — and I promise you, Kubernetes is much more complex than your watch.
So how are most teams solving this problem? Well, the truth is many of them aren’t — they often adopt Kubernetes as part of their digital transformation, only to find out it’s much more complex than they expected. So then they have to hire more engineers and experts to manage it, which in a way defeats the point of Kubernetes in the first place.
What does all of this mean? It means there is a great opportunity for DevOps startups to come in and address the different pain points within the Kubernetes ecosystem. Because the truth is Kubernetes isn’t going anywhere, so any platform or tooling that helps make Kubernetes more secure, simple to use, and easy to troubleshoot will be well appreciated by the software development community.
It’s my belief that Kubernetes is becoming the new Linux: 96.4% of the top 1 million web servers’ operating systems are Linux. Similarly, Kubernetes is trending to become the de facto operating system for modern, cloud-native applications. Kubernetes is already the most popular open-source project within the Cloud Native Computing Foundation (CNCF), with 91% of respondents using Kubernetes — a steady increase from 78% in 2019 and 58% in 2018.
While the technology is proven and adoption is skyrocketing, there are still some fundamental challenges that will undoubtedly be solved by 3rd party solutions. Let’s go deeper and look at 5 reasons why we’ll see a surge of more startups in this space.
1: Containers are the go-to way of building modern apps
Docker revolutionized how developers build and ship applications. Container technology has made it easier to move applications and workloads between clouds. It also provides as much resource isolation as a traditional hypervisor, but with considerable opportunities to improve agility, efficiency, and speed.
And where you see containers, you see Kubernetes to help with orchestration. According to Datadog’s most recent report about container adoption, nearly 90% of all containers are orchestrated. By using an orchestrator to automate aspects of container deployment and maintenance, organizations can not only deploy faster but also ensure that their applications run reliably at scale.
2: The Kubernetes ecosystem is rapidly maturing
All public clouds have already adopted Kubernetes and offer their own distributions, such as Google Kubernetes Engine (GKE), Amazon Elastic Kubernetes Service (EKS), and Azure Kubernetes Service (AKS). Last year, the Steering Committee launched a new contributor-focused website, Kubernetes.dev, which brings contributor documentation, resources, and project event information into one central location. The Certified Kubernetes Security Specialist (CKS) certification program was also recently launched, joining the Certified Kubernetes Administrator (CKA) and Certified Kubernetes Application Developer (CKAD) programs.
3: Security is the canary in the coal mine
Security is a top priority for organizations (and is becoming more of a focus for developers with the rise of DevSecOps). We’ve seen Twistlock and Bridgecrew acquired by Palo Alto Networks. Alcide was acquired by Rapid7. Acquasec is now a unicorn. These are extremely successful security companies that have hyper-focused on modern cloud environments.
Now that the security market for Kubernetes is saturated, we’ll see a second wave of startups that can now focus on Kubernetes performance, ease of use, and troubleshooting. We’ve already seen Pixie Labs acquired by New Relic and there will be plenty more stories like it in the coming years.
4: Digital Transformation of financial services
Technology being adopted and embraced within the financial industry is a good indicator that it is here to stay. They are dealing with highly sensitive data and can’t afford to use technology that hasn’t proven it’s secure and reliable. And yet we continue to see banks adopting cloud technologies and building modern applications.
According to PwC’s COVID-19 CFO Pulse Survey, when they asked finance leaders which programs they wanted to protect and which they were more likely to cut, “digital transformation” fell firmly within the protect camp, along with customer experience and cybersecurity/privacy. Many of the financial technology firms (‘fintechs’) and mobile app-based challenger banks have taken advantage of modern application architectures and DevOps practices that are associated with cloud-native technologies, such as microservices and Kubernetes.
Here’s a great case study about how CapitalOne adopted Kubernetes to help with their digital transformation journey.
5: Open source is past the hype curve
Open source projects have a well-known and often analyzed hype cycle that is well-documented by Gartner and many other industry analysts. It takes open-source projects quite a few stages and years to reach a level of maturity and productivity to deliver on their promise and be truly enterprise-ready.
Once open source projects reach this plateau of productivity, you’ll find that many tools, projects and complementary capabilities are built on top of these to enable them to be even more production-grade. If we can take cues from other open-source projects – many times this is in the form of technology distributions or pre-built plug & play stacks.
There are many good examples of companies with valuations of greater than numerous billions supporting & building upon open source projects. Some of these include popular Linux distributions such as Red Hat Enterprise Linux, Suse, Ubuntu/Canonical, production-grade Apache projects such as Kafka and Spark from Confluent and Databricks respectively, and even smaller yet successful platforms such as Logz.io built upon the Elastic “ELK” stack – Elasticsearch, Logstash, and Kibana).
Kubernetes today has reached a level of maturity where it has achieved adoption in the largest organizations, and exciting new tooling is being built upon it that to has the potential to reach greater than billion-dollar valuations, to take it to the next level and its next phase of growth. Now would be an excellent time to bet on Kubernetes.
