Let's talk about the cloud skills gap

Lydia Leong recently wrote a thought-provoking piece suggesting that cloud adoption will fail because of the skills gap. This certainly isn't (or shouldn't be) news to those of us paying attention. The cloud has become progressively more complex as it has matured. There has been an explosion of cloud services, a rapid expansion of public cloud competitors that are achieving (or exceeding) feature parity for the most common use cases, and a third-party market that now contains more than 1,000 "cloud-native" tools, services, and platforms.

So is it hard to believe that the "cloud have-nots", i.e. the vast majority of developers who haven't had the freedom (or mandate) to experiment with cloud services, face significant headwinds on their own digital transformation journeys? It shouldn't be. I've been working with AWS since 2009, back when there were just a few services. Today, Amazon has over 250 services, and I'd guess that I'm proficient with less than 10% of them, and even that might be an exaggeration.

How much can any one person really know?

I don't think I'm alone here either. I've probably used well over a hundred AWS services, but oftentimes they were for an obscure part of the application. Or even worse, some sort of uninteresting infrastructure configuration. I likely spent hours looking through documentation and blog posts in order to flip some switch, and then almost certainly forgot what I did and why I did it. Time well-spent (I hope) for the sake of that application, but it was the kind of non-reinforced heavy lifting that will probably require a similar level of effort the next time I face the same issue.

What about the plethora of services at Cloudflare, Azure, GCP, and others? Add in offerings from the CNCF's "Cloud Native Landscape" and you might find the average cloud developer with proficiencies in the low single digits. I'm not extraordinary, and I know lots of cloud developers that hold much broader skillsets, but even the smartest person out there likely has a working knowledge of less than 5% of the total landscape. And even if that person does exist, you can't afford them.

And this is where the organizational skills gap that Lydia refers to really kicks in. Organizations of all sizes are typically full of people making the wrong technology decisions. It doesn't mean that the people making these decisions aren't smart, dedicated, masterful technologists, it just means that bias, ignorance, corporate pressure, and other constraints ultimately result in subpar choices. Choices that then force your organization to require resources with specialized skillsets. Even assuming that your organization can upskill or hire resources in the short term, maintaining competency and institutional knowledge in the long term is always a challenge.

What's the solution?

There is no easy or immediate solution. Companies aren't going to repatriate to their own data centers. That time has definitely passed. Cloud is simply the way forward. The "crisis-level skills gap" that Amy Sarah John refers to in 6 long-term planning trends by Gartner for cloud and edge computing suggests a few ideas, but focusing on Kubernetes and talent programs simply aren't going to get you there. There are too many things to learn and too high a turnover in tech. Couple that with a flurry of poor technology choices by upper management and most organizations will find themselves on the refactoring hamster wheel constantly adopting new silver bullets that they'll likely shoot themselves in the foot with.

The long term solution has to go beyond simply training more cloud generalists. Specialization will always be useful, but cloud is about composition and integration, and the complexity has outgrown the average person's (and organization's) capabilities. Right now there is a relatively small group of people that are able to effectively write "machine code" for the cloud. Everyone else is waiting for the cloud equivalent of C and C++.