As I mentioned in this episode of Chaos Lever, re:Invent 2022 happened. And it was fine? Fine, yes. This is the 11th re:Invent and while it’s not the largest ever, attendance has certainly recovered from the pandemic. Depending on who you ask there was somewhere between 60k and an infinite number of people at the conference. The high end estimate came from an AI trained in crypto finance, so I wouldn’t trust anything it has to say relating to reasonable numbers.
There were also plenty of people attending virtually or more accurately, sporadically viewing remotely, in some cases in Las Vegas itself. As much as AWS tries to make getting around easy between the four major locations, it’s still a slog to get from the MGM to the Venetian, and sometimes it’s just easier (and more comfortable) to watch from a screen a few miles away from the actual event.
I was there for a couple days only, less than that in fact. And I focused my efforts on walking the expo hall and chatting with vendors, especially in start-up alley. Eschewing the larger vendor booths in favor of constructions that could actually be called booths and not encampments for a prolonged siege.
Honestly, I’m not sure what one has to gain from wandering into the dense forest of screens and marketing effluvia that populates a Diamond Sponsor- oh, excuse me, Emerald Sponsor (Emerald, like all the extra green you have to pay over Diamond) booth. For example, VMware. No offense meant, but why would I go there? I know who the company is, I know what they do, and they have battalions of marketing people who already keep me abreast of the company’s latest escapades via multiple mediums and formats. Unless I recognize one of the poor souls assigned booth duty, I am unlikely to go near any booth that takes up more than a few square feet.
No, I submit to you the actual vendors of interest can be found in the periphery. And they are desperate for human contact after being ignored by swaths of people. I had excellent conversations with XoSphere, Aiven, HYCU, Structura, and Retool.
But what about AWS itself? What was the big news? Any crazy new services or amazing improvements? Despite well-dressed people talking at Keynotes for many, many hours, there doesn’t seem to be one blockbuster thing people are buzzing about. If I peruse the tech news over the last two weeks, it is as if re:Invent barely happened. The tech zeitgeist is squarely focused on the nightmare of Twitter, the continuing disaster of crypto, and the rise of OpenAI’s weird stuff that will eventually destroy multiple industries and people. Oh, and a new version of Kubernetes (1.26) is out. And maybe that’s the biggest point of all. Open source platforms vs. closed-source primitives.
AWS and the public cloud in general have moved into the boring infrastructure category and out of the innovation seat, and no wonder. Cloud computing, depending on how you measure it, is now about 15 years old. While it has enabled incredible transformation of industries and disrupted traditional companies in countless ways, cloud computing and the big public cloud providers have succeeded in becoming staid and boring. That’s not necessarily bad. One thing that the cloud providers did create was self-service access to basic primitives, like IaaS, for companies to build new solutions on. In fact, that has long been AWS preferred mode of operations, creating primitives that developers can combine in new and interesting ways.
The problem with primitives, is that there can be too many of them and too many ways to combine them. And sometimes developers and infrastructure people appreciate a helping hand. AWS has something ludicrous like 300+ services. If we assume that each of those services represents a handful of primitives, that is an infinite number of possible combinations to get right, or horribly wrong. One of the things I’ve always felt about AWS is that it was like a set of tinker toys without an instruction manual. You can build anything you want, and they’re not going to tell you how. That can be overwhelming to the cloud architect and developer alike.
Consider the developer experience of 10 years ago to what they need to contend with now. It’s not just choosing which programming language to use, but also which framework to use, and if you’re using Javascript today like most folks, there are about a bazillion last I checked. Next you need to deploy that framework in a target environment. Do you use EC2 instances, containers on ECS or EKS, try to package things as Lambda or run them in Fargate? How do you handle your database backend, are you going to use Postgres or something Postgres compatible? And if so, do you go with a proprietary cloud DB, or an open source DB hosted as a platform, or host it yourself in that EKS cluster? Oh, and you’re going to need ingress controllers, or load balancers, or API gateways. And security! Don’t forget about security! It’s enough to make a dev want to throw their hands up and go back to deploying ASP.Net apps on IIS with a basic MSSQL backend.
AWS has always excelled at providing primitives that enable programmers to do whatever they want. They are looking for builders! Unfortunately, developers and infrastructure folks alike are overwhelmed with choice. The obvious answer is to conjure up a platform based on those primitives, providing an opinionated environment for application development and deployment. AWS has tried that in the past with things like Elastic Beanstalk, LightSail, and App Runner, but even then the seams stitching together the underlying primitives were showing. EBS in particular was an imperfect, leaky abstraction, and AWS chose to do that by design. It’s like they knew they had to build a platform, but they didn’t want to seem heavy handed or reduce customer choice, and in the end it’s a half-baked mess.
Where some see a failure, others see opportunity! And there are several vendors today that specialize in providing a platform for app deployment that lacks choice deliberately. The idea is to get the code deployed as quickly as possible, without the developer needing to know too much about the underlying primitives; ideally nothing. Fly, Fastly, Vercel, and Fermyon are all laser focused on getting you from idea to code to running app in the least amount of time possible. Are there tradeoffs and lock-in? Absolutely. But that’s comparatively cheap compared to the hourly cost of engineers who are spending entirely too much time fiddling with primitives and doing undifferentiated heavy lifting on AWS.
Many of the platform companies are actually building their solutions on AWS, Fermyon for instance has started out with a fleet of EC2 instances using custom AMIs. AWS isn’t necessarily losing out on revenue as a result of the new platforms, but the innovation is from the new platforms companies using AWS, not from AWS itself. Again, this is fine and to a certain degree expected.
I’d compare the journey of public cloud to that of another game changer in the industry, VMware. In its halcyon days, VMware was innovating on what a virtual machine could be and how it might be managed. It’s hard to overstate how groundbreaking vMotion and Storage vMotion were. How DRS could rebalance your hypervisor while you slept soundly at night. How developers could get a development environment spun up in days instead of weeks. VMware sure had some sweet primitives to build on.
VMware also tried to crack the platform nut when it comes to application development. Their acquisitions of Spring with their framework and later the Pivotal platform, were supposed to give them platforms that developers would embrace, trading complexity and choice for simplicity and speed. While Spring has its adherents and there are certainly fans of Pivotal, as a whole the introduction of containerization and Kubernetes threw a bit of a wrench in VMware’s plans for complete global dominance. In short, they backed the wrong horse, and were slow to get on the right one with Tanzu. Now that we have Broadcom looming in the distance, VMware is unlikely to become a dominant application platform, forever existing at the infrastructure layer.
That’s a profitable place to be, but it doesn’t drive growth in the way investors have come to expect.
The second major thing to bear in mind about AWS is the change of leadership. Andy Jassy has made the transition to CEO of Amazon proper and handed the mantle of AWS over to Adam Selipsky. And while they might seem like two completely interchangeable, middle-aged, white guys, their presentation style and management style are remarkably different. That difference will be reflected in the way AWS is run going forward. While there is little chance major changes are coming to the organization structure or principles espoused by leadership, there is a certain calcification settling in at AWS. Does that mean their position of cloud dominance could be in danger?
It’s tempting to equate AWS’ position to that of Microsoft in the early 2000s. Dominant at the desktop and server OS levels, Microsoft was blindsided by the one two punch of mobile and Linux. Web 2.0 is built on the back of open source technologies and not Windows Server with its bloated stack and aggressively expensive licensing. Web 2.0 is also built on browsers, something that Microsoft one had dominance in, but was edged out by another company that had embraced open source, good ole Google with their Chrome browser. At the same time, Microsoft failed to get mobile right not just once, or twice, or even three times. I think I can count at least 5 separate failures of Microsoft to break into the mobile ecosystem. Look up the Kin if you’d like a good laugh. They’ve finally thrown in the towel and gone with Android, another Open source based product, and even then the Surface Duo is… not great.
Many analysts assumed at the time Microsoft was on the road to become a has-been by the mid-2010s, based on their trajectory and lack of success in so many areas. It took some hard choices and big shake-ups at the company to turn things around and recapture market share in a completely new way. Services instead of licenses, and platforms instead of primitives.
Do I see AWS falling into a similar spiral in the next few years? Not really, like I said, it’s tempting to compare the two, but I don’t think it makes sense. Microsoft’s problems were a misunderstanding of the future and some serious negativity from developers and IT folks. I believe AWS understands the market it operates in and will continue to grow in both revenue and profit for many years to come. Providing great primitives is useful to companies that want to build a platform, and AWS tends to be sticky for platform companies that don’t care to build for multi-cloud or host their own datacenter. We can have a conversation another time about cloud repatriation and the economic break even point of self hosting.
Directionally, AWS is fine. And they are focusing on what I think the next big frontier is, whether we like it or not, that of machine learning and AI. The data and analytics area of the Expo was booming and it also received its own keynote. I expect AWS will continue to build primitives and host platforms for other vendors, but I wouldn’t expect any major new services to be launched in the next few years. AWS is no longer re:Inventing cloud; they are providing the space for others to do so. The biggest threat to AWS’ dominance is not other cloud providers, it’s a better set of multicloud primitives offered at lower cost.
Creating Your First Terraform Repository
March 4, 2024
Comparing OpenTofu and Terraform
January 22, 2024
January 17, 2024
