This post is in response to the ongoing discussion on the future of open-source, recently re-kindled by Amazon’s Open Distro for Elasticsearch announcement. This analysis is admittedly biased towards specialized data systems. Things like Kubernetes and Docker that are lower in the stack have larger followings and different open-core market dynamics.
Since AWS announced their Open Distro for Elasticsearch, there has been an ongoing discussion about what this means for open-source communities and the “open-core” companies build around them. This has been brewing for a while: AWS has gone after vendors behind open-source tech before (see DocumentDB and Correto). But this story has caused more of an uproar because of Elastic’s size (Oracle was a big company well before AWS announced Correto), and AWS’s stance on open-source (DocumentDB was a response to anti-SaaS provisions added to MongoDB’s license, not a commitment from Amazon to contribute back to the open-source codebase).
Two key questions in this debate are:
Players in the landscape of any given open-source project can be broken down into three groups:
Those who produce open-source software. This often includes:
Those who capture value from open-source software. This often includes:
Open-source software is free to use. Vendors can’t charge for the software itself. So they have to charge for surrounding services and software that make it easier to perform one of the following tasks:
Build applications that use the open-source software. Vendor strategies to capture value here include:
Run applications that use the open-source software. Vendor strategies to capture value here include:
When open-source projects like Elasticsearch, Spark, and Kafka were small, the open-core vendors behind them used their funding to aggressively develop and promote their open-source projects. They reaped the rewards of these efforts with huge profits from software licensing fees (for support, tooling, and enterprise-only features bundled in a per-node price), and later from their own cloud-hosted offerings. For a long time they comfortably captured the lion’s share of value that the open-source project had created.
Over time, as open-source projects gained popularity they started drawing the attention of large cloud providers. AWS, Azure and GCP began to release SaaS offerings built around open-source projects with which they had little prior involvement. After spending billions growing and promoting open-source communities, open-core vendors rightfully felt that cloud providers were encroaching on their territory. They had laid all the groundwork, only to see front-running giants swoop in like predators and capture the resulting value. In their eyes, cloud providers are trying to make the landscape look like this:
Open-core vendors are becoming territorial for good reason. Earlier, we outlined the two ways you can capture value from open-source:
Helping developers build applications is a cost center, not a profit center for open-core vendors. They invest in documentation, marketing events, and developer relations to grow their developer following, and then sell support, security, and operational management tools to central IT to make money.
If a company goes with AWS, most of those operational problems go away. An AWS managed service handles provisioning and scale, and keeps servers up and running. AWS also has sufficient cash to invest in support offerings that can catch up to open-core vendors with aggressive hiring of expertise. The need for central IT to buy deployment tooling and emergency support disappears.
[Note that (1) reactive, break-fix support is still outdated and ineffective, and (2) using a managed services doesn’t remove the need for deep domain expertise to debug, tune, and enforce best practices. Both of these topics are better left to later posts.]
Furthermore, AWS sees this software as a means to drive cloud usage, not as a stand-alone source of revenue. Cloud vendors want to sell more server time, preferably with a higher markup for added services & support. At their scale, they can drive the software support costs of their offerings to near $0 and still grow their profits through server and storage revenue. Long-term, this cost advantage is the biggest threat to the open-core vendor’s market.
Different open-core vendors have taken different approaches to neutralize the threat cloud vendors pose to the central IT revenue source. MongoDB introduced “anti-SaaS” provisions to their license, requiring that anyone building a SaaS offering for MongoDB open-source the codebase behind it. The objective was to prevent users from easily migrating their MongoDB applications to AWS, bypassing the open-core vendor. AWS responded with DocumentDB, providing API compatibility and a path to migration without violating licensing terms. Elastic blurred the lines between what is open vs closed in each release. The objective was to reduce the relative value of the AWS managed offering and retain the status as the go-to expert. AWS responded with the Open Distro for Elasticsearch.
This fight isn’t over, but it’s hard to see how Amazon’s continued encroachment doesn’t harm the prospects of open-core companies. That being said, Mongo’s stock price is up 60% since the DocumentDB announcement, and Elastic is flat since Amazon’s Elasticsearch announcement. But stock price, and even revenue, will probably be lagging indicators of future prospects. The dynamic between AWS and open-core companies may soon look a lot like the Facebook-Snap scenario: an open-core company can grow to a certain size, but will ultimately cede future market growth to major cloud providers when they enter the market. A slowdown in growth may not present until the next wave of the adoption curve is ready to use these technologies and turns to a cheaper and more familiar AWS offering.
Amazon’s stance is that this is a net benefit to users of open-source software. They say they’re driving a truly open Elasticsearch and capturing a fair portion of the value they create. It’s hard to argue against the utility to end-users. Their criticisms of Elastic are valid, and the new initiative is supported by Netflix, a prolific contributor to open-source projects. But this conversation has never been about the popular and well-monetized projects getting all the headlines. It’s always been about the viability of the fledgling open-source projects just getting off the ground and the future open-source projects that have yet to begin.
To anyone who has worked at an open-core company, AWS’s actions are a shock but not a surprise. For years, we’ve been asking the question “What do we do when AWS releases a managed offering?”, and for the most popular open-source projects, this has always seemed inevitable. But we’ve now seen a broad enough sample size of how AWS competes with open-core vendors to know that there’s a ceiling to how far an open-source project can go and still remain truly open.
At a certain point, the optimal decision for an open-core company is to be more closed. How soon that point comes determines which of the following scenarios is more likely:
It’s hard to believe the pessimistic outcome is really going to happen. It is likely that open-core vendor margins will decline and the specter of competition will put a cap on the addressable market. But there will still be ample opportunities for developers to monetize their open-source projects. The nature of what is valuable to communities will continue to evolve as well. Open-source innovation may thrive broadly while increasingly commoditized pockets like infrastructure eventually become controlled by the major cloud vendors.