Let’s talk about the Elastic license change
Is it a bomb under the open source model, or a genius move to protect their business from pirates?
On January 14, 2021 a blog post was published on Elastic’s website, titled: licensing change. In this blog post, they announced a license change in the next release of all their tools (including Elasticsearch and Kibana). The purpose of this change is mainly to block AWS: Elastic wants to make it impossible for AWS to create its own competing managed Elasticsearch service. The announced changes were not very well received in the open source world. In this blog post I tried to give an overview of what happened, and what the consequences are for companies that use Elasticsearch and Kibana. Just a disclaimer, I am not a lawyer. Hence I will not go into the content of the new license model. The aim of this post is mainly to provide an overview of varying opinions on the internet and the possible consequences for users.
The Elastic company
If you do not know the company Elastic at all, it is the company behind one of the largest search engines called Elasticsearch. The company was founded in Amsterdam, so it is a company from Dutch origin. The company is worth billions on the stock exchange in the U.S. Elasticsearch is their main product, but many other products have emerged around it. Elasticsearch started as a fully open source product, just like some of their other projects such as Kibana, Logstash and Beats which also started as fully open sourced projects.
The change in licensing
Elasticsearch started as a full Apache 2.0 project. With the arrival of X-Pack, components were added that followed a closed model. In a blog post in 2018 called Doubling down on open, Elastic announced that it will also make the X-Pack code openly available. Mind you, open, not open source. From that moment on, a dual license was clearly used. Some of the source code was made available under the Apache 2.0 license, another part of the code was made available through the elastic license. The binaries were available through the elastic license.
The license model will change with the next release. The Apache 2.0 license is being replaced by Server Side Public License (SSPL). The SSPL license was developed by MongoDB, through this license they tried to make it more difficult for AWS to offer Mongo as a service. Where do problems arise for the community? Well problems start with it no longer being an open source license. Why SSPL is not an open source license is clearly described here on the website of the Open Source initiative. The bottom line is that the source code will remain available, but not according to the ideas of the Open Source initiative (OSI). Another look at this case is explained very well in this post. The open source initiative can say that they do not recognize a license, but they have no right to say that something is not open source. The writer of this blog post Righteous, Expedient, Wrong ~ OSI swings at Elastic, misses and leaves a mess giving you yet another view of OSI’s claims.
The reason for taking this step is to make it impossible for Amazon to make the AWS Elasticsearch managed service available. Elastic has drawn up agreements with a number of large cloud providers such as Google and Microsoft Azure about their service to offer Elastic products as a managed service. However the negotiations with AWS did not go anywhere. The consequences of this license change are significant for some companies that are reliant on Elasticsearch. This is explained in the blog of Bonsai, a company that has been offering Elasticsearch as a service for years. For many developers the whole ordeal is much more a matter of principle than anything else. This is well articulated by someone like Drew DeVault in his blog Elasticsearch does not belong to Elastic. He calls the license change a slap in the face for the 1,573 developers who contributed to the project by making their code available.
The disagreement between Elastic and AWS
Anyway, back to those licenses. When launched, Elasticsearch was completely open source under the Apache 2.0 license. Previously, Elastic had already decided not to give away everything for free, so they added their own license to part of the code. Something that was not appreciated by a party like AWS, especially because it made it difficult to use the code yourself. This was when a new Elasticsearch distribution emerged called “Open Distro for Elasticsearch”. This is a distribution with all code released under the Apache 2.0 license in conjunction with other open source products. According to Elastic there were some problems with it. The distribution uses a number of open source extensions. Elastic claims that the creators of one of these components copied code from Elastic that is not under the Apache license. Read more about this in Dear Searchguard users. Personally, I do not think it is very appropriate, but it does make sense when keeping the open source concept in mind. Not stealing code, of course, but making a fork. As Elastic itself points out, they have tried to make things move in a different direction with AWS, but their attempts have failed. Read the following article Why License Change AWS for more information on this. Incidentally, Elastic is not alone in their fight against AWS. There are many companies with Open Source products complaining about the way AWS deals with them. An extensive article appeared in the New York Times in 2019: How Amazon Wields Power in the Technology World. It describes how Amazon turns Open Source projects into managed services without benefiting the companies that try to make a living from the projects. In the article they mention the strip mining of open source projects. Examples are given about products such as MongoDB, Splunk, MariaDB, Redis. In the article by ZDNet AWS hits back at open-source software critics, the feeling of the open-source world about AWS is highlighted again.
Back to AWS and Elasticsearch. Amazon is of course not keeping quiet either. They released a statement saying that they have done nothing wrong. They have made patches and extensions available to Elasticsearch. However, they say that this systematically led to nothing. This is why they started Open Distro, which they themselves say is not a fork yet because they have worked according to Open Source Upstream. In other words, they have followed the rules as a good Open Source user should. AWS’s response to the latest license change is that they are now actually going to fork.
AWS is not the only company that has problems with these latest licensing developments. There are plenty of articles about it available online. A good example is Logz.io’s blog Open source Elasticsearch doubling down. Logz.io offers a cloud monitoring solution based on open source tools such as Elasticsearch and Kibana. They indicate that the license change feels like a slap in the face for the open source community, and are working with a number of other companies to start a fork. Yes indeed, one of these companies is AWS. This has been made public in the following article: Stepping up for a truly open source Elasticsearch. Another company that has also been offering Elasticsearch As A Service for a long time is Bonsai. They explain their options for continuing to provide this service in their blog Elasticsearch license changes. This is how they view the path to comply with the SSPL license. At first glance, the text of the license itself does not seem unambiguous enough to fully support it. Then they can choose between making an agreement with Elastic or going for an Open Source fork.
Impact on those working with Elasticsearch daily
In my opinion, the announced changes are a bad thing for Elasticsearch users. How nice would it be if several companies worked on improvements to the Elasticsearch product, according to the Open Source idea. I understand very well that Elastic wants to earn money from their products, they are the ones putting the most hours into development. Just like MongoDB, which faced similar problems a few years ago, and therefore developed a new license. It would be nice if AWS would put some of their money back into the project. Of course they say they already do this by offering improvements, but if they had to buy some sort of license, are we still talking about Open Source? That is exactly what they say at the Open Source Initiative. Fine if you as a company want to earn money through a certain business model, but then you should not advertise with the fact that you offer an open source product. And you have to think about this in advance, not after you have benefited greatly from the open source community.
What is the impact for you if you want to use Elasticsearch in your project? There is no clear answer to that. Elastic has written a FAQ that should help you. If you already had some form of a contract with Elastic, nothing will change. If you previously only used the binaries, not a lot will change either. Maybe you should check whether you comply with the license. When you build a service based on Elastic products, make extensions to this and offer them as a service in the cloud, you will have to talk to Elastic and negotiate with them. Or you must have faith in a fork and continue on that particular path. If you now use a service like AWS Elasticsearch, Bonsai, Logz.io, you will have to pay close attention to their strategy in the near future. Most companies will negotiate with Elastic or switch to a fork. Of course you can also switch to a cloud provider that has partnered with Elastic, or use Elastic’s cloud service itself. These are all very good choices if you want to enjoy all the good things Elasticsearch has to offer.
As a software engineer I am a big fan of the open source model. But as a very experienced Elasticsearch consultant I also feel a lot of sympathy for the position Elastic is in. I furthermore share the opinion of many developers in the community that a company like Elastic should be able to earn enough money from their own cloud service and their support contracts. I will continue to monitor developments closely. For now, I will continue to build projects with Elasticsearch, but I will also not stop with the projects that use AWS Elasticsearch. We will continue to do projects with Apache Solr and look with great interest at tools such as Vespa and Weaviate to see if they offer better alternatives for customers.
References in chronological order
February 27, 2018 – https://www.elastic.co/blog/doubling-down-on-open
September 4, 2019 – https://www.elastic.co/blog/dear-search-guard-users
January 14, 2021 – https://www.elastic.co/blog/licensing-change
January 19, 2021 – https://www.elastic.co/blog/why-license-change-AWS
January 19, 2021 – https://www.elastic.co/blog/license-change-clarification
Januari 19, 2021 – https://drewdevault.com/2021/01/19/Elasticsearch-does-not-belong-to-Elastic.html
January 20, 2021 – https://logz.io/blog/open-source-elasticsearch-doubling-down/
January 21, 2021 – https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/
Also interesting to read
https://www.elastic.co/pricing/faq/licensing
https://opendistro.github.io
https://opensource.org/node/1099
