Implementing features such as auto-complete or go-to-definition for a programming language is not often a simple matter. Traditionally, this work needs to be done for each editor or IDE individually, requiring expertise in both the targeted programming language and the programming language used to implement the IDE of choice.
Some years ago, Microsoft decided to solve this problem by defining a Language Server Protocol (LSP) to allow programming language support to be implemented and distributed independently of a specific editor or IDE.
LSP was originally developed to be integrated into Visual Studio Code, but was rapidly implemented in many other software development tools, becoming an open standard.
This article provides an overview of how LSP works, highlighting the benefits that this technology offers to developers and communities focused on non-mainstream programming languages.
Offering a case study of this technology in action, we look at Klarna, a Swedish fintech company that has benefited significantly from LSP by implementing an Erlang language server compliant with this protocol, known as Erlang LS.
What is the Language Server Protocol?
The Language Server Protocol uses a set of JSON-RPC messages, a remote procedure control where data and commands are encoded in JSON.
LSP messages can be exchanged between two entities: a client (represented by the IDE or the editor), and a server, which can be completely decoupled from the client and allows the implementation of a specific programming language feature support.
Client and server have no constraints in terms of the programming language used to implement their features, as long as the technologies used allow the exchange of the aforementioned JSON-RPC messages.
The language server protocol allows the exchange of messages in both synchronous and asynchronous ways, the latter being directed from both the client and the server to the other entity. To better illustrate how LSP works, here is a simple diagram:
In the above example (taken from the LSP web page on GitHub), the first three messages are notifications. In the first two, the IDE sends asynchronous messages to the language server, which can trigger several new actions.
For example, the server might start a compilation task when it receives a textDocument/didChange notification.
Once the compilation finishes, the server can then send another (asynchronous) notification to the IDE, which uses the message contents to display compilation results (including potential compilation errors or warnings) to the user.
Asynchronous notifications are not the only messages that can be exchanged between the language server and the IDE.
The protocol also includes a series of requests, sent from the client (IDE) to the language server, in a manner similar to the HTTP protocol. A response follows a request sent (synchronously) from the server back to the client. In the above example, the user executes a Goto definition by sending a specific JSON-RPC message, the contents of which might look something like this:
The above request includes the document URI and the position of a word representing a function or variable name. With such a message, the IDE requests that the position where said name is defined is found. The response to this request might look as follows:
LSP is much more complex than shown above, although this example is useful in giving a general idea of how the protocol works. The full specifications are open to view and can be found on the LSP GitHub page.
IDEs and editors that implement LSP can run multiple language servers, which act as separate entities for the actual editor. Language servers may also be developed in any programming language, independently of the language(s) used for building the IDE.
It’s even possible to develop a language server for a specific programming language… using that same language! That might sound improbable, but it has already been achieved many times.
Benefits of LSP
Today, LSP is used by many IDEs and development tools, such as Visual Studio Code, Eclipse, Emacs, Atom, Sublime, JupyterLab, Vim, Theia and many others – a more comprehensive list is available at this link. This is not particularly surprising as this idea has a lot of benefits for IDE developers.
The main benefit of such an approach is the ability to re-use a language server in any other IDE. Once someone writes a server for a particular programming language, it immediately has the potential to support any LSP-compliant IDE or editor, with very little integration effort.
This is especially significant for non-mainstream programming languages, where the community and the market are limited.
In such a context, the cost of developing support for a language that is requested only by a small niche of developers might be uneconomical for IDE producers. The only option available then is to rely on the community and its ability to develop open-source development tools.
Without LSP, the community would likely focus on the few available editors with plugins to implement language-related features, in the process constraining other developers to work on those specific tools.
Alternatively, developing a language server would translate into much wider support, and consequently, a much wider user base.
A less evident advantage of such an approach relates to community growth. Considering less common languages, such as Erlang, Cobol, Elixir, Haskell and so on, the poor accessibility of development tools might be a factor that discourages developers from approaching these less known languages.
In contrast, the availability of a language server might facilitate the learning process for any novice developer, allowing them to use the development environment they are familiar with, and increasing the possibility of communities growing up around less well-known programming languages.
Case Study: how Klarna uses LSP
About Klarna
Klarna is a Swedish unicorn in the Payments industry, currently valued at 31 billion dollars. We chose Klarna as a case study because it has one of the world’s largest Erlang systems.
Klarna’s activities rely on a huge codebase written in Erlang, a good example of what has been referred to above as “non-mainstream programming language”. Klarna’s Erlang codebase is made up of something in the region of 1.5 million lines of code, with around 70 developers employed in maintaining it.
Features such as in-line diagnostics (warnings and errors), the ability to navigate through the code by jumping to definitions, or finding callers for a specific function, combined with the ability to detect unused includes or macros, have boosted productivity dramatically.
Due to the large-scale adoption of Erlang in their codebase, Klarna developers represent a significant part of the Erlang community, organizing events and meetups to engage more people in this technology.
Klarna, which recently launched in Italy, is also a founding sponsor of the Erlang Ecosystem Foundation (EEF) – a non-profit foundation and a constant presence at Erlang conferences around the world.
Language Server Protocol for Erlang
Before LSP was defined, some tools existed to ease the development process in a programming language such as Erlang.
However, the developer community around Erlang has always been relatively small, especially when compared with communities around general-purpose programming languages such as C, Java or Python.
This creates the risk that such development tools might suddenly no longer be maintained, or that a good plugin might not be available for a particular IDE, limiting Erlang development to specific tools.
To solve this lack of development tools, a few years ago Erlang Community Lead and Klarna former employee Roberto Aloi came up with a great idea: writing an Erlang language server that exploited all the advantages of LSP, bringing them into his daily work.
This project, named Erlang LS (Erlang Language Server), is available on GitHub, open-source and released under the Apache License 2.0.
Erlang LS allowed Erlang support to be extended to many IDEs, and is currently used in editors such as Emacs, Spacemacs, Visual Studio Code, Sublime Text 3, IntelliJ, Vim and Theia.
The supported features include code completion, go to definition, signature suggestions (i.e. autocomplete), compiler diagnostics, navigation between files, folding and many others (a more comprehensive list is available at this link).
Today, the availability of ErlangLS allows Klarna developers to adopt their preferred editors with no constraints, thus allowing much more productive development environments.
The community supporting Erlang LS is very active and growing. Other big companies, including Facebook (the WhatsApp server includes a significant codebase written in Erlang, as discussed in a talk at Code BEAM V 2020 – watch below) and Ericsson are watching the project with interest.
ErlangLS provides an excellent example of how the creation of open tools that improve the accessibility of a technology may significantly increase interest in that technology.
Conclusions
Building open language servers that support LSP represents a great option for communities focused on less known or adopted programming languages.
This is the case for many niche functional programming languages or domain-specific languages even though these may be very powerful in specific contexts.
Supporting the development process for such programming languages on modern editors and IDEs such as Visual Studio Code or IntelliJ might be uneconomical for the producer in the short term. Nevertheless, doing so can easily become the first step in growing niche communities.