Migrating from LSP-Mode to Eglot

Recently, I decided to try the now inbuilt LSP client called Eglot. I’ve been using the lsp-mode package for some years and while I don’t have any problems with it, I decided to try the in-house solution. Though, I already tried Eglot in the past, and it didn’t work for me, due to some complications with the language I’ve tried to use it with. At the time it didn’t support Clojure, and while adding it wasn’t hard, some features did not work. Which wasn’t the case with lsp-mode, thus I used it instead.

In this post, I’ll outline some problems I encountered, which are mostly Clojure-related. At this point, I can’t say if I will actually move to Eglot, but I plan to use it for some time to see if I miss any features from lsp-mode. Until then, it will stay in my private configuration.

Configuration and installation

I often see claims that lsp-mode packs too many features. While true, they’re all mostly optional and individually configurable. Over the years, I’ve configured lsp-mode in a fairly minimal way. You see, with Clojure, and other lisps for that matter, language servers are somewhat inferior, because there’s a much more accurate way to obtain information about the code. I, of course, talk about the REPL. Being a dynamic environment, we can ask it about its state, known functions, macros, available modules, etc. Of course, it depends on the implementation of the REPL, and not all are created equally, but the one I’m using (nREPL) is quite capable. Language server has to do all of that statically, using their own parsers, tracking of variables, modules, and so on.

Because of that, I don’t really use most of the language server’s features - only linting, and occasionally go to definition. If it wasn’t for one specific feature missing from CIDER, I wouldn’t use language server at all, I think.

Here’s my lsp-mode configuration:

(use-package lsp-mode
  :ensure t
  :hook ((lsp-mode . lsp-diagnostics-mode)
         (lsp-mode . lsp-completion-mode-maybe))
  :preface
  (defun lsp-completion-mode-maybe ()
    (unless (bound-and-true-p cider-mode)
      (lsp-completion-mode 1)))
  :custom
  (lsp-keymap-prefix "C-c l")
  (lsp-diagnostics-provider :flymake)
  (lsp-completion-provider :none)
  (lsp-session-file (expand-file-name ".lsp-session" user-emacs-directory))
  (lsp-log-io nil)
  (lsp-keep-workspace-alive nil)
  (lsp-idle-delay 0.5)
  ;; core
  (lsp-enable-xref t)
  (lsp-auto-configure nil)
  (lsp-eldoc-enable-hover nil)
  (lsp-enable-dap-auto-configure nil)
  (lsp-enable-file-watchers nil)
  (lsp-enable-folding nil)
  (lsp-enable-imenu nil)
  (lsp-enable-indentation nil)
  (lsp-enable-links nil)
  (lsp-enable-on-type-formatting nil)
  (lsp-enable-suggest-server-download nil)
  (lsp-enable-symbol-highlighting nil)
  (lsp-enable-text-document-color nil)
  ;; completion
  (lsp-completion-enable t)
  (lsp-completion-enable-additional-text-edit nil)
  (lsp-enable-snippet nil)
  (lsp-completion-show-kind nil)
  ;; headerline
  (lsp-headerline-breadcrumb-enable nil)
  (lsp-headerline-breadcrumb-enable-diagnostics nil)
  (lsp-headerline-breadcrumb-enable-symbol-numbers nil)
  (lsp-headerline-breadcrumb-icons-enable nil)
  ;; modeline
  (lsp-modeline-code-actions-enable nil)
  (lsp-modeline-diagnostics-enable nil)
  (lsp-modeline-workspace-status-enable nil)
  (lsp-signature-doc-lines 1)
  ;; lens
  (lsp-lens-enable nil)
  ;; semantic
  (lsp-semantic-tokens-enable nil)
  :init
  (setq lsp-use-plists t))

It’s quite big, actually. Most of it, however, is disabling features I don’t need. But it’s not all of it, here’s the rest:

(use-package lsp-clojure
  :demand t
  :after lsp-mode
  :hook (cider-mode . cider-toggle-lsp-completion-maybe)
  :preface
  (defun cider-toggle-lsp-completion-maybe ()
    (lsp-completion-mode (if (bound-and-true-p cider-mode) -1 1))))

(use-package lsp-clojure
  :no-require
  :hook ((clojure-mode
          clojurec-mode
          clojurescript-mode)
         . lsp))

I have a separate configuration section for lsp-clojure, which is a submodule in the lsp-mode package. It’s separated into two blocks, so I can load one eagerly, and some other settings lazily without using explicit eval-after-load in use-package.

(use-package lsp-java
  :ensure t
  :demand t
  :after lsp-mode
  :when (and openjdk-11-path
             (file-exists-p openjdk-11-path))
  :custom
  (lsp-java-java-path openjdk-11-path))

(use-package lsp-java
  :no-require
  :hook (java-mode . lsp))

Same thing for lsp-java which, however, is an external package. And a similar thing for Scala language server, called Metals:

(use-package lsp-metals
  :ensure t
  :custom
  (lsp-metals-server-args
   '("-J-Dmetals.allow-multiline-string-formatting=off"))
  :hook (scala-mode . lsp))

But that’s not all, I also tweak the lsp-treemacs package, which I don’t really use, but if it occasionally pops up I’d like to see it in a less noisy way:

(use-package lsp-treemacs
  :ensure t
  :defer t
  :custom
  (lsp-treemacs-theme "Iconless"))

Few things to note:

1. I know about the lsp-auto-configure custom, and set it to nil. My problem with such customs is that it’s not always means that it’s properly used in the codebase. Some features may not look at it when the lsp-mode is initialized, and I don’t want to study the codebase to see if it is used or not, so I just prefer more explicit configuration. Yet, I don’t want to update the list of options all the time, so that’s why I set it to nil, so the newly added options are disabled automatically, if they’re properly implemented, that is. If not, they’ll pop in and I’ll notice them.
2. I set lsp-completion-enable to t but I don’t want it to be enabled when I use CIDER, because the run-time completions give me better results. So I disable lsp-completion-mode in a CIDER-specific hook. I also re-enable it, once I quit CIDER session for the current buffer.
3. I don’t need most of the features, even for non-REPL languages, such as Java or Scala. Mainly because I don’t write code in these languages and mostly only read them, so I really only need the navigation part of LSP.

So my personal use-cases for LSP are a bit specific. I actually prefer my editor to have less smart features, so for example my auto-complete isn’t auto at all, I manually trigger it only when I need it. Thus, I don’t have that many language servers configured, as the things I occasionally do with other languages almost never require IDE capabilities.

Now, with that in mind, here’s my Eglot configuration:

(use-package eglot
  :ensure t
  :hook ((( clojure-mode clojurec-mode clojurescript-mode
            java-mode scala-mode)
          . eglot-ensure)
         ((cider-mode eglot-managed-mode) . eglot-disable-in-cider))
  :preface
  (defun eglot-disable-in-cider ()
    (when (eglot-managed-p)
      (if (bound-and-true-p cider-mode)
          (progn
            (remove-hook 'completion-at-point-functions 'eglot-completion-at-point t)
            (remove-hook 'xref-backend-functions 'eglot-xref-backend t))
        (add-hook 'completion-at-point-functions 'eglot-completion-at-point nil t)
        (add-hook 'xref-backend-functions 'eglot-xref-backend nil t))))
  :custom
  (eglot-autoshutdown t)
  (eglot-events-buffer-size 0)
  (eglot-extend-to-xref nil)
  (eglot-ignored-server-capabilities
   '(:hoverProvider
     :documentHighlightProvider
     :documentFormattingProvider
     :documentRangeFormattingProvider
     :documentOnTypeFormattingProvider
     :colorProvider
     :foldingRangeProvider))
  (eglot-stay-out-of '(yasnippet)))

Same as with lsp-mode I disable most things I don’t need LS to do, like syntax highlighting, formatting, hovering, and folding. I, personally, don’t agree with the decision to put syntax highlighting, formatting and folding into the language server to begin with - requiring to send data to the server in order to do these things is bananas. The reason it’s done like that, I think, is because Microsoft’s flagship editor focuses on LSP heavily, and because LSP is their own child, I feel that they simply want to use it, instead of other, often more appropriate solutions. It’s much more sensible to do it with an integrated parser, like Emacs’ own smie or now available tree-sitter, but I digress.

Problems and nuances

First and foremost, if the transition was smooth, this section wouldn’t be here. Though I must say, that things are substantially better today than it was several years ago. Still, I found things that made my workflow a bit harder.

Jar archives

Clojure LSP, and Java for that matter, use a special kind of dependency scheme that points to a jar archive with the dependency source code or class files. So every time you want to go to the definition of some symbol that came from an external library it’s a problem, as Emacs has to do some work to account for that.

The problem here is that such servers respond to the goto request with a path that mostly depends on the server. If it’s pointing to an archive, it also needs to point to a file, and possibly the position. Clojure-lsp does it like this:

"jar:file:///.m2/repository/hiccup/hiccup/1.0.5/hiccup-1.0.5.jar!/hiccup/page.clj"

Emacs doesn’t understand this kind of path, and tries to create a file instead of going into the archive. Thankfully, there’s a package that fixes that:

(use-package jarchive
  :ensure t
  :after eglot
  :config
  (jarchive-setup))

That problem is no more. Though it’s interesting that lsp-mode dealt with it on its own, because unlike Eglot, they have separate sub-packages to handle quirks of each language server they support. When working with Clojure projects this introduced its own problems, as lsp-mode extracted archives into a .cache directory, and I often had problems with it becoming stale, as I often switch versions in Git when working on a project. Switching a version means that dependencies could have changed, but I found that lsp-mode jumped to a cached file instead of updating it. It rarely happened, and maybe it was fixed, but I didn’t notice. Hopefully, this won’t happen because jarchive doesn’t have cache.

XREF, completions

Both lsp-mode and eglot provide completion and go to definition facilities via the standard Emacs API. Completions are provided via the completion-at-point-functions and goto is done via xref-backend-functions. However, lsp-mode also has additional functions available for calling manually, without going through the Xref interface. In the end, Xref uses these functions, so it’s good for you if you don’t want to use Xref, but still want to go to definition. Here’s why that’s important, and why I tweak it in the cider-mode-hook.

As I’ve mentioned, I prefer information from the run-time connection with a REPL when it comes to completion and goto-definition. It often has updated information, not available statically, such as when you’ve required some additional functions in the REPL. However, this doesn’t always work. For example, when I connect to a remotely running service that runs a network REPL, it often doesn’t have any source code information, and REPL can’t provide it. In such cases, I’d like to use information from LSP. And that’s where the problems with Eglot begin.

The way xref-backend-functions work is as follows:

Each function on this hook is called in turn with no arguments, and should return either nil to mean that it is not applicable, or an xref backend, which is a value to be used to dispatch the generic functions.

CIDER’s backend function checks if there’s an nREPL connection for this buffer, and if it is, it returns the backend function. Which is correct behavior, but then, if the REPL for whatever reason can’t provide requested information, Xref bails out and doesn’t try the rest of the backends. Thus, Eglot’s backend is never used.

Backend providers are tried in order, and this order depends on the order in which features are loaded. I don’t enable cider-mode unless I connect to a REPL, but as seen in the config, eglot-ensure runs for Clojure major mode automatically. When I need a REPL, I call cider-jack-in-clj and it adds the backend to Xref. Thus, the resulting order is always (cider--xref-backend eglot-xref-backend t).

So, if CIDER fails to find a location of the given symbol, with lsp-mode I could just call lsp-find-definition manually and bypass Xref. Can’t do so with Eglot. I know about the xref-union package, however it doesn’t seem to do anything for me. It adds its own hook into the global value of the xref-backend-functions and thus its backend is never tried, because of the exact same situation described above. I guess this package should also clean local hooks fist, or maybe put itself as a local backend in order to work.

Completions feature a similar problem, though much less annoying. In lsp-mode there are functions for enabling and disabling the completion via a minor mode. With eglot I can do the same thing in my manipulation completion-at-point-functions, it’s just less user-friendly. A few notes on why I disable completions:

Originally, I added this to workaround the problem when both lsp and cider run as a part of clojure-mode-hook. This results in a different order in which completion backends end up in the completion-at-point-functions list, and thus one takes precedence over the other. I since have moved away from running CIDER as a hook, and turn it on explicitly instead as a part of the connection process, and shut it down once no connections left. This works much better for me, and technically this function should be no longer needed but it isn’t the case.

In reality, their order is kinda arbitrary because of how they’re set up. Once you turn on CIDER, it enables itself automatically for the files in the project you’re working with. Eglot, on the other hand turns itself unconditionally, because it runs in a hook. Here’s how it goes:

1. You open .clj file.

2. Eglot runs via the clojure-mode-hook.

   Buffer local value of the completion-at-point-functions variable is (eglot-completion-at-point t)

3. You call cider-jack-in-clj.

   Buffer local value of the completion-at-point-functions variable is (cider-complete-at-point eglot-completion-at-point t)

   Which means, CIDER’s completion overtakes Eglot’s, exactly how I want it to.

4. You open another .clj file.

   Buffer local value of the completion-at-point-functions variable for this new buffer is now (eglot-completion-at-point cider-complete-at-point t).

   So in the first buffer you get completions from CIDER, but in the second and all other new buffers you get completions from Eglot.

Because of that, I had to write the cider-toggle-lsp-completion-maybe / eglot-disable-in-cider and set them up so that I get the exact completion backend I want when I use CIDER, and get back LS-powered one once CIDER is disabled.

I know that this case is probably not as common with other languages, and Clojure is kinda special because it has two separate sources of truth that don’t often line up. When working with a remote application, the source code doesn’t always match what’s running on the server, because versioning is hard, and sometimes I just forget to do a checkout. Thus, it’s important to get information from the runtime instead, as the statically available information can mismatch actual running code.

Other hiccups

As I’ve mentioned, when programming in Clojure, I mostly use LSP for a very few things, as most of the features are backed up by CIDER. However, there’s one thing that CIDER doesn’t do as well as clojure-lsp - finding all references to a given symbol. There’s a separate package called clj-refactor.el that can do it, but it almost never works for me, and it is quite slow. So lsp-find-references is a very handy feature of lsp-mode.

Eglot has it via the xref-find-references interface. However, it suffers from the same thing as completions and goto definition - CIDER’s Xref backend takes precedence over Eglot’s, and returns much fewer results, because CIDER’s implementation of this feature has some issues. So a dedicated function, like lsp-find-references from lsp-mode would be handy in Eglot too.

Another thing is that Eglot seem to be blocking Emacs during server initialization. The clojure-lsp isn’t super slow to start, but it can take some time to initialize. I don’t recall this experience with lsp-mode, and apparently Eglot has a setting eglot-sync-connect for choosing how long to wait synchronously for initialization, and then continue in the background if the server isn’t initialized by that time. However, setting it to 0 lead to blocking UI completely in my case, so I’m not sure what’s the deal here. Maybe it’s a bug, but Eglot author says that it should not be 0, so I guess it’s not supported or tested properly. Although, the docstring mentions that it can be set to 0 or nil, so I’m not sure what gives.

I’m not sure what’s the problem though, SLY, CIDER, and many other packages I tried that interact with a separate process that takes time to initialize do it without blocking Emacs. I should follow suite, and make Fennel REPL initialization non-blocking, as right now it is synchronous too.

And finally, lsp-mode has its own notion of workspaces/projects, whatever they call them. Every time you try to initialize a server in a given project, it will ask you if you want to do it and adds this project to a list of all projects where you’ve agreed to. Or, you can then choose to ignore certain projects where you don’t want language server to run.

For instance, in Clojure we often use EDN files to store settings, but in Emacs .edn is the same as .clj, i.e. it uses the same clojure-mode and will trigger language server initialization. This isn’t a problem if you’re editing a file within the project, although it is often inconvenient because it paints the screen red as the server doesn’t understand that this file contains data and not the code, so it shows tons of unknown symbol-type errors. But, when such file isn’t a part of the project, and sits somewhere else, like for example ~/.config/clojure-lsp/config.edn, it’s possible to tell lsp-mode to don’t try to initialize while in the ~/.config/clojure-lsp directory. I do so for the ~/.m2 directory as well, as I don’t want language server to be started for each dependency I happened to jump in. This fine-grained control isn’t possible with Eglot as far as I can see - it will try to initialize the sever every time, which is somewhat annoying. Perhaps it can be done by setting eglot-workspace-configuration in a project, but I don’t understand the documentation for this variable well enough to say for sure.

None of the issues above are blocking, but some certainly are annoying. I’ll see if I can fix those, but honestly speaking, lsp-mode still feels like a more complete package for language server support in Emacs. With just enough configuration it can be made minimum, and it is fast enough for general use with most servers I’ve tried. Additionally, an ability to install the server right from the M-x lsp-install-server is amazing, and makes updating existing servers trivial. No need to go to the releases and do the update manually. I kinda wish Eglot had that, but as far as I understand, being an Emacs core package that would not happen, as none of the core packages I know download stuff from the non-GNU places. Oh, well.

So, a quick list of things I’d like Eglot to have:

• Dedicated functions for finding references, jumping to definitions;
• Minor-mode-like functions for turning Xref and completions on and off on a workspace basis;
• A keymap prefix for mnemonic mapping available commands;
• Ability to ignore certain directories or projects from launching language servers.

I will continue testing Eglot out, and maybe update this post in some time once I arrive at a conclusion on which mode I chose to continue using.