Replacing clojure-lsp with clj-kondo and Refactor-nREPL

I’ve been programming in Clojure for the last five years. I don’t write much about it here, largely because I use Clojure at work and rarely for hobby projects, so I don’t have much to share. Even today, the post will be more about Clojure tooling, rather than Clojure itself.

Today, most developers expect the language they work with to have a certain amount of specific tools, like a language server, build system, etc. These tools usually help work on a project with less friction. For instance, most languages require a language server to provide things like autocomplete, jumping to definition, and linting. However, when I do work on a hobby project, I usually prefer a more distraction-free environment. Count me old-school, but I kinda like the simplicity of just having you, your text editor, and the code. I don’t really use these things even with other languages.

Of course, Clojure itself doesn’t need such tooling that much, because there’s the REPL, which already acts like your language-specific tooling. I use Emacs, and it has excellent Clojure support thanks to nREPL and CIDER - it provides most of the features that a language server can, albeit in a bit different way. However, even with nREPL and CIDER, I don’t really use most of the features, maybe only the goto definition thing. Another thing provided by language servers is linting, but since Clojure is a dynamic language and it has a REPL, I already evaluate code all the time, so the REPL usually gives me all errors, and I don’t really need static linting that much either.

That goes for hobby projects. However, when working in a team on a big project, that’s a different story. For the last five years, I’ve been working on medium-sized projects, and the main difference for me is that there’s a lot of code I didn’t write myself. Language server helps here, because not only does it provide linting of such code, it also allows me to avoid loading all of the code into the REPL for CIDER-specific features to kick in. Also, not everyone in those projects used a language server, so every now and then the linter pointed out some potential problems.

Recently, I switched jobs, and now I work on a different team, on a much bigger project. My work setup didn’t change - basically, my Emacs config was ready as is to start working on a new project, or that’s what I thought. However, I noticed that things didn’t work out as planned.

I’m a firm believer that developers should use the slowest hardware they can get their hands on for developing. If your hardware is slow, you’re urged to write optimal code and, subsequently, create tools that work on that hardware. This is where things started to go haywire.

You see, this is a big project, with a lot of files. Usually, that’s not a problem, however, this time it was. In Clojure, we have Clojure-LSP as the language server to use. I have a pretty old laptop. Here are my specs:

It was plenty for anything I did in the past, for hobby projects, that is. At my previous workplace, I had a work laptop that had 24GB of RAM and a slightly faster CPU. Not that I really need an extra eight GB of RAM, I never saw it go above 16 while working on a single project. Sometimes I did work on several projects at the same time, and then it surely helped, but it was rarely the case.

At the current job, I hit SWAP all the time, and it started to bother me a lot lately. The reason is, as you might have guessed is Clojure-LSP. When started in the project I’m working on, it alone takes up around 8-11 GB of RAM. Right now, as I write this post, my editor, Firefox, and a messenger already take up 4.5 GB of RAM, so adding Clojure-LSP to this mix will by itself approach my limit. And then, I start the REPL, which took another 2-3 GB, and we’re in the SWAP territory.

I think it’s obvious that there’s no way my laptop can handle this load without becoming sluggish. RAM isn’t the only bottleneck here, CPU usage spikes up a lot, too, and the temperature is around 75 degrees and up constantly. So I decided to change this.

I’m not alone at this, unfortunately, my colleagues also suffer from Clojure-LSP being a resource hog. And the real problem is that they have even faster machines than mine with more RAM. So even if I upgrade my laptop or buy a new one, it won’t help that much. So, as an experiment, we decided to disable Clojure-LSP and go back to a simpler setup.

Disabling Clojure-LSP

I use the lsp-mode package, so disabling Clojure-LSP was as easy as commenting out the hook that starts the language server, but I went further and removed lsp-mode completely, as I don’t have use for it other than for Clojure. But now, I have no linting, which I’d like to have since this is a complex project. The go-to linter in Clojure world is clj-kondo, so I added flymake-kondor¹, since I use flymake and not flycheck. Fortunately, Clojure-LSP uses clj-kondo internally, so the linting configuration is the same.

I was expecting that linting such a big project would still eat a lot of RAM, however, for some reason, there wasn’t any major spike in RAM usage when just using clj-kondo. Linting works fast, and Emacs no longer freezes every now and then. I guess communication with the language server is much more taxing than simple parsing of stdout.

However, linting isn’t the only thing provided by Clojure-LSP.

Bringing back refactoring

Two main things I noticed I rely on with a language server are symbol renaming and finding references. Both of these tasks can be handled with refactor-nrepl, however, it’s a bit finicky.

Symbol renaming

Before we begin, let me tell you why you might want this feature as part of the tool that does code analysis. Sure, it’s one thing to rename a symbol inside a single namespace, but when you want to rename it across the project, it gets tricky.

One way of doing it is to use grep, and utilize the Emacs capabilities to edit the buffer created by grep directly. While it works, it’s not as precise as with a language server, because the symbol can be different depending on a file. For instance, when renaming a namespaced keyword, you can encounter a problem that the keyword is written as ::foo in the file you’re editing, but as :fully.qualified.namespace.name/foo or ::namespace-alias/foo depending on how the namespace was used. Sure, you can write a regular expression for grep in the first case, but not really for the second one.

Here’s an example: imagine we have several namespaces in our project (can be in different files, can be in a single file like here):

;;; src/project/multimethods.clj
(ns project.multimethods)

(defmulti m ::foo)

;;; src/project/utils.clj
(ns project.utils)

(defn foo [data]
  ,,,)

;;; src/project/other_ns.clj
(ns project.other-ns
  (:require project.multimethods
            [project.utils :refer [foo]]))

(defmethod project.multimethods/a :project.multimethods/foo [data]
  (foo data))

;;; src/project/another_ns.clj
(ns project.another-ns
  (:require [project.multimethods :as mm]))

(defmethod mm/b ::mm/foo [data]
  ,,,)

;;; src/project/unrelated.clj
(ns project.unrelated)

(defn foo [data]
  ,,,)

;;; src/project/yet_another_ns.clj
(ns project.yet-another-ns
  (:require [project.unrelated :as unrelated]))

(defn bar [data]
  (::unrelated/foo data))

It’s a bit verbose, but I tried to make as short of an example that shows all possible ways to use a symbol. Here, if you’re going to rename the ::foo keyword, you can’t really grep with "::foo|:project.multimethods/foo|::[^/]+/foo", because it will also find ::unrelated/foo, which is unrelated. Sure, you can write a pretty generic regular expression, and then meticulously find all relevant symbols in the grep buffer, and rename them using multiple cursors, or the query-replace feature, but it’s a bit much.

Same goes for renaming functions - if you’re trying to rename foo from project.utils, grepping can find foo in the project.unrelated namespace. So grep is not a suitable alternative to Clojure-LSP, as it isn’t capable of doing semantic analysis.

Since we’ve disabled Clojure LSP, we need a different tool to handle this task. Thankfully, there’s the refactor-nrepl project that provides refactoring features via the nREPL integration that we’re using. It has a lot of features, and it works well enough for our project. Well enough, because refactor-nrepl is a bit finicky.

Problems with refactor-nrelp

First of all, refactor-nrepl works. Most of the time, that is.

I think one of the reasons why Clojure LSP used so much RAM was that it did all of the possible analysis in the background. The reason I think it’s true is that it did the renaming almost instantaneously.

When renaming a symbol with refactor-nrepl, I often get a timeout error with CIDER - renaming is a blocking operation, and CIDER tries not to block the editor for too long in some cases. On a large project, renaming a symbol takes a lot of time to fetch all symbol occurrences alone. I guess that’s the trade-off. Perhaps I can configure the timeout to be a bit longer, but we’ll see.

Another feature of Clojure LSP I relied on was finding usages. Refactor nREPL gives that in the form of finding references, which, again, works, but is susceptible to the same timeout problem. And I’m not sure if it is as precise as Clojure LSP’s one. CIDER itself also has a feature for finding usages, but it is also finicky in its own way.

I suppose, because Clojure-LSP appeared, fewer people use refactor-nrepl today than it was before, and the overall advancement in CIDER development may have slowed down because of it. Looking at GitHub graphs, refactor-nrepl development started around 2015, and major activity stopped around 2019, precisely when Clojure-LSP was created.

Which is a bit of a shame, if Clojure-LSP was responsible for slowing down advancement in refactor-nrepl, but thankfully it is still maintained and works. It’s good to have alternatives, and putting all eggs into one basket was never a good way of doing things.

With refactor-nrepl, most of the things I used Clojure LSP are again available to me, but I have to do further testing, because I found some instances where it fails. Maybe I need to configure a bunch of settings. Or maybe you’ll see another post with me going back to Clojure LSP in a few months/weeks. Who knows!

Not a bashing on Clojure LSP

Contrary to what this post may seem like, this was not intended as bashing on Clojure LSP developers. Clojure LSP is a good piece of tech, and certainly helps Clojure developers around the world.

The reason it is problematic to use on this particular project can be due to a lot of factors. First, the project is enormous, lsp-mode reports that it wants to “watch” for around 2300 directories. Disabling file watchers helps, but by a small margin.

Second, the tech stack. Clojure LSP is written in Clojure, which, while it makes sense, might not be the best choice. Clojure is known to be not the best tool for writing utilities. And while Clojure LSP is a server, and not a CLI utility with a fast lifecycle, it may still be sub-optimal. Maybe, once Jank is ready, Clojure LSP could be rewritten in it, making it faster. I don’t know.

A completely unrelated reason, in my opinion, is that Clojure itself is not the best target for LSP. It’s a dynamic language, where we do a lot at runtime in the REPL. Doing static analysis in such a system can be difficult, as there’s no longer one source of truth. nREPL, bridging runtime and source code, while also having a lot of LSP features, seems like a much better fit for languages like Clojure. And, like, nREPL has appeared almost six years before the Language Server Protocol, and in my opinion, it could have been a far better protocol for developing language tooling, especially since it is also language-agnostic.

Anyway, replacing Clojure LSP with plain clj-kondo brought back the joy of writing Clojure, so I’m happy again.