Condition system in Clojure

A while ago I’ve watched this amazing talk: Condition Systems in an Exceptional Language by Chris Houser. And more recently I’ve found one interesting library called farolero, which provides a set of functions and macros that mimic Common Lisp’s condition system. So I was generally interested in the topic and decided to give it a shot and try both approaches. But before we can appreciate the condition system and its error handling capabilities, we need to look at how errors are generally handled in Clojure. So let’s start with the classic Clojure/Java approach of throwing and catching exceptions.

Exception-based error handling

The examples in the talk are based on the examples from the book called “Practical Common Lisp”, and while these are great examples, I still find it a bit overburdened with details. I think the main reason for this is that we have four functions to think about, each of which tries to take its part, and demonstrate how we could design such error handling in our application. Sure this can provide a great way to show how we can handle errors on different levels, but it can also confuse a lot of people who don’t know about the condition system. It was definitively confusing for me at first read at the very least, so let’s use a more simple example with one or two functions instead. We can always add more complexity later if we want to expand.

Let’s say we want to parse a date string, and we want to check if the date is a correct leap date as well. We can do this with this regular expression:

(def ^:const date-re
  (re-pattern (str #"(?:(?:(?:0[1-9]|1[0-9]|2[0-8])-(?:0[1-9]|1[012]))|(?:29|30|31)-(?:0[13578]|1[02])|(?:29|30)-(?:0[4,6,9]|11))-(?:19|[2-9][0-9])\d\d"
                   #"|(?:29-02-(?:19|[2-9][0-9])(?:00|04|08|12|16|20|24|28|32|36|40|44|48|52|56|60|64|68|72|76|80|84|88|92|96))")))

You don’t really need to dig into this regexp, all you need to know is that it handles dates formatted as dd-MM-yyyy, and correctly detects leap dates. Now we can write a function that parses the date string:

(import '[java.time LocalDate])
(import '[java.time.format DateTimeFormatter])

(defn parse-date
  "Parse `date` string into Java `LocalDate`"
  [date]
  (if-let [date (and (string? date) (re-find date-re date))]
    (LocalDate/parse date (DateTimeFormatter/ofPattern "dd-MM-yyyy"))
    (throw (ex-info "invalid date" {:data date}))))

So, if our date is a valid one, we parse it to LocalDate. If date is not valid, we throw an exception, because we don’t know how to deal with invalid dates at this level, so hopefully, the higher scope will catch the exception and deal with it.

Now, we can use this function to convert all valid dates and filter out invalid ones like this:

(keep #(try
         (parse-date %)
         (catch Exception e
           ;; simple error logging
           (println (ex-message e) (:data (ex-data e)))))
      ["29-02-2020" "29-02-2021" "12-34-5678"])
;; invalid date 29-02-2021
;; invalid date 12-34-5678
;; => (#object[java.time.LocalDate 0x194a4280 "2020-02-29"])

When we call this keep we successfully parse the first date, but fail to parse other ones, because there’s no February 29th in 2021, and the last one is plain wrong, so we log it with println and move on. But let’s assume that we’re OK with this wrong leap date, we just want to transform it into March 1st of 2021. Let’s write a function that checks if the given date is indeed February 29th, and transforms it. If the date is not a leap one, we again throw an exception:

(defn reparse-leap-date [date]
  (if-let [[_ year] (and (string? date)
                         (re-find #"29-02-((?::19|[2-9][0-9])\d\d)" date))]
    (let [next-day (str "01-03-" year)]
      (println (format "Wrong leap date: %s, using %s instead" date next-day))
      (parse-date next-day))
    (throw (ex-info "Not a leap date" {:data date}))))

Now we can try to re-parse the date of the initial parse has failed, but we also now have another exception to deal with, so we wrap everything in another try:

(keep #(try
         (parse-date %)
         (catch Exception e
           (try (reparse-leap-date (:data (ex-data e)))
                (catch Exception e
                  (println (ex-message e) (:data (ex-data e)))))))
      ["29-02-2020" "29-02-2021" "12-34-5678"])
;; Wrong leap date: 29-02-2021, using 01-03-2021 instead
;; Not a leap date 12-34-5678
;; => (#object[java.time.LocalDate 0x102d92c4 "2020-02-29"]
;;     #object[java.time.LocalDate 0x2b97cc1f "2021-03-01"])

Now one can say, that we’ve written the code, that successfully recovers from errors while parsing dates. However, the problem here, is that we had to incorporate our error handling into the function we’re passing to keep (in our case an anonymous function). We may want to promote this to a named function and share it with some other places in our program, but such places may not be OK with converting wrong leap dates. Or, the caller of this promoted function will not be OK with it returning nil, when the date is invalid, and instead will have to do extra work around the function checking its return value.

So we may find ourselves with functions like parse-date-or-convert-leap, parse-date-or-log-invalid, e.t.c., or with a lot of anonymous functions, all of which will share the parse-date at it’s core. And throwing exceptions may be a bit costly, as exceptions are expensive, so this may not be the optimal strategy in the first place. So right now our options are either to write a lot of try catch mess, duplicate code, or handle errors somehow differently.

And actually, let’s think a bit more about this example. We’re detecting and signaling the error inside the parse-date function, but handling the actual error outside of this function. This means that we’re not really recovering from the error in the function which signaled it, but rather in the caller scope of that function. So outer function must know how to deal with the error in the inner function, and perhaps have some code that does what was supposedly needed from the function which threw an error.

Ideally, we would want to return to the exact place in the function where the error happened and do something about it instead. And this is what the condition system in Common Lisp allows us to do.

Conditions in pure Clojure

We can implement the condition system in Clojure using only inbuilt features of the language - mainly the dynamic scope. But first, let’s understand what the condition system actually is.

Common Lisp condition system is a very useful feature of the language, often used for error handling, and to do other interesting stuff as well. The main difference between conditions and exceptions is that exceptions are a two-part system, where one side signals the error, and another side decides how to handle it and actually handles it. In Common Lisp, conditions are a three-part system, with an additional thing, called restarts. This can be seen as that our catch block is separated into two parts - one will only decide how to recover from the error, and another will actually do the recovery part. In conventional exception systems, by the time you get an exception, you should already decide how to handle it. Furthermore, you have to deal with the exception outside of the place where it has actually happened, while in the condition system you can return to that place and restart the process.

So, the condition system consists of three main parts: signals, handlers, and restarts. We can think of signals as exceptions in Clojure, but a bit more general. Which means that we already have signals, however, we actually can’t use Java’s exceptions in the condition system to signal errors. An exception will still be used for stack unwinding, and the topmost error handling, where an unhandled signal will pop up as an exception, but ideally this will not happen. And handlers and restarts don’t really exist yet so we’ll have to define those manually.

So how we can do this in Clojure? First, we need to create the signal:

(defn ^:dynamic *invalid-date* [date]
  (throw (ex-info "invalid date" {:data date})))

That’s it. We’ve just moved the throw call into a separate dynamic function, which gave us the facility to replace this function’s behavior later with a custom handler. Now we can introduce some restarts:

(def ^:dynamic *use-value*)
(def ^:dynamic *log-invalid-date*)

This may not make much sense right now, as we’ve just created two more dynamic vars without any behavior, but we will assign some behavior in the function where we would use these restarts. Which we will do in the parse-date function:

(defn parse-date [date]
  (binding [*use-value* identity
            *log-invalid-date* (fn [date]
                                 (println "unable to parse" date))]
    (if-let [date (and (string? date) (re-find date-re date))]
      (LocalDate/parse date (DateTimeFormatter/ofPattern "dd-MM-yyyy"))
      (*invalid-date* date))))

In this binding form we’re assigning default behaviors to our restarts - you can think of it as restart-case in Common Lisp. The *use-value* restart is a pretty common restart, which means we will do the error recovery outside of the function, and pass the correct value for this function to use. The *log-invalid-date* restart is a simple logger that can be also used instead. We could add more restarts here if there were more ways to recover from the error, but let’s just stay with these two for now.

And in the if-let we’re checking if the date is a string and matches our regexp, and if not we’re using our *invalid-date* signal. Right now this signal will throw an exception, so if we will use this function in our keep, we’ll get an unhandled exception:

(keep parse-date ["29-02-2020" "29-02-2021" "12-34-5678"])

But now, we can register a handler to our signal, that will log our errors. We do this by using another binding:

(binding [*invalid-date* (fn [date] (*log-invalid-date* date))]
  (keep (bound-fn [date] (parse-date date))
        ["29-02-2020" "29-02-2021" "12-34-5678"]))
;; unable to parse 29-02-2021
;; unable to parse 12-34-5678
;; => (#object[java.time.LocalDate 0x102d92c4 "2020-02-29"])

Note: we have to use either doall or bound-fn in our keep call, as we’re in dynamic scope, and keep is lazy.

So, here we’re binding our *invalid-date* signal to use a *log-invalid-date* restart and continue. If we run this code, we’ll see two dates being logged and only one date lands on the resulting list of parsed dates. Now, let’s add a recovery, that will try to re-parse our date, and recover from the error.

To do so, we need to change our reparse-leap-date function like this:

(defn reparse-leap-date [date]
  (binding [*use-value* identity
            *log-invalid-date* #(println "unable to parse as leap date" %)]
    (if-let [[_ year] (and (string? date)
                           (re-find #"29-02-((?::19|[2-9][0-9])\d\d)" date))]
      (let [next-day (str "01-03-" year)]
        (println (format "Wrong leap date: %s, using %s instead" date next-day))
        (parse-date next-day))
      (*invalid-date* date))))

Again, we’re imitating the restart-case from Common Lisp with binding, in which we provide two restarts. And as in the previous version, if the date is not February 29th, we signal the error with *invalid-date* signal. So how we would use this function to recover?

One may think that we will just need to rebind *invalid-date* to a function that calls *use-value* restart, and provide the result of reparse-leap-date to this restart as an input. This is mostly true, but actually, there’s a bit of a problem. We need to remember that if our function can’t parse a date as a leap date, we’re signaling the error using the same signal we’ve just rebound. This is a recipe for stack overflow exception because we will call reparse-leap-date over and over again from *invalid-date* signal handler. So, generally speaking, once we’ve entered the handler, we need to bind the original signal’s value back, because we can’t be sure that by handling this signal, we will not receive it again:

(let [decline-invalid-date *invalid-date*]
  (binding [*invalid-date* (fn [date]
                             (binding [*invalid-date* decline-invalid-date]
                               (*use-value* (reparse-leap-date date))))]
    (keep (bound-fn [date] (parse-date date))
          ["29-02-2020" "29-02-2021" "12-34-5678"])))

This is how we invoke our *use-value* restart. It may seem a bit tricky at first glance, and we’re rebinding things in dynamic scope in a kinda special way, so let me explain what happens here.

The first thing we do is store our original signal value in the let binding as decline-invalid-date. We then bind our *invalid-date* signal to an anonymous function that, when called, binds *invalid-date* back to its original value stored in the closure, and executes the *use-value* restart. This restart receives the value from the reparse-leap-date function, which may use *invalid-date* signal, but it won’t be handled with this particular handler anymore, as we’ve just unbound it. This means, that if an error happens in reparse-leap-date, an exception will be thrown, as our *invalid-date* signal is bound to its original function which throws an exception. So to handle the error from the reparse-leap-date function we need another handler around the let block, which will use the logging restart for example:

(binding [*invalid-date* (fn [date] (*log-invalid-date* date))]
  (let [decline-invalid-date *invalid-date*]
    (binding [*invalid-date*
              (fn [date]
                (binding [*invalid-date* decline-invalid-date]
                  (*use-value* (reparse-leap-date date))))]
      (keep (bound-fn [date] (parse-date date))
            ["29-02-2020" "29-02-2021" "12-34-5678"]))))
;; Wrong leap date: 29-02-2021, using 01-03-2021 instead
;; unable to parse as leap date 12-34-5678
;; => (#object[java.time.LocalDate 0x551de37d "2020-02-29"]
;;     #object[java.time.LocalDate 0x6ef81f31 "2021-03-01"])

If we run this code we will get a list of two dates and a message in the log. The first date will be parsed correctly, the second one will be handled with the inner *invalid-date* handler, and the third date will be logged with the outer *invalid-date* handler. And, as a matter of fact, since the *log-invalid-date* restart will be called only from the decline-invalid-date function we just as well could register the handler in the inner binding, like this:

(binding [*invalid-date*
          (fn [date]
            (binding [*invalid-date* (fn [date] (*log-invalid-date* date))]
              (*use-value* (reparse-leap-date date))))]
  (keep (bound-fn [date] (parse-date date))
        ["29-02-2020" "29-02-2021" "12-34-5678"]))

So this is a flexible system.

But another great property of storing the original signal’s value in the let is that we can decline to handle the signal completely. This means, that we can add some logic into our handler, that analyzes its input value, decides whether it can handle or it should decline to handle this, and just propagate the error up the dynamic scope. This is why I’ve called original value decline-invalid-date because by calling it we will decline to handle the *invalid-date* error. And Common Lisp condition system also can do the same thing, except it’s a bit easier there. We do so by not calling the invoke-restart function in the handler, but since in our system there’s no invoke-restart at all, we have to save the signal value manually.

However, even though this system is pretty flexible it is also a bit too boilerplate. We have to do many things, that can be actually wrapped into macros, and the farolero library, I’ve mentioned earlier, provides that. Let’s see how our code above can look with more Common Lisp-like macros.

Farolero conditions and restarts

Farolero is a library, that provides a set of functions and macros that provide a condition system very similar to the one from Common Lisp. This is a fairly new library, but I’ve found it the most interesting one among the others, which provides condition systems for Clojure. First, let’s add it as a dependency. Create deps.edn file with the following contents:

{:deps {org.suskalo/farolero {:mvn/version "1.0.0-RC2"}}}

Alternatively you can use clj -Sdeps '{:deps {org.suskalo/farolero {:mvn/version "1.0.0-RC2"}}}' to start the REPL with this library available for require.

Now let’s require needed stuff, and rewrite our parse-date function:

(require '[farolero.core :refer [restart-case error handler-bind invoke-restart]])

(defn parse-date [date]
  (restart-case (if-let [date (and (string? date) (re-find date-re date))]
                  (LocalDate/parse date (DateTimeFormatter/ofPattern "dd-MM-yyyy"))
                  (error ::invalid-date date))
    (::use-value [value] value)
    (::log-invalid-date [date] (println "unable to parse" date))))

Here we’re using proper restart-case macro, which accepts the expression as the first argument, and restarts we desire to implement for this expression. In the expression we check date with our regular expression and if it doesn’t match, we’re calling error, which signals the ::invalid-date error with one argument date. Similarly to our previous example, we can handle this error by using the ::log-invalid-date restart:

(handler-bind [::invalid-date (fn [_condition date]
                                (invoke-restart ::log-invalid-date date))]
  (keep (bound-fn [date] (parse-date date)) ["29-02-2020" "29-02-2021" "12-34-5678"]))

Executing this code we will, as before, receive a list with one parsed date, and two messages in the log. To actually recover from the first error, we can bind this handler to a ::use-value restart, which we pass the value provided with reparse-leap-date function, but first we need to update it to use restarts as well:

(defn reparse-leap-date [date]
  (restart-case
      (if-let [[_ year] (and (string? date)
                             (re-find #"29-02-((?::19|[2-9][0-9])\d\d)" date))]
        (let [next-day (str "01-03-" year)]
          (println (format "Wrong leap date: %s, using %s instead" date next-day))
          (parse-date next-day))
        (error ::invalid-date date))
    (::use-value [value] value)
    (::log-invalid-date [date] (println "unable to parse as leap date" date))))

Note, that we’re signaling the same ::invalid-date with error function, and we had to deal with this in the dynamic scope solution previously, but this will not be problematic in the case of this library. We now can use this function to provide a value for the ::use-value restart as follows:

(handler-bind [::invalid-date
               (fn [_condition date]
                 (invoke-restart ::log-invalid-date date))]
  (handler-bind [::invalid-date
                 (fn [_condition date]
                   (invoke-restart ::use-value (reparse-leap-date date)))]
    (keep (bound-fn [date] (parse-date date))
          ["29-02-2020" "29-02-2021" "12-34-5678"])))
;; Wrong leap date: 29-02-2021, using 01-03-2021 instead
;; unable to parse as leap date 12-34-5678
;; => (#object[java.time.LocalDate 0x13741d5a "2020-02-29"]
;;     #object[java.time.LocalDate 0x6b69761b "2021-03-01"])

And I’ve already included the outer handler, that logs every unhandled date. This outer handler could have been defined as the inner handler around reparse-leap-date call, but I’ve used it outside just to show that we don’t have to deal with signal rebinding.

Compare this code to our no-libraries-required solution:

(binding [*invalid-date* (fn [date] (*log-invalid-date* date))]
  (let [decline-invalid-date *invalid-date*]
    (binding [*invalid-date*
              (fn [date]
                (binding [*invalid-date* decline-invalid-date]
                  (*use-value* (reparse-leap-date date))))]
      (keep (bound-fn [date] (parse-date date))
            ["29-02-2020" "29-02-2021" "12-34-5678"]))))

The difference is not that big and the structure is mostly the same. However, the code which uses farolero library is still much clearer, because there are no let and nested binding in handlers, which needed to provide ways to decline error handling and unwind the stack. And as a matter of fact, if we’re exiting the handler normally, e.g. not by using invoke-restart, we will decline error handling, and our signal will unwind the stack if there’s no other handler to catch it.

An additional feature, provided by the farolero library, is an interactive debugger, which is mostly the same as in Common Lisp (to my knowledge). This debugger allows us to handle unhandled signals by using provided restarts interactively. For example, if we remove the outermost handler-bind and execute the code as follows:

(handler-bind [::invalid-date
               (fn [_condition date]
                 (invoke-restart ::use-value (reparse-leap-date date)))]
  (keep (bound-fn [date] (parse-date date))
        ["29-02-2020" "29-02-2021" "12-34-5678"]))

We will see an interactive prompt that provides us with some information about the signal, and our restart options:

Debugger level 1 entered on :user/invalid-date
:user/invalid-date was signaled with arguments ("12-34-5678")
0 [:user/use-value] :user/use-value
1 [:user/log-invalid-date] :user/log-invalid-date
2 [:user/use-value] :user/use-value
3 [:user/log-invalid-date] :user/log-invalid-date
4 [:farolero.core/throw] Throw the condition as an exception
user>

Why this might not work for you (but still is an option)

One glaring problem with both of these approaches is that both are somewhat foreign to the language, even though both use just the features provided by the language. The advantage of the condition system in Common Lisp is that it is deeply integrated into the language, and conditions, signals, and restarts are widely used across all code, and not just to handle errors. In Java, and therefore in Clojure exceptions are the primary way of error handling. And actually, in Clojure we’re trying to avoid exceptions as much as possible, mostly using nil to tell that function has failed and then branching on return values.

If you think about it, if-some and try catch are two of the most common error handling patterns in Clojure - the first one is mainly used in plain Clojure, and the second one is when interacting with Java. This is a bit of a shame because both can’t provide all these features that the condition system has, and we can’t really run away from exceptions, given that so many java libraries use them.

Libraries like farolero, or the dynamic scope approach can close this gap a bit, however, as I’ve mentioned, the key advantage of the condition system in Common Lisp is that it is used everywhere. Whereas conditions in Clojure will only be an additional way of error handling, thus one more thing to think of and remember. Hence, some parts of the project will use exceptions, because of some libraries that throw them around, some parts will use nil-punning, and some will use conditions, which only may make it harder to maintain the project. Or you will have to wrap a lot of stuff to use your condition system of choice, which will add a lot of implementation details to the code.

Another point is that there are several ways of using the condition system in Clojure - either via some library or by using a dynamic scope. This is not a problem for a project that is more like an application rather than a library, but libraries should not expose their dependencies unless it’s absolutely necessary. Although by using dynamic vars to provide signals and restarts you’re not taking away traditional exception handling, you only provide an additional way to recover, and there’s no extra dependency at all.

This may seem that I’m mostly against the use of the condition system, but in fact, I think that you should at least try this approach, and understand where it would be the preferred way. Condition system really can make your application very robust, and provide very versatile ways to recover from errors. For example, with farolero you can even fix your application at runtime when an unhandled signal pops up by using the interactive debugger it provides. And since farolero also has some ways of handling exceptions as conditions, this may be a good choice even if you use exceptions in your codebase. But this needs more experimenting, and I’m definitively going to keep an eye on this library, as well as the others.

Andrey Listopadov

Condition system in Clojure

Exception-based error handling

Conditions in pure Clojure

Farolero conditions and restarts

Why this might not work for you (but still is an option)

Further reading