Lua appreciation post

Lua is one of the most pleasant languages that I’ve used so far. Well, I’m not writing in Lua directly, instead, I use Fennel - a compiler for a Clojure/Lisp-like syntax to Lua. Because of that, I actually don’t really know Lua syntax that well, even though it’s really simple, it still has some quirks. That’s something I like about hosted languages and transpilers - you may leverage the power of a good runtime while sticking to the syntax you know well. But I’ll do my best here, describing things I love about Lua and its runtime, and some things that I wish have been a bit better.

The runtime

And the first thing I’d like to talk about is how great the Lua runtime actually is! Lua is small, its runtime is lightweight, and has good enough performance to use in a variety of places. It can be used from games to window managers and text editors, such as NeoVim, Micro, Vis, which all provide a way to use Lua for configuration and scripting. There even is an editor that is almost fully implemented in Lua, called Lite, and it can be extended via it as well. This is because Lua is actually pretty suited for embedding in applications and even can be used on some micro-controllers too, so it is quite versatile!

What I like the most is perhaps how expressive you can be even with so little the runtime gives you. Being very small, Lua supports first-class functions, closures, tail call optimization, collaborative multitasking via coroutines, and more! To be frank it often feels like you’re using Scheme with a different syntax (which is actually can be fixed with Fennel). All these features already make Lua very expressive and powerful, but Lua actually has more things to offer!

The standard library

It’s often said that Lua has no standard library, but I don’t think it’s practically true. Lua, for sure, has a small standard library, but it’s really surprising how far you can get away with just it!

By default, Lua features these modules: io, os, string, utf8, math, coroutine, table, package, debug. Not all of these modules are present at all times though. For example, it’s a common thing to remove the debug module from production systems for better sandboxing. Some systems restrict or even remove the package module, so custom package searchers could not be added.

It’s still much smaller than in other languages, like Python, which Lua is often compared to, but even with all of that that is available, it’s possible to build exciting stuff! For example, Fennel itself is built entirely in Lua, without any other third-party library. As a result, Fennel is a single Lua script, which can be executed as a standalone language, or used to do ahead-of-time compilation from Fennel to Lua (and back!).

The standard library is minimalist, for so to speak. For example, there are no multitasking facilities, as one might expect from a modern language, only basic coroutines. But given the embeddable nature of Lua, this is a good thing, because not all systems have support for real multitasking, or you might need more specific facilities for your particular needs. Coroutines allow you to implement what suits your needs best.

But one of the most powerful things in Lua is tables.

Tables

The main data structure in Lua is a table. A table in Lua is a compound type, containing both sequential and associative parts together in one table. Originally there was only the associative part, but the sequential part was later added for better performance. There are no other data structures in Lua, e.g. no linked lists, no tries, no sets. Furthermore, Lua doesn’t have classes!

So you might wonder how’s tables are the most powerful features of Lua? The answer is simple - a table can be any of that! And Lua provides us with a lot of ways to achieve that.

This is possible because in Lua tables are flexible. Not only tables can contain any other type, like strings, functions, or other tables, it’s behavior can be altered via metatables. So implementing a class is a matter of implementing a proper set of metamethods, using a table as a container. For example, lazy evaluated linked lists are entirely possible by using a combination of closures and metatables. Immutable tables are possible, thanks to inbuilt __index and __newindex metamethods. And because tables can be thrown as errors, and because tables are dynamic, it’s possible to implement a different error handling system, like a condition system from Common Lisp.

Not only that, but Lua provides a table module in its standard library, which provides various functions for working with tables. It contains only necessary stuff, and more complex things can be built on top of it.

Another neat thing about tables is that Lua has a really clear vision of iterators. Lua provides two main functions for table iteration: pairs and ipairs. Their main difference is that ipairs only traverses the sequential part of the table, and pairs traverses table as a whole. And these iterators are just pure functions, that you (theoretically) can share, as there’s no inner state. And because iterators are lazy, it’s not a problem to consume really big resources, such as tables or even files.

All in all, tables are a very flexible part of Lua, which allows the user to implement anything, given enough time and determination of course.

Problems

But not that there aren’t any problems in Lua - there are, as in everything. The first one is that the lack of some things in the standard library often shows when you want to create a script that you can later distribute to other people as an executable. This can be solved by embedding the library code into your application via the package module, and Fennel actually does it, so this can be migrated.

But this raises another problem - the language ecosystem. Lua has a de facto package manager, called LuaRocks, and while Lua definitively rocks, LuaRocks doesn’t (IMO). I mean, it does its job, but it is extremely clunky, and not very user-friendly. Lua actually contributed to this problem, because it has changed its notion of modules during some releases, and also because it doesn’t have proper support for relative require. As a result, a lot of libraries are just a single file, that you can drop directly into your application. Some other libraries are more complex though, and almost require you to use Luarocks for installation, otherwise, you would need to hand-manage LUA_PATH for each such library. Which works, but not as great as it could be.

Varargs

As for language-side problems, I can think of only one that stands out in particular - the support for varargs ... and a multi-value return. There’s a great article, going in-depth of vararg quirks, so I’ll keep it brief. The main problem with varargs is that they’re not a first-class value. You can’t properly store varargs without using a table, and given that varargs obey a bit different scoping rules it’s sometimes awkward to use them with higher-order functions. For a better illustration here’s an implementation of a partial function from Clojure:

Lua 5.4.3  Copyright (C) 1994-2021 Lua.org, PUC-Rio
> function partial(f, ...)
>>   local args = table.pack(...)
>>   return function(...)
>>     local inner = table.pack(...)
>>     table.move(inner, 1, inner.n, 1 + args.n)
>>     table.move(args, 1, args.n, 1, inner)
>>     return f(table.unpack(inner, 1, args.n + inner.n))
>>   end
>> end
> greet = partial(print, "Hello")
> greet("Igor")
Hello	Igor
> greet("Bob", "Alice")
Hello	Bob	Alice

This is a simplified version of currying - we create a function, that takes a function, and some but not all arguments for that function. The result is itself a function, that will call the original function with all additional arguments:

The main issue with that function is that in order to pass the ... args to the inner function, these args have to be packed into a table, which will act as a closure for the returned function. The returned function then needs to pack its own ... into a table, move those to make space for outer arguments, and copy those to the beginning of the table. Then we can unpack the args and pass those to the function. All this packing and unpacking is necessary due to the fact that Lua tables can’t contain nil as value, but varargs can. In Lisps, for example, functions that accept a variable amount of arguments, implicitly put those to a list, which later can be operated directly. And there’s no problem with having nil in a list too.

Another problem is that you can also return multiple values from a function, and this can result in some unexpected errors when chaining function calls.

Personal reasons to like Lua

I fell in love with Lua because of how welcoming as a language it is. There are a lot of things available for you to make facilities you need for your particular application. Want a custom coroutine scheduler? No problem! Want an object system? You can do it too. Need more data structures? You can implement those on top of tables, or even do it in C for better performance. And yeah, Lua has great C FFI.

I know that other languages can do these things, and it’s not a unique feature of Lua. But other languages are often much more complex, both in implementation and semantics, while Lua really keeps things simple. Apart from some quirks regarding multiple values thing there’s not much you need to learn - you can grab Lua and almost instantly start doing things with it. And libraries like Penlight pretty much solve the problem of a small standard library or the lack of “batteries”.

Lua’s embeddable nature may scare some people, but nothing really prevents you from writing ordinary scripts, as some do in Python. It’s definitively not for everyone, but I’ve found it very suitable for my needs.