CLuix: A Common Lisp Approach to Functional Package Management

Hey there,

This time I'd like to talk about an idea I've been carrying around for some time, percolating in my subconscious. Hopefully, I'll get to explore an actual implementation soon.

The main idea is that instead of the approach used by Guix/Nix where things are very different from a standard *nix system but we're trying our best to pretend that things are somewhat normal, we build things so they actually are normal.

How the system works

The way we accomplish this is by having each generation be an actual filesystem image we can mount. We then use overlayfs so that a user can actually change config files if they absolutely want to (though it should be highly discouraged). Also, things like a user's home directory should be completely separate from the system image. This way, we should be able to have our entire system in a single image file and all the user's actual files in the normal filesystem.

Configuration options should also be in their separate image file, that way we could share the system image between multiple hosts (think a fleet of machines all running the same software, but having different hostnames).

This would also make installation super simple since we can just move the base system image over.

An example system definition

Here's how a system definition might look, based on an existing GuixSD definition that I've rewritten to demonstrate the intended syntax for CLuix:

(operating-system
  (host-name "cocz.net")
  (timezone "Europe/Berlin")
  ;; One could also specify locale/kernel and other options here. If not specified, defaults are used.

  ;; The list of user accounts ('root' is implicit).
  (users (user-account (name "benny")
                       (group "users")
                       (home-directory "/home/benny")
                       (supplementary-groups '("wheel"))))

  ;; List of services - can be a simple string for default configuration
  ;; or a service definition when customization is needed
  (services "ntpd"
            (service "ssh" :port 69)
            (service "certbot" :domains '("cocz.net"))
            (service "nginx" :web-root "/home/benny/www" :server-name "cocz.net"))

  ;; Packages to be available for all users in the system
  ;; Simple string for default package, or custom configuration as shown with Emacs
  ;; This example specifies a terminal-only build of Emacs with no X support
  (packages (package "emacs" :no-x)
            "rsync"))

Why Common Lisp?

Other languages might work as well, but generally I needed a language that can perform runtime compilation and interpret code, which eliminates options like C, Rust, Go, and similar compiled languages. Common Lisp stands out as an excellent choice primarily because of SBCL, which is an outstanding CL runtime that in my benchmarks consistently outperforms Node.js/V8. It also handles multithreading much better, and building this entire system on something like CLOS (Common Lisp Object System) could prove extremely powerful for a declarative package management system.

How do packages work?

I'd like to have packages work somewhat similar to Arch Linux, and if possible also reuse makepkg files from Arch, mainly due to the AUR. Although reusing Guix definitions might be better since it's already meant for a declarative system like CLuix.

What's the benefit?

Mainly compatibility and performance. One thing that keeps making me go back to distros like Arch is that NixOS/GuixSD is just inherently slower. I'm not sure whether that's due to wrong defaults in their packages, though I think it might be because figuring out where shared libraries are is much more complicated on these systems. Just a wild guess though, but at least when it comes to software I've written, I can actually see that it's quite a bit slower on a Guix/Nix system. NixOS seemed a little bit faster and the whole package management aspect is much faster, though I still much prefer Guile Scheme over Nix.

What about linux-libre?

Apart from the performance issues, the linux-libre default where one has to jump through many hoops just to get their hardware to work has really left a bitter aftertaste. I mean, I can absolutely understand their position, but the problem is that even though I'm somewhat knowledgeable about these things and try to get hardware that's compatible with linux-libre, I don't believe I have a single device where everything works with linux-libre (I even have 2 machines that won't even boot properly with it...).

It would be so much better if one could choose which kernel to use and basically just include NonGuix as a valid option. Then I might start looking at it again. As it stands, I can only really use GuixSD on seriously outdated hardware where the performance costs make a slow system even slower, which is something I also really dislike. So in the end, I just use other distros like Arch, NetBSD, or Void Linux.

What about Docker?

I'd also love it if, just like with Guix, one could export a system definition to an OCI Image. While Dockerfiles do work, I'd love to be able to reuse system definitions for real hardware, VMs, and containers.

Next Steps

Once I have more time, I'll begin exploring this concept in depth. My plan is to reuse as much as possible from Arch Linux and Guix SD to create a workable prototype. I might initially skip the image/overlayfs approach and instead implement a diffing system that uses pacman to track installed programs and determine what needs to be added, removed, or updated.

This approach could be particularly valuable as a layer running on top of existing Arch systems, serving as an alternative interface to pacman. Adding support for the AUR would make it even more powerful, potentially giving users the best of both worlds: the declarative, reproducible nature of Guix/Nix with the extensive package availability and performance of Arch Linux.

Adios,
べン