Ideas, ideas, and a lack of implementations

by hobbitalastair

One of the things I really like about Linux is the flexibility to do what I want. The Linux kernel itself is just a single piece of software, which can be added to any number of other bits, to create something that is heavily customised, exactly what I want.

On the other hand, it does rely on those pieces of software existing, and doing what I want…

One of the other problems with this approach is that each individual piece of software has to be customised for each individual machine or task. Add in cross compiling, and the fact that each individual has to repeat the process, and it quickly becomes a mess.

This is where a Linux distro comes in. By handling the basic packaging of the software, users can create a system that is fairly close to what they want, with a lot less effort, and duplication of effort. Distros are great!

Takahe Linux is a Linux distro. It fills a certain need, like many other distros, although it is probably too specialized to ever have more than a single person (me) working on it. Yet, I’d like it to be customizable.

How can I achieve this balance?

One way that I quite like is to use a ‘tiered’ configuration system. Systemd uses it; having personally been exposed to it on my Arch Linux systems, I can say that it simplifies management enormously.

The basic approach is to have 3-4 ‘layers’ of configuration files. Note that I said ‘configuration files’ – unfortunately, if you want to customize your systems any further, you will need to start managing your own packages of whatever pieces of software you want. This is OK to some degree – what I do is customise the ‘important’ pieces of software, those that I use frequently, and the I just leave the ‘non-important’ pieces to the distro developers. That way, I can maintain a manageable collection of packages.

These ‘layers’ are a bit like the packages. The idea is that the admin, or even the user, should be able to change exactly what they want to change – and leave the rest untouched. Like only self-packaging important pieces of software, users or admins should be able to modify only what they need to modify, keeping the workload at a manageable level. Provided the distribution or upstream developers provide sane defaults, that is.

An example of a ‘sane’ production system that uses this method is systemd. Yes, it is not perfect. However, it does cover what I see as the main layers:

  • Upstream layer. This is accomplished via presets. I believe that distributions also use presets, so it isn’t perfect.
  • Admin layer. This is accomplished via separate config files, which override the default ones if they are supplied, and by providing mechanisms that can be used to override presets.
  • User layer. This is provided by user-specific services and files, which can override the both the admin and upstream versions.

Personally, I really like this system. It lends itself to a situation where all of the hardware or even deployment dependent settings can be deployed in the form of a package, which can be handled easily by existing tools. It avoids the need for complex configuration management tools, like what I am trying to make, and generally provides a simpler ‘interface’ for tinkerers. For simple, single machine systems, you could even mirror your entire ‘admin layer’ content on github – that way, if the machine fails, or you change the underlying distro, it would be easy to pull down the ‘admin layer’ and instantly have a familiar system, with all of your settings. The current alternative is to keep track of changes to existing configuration files, which is not at all portable or readily scriptable – currently, whole applications exist just to parse and modify configuration files, so that changes can be readily scripted.

Of course, this could be simplified still further if all of the applications shared a default configuration file format – say, like Apple’s plist format – which would help enormously, as well as reducing bugs by sharing code. Of course, some applications might have problems with the chosen format not being flexible enough, or choosing to use a different library 😦

Preferably, the layers would be extensible, as well, with the ability to add more or less as time went on. Currently, here’s my vision:

  • /usr (or /system, or whatever you want to call it) holds the readonly, distribution and upstream specific packages. Distribution and upstream, because I can’t see ‘upstream’ ever packaging things exactly as a distribution wanted… but hopefully, at some point, they could be merged, with upstream doing the packaging.
  • /etc or similar would hold the ‘admin layer’ configuration and other similar data files, with any relevant overrides for the machine. Hopefully, every file in /etc could be overridden, for instance if an admin provided a default config for all of the machines on a network, but wanted to allow, say, root ssh access on a few special machines.
  • /var would hold runtime data that should be permanent. /etc should probably really belong in here as well, and so should /home, and even /root.
  • /proc, /dev, /sys, and /tmp should be mounted at runtime, since they should be recreated dynamically every boot.
  • /mnt and /media are more complex – however, I’d prefer it if both could be rolled into one and pointed to, say, /tmp/mnt, or maybe just /media… because, like /dev, /sys, /tmp, and /proc, mounts might not persist over boots.
  • /opt should not exist. Unfortunately, it does, and is still needed occasionally. The best bet here would be to try to convince all packages to coexist nicely under /usr, and only ever use /opt as a fallback.

I’d also imagined what layers I might need or want:

  • Upstream – the default, as decided by the developers.
  • Distribution – changes that are sane for the distribution, for instance turning off lots of features for an embedded system. Because some of this would be done at compile and package time, it’s currently not really needed as a separate layer, at least until upstream starts using a unified packaging scheme…
  • Admin – there are multiple possible sub-layers
    • Company wide configuration
    • Department wide configuration
    • Use case wide configuration
    • Hardware specific configuration – for instance, the hostname, fstab, ssh keys, …
    • Any other configuration types that might exist…
  • User – things like customized preferences for userspace applications, for instance UI styling…

The end goal is to create an easily managed system, at least with regard to configuration, preferably one which can be deployed via standard mechanisms – I’m thinking of package managers. The last piece of the puzzle, of course, would be using a configuration file for determining which packages should be installed…


Like most of my ideas, this one is likely never to really take off. Fortunately or otherwise, it’s too big a change, and would require rewriting a large chunk of the so-called ‘userspace’, even if some projects are already moving that way. Luckily, I can always experiment 🙂