Disclaimer: I work at Amutable but not on tine (though we did discuss this internally a fair bit, as you might expect).
Nix does have a similar model in some respects (in particular the idea of input-hashed build graphs) and so solves a lot of similar problems but there were a few important factors that made us not go with Nix (or NixOS) and instead go with Buck2.
As the blog post mentions, the ability to build directly from a wide variety of upstream distribution artefacts and maintain temporary deltas was incredibly important to us. Our team has a lot of old-school packaging experience (I've done packaging for (open)SUSE for over a decade now and we have former and current Fedora, RHEL, Ubuntu, and Debian packaging folks with even longer histories) and our view is that distributions do a lot of thankless and incredibly valuable work and the most painful part of maintaining a distribution is the deltas you have to maintain forever after you've hard-forked packages or repackaged them. Buck2 allows us to create models for these concepts (which are in tine) without needing to boil the ocean repackaging upstream packages. But Daan and Martin are far better equipped to get into the weeds on this topic.
On the more general topic of why not use NixOS, as discussed in one of our other recent blog posts[1] we are make heavy use of sysexts and the /nix/store combined with the general linking model of NixOS (while very technically impressive) does not play particularly well with that model. I should also mention that we have a NixOS guy on our team too so you'd better believe NixOS came up in conversation. :D
Nix is very awkward (and slow) to use for fine-grained caching, which is price of entry for a decent build system.
It also comes (practically) shackled to nixpkgs which is a barely-documented cataclysm of packages that extensively bake in assumptions that you are also going to use Nix as your runtime environment.
The principles behind Nix are nice but the implementation is sorely lacking in my experience.
> Nix is very awkward (and slow) to use for fine-grained caching, which is price of entry for a decent build system.
This is sadly true, I've tried setting up a bazel-esque C++ build system in Nix, and the requirement for 1 derivation per .o file makes it very inefficient
> It also comes (practically) shackled to nixpkgs
This isn't true, and I think it is a shame that it seems true. Nixpkgs is both awful, and completely optional. It would be great if there were more resources on how to use Nix without Nixpkgs, or as is commonly requested, a smaller, cleaner and better engineered alternative
I know it technically isn't true and I use the word 'practically' deliberately. The problem is that as soon as you opt into one part of nixpkgs (say a compiler) you end up implicitly opting into large parts of it because of common code in nixpkgs that make assumptions about undocumented magic environment variables etc that other packages (e.g. libc) may make use of or set.
I feel like a core problem though is the assumption you use Nix for everything - as soon as you try and pull in prebuilt pieces for pragmatic reasons, or output something you can use somewhere outside of nix, it starts making life hard.
Is Nix really appropriate for building a complete rootfs? Could I start from scratch and write a nix build script or whatever and end up with a rootfs I could run on some aarch64 SBC or SoM?
And how is the Nix story for going from a rootfs to a bootable image? On e.g Rockchip you need to make an image where there's some firmware blob in one are of the flash which jumps to a uBoot in another area which loads the kernel/initramfs/devicetree, how does that look in Nix land?
How does it handle building vendor kernels or carrying kernel patches instead of mainline?
Yes, most of this is all supported. There is a existing recipe for building an image that you just dd to and SD card and can boot in a Raspberry Pi (and other devices supported by mainline kernel + u-boot).
Kernel is a package just like another; so you just write your own recipe. Patching kernel recipe works just like patching any Nix recipe.
Not sure about the rockchip-proprietary boot parts; you may be on your own.
The huge boon of Yocto is that it solves the Rockchip-proprietary parts. And the Qualcomm-proprietary parts. And the IMX-proprietary parts. It has BSP layers which you can add, you can use either e.g Rockchip's official Yocto BSP which focuses on doing things their way + using a vendor kernel or you can use the Yocto project's community Rockchip BSP which focuses on doing things the Yocto way + using upstream.
Even devices which are supported by mainline Linux + uBoot have their own ways you have to do e.g partitioning or what needs to be in Flash. Lots of Rockchip devices are perfectly supported upstream, but their boot process expects to find a tiny loader at a particular flash offset which can then jump to uBoot. Nothing in the ARM SBC world is standardized so everyone does things like this (tho some have partitioning schemes you need to follow instead of using offsets). That's stuff you can't really ignore if you want to be relevant in this space.
I don't really like Nix but it's probably fine for this - you have files that come in, a build step, and output files. Your derivations (build recipes) are functions that can take arguments to e.g. provide a set of patches to apply. There are recipes for building a kernel in nixpkgs already which you could fork.
The main annoyance I'd have in your case is probably the number of file copies of large images it may do to keep things isolated, but most hermetic build systems will do something similar.
I mean "input files, a build step, output files" is the trivial part of this. I've made my own build systems which work the same way, it's not especially hard. But if you wanted to use my build system to make a Linux rootfs you'd find that to be a monumental effort since you'd have to do everything yourself.
It's everything built around the build script language that's relevant. How easy is it to make a Nix config which produces a rootfs with systemd, glibc, coreutils, an OpenSSL server, pam, a dbus broker, the necessary getty services, etc etc etc which makes it a Linux system? That's the question I'm asking.
Because of the elitist, exclusionist air it has gathered around itself, I guess. Trying to get something to work with zero knowledge about it is a maze of twisty passages, all alike.
After following breadcrumbs across blogs, I found the proper incantation, executed it, got "too many arguments" from shell, along with multi-megabyte error wall, did not find anything related to the issue, wrote my own slop-makefile.
My solution is a single, dependency-free, multiplatform CLI tool, which Just Works (tm). It's like using zip or tar from command line, as simple as possible. 100% AI-free.
The other article on their blog introduces a systemd module that not only collects all hardware information, but also signs it with TPM before transmission [0].
I'm not even sure it makes much sense in datacenter conditions. And once it gets onto computers of the regular users, it would be very easy to repurpose it for spyware.
Only an idiot would think that systemd-report is malware. Seriously, do you not understand that this is a service that you would only enable if you need it? It’s not magically sending data to bad guys.
And signing the reports using the TPM doesn’t make it spyware either. It merely allows you to verify that the report came from one of your own machines, and one with a known state. If a bad guy hacks the machine the signature will reveal that the machine was hacked.
And there’s no way to turn a TPM signature into DRM. Suppose Netflix wanted to verify that your computer could not save a copy of a Netflix stream to disk. They could get a TPM signature on some data and use that to verify that your computer was compliant with the rules, right? Wrong! Every single update you ever install will change the signature. BIOS/UEFI updates, kernel updates, bootloader configuration, rolling updates to packages. It all changes the TPM state and thus the signatures. To verify any of that Netflix would have to know every single combination of OS version and BIOS version and motherboard model and so on. In practice all they’d get is a cacophony of noise.
TPM signatures are only useful for verifying the state of your own computers, and only because you control when and how the upgrades happen. And by “you” I really mean large enterprises with tens of thousands of computers (both real and virtual) to manage and a dedicated staff to keep track of everything.
Why cant we just use Nix for building anything? A unified approach to building stuff.
Disclaimer: I work at Amutable but not on tine (though we did discuss this internally a fair bit, as you might expect).
Nix does have a similar model in some respects (in particular the idea of input-hashed build graphs) and so solves a lot of similar problems but there were a few important factors that made us not go with Nix (or NixOS) and instead go with Buck2.
As the blog post mentions, the ability to build directly from a wide variety of upstream distribution artefacts and maintain temporary deltas was incredibly important to us. Our team has a lot of old-school packaging experience (I've done packaging for (open)SUSE for over a decade now and we have former and current Fedora, RHEL, Ubuntu, and Debian packaging folks with even longer histories) and our view is that distributions do a lot of thankless and incredibly valuable work and the most painful part of maintaining a distribution is the deltas you have to maintain forever after you've hard-forked packages or repackaged them. Buck2 allows us to create models for these concepts (which are in tine) without needing to boil the ocean repackaging upstream packages. But Daan and Martin are far better equipped to get into the weeds on this topic.
On the more general topic of why not use NixOS, as discussed in one of our other recent blog posts[1] we are make heavy use of sysexts and the /nix/store combined with the general linking model of NixOS (while very technically impressive) does not play particularly well with that model. I should also mention that we have a NixOS guy on our team too so you'd better believe NixOS came up in conversation. :D
[1]: https://amutable.com/blog/it-starts-upstream-kernel#image-ba...
Nix is very awkward (and slow) to use for fine-grained caching, which is price of entry for a decent build system.
It also comes (practically) shackled to nixpkgs which is a barely-documented cataclysm of packages that extensively bake in assumptions that you are also going to use Nix as your runtime environment.
The principles behind Nix are nice but the implementation is sorely lacking in my experience.
> Nix is very awkward (and slow) to use for fine-grained caching, which is price of entry for a decent build system.
This is sadly true, I've tried setting up a bazel-esque C++ build system in Nix, and the requirement for 1 derivation per .o file makes it very inefficient
> It also comes (practically) shackled to nixpkgs
This isn't true, and I think it is a shame that it seems true. Nixpkgs is both awful, and completely optional. It would be great if there were more resources on how to use Nix without Nixpkgs, or as is commonly requested, a smaller, cleaner and better engineered alternative
I know it technically isn't true and I use the word 'practically' deliberately. The problem is that as soon as you opt into one part of nixpkgs (say a compiler) you end up implicitly opting into large parts of it because of common code in nixpkgs that make assumptions about undocumented magic environment variables etc that other packages (e.g. libc) may make use of or set.
I feel like a core problem though is the assumption you use Nix for everything - as soon as you try and pull in prebuilt pieces for pragmatic reasons, or output something you can use somewhere outside of nix, it starts making life hard.
Honestly, I'm hoping for someone to use AI to translate the Nix universe of scripts to a saner and more modern, user-friendly language.
Is Nix really appropriate for building a complete rootfs? Could I start from scratch and write a nix build script or whatever and end up with a rootfs I could run on some aarch64 SBC or SoM?
And how is the Nix story for going from a rootfs to a bootable image? On e.g Rockchip you need to make an image where there's some firmware blob in one are of the flash which jumps to a uBoot in another area which loads the kernel/initramfs/devicetree, how does that look in Nix land?
How does it handle building vendor kernels or carrying kernel patches instead of mainline?
Yes, most of this is all supported. There is a existing recipe for building an image that you just dd to and SD card and can boot in a Raspberry Pi (and other devices supported by mainline kernel + u-boot).
Kernel is a package just like another; so you just write your own recipe. Patching kernel recipe works just like patching any Nix recipe.
Not sure about the rockchip-proprietary boot parts; you may be on your own.
The huge boon of Yocto is that it solves the Rockchip-proprietary parts. And the Qualcomm-proprietary parts. And the IMX-proprietary parts. It has BSP layers which you can add, you can use either e.g Rockchip's official Yocto BSP which focuses on doing things their way + using a vendor kernel or you can use the Yocto project's community Rockchip BSP which focuses on doing things the Yocto way + using upstream.
Even devices which are supported by mainline Linux + uBoot have their own ways you have to do e.g partitioning or what needs to be in Flash. Lots of Rockchip devices are perfectly supported upstream, but their boot process expects to find a tiny loader at a particular flash offset which can then jump to uBoot. Nothing in the ARM SBC world is standardized so everyone does things like this (tho some have partitioning schemes you need to follow instead of using offsets). That's stuff you can't really ignore if you want to be relevant in this space.
I don't really like Nix but it's probably fine for this - you have files that come in, a build step, and output files. Your derivations (build recipes) are functions that can take arguments to e.g. provide a set of patches to apply. There are recipes for building a kernel in nixpkgs already which you could fork.
The main annoyance I'd have in your case is probably the number of file copies of large images it may do to keep things isolated, but most hermetic build systems will do something similar.
I mean "input files, a build step, output files" is the trivial part of this. I've made my own build systems which work the same way, it's not especially hard. But if you wanted to use my build system to make a Linux rootfs you'd find that to be a monumental effort since you'd have to do everything yourself.
It's everything built around the build script language that's relevant. How easy is it to make a Nix config which produces a rootfs with systemd, glibc, coreutils, an OpenSSL server, pam, a dbus broker, the necessary getty services, etc etc etc which makes it a Linux system? That's the question I'm asking.
Because of the elitist, exclusionist air it has gathered around itself, I guess. Trying to get something to work with zero knowledge about it is a maze of twisty passages, all alike.
After following breadcrumbs across blogs, I found the proper incantation, executed it, got "too many arguments" from shell, along with multi-megabyte error wall, did not find anything related to the issue, wrote my own slop-makefile.
nix is escape room
Regardless of whether this would actually be a good idea, it's unquestionably an instance of
https://xkcd.com/927/
I've also created a boot loader with a native bootable disk image creator tool.
https://news.ycombinator.com/item?id=49685301
My solution is a single, dependency-free, multiplatform CLI tool, which Just Works (tm). It's like using zip or tar from command line, as simple as possible. 100% AI-free.
The other article on their blog introduces a systemd module that not only collects all hardware information, but also signs it with TPM before transmission [0].
I'm not even sure it makes much sense in datacenter conditions. And once it gets onto computers of the regular users, it would be very easy to repurpose it for spyware.
[0]: https://amutable.com/blog/it-starts-upstream-systemd-report
It’s no different than a Prometheus stat exporter. If you don’t want it then don’t enable it.
Also, most consumer computers don’t have a TPM.
> most consumer computers don’t have a TPM
https://www.techtimes.com/articles/319581/20260703/steam-har...
According to the article, ~70% of Steam users are on Windows 11, so have TPM 2.0. That is already the majority.
Is that really a requirement for Windows 11? I don’t keep track. Perhaps Windows 11 is the spyware.
It is important to not trade one malware for the other one.
Only an idiot would think that systemd-report is malware. Seriously, do you not understand that this is a service that you would only enable if you need it? It’s not magically sending data to bad guys.
And signing the reports using the TPM doesn’t make it spyware either. It merely allows you to verify that the report came from one of your own machines, and one with a known state. If a bad guy hacks the machine the signature will reveal that the machine was hacked.
And there’s no way to turn a TPM signature into DRM. Suppose Netflix wanted to verify that your computer could not save a copy of a Netflix stream to disk. They could get a TPM signature on some data and use that to verify that your computer was compliant with the rules, right? Wrong! Every single update you ever install will change the signature. BIOS/UEFI updates, kernel updates, bootloader configuration, rolling updates to packages. It all changes the TPM state and thus the signatures. To verify any of that Netflix would have to know every single combination of OS version and BIOS version and motherboard model and so on. In practice all they’d get is a cacophony of noise.
TPM signatures are only useful for verifying the state of your own computers, and only because you control when and how the upgrades happen. And by “you” I really mean large enterprises with tens of thousands of computers (both real and virtual) to manage and a dedicated staff to keep track of everything.