<unnamed>

Public Key

npub14zruru6537cpp778f8atyqstd5yhf2h5sj60za7c6jclrk3uqzmqentgff

Profile Code

nprofile1qqs23p7p7d2glvqsl0r5n74jqg9k6zt54t6gfd83wlvdfv03mg7qpdspzfmhxue69uhk7enxvd5xz6tw9ec82cs99zpmg

Publishing to

Last Notes

2026-11-05 17:17:09 GMT

The pool was the easy part
I had two 512 GB NVMe drives doing two stubbornly separate jobs. One held /persist, the only thing that survives a reboot on a workstation whose root is a tmpfs. The other held ~/Storage: VM images, model weights, and the scratch directory I keep promising to clean up. Two fixed partitions, and naturally the one filling up was never the one with room. Neither could lend the other a gigabyte.
The fix is old and dull. Put both drives in one LVM pool, carve a single encrypted volume across the pair, and let /persist and ~/Storage draw from the same ~750 GB instead of starving in separate corners.
Dull, except for one detail. To pool the drive that holds /persist, you have to erase the drive that holds /persist. The only copy. On the laptop I was using to do the erasing.
Here is the part nobody warns you about: the step I’d braced for went off without a hitch, and the step I hadn’t given a second thought to booted me into a desktop with no desktop.
The drive that keeps hanging up on me
Before you wipe your only copy of anything, you make a second copy, and you make sure it’s real. My backup target is a 2 TB Seagate that lives in a drawer. It has opinions. It spins itself down and drops off the USB bus the second you stop talking to it, which it interprets generously as about eight seconds of silence. Plug it in and some combination of the kernel’s own UAS blacklist and the drive’s firmware parks it before you’ve finished typing mount.
So the first hour of a disk migration went to a shell script whose entire job was keeping a hard drive awake: kill USB autosuspend, pin the device the instant it enumerates, and read a single block off it every couple of seconds so it never decides I’ve left the room. A heartbeat for a drive that would rather be asleep.
With it pinned, I did the thing you are supposed to do and almost never do. I read the backup back. Not “does restic say it finished,” but a full restic check --read-data, every pack re-hashed straight off the platter. 11,186 packs, roughly 115 GB, about an hour and a half at the ten-ish megabytes a second this poor drive musters over its crippled link. No errors. A backup you haven’t read back is a rumor, and I was about to bet the machine on this one.
Pinning a UUID to a disk that doesn’t exist yet
The reason the frightening part wasn’t frightening is a trick NixOS makes almost too easy.
The new encrypted volume didn’t exist yet. It would get its identity, a LUKS UUID, during the live-USB surgery, out of a cryptsetup command I hadn’t run. Normally that means you do the surgery, then rebuild the config against the new reality, then cross your fingers that it boots. That is a lot of fingers to cross in a live shell at midnight.
So I picked the UUID first. Generated one, pinned it into the config, and built the entire new system against it while the old machine was still running underneath me. The disk the config described was imaginary, and Nix did not care. It evaluated, built, and produced a bootloader entry pointing at a volume that wouldn’t be born for another hour. Every error that could have ambushed me in the live environment surfaced on the running desktop instead, where I had a browser, a shell, and no adrenaline.
By the time I rebooted into the installer I wasn’t rebuilding anything. I was creating a disk to match a config that already believed in it. Wipe both drives by their stable /dev/disk/by-id names, because the sdX letters shuffle the instant you breathe on a USB port. Stack them into one volume group. Format one LUKS container across the whole thing with the UUID I’d pre-committed. Restore /persist from the Seagate. Reboot.
It came up. One encrypted volume spanning both NVMes, 741 GB usable, /persist and ~/Storage finally sharing a tank. I got to feel good about this for most of a minute.
The desktop that booted into nothing
I logged in to bare bash. No fish, no prompt, no Noctalia bar, just a blinking cursor in a terminal that had clearly read none of my config. Two services had failed, Home-Manager and ollama, and the Home-Manager log had the whole story in one line:
mkdir: cannot create directory '/home/lowcache/Storage/tmp': Permission denied
Permission denied. To me. In my own home directory.
Here is what I’d done. The live installer’s sudo kept refusing me for reasons I had no patience to debug at midnight, so I ran the whole migration as root through sudo su. That worked. It also meant that when I typed mkdir /persist/storage to create the directory ~/Storage now lives inside, root created it, and root kept it. restic restored everything underneath with my ownership intact, so the contents were fine. The one directory I made by hand, the parent of my entire junk drawer, belonged to root.
Home-Manager’s first move on login is to make a couple of scratch directories inside ~/Storage. It tried, as me, to write into a directory owned by root, and it quit. And because on this machine my shell and my whole desktop are Home-Manager’s job, one failed mkdir tore straight through fish and Noctalia and dropped me at a prompt from 2004.
This is the same lesson in a new costume. ~/Storage is now a bind mount of /persist/storage, which means the owner of that single directory now decides whether my desktop loads. I keep finding fresh ways to botch persistence, and this was a good one.
Two chowns back to a working machine
The fix for the big scary migration was one command:
chown lowcache:users /persist/storage
Not recursive, very deliberately. The restored contents already had the right owner, and ~/Storage/libvirt has to stay owned by root and qemu for the VM to work, so a recursive chown would have traded one broken thing for another.
Re-running Home-Manager got me further and hit the sequel:
Existing file '/home/lowcache/.config/kitty' would be clobbered
The restore had quietly turned two of my out-of-store config symlinks into real directories, and Home-Manager refuses to overwrite a real file it didn’t place. Both were copies of what already lives in my repo, so I moved them aside, re-ran activation, and it went green. fish came back. The bar came back. ollama wanted its model directory recreated, and then it came back too.
What it cost
The reinstall-class operation, the one where a wrong keystroke loses the only copy of everything, was a non-event. Verified backup, pre-built config, by-id all the way down. I had engineered the fear out of it.
What actually cost me an hour was ownership. A directory I made as the wrong user because I took a shortcut, on a machine where a single directory’s owner now sits in the boot path. If you run a restore as root on an impermanence box, chown your persist root back to yourself before you reboot, or your next login will be a humbling one.
The pool works. I have 475 GB of headroom I get to stop thinking about. And I have one more entry in the running ledger of ways I’ve broken persistence and then written down, so the next person, usually me, doesn’t.
https://infernalcode.com/posts/the-pool-was-the-easy-part/
#rss #feeds #linksharing #qwibbit