Rendered at 20:17:52 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
ghostly_s 27 minutes ago [-]
I'll continue listening to my OS vendor rather than this random github user with some broken links, thanks.
(I’m not interested in being educated on this, I have better things to do. Bring it up with Debian.)
jenders 21 minutes ago [-]
Ah the old HN hug. I’ll fix it
cesarb 45 minutes ago [-]
> fallocate / chmod / mkswap
Why not "mkswap --size ... --file ...", which does these three things and more? For instance, according to the mkswap man page, "[...] sets the nocow attribute for newly created files [...]" which is a detail that seems to be missing from this gist.
jwilk 2 minutes ago [-]
[delayed]
rini17 1 hours ago [-]
Use zram instead. This from crappy 8G RAM multimedia/browsing minipc I'm on right now. The SSD is quite slow and browser is memory hungry so it was an improvement from on disk swap.
$ zramctl
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 zstd 3G 482.9M 86.7M 91.8M 4 [SWAP]
kees99 1 hours ago [-]
zram is a good choice when data in RAM is very repetetive/compressible. Web browsers is one such load. Android apps is another good one.
When that data doesn't compress well at all, zram is no help and traditional swap-to-storage is a better choice.
And there is a middle-ground option, zswap: swapped out pages will eventually end up on a block device, but are compressed first.
SoftTalker 1 hours ago [-]
For a while since RAM sizes have gotten so big I would set up linux servers without swap at all. I figured what is a 2GB swap file going to do if the system has already exhausted 128GB of real RAM?
I never saw any issues doing this, but I learned that linux ideally wants some swap space, which it will use for some kind of housekeeping stuff.
malwrar 47 minutes ago [-]
Any tips on measuring the impact of not having swap? I never use it on my machines for similar reasons.
ktm5j 41 minutes ago [-]
Worst thing that's ever happened to me is having processes randomly killed by the OOM killer. Could be unpleasant if you're writing an email or document that suddenly disappears because your app got killed.
weinzierl 12 hours ago [-]
There used to be a time when not only the OS required using partitions for optimum performace (swapfiles) but also applications.
In the late 90s databases were regularly set up in a way to store their data on raw partitions. There were other types of applications too that required partitions but for databases it was really common.
The practice really only died with OSes allowing apps to bypass the normal filesystem cache.
In Linux this is O_DIRECT and interestingly Linux was super late to the party because Linus hated it with a passion. Now, he was far from the only one, at least Alan Cox and Andrew Morton were skeptical, others as well. Linus was unusually extreme about it and that particular discussion is probably the single source of the majority of quotes people use against him.
RiverCrochet 2 hours ago [-]
was this before mmap(2)?
topspin 2 hours ago [-]
mmap(2) in Unix predates Linux itself by several years (1988 Sun, 1990 BSD). mmap(2) was present in Linux well before any of the high performance databases, file systems and other applications that use(d) O_DIRECT appeared on Linux.
O_DIRECT was resisted by Torvalds and others, but it's there today, and used by supposedly important platforms. Nvidia even has an API to DMA data to and from GPUs that uses O_DIRECT. There are folks working on new async WAL O_DIRECT for PostgreSQL as well.
So what can be said? mmap(2) isn't sufficient for certain cases. Torvalds has faced and successfully navigated difficult design choices in a pragmatic manner. What else?
The lesson I take is this: there are levels to implementation complexity and value. The wisdom to decide the appropriate abstraction level is crucial. You are unlikely to be equipped well enough to beneficially utilize O_DIRECT, but it's not impossible and some, in fact, are so equipped.
klodolph 6 minutes ago [-]
Well-put
kazinator 2 hours ago [-]
Swap partitions are your set-and-forget permanent installation that covers daily use.
Swap files are for "oh shit, I need to run this rare thing that needs lots of VM". You create them, "swapon", then "swapoff" and remove when done.
2 hours ago [-]
spijdar 2 hours ago [-]
Why use swap files when we could have swap directories? ;-)
Sprite had (IMO) a really interesting solution to swap. Each host had a directory, referenced by its host ID under the `/swap` directory, and inside that directory, individual segments of virtual memory would be saved: https://github.com/OSPreservProject/sprite/blob/master/src/k...
Sprite overcommited memory like a modern OS, and stored segments individually so that process migration worked (if a segment was swapped-out, the path to the swap file would remain the same anywhere on the cluster, so if the process was migrated, you didn't need to swap-in the segment before migration):
Btrfs has a workaround: it can create a cow-disabled subvolume for the swapfile.
AFAIK you need a separate partition for Zfs.
thyrsus 14 minutes ago [-]
Most file systems get cached in ram, which is unlikely to have happy consequences when scarce ram triggers use of the filesystem swap.
ducktective 11 hours ago [-]
Tangential, but does anyone know why in 2026 and on Debian 13, my machine still hangs when some process exhausts RAM?
Is there really no higher-priority kernel process to prevent total freeze of the system and send a SIGKILL to the culprit process when such a scenario happens?
It's because linux is a toy OS. Specifically, it overcommits memory in the hope/assumption that it won't all be used at once, but doesn't have a way to gracefully degrade when applications collectively want to use more memory(+swap) than it actually has. You can turn off overcommit, but applications are designed with the overcommitting feature in mind, so your experience might not be as good as you were hoping for.
Making a massive swap space helps a little bit. It's better to just never let your actual memory usage go above 85% to 90%. It's fine to go above if you're trying to optimize a server with a specific set of processes to wring every last bit of efficiency out of it, but not for general desktop computing.
If it really bothers you OpenBSD doesn't allow overcommit at all and Windows handles this situation much more gracefully, so WSL is an option too. If you don't mind Oracle (i do), solaris also doesn't allow overcommit.
throw0101d 1 hours ago [-]
> It's because linux is a toy OS. Specifically, it overcommits memory
You're right. I'm glad the very next sentence in my comment landed.
The problem with turning it off is that the system and applications have been architected assuming that it will be on, so things like fork/execing a memory heavy processes or allocating memory inside a cgroup (which still pretends overcommit is enabled and there's still no way to disable that assumption) that used to work fine might break with no good way to get them to work again. This comment (and siblings) have more specifics: https://news.ycombinator.com/item?id=27794237#27795199
35 minutes ago [-]
nicman23 2 hours ago [-]
or use the system's oom ?
joshAg 1 hours ago [-]
That's what's breaking the system and causing freezes. You can tune it a bit to minimize when it happens, but not get rid of the issue entirely.
rwmj 47 minutes ago [-]
All I want is the oomkiller to always kill firefox. Somehow that's very difficult to achieve.
SoftTalker 1 hours ago [-]
... which then kills sshd, locking you out of being able to get in and do any recovery.
nicman23 30 minutes ago [-]
the system will restart sshd?
joshAg 14 minutes ago [-]
the system will kill whatever it damn well pleases. You can tune it with priorities to ask it to try to not kill that, and once it kills your sshd once, you'll probably configure it to exempt sshd from being OOM killed at all. That doesn't fix the memory pressure or hanging or stalling, but you'll at least be able to log into the box still instead of dragging out a serial cable.
dev_hugepages 11 hours ago [-]
This is called an oomkiller. The kernel has one but it kicks in very late and the kernel prefers to do page trashing instead of killing processes.
systemd-oomd should be integrated in systemd, you can configure it to your liking and see if it improves your problem.
SoftTalker 1 hours ago [-]
I wish there was an easy way to configure it to say "target user processes first, specifically java (or these days python)" as in my experience they are always the culprits. Processes owned by system accounts or root should be the last ones killed.
miladyincontrol 2 hours ago [-]
Adding to this, glad systemd-oomd finally added solid rulesets in 261
Made it far easier to target any containers that got too hot rather than ever risk anything higher priority.
> First off, this is exactly how systemd-oomd is supposed to behave under memory pressure. The documentation is specific on this; systemd-oomd itself says:
> > [...] If the configured limits are exceeded, systemd-oomd will select a cgroup to terminate, and send SIGKILL to all processes in it. [...]
> By having the user@.service template be enrolled in systemd-oomd, Fedora made the cgroup that systemd-oomd would select to be killed be all of your processes (across all of your sessions, if you have more than one). ...
Maybe *Fedora* has fixed or improved in the last 4 years. Or maybe they don't run Fedora.
BenjiWiebe 2 hours ago [-]
One thing Fedora does now, is use zram.
In my experience it works really well. I wonder why my computer is a bit sluggish, and find out I have several gigs in zram.
If that was in swap on a disk, it would be really painful.
rwmj 46 minutes ago [-]
I'm a Fedora developer and I can assure you that Fedora's behaviour when it runs out of memory is still terrible.
User-space OOM killers never really worked for me and imo are not a proper solution anyway. This option instead lets you make the kernel OOM killer actually work for desktop use.
Currently have it set to `1000` and it works very well for me (don't remember the last time I had a full system freeze due to OOM).
fc417fc802 11 hours ago [-]
> does anyone know why
In a nutshell, overcommit. It's more or less broken by design but it's also incredibly practical so pretty much everyone does it.
Couple that with the fact that it's difficult bordering on impossible to correctly determine the culprit. If you've got 16 GB RAM and the user launches 3 processes each of which attempts to use 8 GB who should you kill?
binarin 39 minutes ago [-]
With some workarounds, I've put chrome and slack into the same RAM-limited cgroup - no more whole system freezes. From my anecdotal evidence, this also answers the question "who should you kill" :)
wongarsu 11 hours ago [-]
Notably windows doesn't use overcommit, and degrades much more gracefully under memory pressure. The biggest tradeoff is the amount of disk space consumed by a page file that also has to reserve space for unused pages that have been allocated but never been swapped in. On linux you can turn overcommit off, but there's too much software written around the assumption that overcommit is on
jeroenhd 13 minutes ago [-]
Windows also does a neat trick Linux lacks: automatically adding more swap, up to a limit. Systems with loads of RAM barely lose any storage to swap, but once they do get hit, they can get many gigabytes of swap space without user interaction. I believe macOS does it too, of course.
I'm sure there are many reasons why Linux can't do that by default, but it's a real shame.
fc417fc802 3 minutes ago [-]
I assume that's primarily because the stance of most distros would require something like that to be strictly opt-in. I don't know if it's possible to trigger a service based on overall swap usage? But given that the oom killer exists I don't see why it couldn't be trivially repurposed to add swap files on the fly.
fc417fc802 11 hours ago [-]
Is that still the case today? Notably (IIUC) overcommit is required for certain security measures. I believe it was chromium that I noticed mmaping somewhere north of 1 TB of memory on startup so that it can do (again IIUC) something akin to ASLR internally.
wongarsu 8 hours ago [-]
On Windows you can achieve something like manual overcommit by calling VirtualAlloc with just MEM_RESERVE. That gives you a continuous space in your process's virtual address space, without actually backing it with any physical pages. Kind of like what a malloc does on linux
But where linux would automagically back those pages once you use them, Windows requires you to actually ask for those pages to be backed by something (physical memory or page file) by calling VirtualAlloc with MEM_COMMIT on the range you actually want to use
fc417fc802 7 hours ago [-]
At a glance that seems like a much more sensible design. I guess it's dead in the water for posix on account of fork being CoW? This is quite the rabbit hole. I wonder if programming languages ought to be designed in such a way to accommodate a preemptive signal indicating allocation failure in place of a page fault? Rather than malloc returning null or etc.
toast0 2 hours ago [-]
> I guess it's dead in the water for posix on account of fork being CoW?
If that's the only issue, it's avoidable for most use cases. Lots of processes that fork are doing fork/exec to run a helper program. If they know they will do that and that they will be a large process, it's often useful to setup a fork/exec helper in early application startup.
However, there are some applications that use CoW more intentionally. Lock -> fork -> (unlock in parent / persist coherent snapshot in child) is a common pattern; I believe redis uses thst pattern and I've seen it mentioned in discussions about MMO servers. I believe postgres uses fork and CoW for transaction isolation ... but postgres also runs on Windows so there must be another way or I don't understand.
For the persist case, you could imagine some sort of flag to fork to allow overcommit and maybe even to let CoW requests stall in the parent rather than fail... the child is expected to do its work and exit in a limited time.
negrel 11 hours ago [-]
OS are designed to fully exploit available resources, Linux tries its best before triggering an OOM kill.
I recommend using the earlyoom if you want more aggresive oom kill:
The README contains a lot of interesting information.
oomedfornight 11 hours ago [-]
A strange behavior I sometimes run into with earlyoom is that I try to start up some buggy software of mine and it seemingly never starts.
It took a long evening to figure out that it gets earlyoom'd immediately because it tries to allocate too much. Previously the very familiar hitching and freezing was a very easy sign of what kind of issue I was dealing with
BoredomIsFun 11 hours ago [-]
It does not if you switch swapp off and use zram instead. I am typing right now on such a setup wityh 16 GiB ram and it occasionally, once a week or so, kills my firefox due to oom.
If you are you using disk swap - not sure why would if you have a SSD, but I once heard some justification for doing that - then install early OOM.
If you know ahead of time which programs / processes are at risk of unacceptably high memory usage, check out "ulimit".
trompetenaccoun 11 hours ago [-]
This has finally been fixed in the latest Ubuntu version (26), it now force closes the culprit.
jillesvangurp 10 hours ago [-]
Ah nice. I was dealing with that in one of our environments where a security update ended up causing apt to use more memory than usual so the oom killer nuked our elasticsearch process to "free up some memory".
And since that happened on all nodes around the same time, it took out the entire cluster. If you are not familiar, random, uncontrolled node restarts in any kind of multi node database or search product are a great way to trigger outages. So, not great.
I've had quite a few encounters with the oomkiller killing processes that were important and didn't need killing. Or as I like to phrase it "killing the one reason this server exists".
These days the way to size a server is "have enough memory to run whatever you need running + at least half a GB for whatever apt might randomly demand at any point". And guess what, memory tends to be expensive in cloud environments so people tend to get vms with as little as half a GB of ram.
dagenix 3 hours ago [-]
How did they fix it?
elektrontamer 11 hours ago [-]
I always install earlyoom for that reason
pestatije 11 hours ago [-]
not tangential at all...try setting up a swap partition
jenders 11 hours ago [-]
Triggered
pndc 11 hours ago [-]
I had a 1GB Debian VM which started freezing (requiring a hard reboot) after a routine aptitude upgrade to apply security patches. It was indeed caused by low memory, but not out of memory as there was still enough swap space remaining.
The culprit turned out to be the kernel itself, and rolling back to a 6.1 series kernel made the problem go away. I see that Linus's love for vibe-coding is already paying dividends.
williebeek 10 hours ago [-]
Remember when distros used complicated partition setups, one for /root, one for /var one for /home, a swap partition etc. Was always a bad choice because one of them would be at 99% while others would linger below 10%
For swap, the best advice is to disable swap on your desktop, unless <8GB RAM. Really, I've never needed it and you probably won't either.
MayeulC 1 hours ago [-]
I tried that a few years ago. The result was worse than swapping. The kernel really tries its best not to kill anything, and you end up with "thrashing" again, where code pages (shared objects, executables, mmaped files) are evicted and need to be loaded again from disk. In practice the slowdown was worse than swapping to disk, and much more sudden. I'd recommend at least zram to soften the OOM landing.
blablabla123 8 hours ago [-]
On the Desktop fine-grained partitioning is a bit questionable but on a server it's still quite practical. E.g. preventing the database or excessive logs from filling the disk space. The latter is also an attack vector actually.
Also generally swap is recommended on all systems. During normal use it should rarely be needed but if it's needed it prevents spurious crashes. E.g. make -j16 shouldn't result in a hard to comprehend crash because I miss 1 GB of RAM.
spudlyo 48 minutes ago [-]
> For swap, the best advice is to disable swap on your desktop, unless <8GB RAM.
That maybe reasonable advice if you never intend to hibernate your desktop. I like to shut my desktop down when I'm not using it, and it's nice to be able to quickly boot it back to the same state it was in when I went to bed.
tacomagick 10 hours ago [-]
Id always suggest at least 2-4G of swap even if you have plenty of memory 32g+ because for that rare edge case where you'd really not want to OOM.
pyreko 1 hours ago [-]
> you probably won't either
Hibernation?
jenders 10 hours ago [-]
Agreed. Buy more RAM but these days that means taking a HELOC
crabbone 8 hours ago [-]
I'm not sure if that's a bad choice... BTW, I think STIG still requires it. So, if you want to be compliant with some (US) government requirements, you'd still have to put different bits of your system on different partitions / devices.
Why I'm not sure this is a bad idea: a pathological process that quickly captures the entire allocated disk space will be contained by this layout and will likely not affect the system as a whole. Consider, for example, a process that excessively logs into /var/log: once the filesystem mounted at /var fills up, it might fail / hang, but you will still be able to use stuff from /bin or /sbin. But, if your entire root filesystem fills up, you probably won't be able to run any applications at all.
greedo 2 hours ago [-]
There are also controls that specify noexec, nodev etc for filesystems.
mindwok 1 hours ago [-]
Typically I see admins create partitions as basically failure domains - if one fills up it doesn't explode things relying on the others. Wouldn't this be worse in that regard? If you end up accidentally filling up your entire disk, you can no longer swap. Vice versa, if you swap aggressively you could end up filling the disk and then break a running process.
topaz0 58 minutes ago [-]
Swapfiles are fixed size just like swap partitions. They do not grow and shrink arbitrarily as you swap more/less.
cfstras 11 hours ago [-]
swap files make setting up hibernation a bit more complicated - in most cases the file has to be contiguous, e.g. you have to defragment it once after allocating.
Then, you need to tell the bootloader the byte offset of the file on the partition.
MBCook 3 hours ago [-]
Does contiguousness matter these days? Especially since all disk locations are lies anyway as SSD firmware has its own layer of indirection too?
jenders 11 hours ago [-]
If you want to share step-by-step, I’ll update the gist.
cfstras 4 hours ago [-]
That’s my point, it’s different for each distro & partitioning setup, so I wouldn’t know what to share here. Having a swap partition makes it as easy as adding `resume=/dev/sdXYZ` or `resume=UUID=…` to your kernel parameters.
simoncion 10 minutes ago [-]
And using LVM for "partitioning" bypasses a lot of this silliness. Do 'lvcreate --contiguous' and you should be good to go if you're using most any non-toy initrd.
Given that every r/w filesystem you're going to use as your '/' on a Linux desktop or server supports online growing (and most support offline shrinking), it's nuts to me to hear people worrying about getting partition sizes correct. With LVM, if you get it wrong you just resize the LV and the filesystem it contains.
Hell, LVM makes swapping out disks a really easy online operation. [0] If you don't have nice hot-swap drive caddies, you might have to power off the box once to add in the new disks and again to remove the old ones, but all the data transfer can happen while the system is in use.
[0] Assuming the new and old disks have the same logical sector size, which they usually do.
koala_man 1 hours ago [-]
> Swap files have had the same performance characteristics as swap partitions for more than 20 years
I thought so too, then I benchmarked it on a (spinning rust) HDD and found it was significantly slower:
tl;dr: "In this test, using a swap file was surprisingly 50%+ slower than simply allocating a swap partition at the start of the drive, in spite of the low fragmentation and Linux’s bypass of the FS layer."
Aurornis 58 minutes ago [-]
> hen I benchmarked it on a (spinning rust) HDD
Yes, but as your article notes HDDs have different performance characteristics depending on where the sectors are physically located on the platter.
If you could force the swapfile to the same location on the platter it shouldn't perform differently.
Most people are going to be using solid state where this doesn't matter.
cellularmitosis 4 minutes ago [-]
mentioning "swap", "HDD" and "performance" in the same paragraph is very confusing to me.
swap on HDD is like 5 orders of magnitude slower than ram. what's the point of even thinking about performance differences of one scheme over the other? This is like arguing that one version of emergency limp mode is faster than another.
Kim_Bruning 12 hours ago [-]
Every rule has an exception. Beware of this advice on zfs, for instance.
Meanwhile zswap (of no relation with zfs) is free performance. Is it a rule that everything starting with a 'z' must be cool?
chuckadams 1 hours ago [-]
> Is it a rule that everything starting with a 'z' must be cool?
They tend to be the last word on the subject :)
optionalsquid 11 hours ago [-]
It also doesn't sound like a particularly good idea on BTRFS, judging by the linked documentation:
Swapfiles on BTRFS seem to work great if you allocate them early (with the command linked in your article) and put them in a different subvolume than the rest of your system.
BTRFS in general seems more I/O constrained than ext4 or xfs so it's probably still worse to put a swapfile on a BTRFS partition, but so far my systems are doing fine.
fc417fc802 11 hours ago [-]
And since you should probably be running a modern CoW filesystem then you probably shouldn't be using swapfiles ... is the conclusion I arrived at anyway. Either install multiple times more RAM than you need and enable zswap or else set aside a swap partition as has always been done.
jenders 10 hours ago [-]
Truth be told—- I don’t use swap and set `vm.swappiness=0` but in these difficult times, swap may be the hero we need.
jck86 10 hours ago [-]
zswap indeed is the real magic almost silver bullet nowadays. Device backed swap is often not needed anymore and it solves a lot of the issues with freezing under memory pressure, though it may not be ideal for servers.
And if I have a separate otherwise unused ssd then using a swap partition simply is easier than creating a file system for it and mounting it somewhere. Block level backups also are easier.
What would be nice about swap files is if they could dynamically grow and shrink without a special setup.
Way too many variables on Linux for a one size fits all.
jenders 11 hours ago [-]
That’s a good call out. I’ve never used ZFS on root. Been a ZFS user on Linux since the FUSE days but it has never seemed particular well suited (O_DIRECT support came late, ARC/page cache redundancy, etc). I can’t guarantee a consistent enough workload to optimize ZFS for and I don’t feel like it’s general purpose enough. Large object storage though? I’m sitting on a PB of ZFS.
12 hours ago [-]
mdspan 3 days ago [-]
Swap files are also much easier to set up than partitions if you're using full disk encryption.
somat 26 minutes ago [-]
For what it's worth(not much) On OpenBSD swap is always encrypted, With or without full-disk encryption. I don't know why linux always has to make the easy things hard.
demomode 13 hours ago [-]
Swap files are very fragile if you are using hibernation. For FDE I prefer LVM-over-LUKS and a dedicate partition
pasc1878 12 hours ago [-]
MacOS seems to have no problem with hibernating and it and its predecessor NeXT have been using swap files for over 30 years.
killerstorm 11 hours ago [-]
MacOS has no problem with hibernation because hardware and software are designed together: it uses TPM to manage swap encryption. I assume "BIOS" is also optimized to enable fast boot on sleep.
Linux can work with TPM but it's never as smooth. (Unless I guess you make it custom for your specific setup.)
jeroenhd 4 minutes ago [-]
These days with cryptenroll and friends, using the TPM is hardly an issue anymore. LVM and swapfiles both work well if you just create one large "everything but EFI" encrypted volume.
You'd need to keep secure boot on, preferably with your own keys loaded, to get all of the benefits, and that's a bit harder to automate.
Windows does all of this and more. It's harder on Linux because of a lack of tooling/interest/business incentives, and because on average Linux users want more control over their hardware than Windows users.
cedilla 11 hours ago [-]
I have FDE with keys in the TPM. It's pretty smooth bit required a one time additional step in Fedora. It's also probably not very secure, but my threat model is simple theft.
avhception 10 hours ago [-]
What's the benefit of using the TPM compared to a simple passphrase at boot, in that scenario?
theblazehen 2 hours ago [-]
A person wouldn't be able to pull the disk to get at the information
gertop 1 hours ago [-]
Windows has no problem with hibernation and FDE and it uses a swap file.
Windows and Linux run on the exact same hardware. Windows hibernation doesn't rely on secret bios features.
Unreliable hibernation is 100% and undeniably a Linux problem, there is no secret advantage or conspiracy to keep it down. It's just that nobody cares enough to make Linux not suck.
fc417fc802 11 hours ago [-]
How so? It's a one time setup procedure of only a few steps to create a tiny "protective" ext4 (or whatever) partition at the front. Thereafter you configure crypttab to mount the partition via UUID with an offset and key. If you don't need to hibernate you can read the key from /dev/urandom.
holowoodman 12 hours ago [-]
Stop using partitions. Use LVM!
Kim_Bruning 10 hours ago [-]
Stop using LVM. Use ZFS!
(ZFS might be described as the love child of lvm and btrfs, 'cept ZFS got there first)
( Very briefly: you hand ZFS a stack of disks (in mirrors or raidz groups, for redundancy) and it makes a zpool. Then you carve the pool into datasets as needed. Each dataset looks like an ordinary directory and takes only the space its files actually use, so you never guess partition sizes again. And each one can have its own compression, snapshot schedule, quota, encryption, and send/receive backups, etc etc etc. )
jenders 10 hours ago [-]
Not a btrfs user but OpenZFS u til recently didn’t even support O_DIRECT and may still duplicate page cache in arc. It’s not well suited as a general purpose FS where you can’t anticipate the workload, like on a desktop.
p_l 32 minutes ago [-]
O_DIRECT is niche use case (originally called rather unkind words when it got copied from Irix to Linux).
Page cache duplication has been greatly lowered in last decade
zbentley 1 hours ago [-]
Anecdotal, but I've been happily using it under Linux desktops for years, and it works quite well. Workloads include: development, VM hosting, steam gaming, web browsing, multimedia playback. OSes include Debian, Proxmox, vanilla Arch, CachyOS, and others. Daily-driver hardware (ignoring servers and less general-purpose desktop stuff) included 2019 chromeboxes, 2015 (!) laptops, current-gen gaming laptops, and desktop towers with handfuls of spinning rust and solid state drives.
It seems to work well in a variety of situations: 4GB/single-slow-SSD ancient systems work just as well as spinning rust bulk storage pools with NVME ARC/ZIL caches for my gaming/server/database datasets, and all-SATA-SSD pools can get to near-NVME performance with bonus redundancy for boot volumes and latency-critical stuff. For personal desktop use, I haven't found dedup worth the squeeze in RAM costs and tuning (it works, but it's generally easier to solve most dedup-compatible problems at a layer closer to the cause).
ZFSBootMenu and the ability to roll back to snapshots and restore/maintenance disks from outside of the primary operating system, without having to think about fallback boot drives or physical backup volumes, is a godsend in the "try random sketchy commands that might trash my installation in order to get a low-level driver problem resolved" and "I could take the time to understand what this curl | bash invocation does, but I have better things to do; I want to be able to reverse it if it breaks stuff" departments.
In general, I strongly recommend ZFS for daily-driver use. Its core primitives are quite flexible, it makes redundancy/backups/drive addition/replacement easy, and it works fine on old and under-resourced systems; the mythos of "it requires ECC and enterprise-grade hardware and tons of RAM/CPU to work at all" was always bunk. The enterprise/SAN features are there if you want them, but are off by default, and the core FS capabilities are widely useful. Even casual desktop Linux users would do well to set it up, since there are a lot of rare-but-real ordinary user needs that, if they come up and you're not running something like ZFS, can't be done at all unless you connect purpose-specific hard drives or reinstall your OS.
Especially now that NVMEs are so expensive, ZFS should be considered for its ability to make RAIDing up a set of slower drives (or mostly slow drives with an NVME cache) very easy. That way, you can make your existing disks into something that performs well enough that you don't need to spend money on new hardware.
Just don't install it via DKMS; get a distro that ships it compiled into the kernel or as an installable kernel-paired module. Many such distros exist. The DKMS edition won't eat your data, but you'll get real tired of failed system updates because the kernel changed some source and the compile failed. That happens often; turns out that the volume of the kernel API surface used by something as massive as the ZFS codebase is quite large.
Edit: upon reading back through this, I'm a bit sheepish that I sound like such a breathless shill. I promise I'm not in the ~pocket~ zpool of big filesystem. I just like it.
holowoodman 10 hours ago [-]
ZFS is overrated and mostly a hobbyist/enthusiast thing.
Performance of zvol is worse than for LVM logical volumes. Last I've looked, zvol for swap is just asking for trouble, up to and including deadlocks. ZFS filesystems do have weird edge cases and incompatibilities, e.g. around free space reporting, quota and the like. And support with software vendors e.g. for enterprise distributions or backup software is spotty at best.
throw0101d 1 hours ago [-]
> ZFS is overrated and mostly a hobbyist/enthusiast thing.
As someone who originally ran it on Solaris when it came out ~20 years ago, and have continued to run it on FreeBSD and Linux, I would disagree.
jenders 10 hours ago [-]
>> ZFS is overrated and mostly a hobbyist/enthusiast thing.
Not true at all.. but most large object storage redundancy has moved to the application layer. There are still workloads that are a great fit but I agree OpenZFS is currently not well suited as a general purpose FS.
rascul 7 hours ago [-]
I'll consider it when zfs in in tree.
greedo 2 hours ago [-]
LVM for a *nix sysadmin is cheat mode for managing storage.
jenders 11 hours ago [-]
How often are you needing to change partition geometry after install, on a server?
holowoodman 11 hours ago [-]
Depends. Some customers do run a lot of "pets, not cattle" servers, and those can get a few extra hard disks if space runs out. Some security guidelines prescribe separate volumes/filesystems for e.g. /home or /var/log, so log spam doesn't kill the whole system as easily. But of course, this means that sometimes the initial sizes aren't right and you need to extend something. In those cases, which happen a few times per year, not using LVM would be a huge problem.
But even for the "frequently reinstalled cattle"-usecase, LVM has benefits, e.g. snapshots for backups, debugging and forensics.
kiln_ash 6 hours ago [-]
systemd-oomd killing your whole session because Fedora enrolled user@.service is nasty. A swap file won't save you from the OOM killer, just delays it.
lightedman 30 minutes ago [-]
Or be a proper computer user and have a full drive on its own dedicated bus for swap. There is literally no point in time where a full swap drive is less performant or useful than swap directories or swap files (and swap files are a relic of the 90s when a single application would need gobs of virtual memory.)
Funny watching someone advocate going back to something that sucked when we have far more performant hardware today which makes most of the other stuff wholly irrelevant.
dwedge 13 hours ago [-]
Interestingly I always use swapfiles and this page made me realise there can be a performance impact if the filesystem is old - the lkml link says as long as the swap file isn't fragmented there's no impact
WalterGR 13 hours ago [-]
In practice, there’s a potentially tremendous impact if you’re using a spinning metal disk. Sectors at the beginning of the disk (which is at the outer edge) are read/written way faster than sectors at the end. File systems tend to prefer writing files at the beginning. If the file system has been used for ‘a long time’ then it’s either fragmented - or it’s been defragmented and the free space is at the slow end of the drive.
One nice thing about swap partitions is that they can be fixed in place even before OS installation.
(Of course, if you’re using swap enough for this to be a huge factor, you probably have bigger problems. But I haven’t done the math.)
gblargg 12 hours ago [-]
Indeed. I'd always put swap near the beginning, then a small root partition, then the rest. This reduced head movement for often-accessed data, and used the higher bandwidth of the outer tracks.
mcculley 11 hours ago [-]
It would be nice if the VFS had a hook for providing swap pages, making this manual futzing less necessary.
monster_truck 1 hours ago [-]
Yeah, no. I don't care if I have 128gb or 2tb of ram, or what I'm doing with it (but especially in live/realtime scenarios), I'm always going to have at least one nvme dedicated entirely to swap.
nicman23 2 hours ago [-]
i cannot hibernate in a swapfile in zfs
Toynbeeidea 2 hours ago [-]
No.
charcircuit 13 hours ago [-]
Linux distros that do use swap files do so statically. It doesn't handle dynamically growing and shrinking it like other operating systems.
10 hours ago [-]
0xbadcafebee 13 hours ago [-]
Unless you use the Linux package that does manage swap files dynamically.
If we are going full modern, why not create a Systemd unit? ( https://www.freedesktop.org/software/systemd/man/latest/syst... ) if you need more info. If you didn't know this: /etc/fstab is not used directly after the pivot. Systemd parses it, creates units for every record in it, and that's what gets executed in reality. I can't imagine why would you want a swap file before pivot.
jenders 10 hours ago [-]
systemd is malware.
SahAssar 2 hours ago [-]
If you are going to make recommendations and promote them to a wide community (linux users) that is predominantly using something that you do not consider advisable for that recommendation (systemd) then it might be good to mention that.
Systemd is probably on 80%+ linux systems that people interact with, and 95%+ that people manually configure.
waynesonfire 11 hours ago [-]
It's a terrible idea to delegate swap, a capability that is expected to operate under severe memory pressure, to a subsystem that fails under such conditions.
(I’m not interested in being educated on this, I have better things to do. Bring it up with Debian.)
Why not "mkswap --size ... --file ...", which does these three things and more? For instance, according to the mkswap man page, "[...] sets the nocow attribute for newly created files [...]" which is a detail that seems to be missing from this gist.
When that data doesn't compress well at all, zram is no help and traditional swap-to-storage is a better choice.
And there is a middle-ground option, zswap: swapped out pages will eventually end up on a block device, but are compressed first.
I never saw any issues doing this, but I learned that linux ideally wants some swap space, which it will use for some kind of housekeeping stuff.
In the late 90s databases were regularly set up in a way to store their data on raw partitions. There were other types of applications too that required partitions but for databases it was really common.
The practice really only died with OSes allowing apps to bypass the normal filesystem cache.
In Linux this is O_DIRECT and interestingly Linux was super late to the party because Linus hated it with a passion. Now, he was far from the only one, at least Alan Cox and Andrew Morton were skeptical, others as well. Linus was unusually extreme about it and that particular discussion is probably the single source of the majority of quotes people use against him.
O_DIRECT was resisted by Torvalds and others, but it's there today, and used by supposedly important platforms. Nvidia even has an API to DMA data to and from GPUs that uses O_DIRECT. There are folks working on new async WAL O_DIRECT for PostgreSQL as well.
So what can be said? mmap(2) isn't sufficient for certain cases. Torvalds has faced and successfully navigated difficult design choices in a pragmatic manner. What else?
The lesson I take is this: there are levels to implementation complexity and value. The wisdom to decide the appropriate abstraction level is crucial. You are unlikely to be equipped well enough to beneficially utilize O_DIRECT, but it's not impossible and some, in fact, are so equipped.
Swap files are for "oh shit, I need to run this rare thing that needs lots of VM". You create them, "swapon", then "swapoff" and remove when done.
Sprite had (IMO) a really interesting solution to swap. Each host had a directory, referenced by its host ID under the `/swap` directory, and inside that directory, individual segments of virtual memory would be saved: https://github.com/OSPreservProject/sprite/blob/master/src/k...
Sprite overcommited memory like a modern OS, and stored segments individually so that process migration worked (if a segment was swapped-out, the path to the swap file would remain the same anywhere on the cluster, so if the process was migrated, you didn't need to swap-in the segment before migration):
Btrfs has a workaround: it can create a cow-disabled subvolume for the swapfile.
AFAIK you need a separate partition for Zfs.
Is there really no higher-priority kernel process to prevent total freeze of the system and send a SIGKILL to the culprit process when such a scenario happens?
It's because linux is a toy OS. Specifically, it overcommits memory in the hope/assumption that it won't all be used at once, but doesn't have a way to gracefully degrade when applications collectively want to use more memory(+swap) than it actually has. You can turn off overcommit, but applications are designed with the overcommitting feature in mind, so your experience might not be as good as you were hoping for.
Making a massive swap space helps a little bit. It's better to just never let your actual memory usage go above 85% to 90%. It's fine to go above if you're trying to optimize a server with a specific set of processes to wring every last bit of efficiency out of it, but not for general desktop computing.
If it really bothers you OpenBSD doesn't allow overcommit at all and Windows handles this situation much more gracefully, so WSL is an option too. If you don't mind Oracle (i do), solaris also doesn't allow overcommit.
…by default. It can be disabled via a sysctl:
* https://www.kernel.org/doc/Documentation/vm/overcommit-accou...
The problem with turning it off is that the system and applications have been architected assuming that it will be on, so things like fork/execing a memory heavy processes or allocating memory inside a cgroup (which still pretends overcommit is enabled and there's still no way to disable that assumption) that used to work fine might break with no good way to get them to work again. This comment (and siblings) have more specifics: https://news.ycombinator.com/item?id=27794237#27795199
systemd-oomd should be integrated in systemd, you can configure it to your liking and see if it improves your problem.
Made it far easier to target any containers that got too hot rather than ever risk anything higher priority.
Chris Siebenmann discusses when the OOM killer triggers: https://utcc.utoronto.ca/~cks/space/blog/linux/OOMKillerWhen
Chris disables systemd-oomd after it obliterates his X session with no explanation: https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdOomdNo...
> > [...] If the configured limits are exceeded, systemd-oomd will select a cgroup to terminate, and send SIGKILL to all processes in it. [...]
> By having the user@.service template be enrolled in systemd-oomd, Fedora made the cgroup that systemd-oomd would select to be killed be all of your processes (across all of your sessions, if you have more than one). ...
Maybe *Fedora* has fixed or improved in the last 4 years. Or maybe they don't run Fedora.
In my experience it works really well. I wonder why my computer is a bit sluggish, and find out I have several gigs in zram.
If that was in swap on a disk, it would be really painful.
User-space OOM killers never really worked for me and imo are not a proper solution anyway. This option instead lets you make the kernel OOM killer actually work for desktop use.
Currently have it set to `1000` and it works very well for me (don't remember the last time I had a full system freeze due to OOM).
In a nutshell, overcommit. It's more or less broken by design but it's also incredibly practical so pretty much everyone does it.
Couple that with the fact that it's difficult bordering on impossible to correctly determine the culprit. If you've got 16 GB RAM and the user launches 3 processes each of which attempts to use 8 GB who should you kill?
I'm sure there are many reasons why Linux can't do that by default, but it's a real shame.
But where linux would automagically back those pages once you use them, Windows requires you to actually ask for those pages to be backed by something (physical memory or page file) by calling VirtualAlloc with MEM_COMMIT on the range you actually want to use
If that's the only issue, it's avoidable for most use cases. Lots of processes that fork are doing fork/exec to run a helper program. If they know they will do that and that they will be a large process, it's often useful to setup a fork/exec helper in early application startup.
However, there are some applications that use CoW more intentionally. Lock -> fork -> (unlock in parent / persist coherent snapshot in child) is a common pattern; I believe redis uses thst pattern and I've seen it mentioned in discussions about MMO servers. I believe postgres uses fork and CoW for transaction isolation ... but postgres also runs on Windows so there must be another way or I don't understand.
For the persist case, you could imagine some sort of flag to fork to allow overcommit and maybe even to let CoW requests stall in the parent rather than fail... the child is expected to do its work and exit in a limited time.
I recommend using the earlyoom if you want more aggresive oom kill:
https://github.com/rfjakob/earlyoom
The README contains a lot of interesting information.
It took a long evening to figure out that it gets earlyoom'd immediately because it tries to allocate too much. Previously the very familiar hitching and freezing was a very easy sign of what kind of issue I was dealing with
If you are you using disk swap - not sure why would if you have a SSD, but I once heard some justification for doing that - then install early OOM.
Or zswap instead of zram.
https://chrisdown.name/2026/03/24/zswap-vs-zram-when-to-use-...
https://linuxblog.io/zswap-better-than-zram/
If you know ahead of time which programs / processes are at risk of unacceptably high memory usage, check out "ulimit".
And since that happened on all nodes around the same time, it took out the entire cluster. If you are not familiar, random, uncontrolled node restarts in any kind of multi node database or search product are a great way to trigger outages. So, not great.
I've had quite a few encounters with the oomkiller killing processes that were important and didn't need killing. Or as I like to phrase it "killing the one reason this server exists".
These days the way to size a server is "have enough memory to run whatever you need running + at least half a GB for whatever apt might randomly demand at any point". And guess what, memory tends to be expensive in cloud environments so people tend to get vms with as little as half a GB of ram.
The culprit turned out to be the kernel itself, and rolling back to a 6.1 series kernel made the problem go away. I see that Linus's love for vibe-coding is already paying dividends.
For swap, the best advice is to disable swap on your desktop, unless <8GB RAM. Really, I've never needed it and you probably won't either.
Also generally swap is recommended on all systems. During normal use it should rarely be needed but if it's needed it prevents spurious crashes. E.g. make -j16 shouldn't result in a hard to comprehend crash because I miss 1 GB of RAM.
That maybe reasonable advice if you never intend to hibernate your desktop. I like to shut my desktop down when I'm not using it, and it's nice to be able to quickly boot it back to the same state it was in when I went to bed.
Hibernation?
Why I'm not sure this is a bad idea: a pathological process that quickly captures the entire allocated disk space will be contained by this layout and will likely not affect the system as a whole. Consider, for example, a process that excessively logs into /var/log: once the filesystem mounted at /var fills up, it might fail / hang, but you will still be able to use stuff from /bin or /sbin. But, if your entire root filesystem fills up, you probably won't be able to run any applications at all.
Given that every r/w filesystem you're going to use as your '/' on a Linux desktop or server supports online growing (and most support offline shrinking), it's nuts to me to hear people worrying about getting partition sizes correct. With LVM, if you get it wrong you just resize the LV and the filesystem it contains.
Hell, LVM makes swapping out disks a really easy online operation. [0] If you don't have nice hot-swap drive caddies, you might have to power off the box once to add in the new disks and again to remove the old ones, but all the data transfer can happen while the system is in use.
[0] Assuming the new and old disks have the same logical sector size, which they usually do.
I thought so too, then I benchmarked it on a (spinning rust) HDD and found it was significantly slower:
https://www.vidarholen.net/contents/blog/?p=1110
tl;dr: "In this test, using a swap file was surprisingly 50%+ slower than simply allocating a swap partition at the start of the drive, in spite of the low fragmentation and Linux’s bypass of the FS layer."
Yes, but as your article notes HDDs have different performance characteristics depending on where the sectors are physically located on the platter.
If you could force the swapfile to the same location on the platter it shouldn't perform differently.
Most people are going to be using solid state where this doesn't matter.
swap on HDD is like 5 orders of magnitude slower than ram. what's the point of even thinking about performance differences of one scheme over the other? This is like arguing that one version of emergency limp mode is faster than another.
Meanwhile zswap (of no relation with zfs) is free performance. Is it a rule that everything starting with a 'z' must be cool?
They tend to be the last word on the subject :)
https://btrfs.readthedocs.io/en/latest/Swapfile.html
BTRFS in general seems more I/O constrained than ext4 or xfs so it's probably still worse to put a swapfile on a BTRFS partition, but so far my systems are doing fine.
And if I have a separate otherwise unused ssd then using a swap partition simply is easier than creating a file system for it and mounting it somewhere. Block level backups also are easier.
What would be nice about swap files is if they could dynamically grow and shrink without a special setup.
Way too many variables on Linux for a one size fits all.
Linux can work with TPM but it's never as smooth. (Unless I guess you make it custom for your specific setup.)
You'd need to keep secure boot on, preferably with your own keys loaded, to get all of the benefits, and that's a bit harder to automate.
Windows does all of this and more. It's harder on Linux because of a lack of tooling/interest/business incentives, and because on average Linux users want more control over their hardware than Windows users.
Windows and Linux run on the exact same hardware. Windows hibernation doesn't rely on secret bios features.
Unreliable hibernation is 100% and undeniably a Linux problem, there is no secret advantage or conspiracy to keep it down. It's just that nobody cares enough to make Linux not suck.
(ZFS might be described as the love child of lvm and btrfs, 'cept ZFS got there first)
( Very briefly: you hand ZFS a stack of disks (in mirrors or raidz groups, for redundancy) and it makes a zpool. Then you carve the pool into datasets as needed. Each dataset looks like an ordinary directory and takes only the space its files actually use, so you never guess partition sizes again. And each one can have its own compression, snapshot schedule, quota, encryption, and send/receive backups, etc etc etc. )
Page cache duplication has been greatly lowered in last decade
It seems to work well in a variety of situations: 4GB/single-slow-SSD ancient systems work just as well as spinning rust bulk storage pools with NVME ARC/ZIL caches for my gaming/server/database datasets, and all-SATA-SSD pools can get to near-NVME performance with bonus redundancy for boot volumes and latency-critical stuff. For personal desktop use, I haven't found dedup worth the squeeze in RAM costs and tuning (it works, but it's generally easier to solve most dedup-compatible problems at a layer closer to the cause).
ZFSBootMenu and the ability to roll back to snapshots and restore/maintenance disks from outside of the primary operating system, without having to think about fallback boot drives or physical backup volumes, is a godsend in the "try random sketchy commands that might trash my installation in order to get a low-level driver problem resolved" and "I could take the time to understand what this curl | bash invocation does, but I have better things to do; I want to be able to reverse it if it breaks stuff" departments.
In general, I strongly recommend ZFS for daily-driver use. Its core primitives are quite flexible, it makes redundancy/backups/drive addition/replacement easy, and it works fine on old and under-resourced systems; the mythos of "it requires ECC and enterprise-grade hardware and tons of RAM/CPU to work at all" was always bunk. The enterprise/SAN features are there if you want them, but are off by default, and the core FS capabilities are widely useful. Even casual desktop Linux users would do well to set it up, since there are a lot of rare-but-real ordinary user needs that, if they come up and you're not running something like ZFS, can't be done at all unless you connect purpose-specific hard drives or reinstall your OS.
Especially now that NVMEs are so expensive, ZFS should be considered for its ability to make RAIDing up a set of slower drives (or mostly slow drives with an NVME cache) very easy. That way, you can make your existing disks into something that performs well enough that you don't need to spend money on new hardware.
Just don't install it via DKMS; get a distro that ships it compiled into the kernel or as an installable kernel-paired module. Many such distros exist. The DKMS edition won't eat your data, but you'll get real tired of failed system updates because the kernel changed some source and the compile failed. That happens often; turns out that the volume of the kernel API surface used by something as massive as the ZFS codebase is quite large.
Edit: upon reading back through this, I'm a bit sheepish that I sound like such a breathless shill. I promise I'm not in the ~pocket~ zpool of big filesystem. I just like it.
Performance of zvol is worse than for LVM logical volumes. Last I've looked, zvol for swap is just asking for trouble, up to and including deadlocks. ZFS filesystems do have weird edge cases and incompatibilities, e.g. around free space reporting, quota and the like. And support with software vendors e.g. for enterprise distributions or backup software is spotty at best.
As someone who originally ran it on Solaris when it came out ~20 years ago, and have continued to run it on FreeBSD and Linux, I would disagree.
Not true at all.. but most large object storage redundancy has moved to the application layer. There are still workloads that are a great fit but I agree OpenZFS is currently not well suited as a general purpose FS.
But even for the "frequently reinstalled cattle"-usecase, LVM has benefits, e.g. snapshots for backups, debugging and forensics.
Funny watching someone advocate going back to something that sucked when we have far more performant hardware today which makes most of the other stuff wholly irrelevant.
One nice thing about swap partitions is that they can be fixed in place even before OS installation.
(Of course, if you’re using swap enough for this to be a huge factor, you probably have bigger problems. But I haven’t done the math.)
Systemd is probably on 80%+ linux systems that people interact with, and 95%+ that people manually configure.