When ZFS runs out of space

I'm one of the hopefully not so few people anymore who proudly run ZFS on their servers. I even discovered the value of regular snapshots I'm doing with freebsd-snapshot. A good thing to save you from some of your accidental rm commands and evil destructive updates.

But there are also some issues to the whole snapshot story. Once you run out of space on a pool things can get a bit more complicated with all that snapshots around. There are two defining points to understand in such a situation:

• Data in a pool can only be freed if it's not referenced any more by any snapshot, clone or data set
• Snapshots are read-only

Thus, cleaning up space by removing files only helps if they are not referenced by a snapshot. But that's likely not the case if you are making multiple snapshots per day. And removing parts of a snapshot is not possible, you can only destroy the snapshot as a whole. So this comes down to painfully searching snapshots and files you don't need anymore and delete them manually. Alternatively you could convert snapshots into clones, which are again writable, and delete single files from them. But I don't think this is a very clean way, and it would surely break you snapshot cycle, leaving you with a bunch of clones you would have to cleanup later.

So, isn't there any better solution? Well, I currently don't see any with ZFS, maybe you know one ... But I can imagine something that would help a lot in such situations. Writable snapshots, as BTRFS has it on its feature list. And on top of that, how cool would it be to have some snapshot-aware file utilities. I can already imagine the line on my command prompt: rm --snapshot all ~/firefox*.core *flip* and there is the free space again ...