Send/Receive Bookmarks

[Haravikk]

Describe the feature would like to see added to OpenZFS

I would like to see the addition of a flag to zfs send to also send bookmarks, and a corresponding flag for zfs receive to receive/recreate them (otherwise it will ignore them).

This will require zfs send to transfer the bookmark in a way that zfs receive can set the correct createtxg value (since these will differ between sender/receiver) for the new bookmark.

Timing

Depending upon how easily a bookmark can be recreated on the receiving side, it may not be possible to send all bookmarks in an incremental send stream – this is because a bookmark could be created against an earlier snapshot that no longer exists. In such cases it's probably best to just issue a warning and continue, as there's nothing that can be done about it.

How will this feature improve OpenZFS?

Bookmarks can be extremely useful in managing ZFS without the overhead of snapshots, but currently they are not transferred even in a full replication stream, so it isn't possible to fully clone an existing pool with bookmarks without also running a script to recreate them (which may not even be possible if the original snapshots are gone).

[amotin]

The whole point of bookmark is to point the dataset state for purposes of replication without holding actual data as done by snapshots. So bookmarks still coexisting with snapshots should be trivial to replicate, but I am not sure they are really useful in that case, since the expected workflow includes (possible) snapshot deletion to free the space after creating bookmark. If the snapshot was deleted though, replication of such bookmark would probably means sending first all the changes before the bookmark, then making receiving side to commit txg to create the bookmark at that state and then send changes after the bookmark. Seems like it should be possible, just needs some work.

[Haravikk]

Yeah, would it perhaps make sense to split into separate stages?

First send bookmarks that have corresponding snapshots, since this is the simplest case – in this event "orphan" bookmarks (no snapshot) would be skipped with a warning. Then handling of orphan bookmarks could be added later?

[amotin]

It would be unpredictable and confusing to users. And proper implementation would cover it also.