14753 convert zpool-features.7 to use mdoc

Reviewed by: Andy Fiddaman <andy@omnios.org>
Approved by: Richard Lowe <richlowe@richlowe.net>

1 files changed, 630 insertions, 790 deletions

diff --git a/usr/src/man/man7/zpool-features.7 b/usr/src/man/man7/zpool-features.7
index 146bfc5262..aabc9bc4ad 100644
--- a/usr/src/man/man7/zpool-features.7
+++ b/usr/src/man/man7/zpool-features.7
@@ -1,4 +1,4 @@
-'\" te
+.\"
 .\" Copyright (c) 2013, 2017 by Delphix. All rights reserved.
 .\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
 .\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
@@ -15,821 +15,661 @@
 .\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your
 .\" own identifying information:
 .\" Portions Copyright [yyyy] [name of copyright owner]
-.TH ZPOOL-FEATURES 7 "May 15, 2019"
-.SH NAME
-zpool\-features \- ZFS pool feature descriptions
-.SH DESCRIPTION
-.LP
-ZFS pool on\-disk format versions are specified via "features" which replace
-the old on\-disk format numbers (the last supported on\-disk format number is
-28). To enable a feature on a pool use the \fBupgrade\fR subcommand of the
-\fBzpool\fR(8) command, or set the \fBfeature@\fR\fIfeature_name\fR property
-to \fBenabled\fR.
-.sp
-.LP
+.\" Copyright (c) 2019, Klara Inc.
+.\" Copyright (c) 2019, Allan Jude
+.Dd June 21, 2022
+.Dt ZPOOL-FEATURES 7
+.Os
+.
+.Sh NAME
+.Nm zpool-features
+.Nd description of ZFS pool features
+.
+.Sh DESCRIPTION
+ZFS pool on-disk format versions are specified via
+.Dq features
+which replace the old on-disk format numbers
+.Pq the last supported on-disk format number is 28 .
+To enable a feature on a pool use the
+.Nm zpool Cm upgrade ,
+or set the
+.Sy feature Ns @ Ns Ar feature-name
+property to
+.Sy enabled .
+.Pp
 The pool format does not affect file system version compatibility or the ability
 to send file systems between pools.
-.sp
-.LP
-Since most features can be enabled independently of each other the on\-disk
+.Pp
+Since most features can be enabled independently of each other, the on-disk
 format of the pool is specified by the set of all features marked as
-\fBactive\fR on the pool. If the pool was created by another software version
+.Sy active
+on the pool.
+If the pool was created by another software version
 this set may include unsupported features.
-.SS "Identifying features"
-.LP
-Every feature has a guid of the form \fIcom.example:feature_name\fR. The reverse
-DNS name ensures that the feature's guid is unique across all ZFS
-implementations. When unsupported features are encountered on a pool they will
-be identified by their guids. Refer to the documentation for the ZFS
+.
+.Ss Identifying features
+Every feature has a GUID of the form
+.Ar com.example : Ns Ar feature-name .
+The reversed DNS name ensures that the feature's GUID is unique across all ZFS
+implementations.
+When unsupported features are encountered on a pool they will
+be identified by their GUIDs.
+Refer to the documentation for the ZFS
 implementation that created the pool for information about those features.
-.sp
-.LP
-Each supported feature also has a short name. By convention a feature's short
-name is the portion of its guid which follows the ':' (e.g.
-\fIcom.example:feature_name\fR would have the short name \fIfeature_name\fR),
+.Pp
+Each supported feature also has a short name.
+By convention a feature's short name is the portion of its GUID which follows the
+.Sq \&:
+.Po
+i.e.
+.Ar com.example : Ns Ar feature-name
+would have the short name
+.Ar feature-name
+.Pc ,
 however a feature's short name may differ across ZFS implementations if
 following the convention would result in name conflicts.
-.SS "Feature states"
-.LP
+.
+.Ss Feature states
 Features can be in one of three states:
-.sp
-.ne 2
-.na
-\fB\fBactive\fR\fR
-.ad
-.RS 12n
-This feature's on\-disk format changes are in effect on the pool. Support for
-this feature is required to import the pool in read\-write mode. If this
-feature is not read-only compatible, support is also required to import the pool
-in read\-only mode (see "Read\-only compatibility").
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBenabled\fR\fR
-.ad
-.RS 12n
+.Bl -tag -width "disabled"
+.It Sy active
+This feature's on-disk format changes are in effect on the pool.
+Support for this feature is required to import the pool in read-write mode.
+If this feature is not read-only compatible,
+support is also required to import the pool in read-only mode
+.Pq see Sx Read-only compatibility .
+.It Sy enabled
 An administrator has marked this feature as enabled on the pool, but the
-feature's on\-disk format changes have not been made yet. The pool can still be
-imported by software that does not support this feature, but changes may be made
-to the on\-disk format at any time which will move the feature to the
-\fBactive\fR state. Some features may support returning to the \fBenabled\fR
-state after becoming \fBactive\fR. See feature\-specific documentation for
-details.
-.RE
-
-.sp
-.ne 2
-.na
-\fBdisabled\fR
-.ad
-.RS 12n
-This feature's on\-disk format changes have not been made and will not be made
-unless an administrator moves the feature to the \fBenabled\fR state. Features
-cannot be disabled once they have been enabled.
-.RE
-
-.sp
-.LP
+feature's on-disk format changes have not been made yet.
+The pool can still be imported by software that does not support this feature,
+but changes may be made to the on-disk format at any time
+which will move the feature to the
+.Sy active
+state.
+Some features may support returning to the
+.Sy enabled
+state after becoming
+.Sy active .
+See feature-specific documentation for details.
+.It Sy disabled
+This feature's on-disk format changes have not been made and will not be made
+unless an administrator moves the feature to the
+.Sy enabled
+state.
+Features cannot be disabled once they have been enabled.
+.El
+.Pp
 The state of supported features is exposed through pool properties of the form
-\fIfeature@short_name\fR.
-.SS "Read\-only compatibility"
-.LP
-Some features may make on\-disk format changes that do not interfere with other
-software's ability to read from the pool. These features are referred to as
-"read\-only compatible". If all unsupported features on a pool are read\-only
-compatible, the pool can be imported in read\-only mode by setting the
-\fBreadonly\fR property during import (see \fBzpool\fR(8) for details on
-importing pools).
-.SS "Unsupported features"
-.LP
-For each unsupported feature enabled on an imported pool a pool property
-named \fIunsupported@feature_guid\fR will indicate why the import was allowed
-despite the unsupported feature. Possible values for this property are:
-
-.sp
-.ne 2
-.na
-\fB\fBinactive\fR\fR
-.ad
-.RS 12n
-The feature is in the \fBenabled\fR state and therefore the pool's on\-disk
+.Sy feature Ns @ Ns Ar short-name .
+.
+.Ss Read-only compatibility
+Some features may make on-disk format changes that do not interfere with other
+software's ability to read from the pool.
+These features are referred to as
+.Dq read-only compatible .
+If all unsupported features on a pool are read-only compatible,
+the pool can be imported in read-only mode by setting the
+.Sy readonly
+property during import
+.Po see
+.Xr zpool 8
+for details on importing pools
+.Pc .
+.
+.Ss Unsupported features
+For each unsupported feature enabled on an imported pool, a pool property
+named
+.Sy unsupported Ns @ Ns Ar feature-name
+will indicate why the import was allowed despite the unsupported feature.
+Possible values for this property are:
+.Bl -tag -width "readonly"
+.It Sy inactive
+The feature is in the
+.Sy enabled
+state and therefore the pool's on-disk
 format is still compatible with software that does not support this feature.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBreadonly\fR\fR
-.ad
-.RS 12n
-The feature is read\-only compatible and the pool has been imported in
-read\-only mode.
-.RE
-
-.SS "Feature dependencies"
-.LP
-Some features depend on other features being enabled in order to function
-properly. Enabling a feature will automatically enable any features it
-depends on.
-.SH FEATURES
-.LP
+.It Sy readonly
+The feature is read-only compatible and the pool has been imported in
+read-only mode.
+.El
+.
+.Ss Feature dependencies
+Some features depend on other features being enabled in order to function.
+Enabling a feature will automatically enable any features it depends on.
+.
+.de feature
+.It Sy \\$2
+.Bl -tag -compact -width "READ-ONLY COMPATIBLE"
+.It GUID
+.Sy \\$1:\\$2
+.if !"\\$4"" \{\
+.It DEPENDENCIES
+\fB\\$4\fP\c
+.if !"\\$5"" , \fB\\$5\fP\c
+.if !"\\$6"" , \fB\\$6\fP\c
+.if !"\\$7"" , \fB\\$7\fP\c
+.if !"\\$8"" , \fB\\$8\fP\c
+.if !"\\$9"" , \fB\\$9\fP\c
+.\}
+.It READ-ONLY COMPATIBLE
+\\$3
+.El
+.Pp
+..
+.
+.ds instant-never \
+.No This feature becomes Sy active No as soon as it is enabled \
+and will never return to being Sy enabled .
+.
+.ds remount-upgrade \
+.No Each filesystem will be upgraded automatically when remounted, \
+or when a new file is created under that filesystem. \
+The upgrade can also be triggered on filesystems via \
+Nm zfs Cm set Sy version Ns = Ns Sy current Ar fs . \
+No The upgrade process runs in the background and may take a while to complete \
+for filesystems containing large amounts of files.
+.
+.de checksum-spiel
+When the
+.Sy \\$1
+feature is set to
+.Sy enabled ,
+the administrator can turn on the
+.Sy \\$1
+checksum on any dataset using
+.Nm zfs Cm set Sy checksum Ns = Ns Sy \\$1 Ar dset
+.Po see Xr zfs 8 Pc .
+This feature becomes
+.Sy active
+once a
+.Sy checksum
+property has been set to
+.Sy \\$1 ,
+and will return to being
+.Sy enabled
+once all filesystems that have ever had their checksum set to
+.Sy \\$1
+are destroyed.
+..
+.
+.Sh FEATURES
 The following features are supported on this system:
-.sp
-.ne 2
-.na
-\fB\fBasync_destroy\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:async_destroy
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	none
-.TE
-
+.Bl -tag -width Ds
+.feature org.zfsonlinux allocation_classes yes
+This feature enables support for separate allocation classes.
+.Pp
+This feature becomes
+.Sy active
+when a dedicated allocation class vdev
+.Pq dedup or special
+is created with the
+.Nm zpool Cm create No or Nm zpool Cm add No commands .
+With device removal, it can be returned to the
+.Sy enabled
+state if all the dedicated allocation class vdevs are removed.
+.
+.feature com.delphix async_destroy yes
 Destroying a file system requires traversing all of its data in order to
-return its used space to the pool. Without \fBasync_destroy\fR the file system
-is not fully removed until all space has been reclaimed. If the destroy
-operation is interrupted by a reboot or power outage the next attempt to open
-the pool will need to complete the destroy operation synchronously.
-
-When \fBasync_destroy\fR is enabled the file system's data will be reclaimed
-by a background process, allowing the destroy operation to complete without
-traversing the entire file system. The background process is able to resume
+return its used space to the pool.
+Without
+.Sy async_destroy ,
+the file system is not fully removed until all space has been reclaimed.
+If the destroy operation is interrupted by a reboot or power outage,
+the next attempt to open the pool will need to complete the destroy
+operation synchronously.
+.Pp
+When
+.Sy async_destroy
+is enabled, the file system's data will be reclaimed by a background process,
+allowing the destroy operation to complete
+without traversing the entire file system.
+The background process is able to resume
 interrupted destroys after the pool has been opened, eliminating the need
-to finish interrupted destroys as part of the open operation. The amount
-of space remaining to be reclaimed by the background process is available
-through the \fBfreeing\fR property.
-
-This feature is only \fBactive\fR while \fBfreeing\fR is non\-zero.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBempty_bpobj\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:empty_bpobj
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	none
-.TE
-
+to finish interrupted destroys as part of the open operation.
+The amount of space remaining to be reclaimed by the background process
+is available through the
+.Sy freeing
+property.
+.Pp
+This feature is only
+.Sy active
+while
+.Sy freeing
+is non-zero.
+.
+.feature com.delphix bookmarks yes extensible_dataset
+This feature enables use of the
+.Nm zfs Cm bookmark
+command.
+.Pp
+This feature is
+.Sy active
+while any bookmarks exist in the pool.
+All bookmarks in the pool can be listed by running
+.Nm zfs Cm list Fl t Sy bookmark Fl r Ar poolname .
+.
+.feature com.datto bookmark_v2 no bookmark extensible_dataset
+This feature enables the creation and management of larger bookmarks which are
+needed for other features in ZFS.
+.Pp
+This feature becomes
+.Sy active
+when a v2 bookmark is created and will be returned to the
+.Sy enabled
+state when all v2 bookmarks are destroyed.
+.
+.feature com.delphix device_removal no
+This feature enables the
+.Nm zpool Cm remove
+command to remove top-level vdevs,
+evacuating them to reduce the total size of the pool.
+.Pp
+This feature becomes
+.Sy active
+when the
+.Nm zpool Cm remove
+command is used
+on a top-level vdev, and will never return to being
+.Sy enabled .
+.
+.feature org.illumos edonr no extensible_dataset
+This feature enables the use of the Edon-R hash algorithm for checksum,
+including for nopwrite
+.Po if compression is also enabled, an overwrite of
+a block whose checksum matches the data being written will be ignored
+.Pc .
+In an abundance of caution, Edon-R requires verification when used with
+dedup:
+.Nm zfs Cm set Sy dedup Ns = Ns Sy edonr , Ns Sy verify
+.Po see Xr zfs 8 Pc .
+.Pp
+Edon-R is a very high-performance hash algorithm that was part
+of the NIST SHA-3 competition.
+It provides extremely high hash performance
+.Pq over 350% faster than SHA-256 ,
+but was not selected because of its unsuitability
+as a general purpose secure hash algorithm.
+This implementation utilizes the new salted checksumming functionality
+in ZFS, which means that the checksum is pre-seeded with a secret
+256-bit random key
+.Pq stored on the pool
+before being fed the data block to be checksummed.
+Thus the produced checksums are unique to a given pool,
+preventing hash collision attacks on systems with dedup.
+.Pp
+.checksum-spiel edonr
+.
+.feature com.delphix embedded_data no
+This feature improves the performance and compression ratio of
+highly-compressible blocks.
+Blocks whose contents can compress to 112 bytes
+or smaller can take advantage of this feature.
+.Pp
+When this feature is enabled, the contents of highly-compressible blocks are
+stored in the block
+.Dq pointer
+itself
+.Po a misnomer in this case, as it contains
+the compressed data, rather than a pointer to its location on disk
+.Pc .
+Thus the space of the block
+.Pq one sector, typically 512 B or 4 KiB
+is saved, and no additional I/O is needed to read and write the data block.
+.
+\*[instant-never]
+.
+.feature com.delphix empty_bpobj yes
 This feature increases the performance of creating and using a large
 number of snapshots of a single filesystem or volume, and also reduces
 the disk space required.
-
+.Pp
 When there are many snapshots, each snapshot uses many Block Pointer
-Objects (bpobj's) to track blocks associated with that snapshot.
-However, in common use cases, most of these bpobj's are empty.  This
-feature allows us to create each bpobj on-demand, thus eliminating the
-empty bpobjs.
-
-This feature is \fBactive\fR while there are any filesystems, volumes,
+Objects
+.Pq bpobjs
+to track blocks associated with that snapshot.
+However, in common use cases, most of these bpobjs are empty.
+This feature allows us to create each bpobj on-demand,
+thus eliminating the empty bpobjs.
+.Pp
+This feature is
+.Sy active
+while there are any filesystems, volumes,
 or snapshots which were created after enabling this feature.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBfilesystem_limits\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.joyent:filesystem_limits
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	extensible_dataset
-.TE
-
-This feature enables filesystem and snapshot limits. These limits can be used
-to control how many filesystems and/or snapshots can be created at the point in
-the tree on which the limits are set.
-
-This feature is \fBactive\fR once either of the limit properties has been
-set on a dataset. Once activated the feature is never deactivated.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBlz4_compress\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	org.illumos:lz4_compress
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	none
-.TE
-
-\fBlz4\fR is a high-performance real-time compression algorithm that
-features significantly faster compression and decompression as well as a
-higher compression ratio than the older \fBlzjb\fR compression.
-Typically, \fBlz4\fR compression is approximately 50% faster on
-compressible data and 200% faster on incompressible data than
-\fBlzjb\fR. It is also approximately 80% faster on decompression, while
-giving approximately 10% better compression ratio.
-
-When the \fBlz4_compress\fR feature is set to \fBenabled\fR, the
-administrator can turn on \fBlz4\fR compression on any dataset on the
-pool using the \fBzfs\fR(8) command. Also, all newly written metadata
-will be compressed with \fBlz4\fR algorithm. Since this feature is not
-read-only compatible, this operation will render the pool unimportable
-on systems without support for the \fBlz4_compress\fR feature. Booting
-off of \fBlz4\fR-compressed root pools is supported.
-
-This feature becomes \fBactive\fR as soon as it is enabled and will
-never return to being \fBenabled\fR.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBspacemap_histogram\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:spacemap_histogram
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	none
-.TE
-
-This features allows ZFS to maintain more information about how free space
-is organized within the pool. If this feature is \fBenabled\fR, ZFS will
-set this feature to \fBactive\fR when a new space map object is created or
-an existing space map is upgraded to the new format. Once the feature is
-\fBactive\fR, it will remain in that state until the pool is destroyed.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBmulti_vdev_crash_dump\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.joyent:multi_vdev_crash_dump
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	none
-.TE
-
-This feature allows a dump device to be configured with a pool comprised
-of multiple vdevs.  Those vdevs may be arranged in any mirrored or raidz
-configuration.
-
-When the \fBmulti_vdev_crash_dump\fR feature is set to \fBenabled\fR,
-the administrator can use the \fBdumpadm\fR(8) command to configure a
-dump device on a pool comprised of multiple vdevs.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBextensible_dataset\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:extensible_dataset
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	none
-.TE
-
+.
+.feature com.delphix enabled_txg yes
+Once this feature is enabled, ZFS records the transaction group number
+in which new features are enabled.
+This has no user-visible impact, but other features may depend on this feature.
+.Pp
+This feature becomes
+.Sy active
+as soon as it is enabled and will never return to being
+.Sy enabled .
+.
+.feature com.datto encryption no bookmark_v2 extensible_dataset
+This feature enables the creation and management of natively encrypted datasets.
+.Pp
+This feature becomes
+.Sy active
+when an encrypted dataset is created and will be returned to the
+.Sy enabled
+state when all datasets that use this feature are destroyed.
+.
+.feature com.delphix extensible_dataset no
 This feature allows more flexible use of internal ZFS data structures,
 and exists for other features to depend on.
-
-This feature will be \fBactive\fR when the first dependent feature uses it,
-and will be returned to the \fBenabled\fR state when all datasets that use
-this feature are destroyed.
-
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBbookmarks\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:bookmarks
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	extensible_dataset
-.TE
-
-This feature enables use of the \fBzfs bookmark\fR subcommand.
-
-This feature is \fBactive\fR while any bookmarks exist in the pool.
-All bookmarks in the pool can be listed by running
-\fBzfs list -t bookmark -r \fIpoolname\fR\fR.
-
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBenabled_txg\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:enabled_txg
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	none
-.TE
-
-Once this feature is enabled ZFS records the transaction group number
-in which new features are enabled. This has no user-visible impact,
-but other features may depend on this feature.
-
-This feature becomes \fBactive\fR as soon as it is enabled and will
-never return to being \fBenabled\fR.
-
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBhole_birth\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:hole_birth
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	enabled_txg
-.TE
-
-This feature improves performance of incremental sends ("zfs send -i")
-and receives for objects with many holes. The most common case of
-hole-filled objects is zvols.
-
-An incremental send stream from snapshot \fBA\fR to snapshot \fBB\fR
-contains information about every block that changed between \fBA\fR and
-\fBB\fR. Blocks which did not change between those snapshots can be
+.Pp
+This feature will be
+.Sy active
+when the first dependent feature uses it, and will be returned to the
+.Sy enabled
+state when all datasets that use this feature are destroyed.
+.
+.feature com.joyent filesystem_limits yes extensible_dataset
+This feature enables filesystem and snapshot limits.
+These limits can be used to control how many filesystems and/or snapshots
+can be created at the point in the tree on which the limits are set.
+.Pp
+This feature is
+.Sy active
+once either of the limit properties has been set on a dataset
+and will never return to being
+.Sy enabled .
+.
+.feature com.delphix hole_birth no enabled_txg
+This feature has/had bugs, the result of which is that, if you do a
+.Nm zfs Cm send Fl i
+.Pq or Fl R , No since it uses Fl i
+from an affected dataset, the receiving party will not see any checksum
+or other errors, but the resulting destination snapshot
+will not match the source.
+Its use by
+.Nm zfs Cm send Fl i
+has been disabled by default
+.Po
+see
+.Sy send_holes_without_birth_time
+in
+.Xr zfs 4
+.Pc .
+.Pp
+This feature improves performance of incremental sends
+.Pq Nm zfs Cm send Fl i
+and receives for objects with many holes.
+The most common case of hole-filled objects is zvols.
+.Pp
+An incremental send stream from snapshot
+.Sy A No to snapshot Sy B
+contains information about every block that changed between
+.Sy A No and Sy B .
+Blocks which did not change between those snapshots can be
 identified and omitted from the stream using a piece of metadata called
-the 'block birth time', but birth times are not recorded for holes (blocks
-filled only with zeroes). Since holes created after \fBA\fR cannot be
-distinguished from holes created before \fBA\fR, information about every
-hole in the entire filesystem or zvol is included in the send stream.
-
-For workloads where holes are rare this is not a problem. However, when
-incrementally replicating filesystems or zvols with many holes (for
-example a zvol formatted with another filesystem) a lot of time will
-be spent sending and receiving unnecessary information about holes that
+the
+.Dq block birth time ,
+but birth times are not recorded for holes
+.Pq blocks filled only with zeroes .
+Since holes created after
+.Sy A No cannot be distinguished from holes created before Sy A ,
+information about every hole in the entire filesystem or zvol
+is included in the send stream.
+.Pp
+For workloads where holes are rare this is not a problem.
+However, when incrementally replicating filesystems or zvols with many holes
+.Pq for example a zvol formatted with another filesystem
+a lot of time will be spent sending and receiving unnecessary information
+about holes that already exist on the receiving side.
+.Pp
+Once the
+.Sy hole_birth
+feature has been enabled the block birth times
+of all new holes will be recorded.
+Incremental sends between snapshots created after this feature is enabled
+will use this new metadata to avoid sending information about holes that
 already exist on the receiving side.
-
-Once the \fBhole_birth\fR feature has been enabled the block birth times
-of all new holes will be recorded. Incremental sends between snapshots
-created after this feature is enabled will use this new metadata to avoid
-sending information about holes that already exist on the receiving side.
-
-This feature becomes \fBactive\fR as soon as it is enabled and will
-never return to being \fBenabled\fR.
-
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBembedded_data\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:embedded_data
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	none
-.TE
-
-This feature improves the performance and compression ratio of
-highly-compressible blocks.  Blocks whose contents can compress to 112 bytes
-or smaller can take advantage of this feature.
-
-When this feature is enabled, the contents of highly-compressible blocks are
-stored in the block "pointer" itself (a misnomer in this case, as it contains
-the compresseed data, rather than a pointer to its location on disk).  Thus
-the space of the block (one sector, typically 512 bytes or 4KB) is saved,
-and no additional i/o is needed to read and write the data block.
-
-This feature becomes \fBactive\fR as soon as it is enabled and will
-never return to being \fBenabled\fR.
-
-.RE
-.sp
-.ne 2
-.na
-\fB\fBdevice_removal\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:device_removal
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	none
-.TE
-
-This feature enables the "zpool remove" subcommand to remove top-level
-vdevs, evacuating them to reduce the total size of the pool.
-
-This feature becomes \fBactive\fR when the "zpool remove" command is used
-on a top-level vdev, and will never return to being \fBenabled\fR.
-
-.RE
-.sp
-.ne 2
-.na
-\fB\fBobsolete_counts\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:obsolete_counts
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	device_removal
-.TE
-
-This feature is an enhancement of device_removal, which will over time
-reduce the memory used to track removed devices.  When indirect blocks
-are freed or remapped, we note that their part of the indirect mapping
-is "obsolete", i.e. no longer needed.  See also the \fBzfs remap\fR
-subcommand in \fBzfs\fR(8).
-
-This feature becomes \fBactive\fR when the "zpool remove" command is
-used on a top-level vdev, and will never return to being \fBenabled\fR.
-
-.RE
-.sp
-.ne 2
-.na
-\fB\fBzpool_checkpoint\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:zpool_checkpoint
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	none
-.TE
-
-This feature enables the "zpool checkpoint" subcommand that can
-checkpoint the state of the pool at the time it was issued and later
-rewind back to it or discard it.
-
-This feature becomes \fBactive\fR when the "zpool checkpoint" command
-is used to checkpoint the pool.
-The feature will only return back to being \fBenabled\fR when the pool
-is rewound or the checkpoint has been discarded.
-
-.RE
-.sp
-.ne 2
-.na
-\fB\fBspacemap_v2\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:spacemap_v2
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	none
-.TE
-
+.Pp
+\*[instant-never]
+.
+.feature org.open-zfs large_blocks no extensible_dataset
+This feature allows the record size on a dataset to be set larger than 128 KiB.
+.Pp
+This feature becomes
+.Sy active
+once a dataset contains a file with a block size larger than 128 KiB,
+and will return to being
+.Sy enabled
+once all filesystems that have ever had their recordsize larger than 128 KiB
+are destroyed.
+.
+.feature org.zfsonlinux large_dnode no extensible_dataset
+This feature allows the size of dnodes in a dataset to be set larger than 512 B.
+.
+This feature becomes
+.Sy active
+once a dataset contains an object with a dnode larger than 512 B,
+which occurs as a result of setting the
+.Sy dnodesize
+dataset property to a value other than
+.Sy legacy .
+The feature will return to being
+.Sy enabled
+once all filesystems that have ever contained a dnode larger than 512 B
+are destroyed.
+Large dnodes allow more data to be stored in the bonus buffer,
+thus potentially improving performance by avoiding the use of spill blocks.
+.
+.feature com.delphix log_spacemap yes com.delphix:spacemap_v2
+This feature improves performance for heavily-fragmented pools,
+especially when workloads are heavy in random-writes.
+It does so by logging all the metaslab changes on a single spacemap every TXG
+instead of scattering multiple writes to all the metaslab spacemaps.
+.Pp
+\*[instant-never]
+.
+.feature org.illumos lz4_compress no
+.Sy lz4
+is a high-performance real-time compression algorithm that
+features significantly faster compression and decompression as well as a
+higher compression ratio than the older
+.Sy lzjb
+compression.
+Typically,
+.Sy lz4
+compression is approximately 50% faster on compressible data and 200% faster
+on incompressible data than
+.Sy lzjb .
+It is also approximately 80% faster on decompression,
+while giving approximately a 10% better compression ratio.
+.Pp
+When the
+.Sy lz4_compress
+feature is set to
+.Sy enabled ,
+the administrator can turn on
+.Sy lz4
+compression on any dataset on the pool using the
+.Xr zfs 8
+command.
+All newly written metadata will be compressed with the
+.Sy lz4
+algorithm.
+.Pp
+\*[instant-never]
+.
+.feature com.joyent multi_vdev_crash_dump no
+This feature allows a dump device to be configured with a pool comprised
+of multiple vdevs.
+Those vdevs may be arranged in any mirrored or raidz configuration.
+.Pp
+When the
+.Sy multi_vdev_crash_dump
+feature is set to
+.Sy enabled ,
+the administrator can use
+.Xr dumpadm 8
+to configure a dump device on a pool comprised of multiple vdevs.
+.Pp
+Under
+.Fx
+and Linux this feature is unused, but registered for compatibility.
+New pools created on these systems will have the feature
+.Sy enabled
+but will never transition to
+.Sy active ,
+as this functionality is not required for crash dump support.
+Existing pools where this feature is
+.Sy active
+can be imported.
+.
+.feature com.delphix obsolete_counts yes device_removal
+This feature is an enhancement of
+.Sy device_removal ,
+which will over time reduce the memory used to track removed devices.
+When indirect blocks are freed or remapped,
+we note that their part of the indirect mapping is
+.Dq obsolete
+– no longer needed.
+.Pp
+This feature becomes
+.Sy active
+when the
+.Nm zpool Cm remove
+command is used on a top-level vdev, and will never return to being
+.Sy enabled .
+.
+.feature org.zfsonlinux project_quota yes extensible_dataset
+This feature allows administrators to account the spaces and objects usage
+information against the project identifier
+.Pq ID .
+.Pp
+The project ID is an object-based attribute.
+When upgrading an existing filesystem,
+objects without a project ID will be assigned a zero project ID.
+When this feature is enabled, newly created objects inherit
+their parent directories' project ID if the parent's inherit flag is set
+.Pq via Nm chattr Sy [+-]P No or Nm zfs Cm project Fl s Ns | Ns Fl C .
+Otherwise, the new object's project ID will be zero.
+An object's project ID can be changed at any time by the owner
+.Pq or privileged user
+via
+.Nm chattr Fl p Ar prjid
+or
+.Nm zfs Cm project Fl p Ar prjid .
+.Pp
+This feature will become
+.Sy active
+as soon as it is enabled and will never return to being
+.Sy disabled .
+\*[remount-upgrade]
+.
+.feature com.datto resilver_defer yes
+This feature allows ZFS to postpone new resilvers if an existing one is already
+in progress.
+Without this feature, any new resilvers will cause the currently
+running one to be immediately restarted from the beginning.
+.Pp
+This feature becomes
+.Sy active
+once a resilver has been deferred, and returns to being
+.Sy enabled
+when the deferred resilver begins.
+.
+.feature org.illumos sha512 no extensible_dataset
+This feature enables the use of the SHA-512/256 truncated hash algorithm
+.Pq FIPS 180-4
+for checksum and dedup.
+The native 64-bit arithmetic of SHA-512 provides an approximate 50%
+performance boost over SHA-256 on 64-bit hardware
+and is thus a good minimum-change replacement candidate
+for systems where hash performance is important,
+but these systems cannot for whatever reason utilize the faster
+.Sy skein No and Sy edonr
+algorithms.
+.Pp
+.checksum-spiel sha512
+.
+.feature org.illumos skein no extensible_dataset
+This feature enables the use of the Skein hash algorithm for checksum and dedup.
+Skein is a high-performance secure hash algorithm that was a
+finalist in the NIST SHA-3 competition.
+It provides a very high security margin and high performance on 64-bit hardware
+.Pq 80% faster than SHA-256 .
+This implementation also utilizes the new salted checksumming
+functionality in ZFS, which means that the checksum is pre-seeded with a
+secret 256-bit random key
+.Pq stored on the pool
+before being fed the data block to be checksummed.
+Thus the produced checksums are unique to a given pool,
+preventing hash collision attacks on systems with dedup.
+.Pp
+.checksum-spiel skein
+.
+.feature com.delphix spacemap_histogram yes
+This features allows ZFS to maintain more information about how free space
+is organized within the pool.
+If this feature is
+.Sy enabled ,
+it will be activated when a new space map object is created, or
+an existing space map is upgraded to the new format,
+and never returns back to being
+.Sy enabled .
+.
+.feature com.delphix spacemap_v2 yes
 This feature enables the use of the new space map encoding which
-consists of two words (instead of one) whenever it is advantageous.
+consists of two words
+.Pq instead of one
+whenever it is advantageous.
 The new encoding allows space maps to represent large regions of
 space more efficiently on-disk while also increasing their maximum
 addressable offset.
-
-This feature becomes \fBactive\fR once it is \fBenabled\fR, and never
-returns back to being \fBenabled\fR.
-
-.RE
-.sp
-.ne 2
-.na
-\fB\fBlarge_blocks\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	org.open-zfs:large_block
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	extensible_dataset
-.TE
-
-The \fBlarge_block\fR feature allows the record size on a dataset to be
-set larger than 128KB.
-
-This feature becomes \fBactive\fR once a \fBrecordsize\fR property has been
-set larger than 128KB, and will return to being \fBenabled\fR once all
-filesystems that have ever had their recordsize larger than 128KB are destroyed.
-.RE
-
-.ne 2
-.na
-\fB\fBlarge_dnode\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	org.zfsonlinux:large_dnode
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	extensible_dataset
-.TE
-
-The \fBlarge_dnode\fR feature allows the size of dnodes in a dataset to be
-set larger than 512B.
-
-This feature becomes \fBactive\fR  once a dataset contains an object with a
-dnode larger than 512B, which occurs as a result of setting the \fBdnodesize\fR
-dataset property to a value other than \fBlegacy\fR. The feature will return to
-being \fBenabled\fR once all filesystems that have ever contained a dnode larger
-than 512B are destroyed. Large dnodes allow more data to be stored in the
-bonus buffer, thus potentially improving performance by avoiding the use of
-spill blocks.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBsha512\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	org.illumos:sha512
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	extensible_dataset
-.TE
-
-This feature enables the use of the SHA-512/256 truncated hash algorithm
-(FIPS 180-4) for checksum and dedup. The native 64-bit arithmetic of
-SHA-512 provides an approximate 50% performance boost over SHA-256 on
-64-bit hardware and is thus a good minimum-change replacement candidate
-for systems where hash performance is important, but these systems
-cannot for whatever reason utilize the faster \fBskein\fR and
-\fBedonr\fR algorithms.
-
-When the \fBsha512\fR feature is set to \fBenabled\fR, the administrator
-can turn on the \fBsha512\fR checksum on any dataset using the
-\fBzfs set checksum=sha512\fR command.  This feature becomes
-\fBactive\fR once a \fBchecksum\fR property has been set to \fBsha512\fR,
-and will return to being \fBenabled\fR once all filesystems that have
-ever had their checksum set to \fBsha512\fR are destroyed.
-
-Booting off of pools utilizing SHA-512/256 is supported.
-
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBskein\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	org.illumos:skein
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	extensible_dataset
-.TE
-
-This feature enables the use of the Skein hash algorithm for checksum
-and dedup. Skein is a high-performance secure hash algorithm that was a
-finalist in the NIST SHA-3 competition. It provides a very high security
-margin and high performance on 64-bit hardware (80% faster than
-SHA-256). This implementation also utilizes the new salted checksumming
-functionality in ZFS, which means that the checksum is pre-seeded with a
-secret 256-bit random key (stored on the pool) before being fed the data
-block to be checksummed. Thus the produced checksums are unique to a
-given pool, preventing hash collision attacks on systems with dedup.
-
-When the \fBskein\fR feature is set to \fBenabled\fR, the administrator
-can turn on the \fBskein\fR checksum on any dataset using the
-\fBzfs set checksum=skein\fR command.  This feature becomes
-\fBactive\fR once a \fBchecksum\fR property has been set to \fBskein\fR,
-and will return to being \fBenabled\fR once all filesystems that have
-ever had their checksum set to \fBskein\fR are destroyed.
-
-Booting off of pools using \fBskein\fR is supported.
-
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBbookmark_v2\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.datto:bookmark_v2
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	extensible_dataset
-.TE
-
-This feature enables the creation and management of larger bookmarks which are
-needed for other features in ZFS.
-
-This feature becomes \fBactive\fR when a v2 bookmark is created and will be
-returned to the \fBenabled\fR state when all v2 bookmarks are destroyed.
-
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBedonr\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	org.illumos:edonr
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	extensible_dataset
-.TE
-
-This feature enables the use of the Edon-R hash algorithm for checksum,
-including for nopwrite (if compression is also enabled, an overwrite of
-a block whose checksum matches the data being written will be ignored).
-In an abundance of caution, Edon-R can not be used with dedup
-(without verification).
-
-Edon-R is a very high-performance hash algorithm that was part
-of the NIST SHA-3 competition. It provides extremely high hash
-performance (over 350% faster than SHA-256), but was not selected
-because of its unsuitability as a general purpose secure hash algorithm.
-This implementation utilizes the new salted checksumming functionality
-in ZFS, which means that the checksum is pre-seeded with a secret
-256-bit random key (stored on the pool) before being fed the data block
-to be checksummed. Thus the produced checksums are unique to a given
-pool.
-
-When the \fBedonr\fR feature is set to \fBenabled\fR, the administrator
-can turn on the \fBedonr\fR checksum on any dataset using the
-\fBzfs set checksum=edonr\fR command.  This feature becomes
-\fBactive\fR once a \fBchecksum\fR property has been set to \fBedonr\fR,
-and will return to being \fBenabled\fR once all filesystems that have
-ever had their checksum set to \fBedonr\fR are destroyed.
-
-Booting off of pools using \fBedonr\fR is supported.
-
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBallocation_classes\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.intel:allocation_classes
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	none
-.TE
-
-This feature enables support for separate allocation classes.
-
-This feature becomes \fBactive\fR when a dedicated allocation class vdev
-(dedup or special) is created with zpool create or zpool add. With device
-removal, it can be returned to the \fBenabled\fR state if all the top-level
-vdevs from an allocation class are removed.
-.RE
-
-.sp
-.ne 2
-.na
-\fB\fBencryption\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.datto:encryption
-READ\-ONLY COMPATIBLE	no
-DEPENDENCIES	extensible_dataset
-.TE
-
-This feature enables the creation and management of natively encrypted datasets.
-
-This feature becomes \fBactive\fR when an encrypted dataset is created
-and will be returned to the \fBenabled\fR state when all datasets that
-use this feature are destroyed.
-
-.RE
-.sp
-.ne 2
-.na
-\fB\fBresilver_defer\fR\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.datto:resilver_defer
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	none
-.TE
-
-This feature allows zfs to postpone new resilvers if an existing one is already
-in progress. Without this feature, any new resilvers will cause the currently
-running one to be immediately restarted from the beginning.
-
-This feature becomes \fBactive\fR once a resilver has been deferred, and
-returns to being \fBenabled\fR when the deferred resilver begins.
-.RE
-
-.sp
-.ne 2
-.na
-\fBuserobj_accounting\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	org.zfsonlinux:userobj_accounting
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	extensible_dataset
-.TE
-
+.Pp
+This feature becomes
+.Sy active
+once it is
+.Sy enabled ,
+and never returns back to being
+.Sy enabled .
+.
+.feature org.zfsonlinux userobj_accounting yes extensible_dataset
 This feature allows administrators to account the object usage information
 by user and group.
-
-This feature becomes \fBactive\fR as soon as it is enabled and will never
-return to being \fBenabled\fR.
-Each filesystem will be upgraded automatically when remounted, or when new
-files are created under that filesystem.
-The upgrade can also be started manually on filesystems by running
-`zfs set version=current <pool/fs>`.
-The upgrade process runs in the background and may take a while to complete
-for filesystems containing a large number of files.
-.RE
-
-.sp
-.ne 2
-.na
-\fBproject_quota\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	org.zfsonlinux:project_quota
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	extensible_dataset
-.TE
-
-This feature allows administrators to account the space and object usage
-information against the project identifier (ID).
-
-The project ID is a new object-based attribute.
-When upgrading an existing filesystem, an object without a project ID
-attribute will be assigned a zero project ID.
-After this feature is enabled, a newly created object will inherit
-its parent directory's project ID if the parent's inherit flag is set (via
-\fBzfs project [-s|-C]\fR).
-Otherwise, the new object's project ID will be set as zero.
-An object's project ID can be changed at any time by the owner (or privileged
-user) via \fBzfs project -p $prjid\fR.
-
-This feature will become \fBactive\fR as soon as it is enabled and will never
-return to being \fBdisabled\fR.
-Each filesystem will be upgraded automatically when remounted or when a new file
-is created under that filesystem.
-The upgrade can also be triggered on filesystems via `zfs set version=current
-<pool/fs>`.
-The upgrade process runs in the background and may take a while to complete
-for the filesystems containing a large number of files.
-.RE
-
-.sp
-.ne 2
-.na
-\fBlog_spacemap\fR
-.ad
-.RS 4n
-.TS
-l l .
-GUID	com.delphix:log_spacemap
-READ\-ONLY COMPATIBLE	yes
-DEPENDENCIES	com.delphix:spacemap_v2
-.TE
-
-This feature improves performance for heavily-fragmented pools,
-especially when workloads are heavy in random-writes.
-It does so by logging all the metaslab changes on a single spacemap every TXG
-instead of scattering multiple writes to all the metaslab spacemaps.
-
-This feature becomes \fBactive\fR as soon as it is enabled and will never
-return to being \fBenabled\fR.
-.RE
-
-.SH "SEE ALSO"
-.BR zfs (8),
-.BR zpool (8)
+.Pp
+\*[instant-never]
+\*[remount-upgrade]
+.
+.feature org.openzfs zilsaxattr yes extensible_dataset
+This feature enables
+.Sy xattr Ns = Ns Sy sa
+extended attribute logging in the ZIL.
+If enabled, extended attribute changes
+.Pq both Sy xattrdir Ns = Ns Sy dir No and Sy xattr Ns = Ns Sy sa
+are guaranteed to be durable if either the dataset had
+.Sy sync Ns = Ns Sy always
+set at the time the changes were made, or
+.Xr sync 2
+is called on the dataset after the changes were made.
+.Pp
+This feature becomes
+.Sy active
+when a ZIL is created for at least one dataset and will be returned to the
+.Sy enabled
+state when it is destroyed for all datasets that use this feature.
+.
+.feature com.delphix zpool_checkpoint yes
+This feature enables the
+.Nm zpool Cm checkpoint
+command that can checkpoint the state of the pool
+at the time it was issued and later rewind back to it or discard it.
+.Pp
+This feature becomes
+.Sy active
+when the
+.Nm zpool Cm checkpoint
+command is used to checkpoint the pool.
+The feature will only return back to being
+.Sy enabled
+when the pool is rewound or the checkpoint has been discarded.
+.El
+.
+.Sh SEE ALSO
+.Xr zfs 8 ,
+.Xr zpool 8