10499 Multi-modifier protection (MMP)

Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Portions contributed by: Tim Chase <tim@chase2k.com>
Portions contributed by: sanjeevbagewadi <sanjeev.bagewadi@gmail.com>
Portions contributed by: John L. Hammond <john.hammond@intel.com>
Portions contributed by: Giuseppe Di Natale <dinatale2@llnl.gov>
Portions contributed by: Prakash Surya <surya1@llnl.gov>
Reviewed by: George Melikov <mail@gmelikov.ru>
Reviewed by: Tom Caputi <tcaputi@datto.com>
Reviewed by: Kash Pande <kash@tripleback.net>
Reviewed by: loli10K <ezomori.nozomu@gmail.com>
Reviewed by: George Melikov <mail@gmelikov.ru>
Reviewed by: Tony Hutter <hutter2@llnl.gov>
Reviewed by: Gu Zheng <guzheng2331314@163.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Ned Bass <bass6@llnl.gov>
Reviewed by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Andy Stormont <astormont@racktopsystems.com>
Reviewed by: Toomas Soome <tsoome@me.com>
Reviewed by: Kody Kantor <kody.kantor@joyent.com>
Approved by: Dan McDonald <danmcd@joyent.com>

1 files changed, 52 insertions, 2 deletions

diff --git a/usr/src/man/man1m/zpool.1m b/usr/src/man/man1m/zpool.1m
index 5c1c7f1cc7..29935e5d77 100644
--- a/usr/src/man/man1m/zpool.1m
+++ b/usr/src/man/man1m/zpool.1m
@@ -409,6 +409,11 @@ If a pool has a shared spare that is currently being used, the pool can not be
 exported since other pools may use this shared spare, which may lead to
 potential data corruption.
 .Pp
+Shared spares add some risk.
+If the pools are imported on different hosts, and both pools suffer a device
+failure at the same time, both could attempt to use the spare at the same time.
+This may not be detected, resulting in data corruption.
+.Pp
 An in-progress spare replacement can be cancelled by detaching the hot spare.
 If the original faulted device is detached, then the hot spare assumes its
 place in the configuration, and is removed from the spare list of all active
@@ -731,6 +736,31 @@ The default value is
 .Sy off .
 This property can also be referred to by its shortened name,
 .Sy listsnaps .
+.It Sy multihost Ns = Ns Sy on Ns | Ns Sy off
+Controls whether a pool activity check should be performed during
+.Nm zpool Cm import .
+When a pool is determined to be active it cannot be imported, even with the
+.Fl f
+option.
+This property is intended to be used in failover configurations
+where multiple hosts have access to a pool on shared storage.
+.sp
+Multihost provides protection on import only.
+It does not protect against an
+individual device being used in multiple pools, regardless of the type of vdev.
+See the discussion under
+.Sy zpool create.
+.sp
+When this property is on, periodic writes to storage occur to show the pool is
+in use.
+See
+.Sy zfs_multihost_interval
+in the
+.Xr zfs-module-parameters 5
+man page.
+In order to enable this property each host must set a unique hostid.
+The default value is
+.Sy off .
 .It Sy version Ns = Ns Ar version
 The current on-disk version of the pool.
 This can be increased, but never decreased.
@@ -862,6 +892,10 @@ Clears device errors in a pool.
 If no arguments are specified, all device errors within the pool are cleared.
 If one or more devices is specified, only those errors associated with the
 specified device or devices are cleared.
+If multihost is enabled, and the pool has been suspended, this will not
+resume I/O.
+While the pool was suspended, it may have been imported on
+another host, and resuming I/O could result in pool damage.
 .It Xo
 .Nm
 .Cm create
@@ -897,8 +931,24 @@ specification is described in the
 .Sx Virtual Devices
 section.
 .Pp
-The command verifies that each device specified is accessible and not currently
-in use by another subsystem.
+The command attempts to verify that each device specified is accessible and not
+currently in use by another subsystem.
+However this check is not robust enough
+to detect simultaneous attempts to use a new device in different pools, even if
+.Sy multihost
+is
+.Sy enabled.
+The
+administrator must ensure that simultaneous invocations of any combination of
+.Sy zpool replace ,
+.Sy zpool create ,
+.Sy zpool add ,
+or
+.Sy zpool labelclear ,
+do not refer to the same device.
+Using the same device in two pools will
+result in pool corruption.
+.sp
 There are some uses, such as being currently mounted, or specified as the
 dedicated dump device, that prevents a device from ever being used by ZFS.
 Other uses, such as having a preexisting UFS file system, can be overridden with