13500 Want support for overlay networks

Portions contributed by: Dan McDonald <danmcd@joyent.com>
Portions contributed by: Jason King <jason.king@joyent.com>
Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Portions contributed by: Mike Zeller <mike@mikezeller.net>
Portions contributed by: Andy Fiddaman <omnios@citrus-it.co.uk>
Reviewed by: Andy Fiddaman <andy@omnios.org>
Reviewed by: Dan McDonald <danmcd@joyent.com>
Reviewed by: Jason King <jbk@joyent.com>
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Joshua M. Clulow <jmc@joyent.com>
Reviewed by: Mike Gerdts <mike.gerdts@joyent.com>
Reviewed by: Mike Zeller <mike.zeller@joyent.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: Rob Gulewich <robert.gulewich@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Ryan Zezeski <rpz@joyent.com>
Approved by: Joshua M. Clulow <josh@sysmgr.org>

2 files changed, 522 insertions, 0 deletions

diff --git a/usr/src/man/man5/Makefile b/usr/src/man/man5/Makefile
index 9eb12d0164..ea0520872a 100644
--- a/usr/src/man/man5/Makefile
+++ b/usr/src/man/man5/Makefile
@@ -83,6 +83,7 @@ _MANFILES=	Intro.5			\
 		ms.5			\
 		mutex.5			\
 		nfssec.5		\
+		overlay.5		\
 		pam_allow.5		\
 		pam_authtok_check.5	\
 		pam_authtok_get.5	\
diff --git a/usr/src/man/man5/overlay.5 b/usr/src/man/man5/overlay.5
new file mode 100644
index 0000000000..41d1b18739
--- /dev/null
+++ b/usr/src/man/man5/overlay.5
@@ -0,0 +1,521 @@
+.\"
+.\" This file and its contents are supplied under the terms of the
+.\" Common Development and Distribution License ("CDDL"), version 1.0.
+.\" You may only use this file in accordance with the terms of version
+.\" 1.0 of the CDDL.
+.\"
+.\" A full copy of the text of the CDDL should have accompanied this
+.\" source.  A copy of the CDDL is also available via the Internet at
+.\" http://www.illumos.org/license/CDDL.
+.\"
+.\"
+.\" Copyright 2015 Joyent, Inc.
+.\"
+.Dd Apr 09, 2015
+.Dt OVERLAY 5
+.Os
+.Sh NAME
+.Nm overlay
+.Nd Overlay Devices
+.Sh DESCRIPTION
+Overlay devices are a GLDv3 device that allows users to create overlay
+networks that can be used to form the basis of network virtualization
+and software defined networking.
+Overlay networks allow a single physical network, often called an
+.Sy underlay
+network, to provide the means for creating multiple logical, isolated,
+and discrete layer two and layer three networks on top of it.
+.Pp
+Overlay devices are administered through
+.Xr dladm 1M .
+Overlay devices themselves cannot be plumbed up with
+.Sy IP ,
+.Sy vnd ,
+or any other protocol.
+Instead, like an
+.Sy etherstub ,
+they allow for VNICs to be created on top of them.
+Like an
+.Sy etherstub ,
+an overlay device acts as a local switch; however, when it encounters a
+non-local destination address, it instead looks up where it should send
+the packet, encapsulates it, and sends it out another interface in the
+system.
+.Pp
+A single overlay device encapsulates the logic to answer two different,
+but related, questions:
+.Pp
+.Bl -enum -offset indent -compact
+.It
+How should a packet be transformed and put on the wire?
+.It
+Where should a transformed packet be sent?
+.El
+.Pp
+Each of these questions is answered by a plugin.
+The first question is answered by what's called an
+.Em encapsulation plugin .
+The second question is answered by what's called a
+.Em search plugin .
+Packets are encapsulated and decapsulated using the encapsulation plugin
+by the kernel.
+The search plugins are all user land plugins that are consumed by the
+varpd service whose FMRI is
+.Em svc:/network/varpd:default .
+This separation allows for the kernel to be responsible for the data
+path, while having the search plugins in userland allows the system to
+provide a much more expressive interface.
+.Ss Overlay Types
+Overlay devices come in two different flavors, one where all packets are
+always sent to a single address, the other, where the destination of a
+packet varies based on the target MAC address of the packet.
+This information is maintained in a
+.Em target table ,
+which is independent and unique to each overlay device.
+We call the plugins that send traffic to a single location, for example
+a single unicast or multicast IP address, a
+.Sy point to point
+overlay and the overlay devices that can send traffic to different
+locations based on the MAC address of that packet a
+.Sy dynamic
+overlay.
+The plugin type is determined based on the type of the
+.Sy search plugin .
+These are all fully listed in the section
+.Sx Plugins and their Properties .
+.Ss Overlay Destination
+Both encapsulation and search plugins define the kinds of destinations
+that they know how to support.
+An encapsulation plugin always has a single destination type that's
+determined based on how the encapsulation is defined.
+A search plugin, on the other hand, can support multiple combinations of
+destinations.
+A search plugin must support the destination type of the encapsulation
+device.
+The destination may require any of the following three pieces of
+information, depending on the encapsulation plugin:
+.Bl -hang -width Ds
+.It Sy MAC Address
+.Bd -filled -compact
+An Ethernet MAC address is required to determine the destination.
+.Ed
+.It Sy IP Address
+.Bd -filled -compact
+An IP address is required.
+Both IPv4 and IPv6 addresses are supported.
+.Ed
+.It Sy Port
+.Bd -filled -compact
+An IP protocol level (TCP, UDP, SCTP, etc.) port is required.
+.Ed
+.El
+.Pp
+The list of destination types that are supported by both the search and
+encapsulation plugins is listed in the section
+.Sx Plugins and their Properties .
+.Ss varpd
+The varpd service, mentioned above, is responsible for providing the
+virtual ARP daemon.
+Its responsibility is conceptually similar to ARP.
+It runs all instances of search plugins in the system and is responsible
+for answering the kernel's ARP-like questions for where packets should
+be sent.
+.Pp
+The varpd service, svc:/network/varpd:default, must be enabled for
+overlay devices to function.
+If it is disabled while there are active devices, then most overlay
+devices will not function correctly and likely will end up dropping
+traffic.
+.Sh PLUGINS AND PROPERTIES
+Properties fall into three categories in the system:
+.Bl -enum -offset indent -compact
+.It
+Generic properties all overlay devices have
+.It
+Properties specific to the encapsulation plugin
+.It
+Properties specific to the search plugin
+.El
+.Pp
+Each property in the system has the following attributes, which mirror
+the traditional
+.Xr dladm 1M
+link properties:
+.Bl -hang -width Ds
+.It Sy Name
+.Bd -filled -compact
+The name of a property is namespaced by its module and always structured
+and referred to as as module/property.
+This allows for both an encapsulation and search plugin to have a
+property with the same name.
+Properties that are valid for all overlay devices and not specific to a
+module do not generally use a module prefix.
+.Pp
+For example, the property
+.Sy vxlan/listen_ip
+is associated with the
+.Sy vxlan
+encapsulation module.
+.Ed
+.It Sy Type
+.Bd -filled -compact
+Each property in the system has a type.
+.Xr dladm 1M
+takes care of converting between the internal representation and a
+value, but the type influences the acceptable input range.
+The types are:
+.Bl -hang -width Ds
+.It Sy INT
+A signed integer that is up to eight bytes long
+.Pq Sy int64_t .
+.It Sy UINT
+An unsigned integer that is up to eight bytes long
+.Pq Sy uint64_t .
+.It Sy IP
+Either an IPv4 or IPv6 address in traditional string form.
+For example, 192.168.128.23 or 2001:470:8af4::1:1.
+IPv4 addresses may also be encoded as IPv4-mapped IPv6 addresses.
+.It Sy STRING
+A string of ASCII or UTF-8 encoded characters terminated with a
+.Sy NUL
+byte.
+The maximum string length, including the terminator, is currently
+256 bytes.
+.El
+.Ed
+.It Sy Permissions
+.Bd -filled -compact
+Each property has permissions associated with it, which indicate whether
+the system considers them read-only properties or read-write properties.
+A read-only property can never be updated once the device is created.
+This generally includes things like the overlay's encapsulation module.
+.Ed
+.It Sy Required
+.Bd -filled -compact
+This property indicates whether the property is required for the given
+plugin.
+If it is not specified during a call to
+.Sy dladm create-overlay ,
+then the overlay cannot be successfully created.
+Properties which have a
+.Sy default
+will use that value if one is not specified rather than cause the
+overlay creation to fail.
+.Ed
+.It Sy Current Value
+.Bd -filled -compact
+The current value of a property, if the property has a value set.
+Required properties always have a value set.
+.Ed
+.It Sy Default Value
+.Bd -filled -compact
+The default value is an optional part of a given property.
+If a property does define a default value, then it will be used when an
+overlay is created and no other value is given.
+.Ed
+.It Sy Value ranges
+.Bd -filled -compact
+Value ranges are an optional part of a given property.
+They indicate a range or set of values that are valid and may be set for
+a property.
+A property may not declare such a range as it may be impractical or
+unknown.
+For example, most properties based on IP addresses will not
+declare a range.
+.Ed
+.El
+.Pp
+The following sections describe both the modules and the properties that
+exist for each module, noting their name, type, permissions, whether or
+not they are required, and if there is a default value.
+In addition, the effects of each property will be described.
+.Ss Encapsulation Plugins
+.Bl -hang -width Ds
+.It Sy vxlan
+The
+.Sy vxlan
+module is a UDP based encapsulation method.
+It takes a frame that would be put on the wire, wraps it up in a VXLAN
+header and places it in a UDP packet that gets sent out on the
+underlying network.
+For more details about the specific format of the VXLAN header, see
+.Xr vxlan 7P .
+.Pp
+The
+.Sy vxlan
+module requires both an
+.Sy IP address
+and
+.Sy port
+to address it.
+It has a 24-bit virtual network ID space, allowing for
+virtual network identifiers that range from
+.Sy 0
+-
+.Sy 16777215 .
+.Pp
+The
+.Sy vxlan
+module has the following properties:
+.Bl -hang -width Ds
+.It Sy vxlan/listen_ip
+.Bd -filled -compact
+Type:
+.Sy IP |
+Permissions:
+.Sy Read/Write |
+.Sy Required
+.Ed
+.Bd -filled
+The
+.Sy vxlan/listen_ip
+property determines the IP address that the system will accept VXLAN
+encapsulated packets on for this overlay.
+.Ed
+.It Sy vxlan/listen_port
+.Bd -filled -compact
+Type:
+.Sy UINT |
+Permissions:
+.Sy Read/Write |
+.Sy Required
+.Ed
+.Bd -filled -compact
+Default Value:
+.Sy 4789 |
+Range:
+.Sy 0 - 65535
+.Ed
+.Bd -filled
+The
+.Sy vxlan/listen_port
+property determines the UDP port that the system will listen on for
+VXLAN traffic for this overlay.
+The default value is
+.Sy 4789 ,
+the IANA assigned port for VXLAN.
+.Ed
+.El
+.Pp
+The
+.Sy vxlan/listen_ip
+and
+.Sy vxlan/listen_port
+properties determine how the system will accept VXLAN encapsulated
+packets for this interface.
+It does not determine the interface that packets will be sent out over.
+Multiple overlays that all use VXLAN can share the same IP and port
+combination, as the virtual network identifier can be used to tell the
+different overlays apart.
+.El
+.Ss Search Plugins
+Because search plugins may support multiple destinations, they may have
+more properties listed than necessarily show up for a given overlay.
+For example, the
+.Sy direct
+plugin supports destinations that are identified by both an IP address
+and a port, or just an IP address.
+In cases where the device is created over an overlay that only uses an
+IP address for its destination, then it will not have the
+.Sy direct/dest_port
+property.
+.Bl -hang -width Ds
+.It Sy direct
+The
+.Sy direct
+plugin is a point to point module that can be used to create an overlay
+that forwards all non-local traffic to a single destination.
+It supports destinations that are a combination of an
+.Sy IP Address
+and a
+.Sy port .
+.Pp
+The
+.Sy direct
+plugin has the following properties:
+.Bl -hang -width Ds
+.It Sy direct/dest_ip
+.Bd -filled -compact
+Type:
+.Sy IP |
+Permissions:
+.Sy Read/Write |
+.Sy Required
+.Ed
+.Bd -filled
+The
+.Sy direct/dest_ip
+property indicates the IP address that all traffic will be sent out.
+Traffic will be sent out the corresponding interface based on
+traditional IP routing rules and the configuration of the networking
+stack of the global zone.
+.Ed
+.It Sy direct/dest_port
+.Bd -filled -compact
+Type:
+.Sy UINT |
+Permissions:
+.Sy Read/Write |
+.Sy Required
+.Ed
+.Bd -filled -compact
+Default Value:
+.Sy - |
+Range:
+.Sy 0 - 65535
+.Ed
+.Bd -filled
+The
+.Sy direct/dest_port
+property indicates the TCP or UDP port that all traffic will be directed
+to.
+.Ed
+.El
+.It Sy files
+The
+.Sy files
+plugin implements a
+.Sy dynamic
+plugin that specifies where traffic should be sent based on a file.
+It is a glorified version of /etc/ethers.
+The
+.Sy dynamic
+plugin does not support broadcast or multicast traffic, but it has
+support for proxy ARP, NDP, and DHCPv4.
+For the full details of the file format, see
+.Xr overlay_files 4 .
+.Pp
+The
+.Sy files
+plugin has the following property:
+.Bl -hang -width Ds
+.It Sy files/config
+.Bd -filled -compact
+Type:
+.Sy String |
+Permissions:
+.Sy Read/Write |
+.Sy Required
+.Ed
+.Bd -filled
+The
+.Sy files/config
+property specifies an absolute path to a file to read.
+The file is a JSON file that is formatted according to
+.Xr overlay_files 4 .
+.Ed
+.El
+.El
+.Ss General Properties
+Each overlay has the following properties which are used to give
+additional information about the system.
+None of these properties may be specified as part of a
+.Sy dladm create-overlay ,
+instead they come from other arguments or from internal parts of the
+system.
+.Bl -hang -width Ds
+.It Sy encap
+.Bd -filled -compact
+.Sy String |
+Permissions:
+.Sy Read Only
+.Ed
+.Bd -filled
+The
+.Sy encap
+property contains the name of the encapsulation module that's in use.
+.Ed
+.It Sy mtu
+.Bd -filled -compact
+.Sy UINT |
+Permissions:
+.Sy Read/Write
+.Ed
+.Bd -filled -compact
+Default Value:
+.Sy 1400 |
+Range:
+.Sy 576 - 9000
+.Ed
+.Bd -filled
+The
+.Sy mtu
+property describes the maximum transmission unit of the overlay.
+The default value is
+.Sy 1400
+bytes, which ensures that in a traditional deployment with an MTU of
+1500 bytes, the overhead that is added from encapsulation is all
+accounted for.
+It is the administrator's responsibility to ensure that
+the device's MTU and the encapsulation overhead does not exceed that of
+the interfaces that the encapsulated traffic will be sent out of.
+.Pp
+To modify the
+.Sy mtu
+property, use
+.Sy dladm set-linkprop .
+.Ed
+.It Sy search
+.Bd -filled -compact
+.Sy String |
+Permissions:
+.Sy Read Only
+.Ed
+.Bd -filled
+The
+.Sy search
+property contains the name of the search plugin that's in use.
+.Ed
+.It Sy varpd/id
+.Bd -filled -compact
+.Sy String |
+Permissions:
+.Sy Read Only
+.Ed
+.Bd -filled
+The
+.Sy varpd/id
+property indicates the identifier which the
+.Sy varpd
+service uses for this overlay.
+.Ed
+.It Sy vnetid
+.Bd -filled -compact
+.Sy UINT |
+Permissions:
+.Sy Read/Write
+.Ed
+.Bd -filled
+The
+.Sy vnetid
+property has the virtual network identifier that belongs to this overlay.
+The valid range for the virtual network identifier depends on the
+encapsulation engine.
+.Ed
+.El
+.Sh FMA INTEGRATION
+Overlay devices are wired into FMA, the illumos fault management
+architecture, and generates error reports depending on the
+.Sy search
+plugin in use.
+Due to limitations in FMA today, when a single overlay
+enters a degraded state, meaning that it cannot properly perform look
+ups or another error occurred, then it degrades the overall
+.Sy overlay
+pseudo-device driver.
+.Pp
+For more fine-grained information about which overlay is actually in a
+.Em degraded
+state, one should run
+.Sy dladm show-overlay -f .
+In addition, for each overlay in a degraded state a more useful
+diagnostic message is provided which describes the reason that caused
+this overlay to enter into a degraded state.
+.Pp
+The overlay driver is self-healing.
+If the problem corrects itself on its own, it will clear the fault on
+the corresponding device.
+.Sh SEE ALSO
+.Xr dladm 1M ,
+.Xr overlay_files 4 ,
+.Xr vxlan 7P