PSARC/2009/331 IP Datapath Refactoring

PSARC/2008/522 EOF of 2001/070 IPsec HW Acceleration support
PSARC/2009/495 netstat -r flags for blackhole and reject routes
PSARC 2009/496 EOF of XRESOLV
PSARC/2009/494 IP_DONTFRAG socket option
PSARC/2009/515 fragmentation controls for ping and traceroute
6798716 ip_newroute delenda est
6798739 ARP and IP are too separate
6807265 IPv4 ip2mac() support
6756382 Please remove Venus IPsec HWACCEL code
6880632 sendto/sendmsg never returns EHOSTUNREACH in Solaris
6748582 sendmsg() return OK, but doesn't send message using IPv4-mapped x IPv6 addr
1119790 TCP and path mtu discovery
4637227 should support equal-cost multi-path (ECMP)
5078568 getsockopt() for IPV6_PATHMTU on a non-connected socket should not succeed
6419648 "AR* contract private note" should be removed as part of ATM SW EOL
6274715 Arp could keep the old entry in the cache while it waits for an arp response
6605615 Remove duplicated TCP/IP opt_set/opt_get code; use conn_t
6874677 IP_TTL can be used to send with ttl zero
4034090 arp should not let you delete your own entry
6882140 Implement IP_DONTFRAG socket option
6883858 Implement ping -D option; traceroute -F should work for IPv6 and shared-IP zones
1119792 TCP/IP black hole detection is broken on receiver
4078796 Directed broadcast forwarding code has problems
4104337 restrict the IPPROTO_IP and IPPROTO_IPV6 options based on the socket family
4203747 Source address selection for source routed packets
4230259 pmtu is increased every ip_ire_pathmtu_interval timer value.
4300533 When sticky option ipv6_pktinfo set to bogus address subsequent connect time out
4471035 ire_delete_cache_gw is called through ire_walk unnecessarily
4514572 SO_DONTROUTE socket option doesn't work with IPv6
4524980 tcp_lookup_ipv4() should compare the ifindex against tcpb->tcpb_bound_if
4532714 machine fails to switch quickly among failed default routes
4634219 IPv6 path mtu discovery is broken when using routing header
4691581 udp broadcast handling causes too many replicas
4708405 mcast is broken on machines when all interfaces are IFF_POINTOPOINT
4770457 netstat/route: source address of interface routes pretends to be gateway address
4786974 use routing table to determine routes/interface for multicast
4792619 An ip_fanout_udp_ipc_v6() routine might lead to some simpler code
4816115 Nuke ipsec_out_use_global_policy
4862844 ipsec offload corner case
4867533 tcp_rq and tcp_wq are redundant
4868589 NCEs should be shared across an IPMP group
4872093 unplumbing an improper virtual interface panics in ip_newroute_get_dst_ill()
4901671 FireEngine needs some cleanup
4907617 IPsec identity latching should be done before sending SYN-ACK
4941461 scopeid and IPV6_PKTINFO with UDP/ICMP connect() does not work properly
4944981 ip does nothing with IP6I_NEXTHOP
4963353 IPv4 and IPv6 proto fanout codes could be brought closer
4963360 consider passing zoneid using ip6i_t instead of ipsec_out_t in NDP
4963734 new ip6_asp locking is used incorrectly in ip_newroute_v6()
5008315 IPv6 code passes ip6i_t to IPsec code instead of ip6_t
5009636 memory leak in ip_fanout_proto_v6()
5092337 tcp/udp option handling can use some cleanup
5035841 Solaris can fail to create a valid broadcast ire
5043747 ar_query_xmit: Could not find the ace
5051574 tcp_check_policy is missing some checks
6305037 full hardware checksum is discarded when there're more than 2 mblks in the chain
6311149 ip.c needs to be put through a woodchipper
4708860 Unable to reassemble CGTP fragmented multicast packets
6224628 Large IPv6 packets with IPsec protection sometimes have length mismatch.
6213243 Solaris does not currently support Dead Gateway Detection
5029091 duplicate code in IP's input path for TCP/UDP/SCTP
4674643 through IPv6 CGTP routes, the very first packet is sent only after a while
6207318 Multiple default routes do not round robin connections to routers.
4823410 IP has an inconsistent view of link mtu
5105520 adding interface route to down interface causes ifconfig hang
5105707 advanced sockets API introduced some dead code
6318399 IP option handling for icmp and udp is too complicated
6321434 Every dropped packet in IP should use ip_drop_packet()
6341693 ifconfig mtu should operate on the physical interface, not individual ipif's
6352430 The credentials attached to an mblk are not particularly useful
6357894 uninitialised ipp_hoplimit needs to be cleaned up.
6363568 ip_xmit_v6() may be missing IRE releases in error cases
6364828 ip_rput_forward needs a makeover
6384416 System panics when running as multicast forwarder using multicast tunnels
6402382 TX: UDP v6 slowpath is not modified to handle mac_exempt conns
6418413 assertion failed ipha->ipha_ident == 0||ipha->ipha_ident == 0xFFFF
6420916 assertion failures in ipv6 wput path
6430851 use of b_prev to store ifindex is not 100% safe
6446106 IPv6 packets stored in nce->nce_qd_mp will be sent with incorrect tcp/udp checksums
6453711 SCTP OOTB sent as if genetated by global zone
6465212 ARP/IP merge should remove ire_freemblk.esballoc
6490163 ip_input() could misbehave if the first mblk's size is not big enough
6496664 missing ipif_refrele leads to reference leak and deferred crash in ip_wput_ipsec_out_v6
6504856 memory leak in ip_fanout_proto_v6() when using link local outer tunnel addresses
6507765 IRE cache hash function performs badly
6510186 IP_FORWARD_PROG bit is easily overlooked
6514727 cgtp ipv6 failure on snv54
6528286 MULTIRT (CGTP) should offload checksum to hardware
6533904 SCTP: doesn't support traffic class for IPv6
6539415 TX: ipif source selection is flawed for unlabeled gateways
6539851 plumbed unworking nic blocks sending broadcast packets
6564468 non-solaris SCTP stack over rawip socket: netstat command counts rawipInData not rawipOutDatagrams
6568511 ipIfStatsOutDiscards not bumped when discarding an ipsec packet on the wrong NIC
6584162 tcp_g_q_inactive() makes incorrect use of taskq_dispatch()
6603974 round-robin default with many interfaces causes infinite temporary IRE thrashing
6611750 ilm_lookup_ill_index_v4 was born an orphan
6618423 ip_wput_frag_mdt sends out packets that void pfhooks
6620964 IRE max bucket count calculations performed in ip_ire_init() are flawed
6626266 various _broadcasts seem redundant
6638182 IP_PKTINFO + SO_DONTROUTE + CIPSO IP option == panic
6647710 IPv6 possible DoS vulnerability
6657357 nce should be kmem_cache alloc'ed from an nce_cache.
6685131 ilg_add -> conn_ilg_alloc interacting with conn_ilg[] walkers can cause panic.
6730298 adding 0.0.0.0 key with mask != 0 causes  'route delete default' to fail
6730976 vni and ipv6 doesn't quite work.
6740956 assertion failed: mp->b_next == 0L && mp->b_prev == 0L in nce_queue_mp_common()
6748515 BUMP_MIB() is occasionally done on the wrong ill
6753250 ip_output_v6() `notv6' error path has an errant ill_refrele()
6756411 NULL-pointer dereference in ip_wput_local()
6769582 IP must forward packet returned from FW-HOOK
6781525 bogus usesrc usage leads directly to panic
6422839 System paniced in ip_multicast_loopback due to NULL pointer dereference
6785521 initial IPv6 DAD solicitation is dropped in ip_newroute_ipif_v6()
6787370 ipnet devices not seeing forwarded IP packets on outgoing interface
6791187 ip*dbg() calls in ip_output_options() claim to originate from ip_wput()
6794047 nce_fp_mp prevents sharing of NCEs across an IPMP group
6797926 many unnecessary ip0dbg() in ip_rput_data_v6
6846919 Packet queued for ND gets sent in the clear.
6856591 ping doesn't send packets with DF set
6861113 arp module has incorrect dependency path for hook module
6865664 IPV6_NEXTHOP does not work with TCP socket
6874681 No ICMP time exceeded when a router receives packet with ttl = 0
6880977 ip_wput_ire() uses over 1k of stack
6595433 IPsec performance could be significantly better when calling hw crypto provider synchronously
6848397 ifconfig down of an interface can hang.
6849602 IPV6_PATHMTU size issue for UDP
6885359 Add compile-time option for testing pure IPsec overhead
6889268 Odd loopback source address selection with IPMP
6895420 assertion failed: connp->conn_helper_info == NULL
6851189 Routing-related panic occurred during reboot on T2000 system running snv_117
6896174 Post-async-encryption, AH+ESP packets may have misinitialized ipha/ip6
6896687 iptun presents IPv6 with an MTU < 1280
6897006 assertion failed: ipif->ipif_id != 0 in ip_sioctl_slifzone_restart

1 files changed, 170 insertions, 380 deletions

diff --git a/usr/src/uts/common/inet/ip/tn_ipopt.c b/usr/src/uts/common/inet/ip/tn_ipopt.c
index 359b8d4623..1ce050ec69 100644
--- a/usr/src/uts/common/inet/ip/tn_ipopt.c
+++ b/usr/src/uts/common/inet/ip/tn_ipopt.c
@@ -271,38 +271,40 @@ tsol_get_option_v6(mblk_t *mp, tsol_ip_label_t *label_type, uchar_t **buffer)
  * tsol_check_dest()
  *
  * This routine verifies if a destination is allowed to recieve messages
- * based on the message cred's security label. If any adjustments to
- * the cred are needed due to the connection's MAC mode or
- * the destination's ability to receive labels, an "effective cred"
- * will be returned.
+ * based on the security label. If any adjustments to the label are needed
+ * due to the connection's MAC mode or the destination's ability
+ * to receive labels, an "effective label" will be returned.
+ *
+ * zone_is_global is set if the actual zoneid is global. That is, it is
+ * not set for an exclusive-IP zone.
  *
- * On successful return, effective_cred will point to the new creds needed
- * or will be NULL if new creds aren't needed. On error, effective_cred
- * is NULL.
+ * On successful return, effective_tsl will point to the new label needed
+ * or will be NULL if a new label isn't needed. On error, effective_tsl will
+ * point to NULL.
  *
  * Returns:
- *	0		Have or constructed appropriate credentials
- *	EHOSTUNREACH	The credentials failed the remote host accreditation
+ *      0		Label (was|is now) correct
+ *	EHOSTUNREACH	The label failed the remote host accreditation
  *      ENOMEM		Memory allocation failure
  */
 int
-tsol_check_dest(const cred_t *credp, const void *dst, uchar_t version,
-    uint_t mac_mode, cred_t **effective_cred)
+tsol_check_dest(const ts_label_t *tsl, const void *dst,
+    uchar_t version, uint_t mac_mode, boolean_t zone_is_global,
+    ts_label_t **effective_tsl)
 {
-	ts_label_t	*tsl, *newtsl = NULL;
+	ts_label_t	*newtsl = NULL;
 	tsol_tpc_t	*dst_rhtp;
-	zoneid_t	zoneid;
 
-	if (effective_cred != NULL)
-		*effective_cred = NULL;
+	if (effective_tsl != NULL)
+		*effective_tsl = NULL;
 	ASSERT(version == IPV4_VERSION ||
 	    (version == IPV6_VERSION &&
 	    !IN6_IS_ADDR_V4MAPPED((in6_addr_t *)dst)));
 
 	/* Always pass kernel level communication (NULL label) */
-	if ((tsl = crgetlabel(credp)) == NULL) {
+	if (tsl == NULL) {
 		DTRACE_PROBE2(tx__tnopt__log__info__labeling__mac__allownull,
-		    char *, "destination ip(1) with null cred was passed",
+		    char *, "destination ip(1) with null label was passed",
 		    ipaddr_t, dst);
 		return (0);
 	}
@@ -358,9 +360,8 @@ tsol_check_dest(const cred_t *credp, const void *dst, uchar_t version,
 		}
 		if (!blequal(&dst_rhtp->tpc_tp.tp_def_label,
 		    &tsl->tsl_label)) {
-			zoneid = crgetzoneid(credp);
 			if (mac_mode != CONN_MAC_AWARE ||
-			    !(zoneid == GLOBAL_ZONEID ||
+			    !(zone_is_global ||
 			    bldominates(&tsl->tsl_label,
 			    &dst_rhtp->tpc_tp.tp_def_label))) {
 				DTRACE_PROBE4(
@@ -438,51 +439,43 @@ tsol_check_dest(const cred_t *credp, const void *dst, uchar_t version,
 	}
 
 	/*
-	 * Generate a new cred if we modified the security label or
-	 * label flags.
+	 * Return the new label.
 	 */
 	if (newtsl != NULL) {
-		if (effective_cred != NULL) {
-			*effective_cred = copycred_from_tslabel(credp,
-			    newtsl, KM_NOSLEEP);
-		}
-		label_rele(newtsl);
-		if (effective_cred != NULL && *effective_cred == NULL) {
-			TPC_RELE(dst_rhtp);
-			return (ENOMEM);
-		}
+		if (effective_tsl != NULL)
+			*effective_tsl = newtsl;
+		else
+			label_rele(newtsl);
 	}
 	TPC_RELE(dst_rhtp);
 	return (0);
 }
 
 /*
- * tsol_compute_label()
+ * tsol_compute_label_v4()
  *
  * This routine computes the IP label that should be on a packet based on the
  * connection and destination information.
  *
+ * The zoneid is the IP zoneid (i.e., GLOBAL_ZONEID for exlusive-IP zones).
+ *
  * Returns:
  *      0		Fetched label
  *	EHOSTUNREACH	No route to destination
  *	EINVAL		Label cannot be computed
  */
 int
-tsol_compute_label(const cred_t *credp, ipaddr_t dst, uchar_t *opt_storage,
-    ip_stack_t *ipst)
+tsol_compute_label_v4(const ts_label_t *tsl, zoneid_t zoneid, ipaddr_t dst,
+    uchar_t *opt_storage, ip_stack_t *ipst)
 {
 	uint_t		sec_opt_len;
-	ts_label_t	*tsl;
-	ire_t		*ire, *sire = NULL;
-	tsol_ire_gw_secattr_t *attrp;
-	zoneid_t	zoneid, ip_zoneid;
-
-	ASSERT(credp != NULL);
+	ire_t		*ire;
+	tsol_ire_gw_secattr_t *attrp = NULL;
 
 	if (opt_storage != NULL)
 		opt_storage[IPOPT_OLEN] = 0;
 
-	if ((tsl = crgetlabel(credp)) == NULL)
+	if (tsl == NULL)
 		return (0);
 
 	/* always pass multicast */
@@ -493,67 +486,44 @@ tsol_compute_label(const cred_t *credp, ipaddr_t dst, uchar_t *opt_storage,
 		return (0);
 
 	if (tsl->tsl_flags & TSLF_UNLABELED) {
-
 		/*
 		 * The destination is unlabeled. Only add a label if the
 		 * destination is not a broadcast/local/loopback address,
 		 * the destination is not on the same subnet, and the
 		 * next-hop gateway is labeled.
-		 *
-		 * For exclusive stacks we set the zoneid to zero
-		 * to operate as if we are in the global zone for
-		 * IRE lookups.
 		 */
-		zoneid = crgetzoneid(credp);
-		if (ipst->ips_netstack->netstack_stackid != GLOBAL_NETSTACKID)
-			ip_zoneid = GLOBAL_ZONEID;
-		else
-			ip_zoneid = zoneid;
-
-		ire = ire_cache_lookup(dst, ip_zoneid, tsl, ipst);
-
-		if (ire != NULL && (ire->ire_type & (IRE_BROADCAST | IRE_LOCAL |
-		    IRE_LOOPBACK | IRE_INTERFACE)) != 0) {
-			IRE_REFRELE(ire);
-			return (0);
-		} else if (ire == NULL) {
-			ire = ire_ftable_lookup(dst, 0, 0, 0, NULL, &sire,
-			    ip_zoneid, 0, tsl, (MATCH_IRE_RECURSIVE |
-			    MATCH_IRE_DEFAULT | MATCH_IRE_SECATTR), ipst);
-		}
-
-		/* no route to destination */
-		if (ire == NULL) {
+		ire = ire_route_recursive_v4(dst, 0, NULL, zoneid, tsl,
+		    MATCH_IRE_SECATTR, B_TRUE, 0, ipst, NULL, &attrp, NULL);
+		ASSERT(ire != NULL);
+		if (ire->ire_flags & (RTF_REJECT|RTF_BLACKHOLE)) {
+			/* no route to destination */
+			ire_refrele(ire);
 			DTRACE_PROBE3(
 			    tx__tnopt__log__info__labeling__routedst__v4,
 			    char *, "No route to unlabeled dest ip(1) with "
-			    "creds(2).", ipaddr_t, dst, cred_t *, credp);
+			    "with label(2).", ipaddr_t, dst, ts_label_t *, tsl);
 			return (EHOSTUNREACH);
 		}
+		if (ire->ire_type & (IRE_BROADCAST | IRE_LOCAL | IRE_LOOPBACK |
+		    IRE_INTERFACE)) {
+			ire_refrele(ire);
+			return (0);
+		}
 
 		/*
-		 * Prefix IRE from f-table lookup means that the destination
-		 * is not directly connected; check the next-hop attributes.
+		 * ire_route_recursive gives us the first attrp it finds
+		 * in the recursive lookup.
 		 */
-		if (sire != NULL) {
-			ASSERT(ire != NULL);
-			IRE_REFRELE(ire);
-			ire = sire;
-		}
-
 		/*
 		 * Return now if next hop gateway is unlabeled. There is
 		 * no need to generate a CIPSO option for this message.
 		 */
-		attrp = ire->ire_gw_secattr;
 		if (attrp == NULL || attrp->igsa_rhc == NULL ||
 		    attrp->igsa_rhc->rhc_tpc->tpc_tp.host_type == UNLABELED) {
-			IRE_REFRELE(ire);
+			ire_refrele(ire);
 			return (0);
 		}
-
-		IRE_REFRELE(ire);
-
+		ire_refrele(ire);
 	}
 
 	/* compute the CIPSO option */
@@ -562,8 +532,8 @@ tsol_compute_label(const cred_t *credp, ipaddr_t dst, uchar_t *opt_storage,
 
 	if (sec_opt_len == 0) {
 		DTRACE_PROBE3(tx__tnopt__log__error__labeling__lostops__v4,
-		    char *, "options lack length for dest ip(1) with creds(2).",
-		    ipaddr_t, dst, cred_t *, credp);
+		    char *, "options lack length for dest ip(1) with label(2).",
+		    ipaddr_t, dst, ts_label_t *, tsl);
 		return (EINVAL);
 	}
 
@@ -575,6 +545,9 @@ tsol_compute_label(const cred_t *credp, ipaddr_t dst, uchar_t *opt_storage,
  * header, move the 'buflen' bytes back to fill the gap, and return the number
  * of bytes removed (as zero or negative number).  Assumes that the headers are
  * sane.
+ *
+ * Note that tsol_remove_secopt does not adjust ipha_length but
+ * tsol_remove_secopt_v6 does adjust ip6_plen.
  */
 int
 tsol_remove_secopt(ipha_t *ipha, int buflen)
@@ -659,6 +632,9 @@ tsol_remove_secopt(ipha_t *ipha, int buflen)
  * option cannot be inserted.  (Note that negative return values are possible
  * when noops must be compressed, and that only -1 indicates error.  Successful
  * return value is always evenly divisible by 4, by definition.)
+ *
+ * Note that tsol_prepend_option does not adjust ipha_length but
+ * tsol_prepend_option_v6 does adjust ip6_plen.
  */
 int
 tsol_prepend_option(uchar_t *optbuf, ipha_t *ipha, int buflen)
@@ -810,28 +786,39 @@ tsol_prepend_option(uchar_t *optbuf, ipha_t *ipha, int buflen)
 }
 
 /*
- * tsol_check_label()
+ * tsol_check_label_v4()
  *
  * This routine computes the IP label that should be on the packet based on the
- * connection and destination information.  If the label is there, it returns
- * zero, so the caller knows that the label is syncronized, and further calls
- * are not required.  If the label isn't right, then the right one is inserted.
+ * connection and destination information.  It's called by the IP forwarding
+ * logic and by ip_output_simple. The ULPs generate the labels before calling
+ * conn_ip_output. If any adjustments to
+ * the label are needed due to the connection's MAC-exempt status or
+ * the destination's ability to receive labels, an "effective label"
+ * will be returned.
  *
  * The packet's header is clear before entering IPsec's engine.
  *
+ * The zoneid is the IP zoneid (i.e., GLOBAL_ZONEID for exlusive-IP zones).
+ * zone_is_global is set if the actual zoneid is global.
+ *
+ * On successful return, effective_tslp will point to the new label needed
+ * or will be NULL if a new label isn't needed. On error, effective_tsl will
+ * point to NULL.
+ *
  * Returns:
- *      0		Label on packet (was|is now) correct
+ *      0		Label on (was|is now) correct
  *      EACCES		The packet failed the remote host accreditation.
  *      ENOMEM		Memory allocation failure.
  *	EINVAL		Label cannot be computed
  */
 int
-tsol_check_label(const cred_t *credp, mblk_t **mpp, uint_t mac_mode,
-    ip_stack_t *ipst, pid_t pid)
+tsol_check_label_v4(const ts_label_t *tsl, zoneid_t zoneid, mblk_t **mpp,
+    uint_t mac_mode, boolean_t zone_is_global, ip_stack_t *ipst,
+    ts_label_t **effective_tslp)
 {
 	mblk_t *mp = *mpp;
 	ipha_t  *ipha;
-	cred_t *effective_cred = NULL;
+	ts_label_t *effective_tsl = NULL;
 	uchar_t opt_storage[IP_MAX_OPT_LENGTH];
 	uint_t hlen;
 	uint_t sec_opt_len;
@@ -839,19 +826,18 @@ tsol_check_label(const cred_t *credp, mblk_t **mpp, uint_t mac_mode,
 	int delta_remove = 0, delta_add, adjust;
 	int retv;
 
+	*effective_tslp = NULL;
 	opt_storage[IPOPT_OPTVAL] = 0;
 
 	ipha = (ipha_t *)mp->b_rptr;
 
 	/*
 	 * Verify the destination is allowed to receive packets at
-	 * the security label of the message data. check_dest()
-	 * may create a new effective cred with a modified label
-	 * or label flags. Apply any such cred to the message block
-	 * for use in future routing decisions.
+	 * the security label of the message data. tsol_check_dest()
+	 * may create a new effective label or label flags.
 	 */
-	retv = tsol_check_dest(credp, &ipha->ipha_dst, IPV4_VERSION,
-	    mac_mode, &effective_cred);
+	retv = tsol_check_dest(tsl, &ipha->ipha_dst, IPV4_VERSION,
+	    mac_mode, zone_is_global, &effective_tsl);
 	if (retv != 0)
 		return (retv);
 
@@ -859,16 +845,15 @@ tsol_check_label(const cred_t *credp, mblk_t **mpp, uint_t mac_mode,
 	 * Calculate the security label to be placed in the text
 	 * of the message (if any).
 	 */
-	if (effective_cred != NULL) {
-		if ((retv = tsol_compute_label(effective_cred,
+	if (effective_tsl != NULL) {
+		if ((retv = tsol_compute_label_v4(effective_tsl, zoneid,
 		    ipha->ipha_dst, opt_storage, ipst)) != 0) {
-			crfree(effective_cred);
+			label_rele(effective_tsl);
 			return (retv);
 		}
-		mblk_setcred(mp, effective_cred, pid);
-		crfree(effective_cred);
+		*effective_tslp = effective_tsl;
 	} else {
-		if ((retv = tsol_compute_label(credp,
+		if ((retv = tsol_compute_label_v4(tsl, zoneid,
 		    ipha->ipha_dst, opt_storage, ipst)) != 0) {
 			return (retv);
 		}
@@ -890,10 +875,6 @@ tsol_check_label(const cred_t *credp, mblk_t **mpp, uint_t mac_mode,
 			return (0);
 	}
 
-	if (msg_getcred(mp, NULL) == NULL) {
-		mblk_setcred(mp, (cred_t *)credp, NOPID);
-	}
-
 	/*
 	 * If there is an option there, then it must be the wrong one; delete.
 	 */
@@ -918,8 +899,13 @@ tsol_check_label(const cred_t *credp, mblk_t **mpp, uint_t mac_mode,
 			copylen = 256;
 		new_mp = allocb_tmpl(hlen + copylen +
 		    (mp->b_rptr - mp->b_datap->db_base), mp);
-		if (new_mp == NULL)
+		if (new_mp == NULL) {
+			if (effective_tsl != NULL) {
+				label_rele(effective_tsl);
+				*effective_tslp = NULL;
+			}
 			return (ENOMEM);
+		}
 
 		/* keep the bias */
 		new_mp->b_rptr += mp->b_rptr - mp->b_datap->db_base;
@@ -948,6 +934,10 @@ tsol_check_label(const cred_t *credp, mblk_t **mpp, uint_t mac_mode,
 	return (0);
 
 param_prob:
+	if (effective_tsl != NULL) {
+		label_rele(effective_tsl);
+		*effective_tslp = NULL;
+	}
 	return (EINVAL);
 }
 
@@ -972,19 +962,17 @@ param_prob:
  * i.e starting from the IP6OPT_LS but not including the pad at the end.
  * The user must prepend two octets (either padding or next header / length)
  * and append padding out to the next 8 octet boundary.
+ *
+ * The zoneid is the IP zoneid (i.e., GLOBAL_ZONEID for exlusive-IP zones).
  */
 int
-tsol_compute_label_v6(const cred_t *credp, const in6_addr_t *dst,
-    uchar_t *opt_storage, ip_stack_t *ipst)
+tsol_compute_label_v6(const ts_label_t *tsl, zoneid_t zoneid,
+    const in6_addr_t *dst, uchar_t *opt_storage, ip_stack_t *ipst)
 {
-	ts_label_t	*tsl;
 	uint_t		sec_opt_len;
 	uint32_t	doi;
-	zoneid_t	zoneid, ip_zoneid;
-	ire_t		*ire, *sire;
-	tsol_ire_gw_secattr_t *attrp;
-
-	ASSERT(credp != NULL);
+	ire_t		*ire;
+	tsol_ire_gw_secattr_t *attrp = NULL;
 
 	if (ip6opt_ls == 0)
 		return (EINVAL);
@@ -992,15 +980,13 @@ tsol_compute_label_v6(const cred_t *credp, const in6_addr_t *dst,
 	if (opt_storage != NULL)
 		opt_storage[IPOPT_OLEN] = 0;
 
-	if ((tsl = crgetlabel(credp)) == NULL)
+	if (tsl == NULL)
 		return (0);
 
 	/* Always pass multicast */
 	if (IN6_IS_ADDR_MULTICAST(dst))
 		return (0);
 
-	zoneid = crgetzoneid(credp);
-
 	/*
 	 * Fill in a V6 label.  If a new format is added here, make certain
 	 * that the maximum size of this label is reflected in sys/tsol/tnet.h
@@ -1012,62 +998,41 @@ tsol_compute_label_v6(const cred_t *credp, const in6_addr_t *dst,
 	if (tsl->tsl_flags & TSLF_UNLABELED) {
 		/*
 		 * The destination is unlabeled. Only add a label if the
-		 * destination is not broadcast/local/loopback address,
+		 * destination is not a broadcast/local/loopback address,
 		 * the destination is not on the same subnet, and the
 		 * next-hop gateway is labeled.
-		 *
-		 * For exclusive stacks we set the zoneid to zero to
-		 * operate as if we are in the global zone when
-		 * performing IRE lookups and conn_t comparisons.
 		 */
-		if (ipst->ips_netstack->netstack_stackid != GLOBAL_NETSTACKID)
-			ip_zoneid = GLOBAL_ZONEID;
-		else
-			ip_zoneid = zoneid;
-
-		sire = NULL;
-		ire = ire_cache_lookup_v6(dst, ip_zoneid, tsl, ipst);
-
-		if (ire != NULL && (ire->ire_type & (IRE_LOCAL |
-		    IRE_LOOPBACK | IRE_INTERFACE)) != 0) {
-			IRE_REFRELE(ire);
-			return (0);
-		} else if (ire == NULL) {
-			ire = ire_ftable_lookup_v6(dst, NULL, NULL, 0, NULL,
-			    &sire, ip_zoneid, 0, tsl, (MATCH_IRE_RECURSIVE |
-			    MATCH_IRE_DEFAULT | MATCH_IRE_SECATTR), ipst);
-		}
-
-		/* no route to destination */
-		if (ire == NULL) {
+		ire = ire_route_recursive_v6(dst, 0, NULL, zoneid, tsl,
+		    MATCH_IRE_SECATTR, B_TRUE, 0, ipst, NULL, &attrp, NULL);
+		ASSERT(ire != NULL);
+		if (ire->ire_flags & (RTF_REJECT|RTF_BLACKHOLE)) {
+			/* no route to destination */
+			ire_refrele(ire);
 			DTRACE_PROBE3(
 			    tx__tnopt__log__info__labeling__routedst__v6,
 			    char *, "No route to unlabeled dest ip6(1) with "
-			    "creds(2).", in6_addr_t *, dst, cred_t *, credp);
+			    "label(2).", in6_addr_t *, dst, ts_label_t *, tsl);
 			return (EHOSTUNREACH);
 		}
-
+		if (ire->ire_type & (IRE_LOCAL | IRE_LOOPBACK |
+		    IRE_INTERFACE)) {
+			ire_refrele(ire);
+			return (0);
+		}
 		/*
-		 * Prefix IRE from f-table lookup means that the destination
-		 * is not directly connected; check the next-hop attributes.
+		 * ire_route_recursive gives us the first attrp it finds
+		 * in the recursive lookup.
 		 */
-		if (sire != NULL) {
-			ASSERT(ire != NULL);
-			IRE_REFRELE(ire);
-			ire = sire;
-		}
-
 		/*
 		 * Return now if next hop gateway is unlabeled. There is
 		 * no need to generate a CIPSO option for this message.
 		 */
-		attrp = ire->ire_gw_secattr;
 		if (attrp == NULL || attrp->igsa_rhc == NULL ||
 		    attrp->igsa_rhc->rhc_tpc->tpc_tp.host_type == UNLABELED) {
-			IRE_REFRELE(ire);
+			ire_refrele(ire);
 			return (0);
 		}
-		IRE_REFRELE(ire);
+		ire_refrele(ire);
 	}
 
 	/* compute the CIPSO option */
@@ -1079,7 +1044,7 @@ tsol_compute_label_v6(const cred_t *credp, const in6_addr_t *dst,
 	if (sec_opt_len == 0) {
 		DTRACE_PROBE3(tx__tnopt__log__error__labeling__lostops__v6,
 		    char *, "options lack length for dest ip6(1) with "
-		    "creds(2).", in6_addr_t *, dst, cred_t *, credp);
+		    "label(2).", in6_addr_t *, dst, ts_label_t *, tsl);
 		return (EINVAL);
 	}
 
@@ -1188,6 +1153,9 @@ tsol_find_secopt_v6(
  * Header and data following the label option that is deleted are copied
  * (i.e. slid backward) to the right position, and returns the number
  * of bytes removed (as zero or negative number.)
+ *
+ * Note that tsol_remove_secopt does not adjust ipha_length but
+ * tsol_remove_secopt_v6 does adjust ip6_plen.
  */
 int
 tsol_remove_secopt_v6(ip6_t *ip6h, int buflen)
@@ -1286,6 +1254,9 @@ tsol_remove_secopt_v6(ip6_t *ip6h, int buflen)
  * extra option being added. Header and data following the position where
  * the label option is inserted are copied (i.e. slid forward) to the right
  * position.
+ *
+ * Note that tsol_prepend_option does not adjust ipha_length but
+ * tsol_prepend_option_v6 does adjust ip6_plen.
  */
 int
 tsol_prepend_option_v6(uchar_t *optbuf, ip6_t *ip6h, int buflen)
@@ -1368,22 +1339,36 @@ tsol_prepend_option_v6(uchar_t *optbuf, ip6_t *ip6h, int buflen)
  * tsol_check_label_v6()
  *
  * This routine computes the IP label that should be on the packet based on the
- * connection and destination information.  It's called only by the IP
- * forwarding logic, because all internal modules atop IP know how to generate
- * their own labels.
+ * connection and destination information.  It's called by the IP forwarding
+ * logic and by ip_output_simple. The ULPs generate the labels before calling
+ * conn_ip_output. If any adjustments to
+ * the label are needed due to the connection's MAC-exempt status or
+ * the destination's ability to receive labels, an "effective label"
+ * will be returned.
+ *
+ * The packet's header is clear before entering IPsec's engine.
+ *
+ * The zoneid is the IP zoneid (i.e., GLOBAL_ZONEID for exlusive-IP zones).
+ * zone_is_global is set if the actual zoneid is global.
+ *
+ * On successful return, effective_tslp will point to the new label needed
+ * or will be NULL if a new label isn't needed. On error, effective_tsl will
+ * point to NULL.
  *
  * Returns:
- *      0		Label on packet was already correct
+ *      0		Label on (was|is now) correct
  *      EACCES		The packet failed the remote host accreditation.
  *      ENOMEM		Memory allocation failure.
+ *	EINVAL		Label cannot be computed
  */
 int
-tsol_check_label_v6(const cred_t *credp, mblk_t **mpp, uint_t mode,
-    ip_stack_t *ipst, pid_t pid)
+tsol_check_label_v6(const ts_label_t *tsl, zoneid_t zoneid, mblk_t **mpp,
+    uint_t mac_mode, boolean_t zone_is_global, ip_stack_t *ipst,
+    ts_label_t **effective_tslp)
 {
 	mblk_t *mp = *mpp;
 	ip6_t  *ip6h;
-	cred_t *effective_cred;
+	ts_label_t *effective_tsl = NULL;
 	/*
 	 * Label option length is limited to IP_MAX_OPT_LENGTH for
 	 * symmetry with IPv4. Can be relaxed if needed
@@ -1399,16 +1384,16 @@ tsol_check_label_v6(const cred_t *credp, mblk_t **mpp, uint_t mode,
 	uint_t	hbhlen;
 	boolean_t hbh_needed;
 
+	*effective_tslp = NULL;
+
 	/*
 	 * Verify the destination is allowed to receive packets at
-	 * the security label of the message data. check_dest()
-	 * may create a new effective cred with a modified label
-	 * or label flags. Apply any such cred to the message block
-	 * for use in future routing decisions.
+	 * the security label of the message data. tsol_check_dest()
+	 * may create a new effective label or label flags.
 	 */
 	ip6h = (ip6_t *)mp->b_rptr;
-	retv = tsol_check_dest(credp, &ip6h->ip6_dst, IPV6_VERSION,
-	    mode, &effective_cred);
+	retv = tsol_check_dest(tsl, &ip6h->ip6_dst, IPV6_VERSION,
+	    mac_mode, zone_is_global, &effective_tsl);
 	if (retv != 0)
 		return (retv);
 
@@ -1416,16 +1401,15 @@ tsol_check_label_v6(const cred_t *credp, mblk_t **mpp, uint_t mode,
 	 * Calculate the security label to be placed in the text
 	 * of the message (if any).
 	 */
-	if (effective_cred != NULL) {
-		if ((retv = tsol_compute_label_v6(effective_cred,
+	if (effective_tsl != NULL) {
+		if ((retv = tsol_compute_label_v6(effective_tsl, zoneid,
 		    &ip6h->ip6_dst, opt_storage, ipst)) != 0) {
-			crfree(effective_cred);
+			label_rele(effective_tsl);
 			return (retv);
 		}
-		mblk_setcred(mp, effective_cred, pid);
-		crfree(effective_cred);
+		*effective_tslp = effective_tsl;
 	} else {
-		if ((retv = tsol_compute_label_v6(credp,
+		if ((retv = tsol_compute_label_v6(tsl, zoneid,
 		    &ip6h->ip6_dst, opt_storage, ipst)) != 0)
 			return (retv);
 	}
@@ -1457,10 +1441,6 @@ tsol_check_label_v6(const cred_t *credp, mblk_t **mpp, uint_t mode,
 		return (0);
 	}
 
-	if (msg_getcred(mp, NULL) == NULL) {
-		mblk_setcred(mp, (cred_t *)credp, NOPID);
-	}
-
 	if (secopt != NULL && sec_opt_len != 0 &&
 	    (bcmp(opt_storage, secopt, sec_opt_len + 2) == 0)) {
 		/* The packet has the correct label already */
@@ -1499,8 +1479,13 @@ tsol_check_label_v6(const cred_t *credp, mblk_t **mpp, uint_t mode,
 			copylen = hdr_len;
 		new_mp = allocb_tmpl(hlen + copylen +
 		    (mp->b_rptr - mp->b_datap->db_base), mp);
-		if (new_mp == NULL)
+		if (new_mp == NULL) {
+			if (effective_tsl != NULL) {
+				label_rele(effective_tsl);
+				*effective_tslp = NULL;
+			}
 			return (ENOMEM);
+		}
 
 		/* keep the bias */
 		new_mp->b_rptr += mp->b_rptr - mp->b_datap->db_base;
@@ -1522,208 +1507,13 @@ tsol_check_label_v6(const cred_t *credp, mblk_t **mpp, uint_t mode,
 	ASSERT(mp->b_wptr + delta_add <= DB_LIM(mp));
 	mp->b_wptr += delta_add;
 
+	/* tsol_prepend_option_v6 has adjusted ip6_plen */
 	return (0);
 
 param_prob:
-	return (EINVAL);
-}
-
-/*
- * Update the given IPv6 "sticky options" structure to contain the provided
- * label, which is encoded as an IPv6 option.  Existing label is removed if
- * necessary, and storage is allocated/freed/resized.
- *
- * Returns 0 on success, errno on failure.
- */
-int
-tsol_update_sticky(ip6_pkt_t *ipp, uint_t *labellen, const uchar_t *labelopt)
-{
-	int rawlen, optlen, newlen;
-	uchar_t *newopts;
-
-	/*
-	 * rawlen is the size of the IPv6 label to be inserted from labelopt.
-	 * optlen is the total length of that option, including any necessary
-	 * headers and padding.  newlen is the new size of the total hop-by-hop
-	 * options buffer, including user options.
-	 */
-	ASSERT(*labellen <= ipp->ipp_hopoptslen);
-	ASSERT((ipp->ipp_hopopts == NULL && ipp->ipp_hopoptslen == 0) ||
-	    (ipp->ipp_hopopts != NULL && ipp->ipp_hopoptslen != 0));
-
-	if ((rawlen = labelopt[1]) != 0) {
-		rawlen += 2;	/* add in header size */
-		optlen = (2 + rawlen + 7) & ~7;
-	} else {
-		optlen = 0;
-	}
-	newlen = ipp->ipp_hopoptslen + optlen - *labellen;
-	if (newlen == 0 && ipp->ipp_hopopts != NULL) {
-		/* Deleting all existing hop-by-hop options */
-		kmem_free(ipp->ipp_hopopts, ipp->ipp_hopoptslen);
-		ipp->ipp_hopopts = NULL;
-		ipp->ipp_fields &= ~IPPF_HOPOPTS;
-	} else if (optlen != *labellen) {
-		/* If the label not same size as last time, then reallocate */
-		if (newlen > IP6_MAX_OPT_LENGTH)
-			return (EHOSTUNREACH);
-		newopts = kmem_alloc(newlen, KM_NOSLEEP);
-		if (newopts == NULL)
-			return (ENOMEM);
-		/*
-		 * If the user has hop-by-hop stickyoptions set, then copy his
-		 * options in after the security label.
-		 */
-		if (ipp->ipp_hopoptslen > *labellen) {
-			bcopy(ipp->ipp_hopopts + *labellen, newopts + optlen,
-			    ipp->ipp_hopoptslen - *labellen);
-			/*
-			 * Stomp out any header gunk here - this was the
-			 * previous next-header and option length field.
-			 */
-			newopts[optlen] = IP6OPT_PADN;
-			newopts[optlen + 1] = 0;
-		}
-		if (ipp->ipp_hopopts != NULL)
-			kmem_free(ipp->ipp_hopopts, ipp->ipp_hopoptslen);
-		ipp->ipp_hopopts = (ip6_hbh_t *)newopts;
-	}
-	ipp->ipp_hopoptslen = newlen;
-	*labellen = optlen;
-
-	newopts = (uchar_t *)ipp->ipp_hopopts;
-
-	/* If there are any options, then fix up reported length */
-	if (newlen > 0) {
-		newopts[1] = (newlen + 7) / 8 - 1;
-		ipp->ipp_fields |= IPPF_HOPOPTS;
-	}
-
-	/* If there's a label, then insert it now */
-	if (optlen > 0) {
-		/* skip next-header and length fields */
-		newopts += 2;
-		bcopy(labelopt, newopts, rawlen);
-		newopts += rawlen;
-		/* make sure padding comes out right */
-		optlen -= 2 + rawlen;
-		if (optlen == 1) {
-			newopts[0] = IP6OPT_PAD1;
-		} else if (optlen > 1) {
-			newopts[0] = IP6OPT_PADN;
-			optlen -=  2;
-			newopts[1] = optlen;
-			if (optlen > 0)
-				bzero(newopts + 2, optlen);
-		}
-	}
-	return (0);
-}
-
-int
-tsol_update_options(uchar_t **opts, uint_t *totlen, uint_t *labellen,
-    const uchar_t *labelopt)
-{
-	int optlen, newlen;
-	uchar_t *newopts;
-
-	optlen = (labelopt[IPOPT_OLEN] + 3) & ~3;
-	newlen = *totlen + optlen - *labellen;
-	if (optlen > *labellen) {
-		if (newlen > IP_MAX_OPT_LENGTH)
-			return (EHOSTUNREACH);
-		newopts = (uchar_t *)mi_alloc(newlen, BPRI_HI);
-		if (newopts == NULL)
-			return (ENOMEM);
-		if (*totlen > *labellen) {
-			bcopy(*opts + *labellen, newopts + optlen,
-			    *totlen - *labellen);
-		}
-		if (*opts != NULL)
-			mi_free((char *)*opts);
-		*opts = newopts;
-	} else if (optlen < *labellen) {
-		if (newlen == 0 && *opts != NULL) {
-			mi_free((char *)*opts);
-			*opts = NULL;
-		}
-		if (*totlen > *labellen) {
-			ovbcopy(*opts + *labellen, *opts + optlen,
-			    *totlen - *labellen);
-		}
-	}
-	*totlen = newlen;
-	*labellen = optlen;
-	if (optlen > 0) {
-		newopts = *opts;
-		bcopy(labelopt, newopts, optlen);
-		/* check if there are user-supplied options that follow */
-		if (optlen < newlen) {
-			/* compute amount of embedded alignment needed */
-			optlen -= newopts[IPOPT_OLEN];
-			newopts += newopts[IPOPT_OLEN];
-			while (--optlen >= 0)
-				*newopts++ = IPOPT_NOP;
-		} else if (optlen != newopts[IPOPT_OLEN]) {
-			/*
-			 * The label option is the only option and it is
-			 * not a multiple of 4 bytes.
-			 */
-			optlen -= newopts[IPOPT_OLEN];
-			newopts += newopts[IPOPT_OLEN];
-			while (--optlen >= 0)
-				*newopts++ = IPOPT_EOL;
-		}
+	if (effective_tsl != NULL) {
+		label_rele(effective_tsl);
+		*effective_tslp = NULL;
 	}
-	return (0);
-}
-
-/*
- * This does the bulk of the processing for setting IPPROTO_IP {T_,}IP_OPTIONS.
- */
-boolean_t
-tsol_option_set(uchar_t **opts, uint_t *optlen, uint_t labellen,
-    const uchar_t *useropts, uint_t userlen)
-{
-	int newlen;
-	uchar_t *newopts;
-
-	newlen = userlen + labellen;
-	if (newlen > *optlen) {
-		/* need more room */
-		newopts = (uchar_t *)mi_alloc(newlen, BPRI_HI);
-		if (newopts == NULL)
-			return (B_FALSE);
-		/*
-		 * The supplied *opts can't be NULL in this case,
-		 * since there's an existing label.
-		 */
-		if (labellen > 0)
-			bcopy(*opts, newopts, labellen);
-		if (*opts != NULL)
-			mi_free((char *)*opts);
-		*opts = newopts;
-	}
-
-	if (newlen == 0) {
-		/* special case -- no remaining IP options at all */
-		if (*opts != NULL) {
-			mi_free((char *)*opts);
-			*opts = NULL;
-		}
-	} else if (userlen > 0) {
-		/* merge in the user's options */
-		newopts = *opts;
-		if (labellen > 0) {
-			int extra = labellen - newopts[IPOPT_OLEN];
-
-			newopts += newopts[IPOPT_OLEN];
-			while (--extra >= 0)
-				*newopts++ = IPOPT_NOP;
-		}
-		bcopy(useropts, newopts, userlen);
-	}
-
-	*optlen = newlen;
-	return (B_TRUE);
+	return (EINVAL);
 }