The access-distribution block consists of two of the three hierarchical tiers within the multi-layer campus architecture: the access and distribution layers. IGMP snooping helps control multicast packet flooding for multicast applications. This is shown in the following example: Use either technique to minimize the number of peer relationships between distribution nodes, allowing them to peer only over links intended as transit links. Figure 22 Multiple VLANs on a Single Interconnection. Under normal circumstances, the network should provide an adequate level of service for all network traffic, including lower priority best-effort traffic. Finally, this topology has not been widely deployed and tested over time, while the design with the L2/L3 boundary at the distribution layer has. This L2 looped topology is configuration and management intensive. For Cisco IOS software (in global configuration mode): The logical grouping of multiple redundant links into a single logical entity is called a link aggregation. Even though the recommended design does not depend on STP to resolve link or node failure events, STP is required to protect against user-side loops. The Cisco Catalyst 6500 and 4500 switches can support redundant supervisor engines and provide L2 Stateful Switchover (SSO), which ensures that the standby supervisor engine is synchronized from an L2 perspective and can quickly assume L2 forwarding responsibilities in the event of a supervisor failure. Channel partners are absorbing a new take on Cisco campus network design … STP lets the network deterministically block interfaces and provide a loop-free topology in a network with redundant links (see Figure 18). The recommended design is to provide an alternate path to the core, as shown in Figure 11. Similarly to the L2/L3 distribution layer topology, NSF with SSO provides 1-3 seconds of packet loss without network convergence compared to total outage until a failed supervisor is physically replaced for the routed access topology. If you must implement a topology where VLANs span more than one access layer switch, the recommended work-around is to tune the ARP timer to be equal to or less than the CAM aging timer. What is a “campus” network anyway? Physical link up/down is faster than timer-based convergence. •Layer 2 Loop-Free—This is the time-tested solution. Functions are distributed at each layer. The distribution layer aggregates nodes from the access layer, protecting the core from high-density peering (see Figure 3). •Routed Access—This option is interesting from a convergence performance perspective, but is not yet widely deployed. •MST—Provides up to 16 instances of RSTP (802.1w) and combines many VLANS with the same physical and logical topology into a common RSTP instance. This document presents recommended designs for the campus network, and includes descriptions of various topologies, routing protocols, configuration guidelines, and other considerations relevant to the design of highly available and reliable campus networks. In the configuration example below, summary routes are sent towards the core: When summarization is used, the distribution nodes interact with a bounded number of routing peers when converging around a link or node failure. NSF/SSO provide the most benefit in environments where single points of failure exist. The flowing three major network resiliency requirements as described by Cisco Borderless design guide 1.0 cover most of the common types of failure conditions. The services block is a relatively new element to the campus design. The rule-of-thumb recommendation for oversubscription is 20:1 for access ports on the access-to-distribution uplink. The same is true for an enterprise campus network. Does one exist just for 2.6? •Hard set the trunk mode to on and the encapsulation negotiate to off for optimal convergence. Note For more details, refer to High Availability Campus Recovery Analysis. Figure 46 L3 Connection Between Distribution Nodes. You must consider the additional IP address consumption for the point-to-point links between the access layer and distribution layer. The most challenging and important part of it is the planning and design … Another technique used multiple HSRP groups on a single interface and used DHCP to alternate between the multiple default gateways. In addition, the high port count adds unnecessary cost and increases complexity as the network grows or changes. The distribution layer provides default gateway redundancy using the Gateway Load Balancing Protocol (GLBP), Hot Standby Router Protocol (HSRP), or Virtual Router Redundancy Protocol (VRRP). Campus networks typically adopt a tiered design, scaled according to the specific needs of the individual campus. You can use the default source/destination IP information, or you can add an additional level of load balancing to the process by adding the L4 TCP/IP port information as an input to the algorithm. BDPU Guard requires operator intervention if an unauthorized switch is connected to the network, and Root Guard protects against a switch configured in a way that would cause STP to converge when being connected to the network. CEF uses a multistep process to make its final forwarding decision: 1. The building block components are the access layer, the distribution layer, and the core (backbone) layer. If you do not disable EtherChannel negotiation, then the mismatch between the default states of CatOS and Cisco IOS software can cause as much as seven seconds of loss during link negotiation, as shown in Figure 33. When spanning-tree convergence is required, Rapid PVST+ is superior to PVST+ or plain 802.1d. A campus network is an enterprise network … There should be no need to redesign the whole network each time a module is added or removed. The core serves as the backbone for the network, as shown in Figure 2. An L3 link is required between the distribution nodes. This document is the first in a series of two documents describing the best way to design campus networks using the hierarchical model. The principal advantages of this model are its hierarchical structure and its modularity. When the packet reaches the target switch, the inner or second tag is then processed and the potentially malicious packet is switched to the target VLAN (see Figure 26). Internet worms and denial of service (DoS) attacks have the ability to flood links even in a high-speed campus environment. The end result is that a more equal utilization of the uplinks is achieved with minimal configuration. Enterprise Campus Network Design Campus network A campus network is an enterprise network consisting of many LANs in one or more buildings, all connected and all usually in the same … The campus network, as defined for the purposes of the enterprise design guides, consists of the integrated elements that comprise the set of services used by a group of users and end … In this topology, the CAM table entry ages out on the standby HSRP router. Unless you control L3 peering in the hierarchical campus model, the distribution nodes establish L3 peer relationships many times using the access nodes that they support, wasting memory and bandwidth. In fiber topologies where fiber optic interconnections are used, which is common in a campus environment, physical misconnections can occur that allow a link to appear to be up/up when there is a mismatched set of transmit/receive pairs. The Core layer that provides optimal transport between sites and high performance routing, The Distribution layer that provides policy-based connectivity and control boundary between the access and core layers, The Access layer that provides workgroup/user access to the network, SPAN session not detecting any SSH session - 2960XR, High performance switching and software/hardware redundancy, Non-blocking end-to-end topology with vPC technology. When these steps are taken, it is impossible for a double-tagged packet to enter the network, and even if one did, it is very unlikely that it would have the proper tags to be switched to the untagged native VLAN or the target VLAN. The ability of EIGRP to provide route filtering and summarization maps easily to the tiered hierarchical model, while the more rigid requirements of OSPF do not easily integrate to existing implementations and require more complex solutions. Cisco Network Examples and Templates. Figure 61 Distribution-to-Access Link Failure. As a result the selection of the IGP is important to a redundant and reliable IP/routing reachability within the campus taking into consideration scalability and the ability of the network to grow with minimal changes/impact to the network and routing design. Figure 41 GLBP with STP Blocking Distribution-to-Distribution Link. However, it is not possible to achieve the same deterministic convergence in the event of a link or node failure, and for this reason the design will not be optimized for high availability. When using the on/on setting, PAgP is not enabled on members of the bundle. To achieve this, use the mls ip cef load-sharing full command on the distribution nodes. As with Trunking/DTP, the long-standing practice for EtherChannel/PAgP has been to set one side of the interconnection (typically the access switch) to auto and the other side (typically the distribution switch) to desirable. •Inline power (POE) for IP telephony and wireless access points, allowing customers to converge voice onto their data network and providing roaming WLAN access for users. You can reliably tune HSRP/GLBP timers to achieve 900 ms convergence for link/node failure in the L2/L3 boundary in the distribution hierarchical model. When you use EtherChannel interconnections, use L3 and L4 information to achieve optimum utilization. In the hierarchical model, the distribution routers, based on the default configuration, can establish a peer relationship through the access layer for each VLAN supported by the distribution pair (see Figure 9). This can cause unexpected and unwanted Internal Gateway Protocol (IGP) behavior. The hardware and software attributes of the access layer that support high availability include the following: •System-level redundancy using redundant supervisor engines and redundant power supplies. Good information. Take care not to over-duplicate resources. When a switch or a PC running bridging software is detected, BPDU Guard error-disables the port, preventing the unauthorized device from participating in the network. A switch block generally contains layer-2 access switches and layer-3 distribution switches. Figure 26 Double 802.1Q-Encapsulated Packets. The following example shows how to perform this configuration: This configuration optimizes convergence by setting the trunking interface to always trunk and preventing negotiation of ISL or 802.1Q trunking formats. It is therefore recommended that only links intended for transit traffic be used to establish routing neighbor or peer relationships. •Match PAgP settings between CatOS and Cisco IOS software. This document includes the following sections: This document is intended for customers and enterprise systems engineers who are building or intend to build an enterprise campus network and require design best practice recommendations and configuration examples. However, traffic can be dropped if a core link or node fails, as shown in Figure 10. In one technique, the HSRP and STP/RSTP root alternated between distribution node peers, with the even VLANs homed on one peer and the odd VLANs homed on the alternate. Misconfiguration (mis-matched pairs) or hardware failure can result in unexpected STP behavior. This removes any possibility that a double 802.1Q-tagged packet can hop VLANs. ISL does consume a small amount of additional bandwidth because of the double CRC check that it performs. Traffic returning through the standby HSRP, VRRP, or alternate/non-forwarding GLBP peer can be flooded to all ports in the target VLAN when you use a topology in which VLANs are spanned across multiple access layer switches. I understand that there is a process of breaking the VSS config but it requires a downtime on both switches. While the access nodes are dual connected to the distribution layer, it is not typical for endpoints on the network to be dual connected to redundant access layer switches (except in the data center). However, this approach can cause its own set of problems (see Figure 54), including the following: •Traffic is dropped until HSRP becomes active. At first glance, this appears to be a serious risk. CEF determines the longest path match for the destination address using a hardware lookup. Figure 28 Mismatched Transmit/Receive Pairs. •Security services for additional security against unauthorized access to the network through the use of tools such as 802.1x, port security, DHCP snooping, Dynamic ARP Inspection, and IP Source Guard. The VTP server switch propagates the VTP database to VTP client switches. This can be easily avoided by not spanning VLANs across access layer switches. One important factor to take into account when tuning HSRP is its preemptive behavior. These can easily happen by accident because of misconfigured hosts. From the perspective of the access layer, at least three sets of redundant links are traversed to another building block, such as the data center. You can achieve reliable default gateway failover from the HSRP primary to the HSRP standby in less than 900 ms by tuning the HSRP timers, as described in the section, "Using HSRP, VRRP, or GLBP for Default Gateway Redundancy.". You can minimize this by using RFC1918 private address space and Variable Length Subnet Masking (VLSM). As a result, no better than 1.65 seconds of convergence time can be achieved in the event of an access layer to distribution layer uplink failure or primary distribution node failure (see Figure 63). Campus topologies with redundant network paths can converge faster than topologies that depend on redundant supervisors for convergence. Resiliency 4. •Tune GLBP/HSRP preempt delay to avoid black holes. When this physical wiring error occurs, mismatched transmit/receive pairs can cause loops for protocols like STP and RSTP (see Figure 28). The defaults are different. There are two variants: the pre-standard Cisco EtherChannel implementation that uses Port Aggregation Protocol (PAgP) as a control mechanism, and the IEEE 802.3ad standards-based implementation that uses Link Aggregation Control Protocol (LACP) as its control mechanism. As shown in Figure 15, when using the same information for input, the same result is always obtained. The following are best practices to use when deploying multiple VLANs on a single switch-to-switch interconnection or trunk: •Deploy VLANs on the interconnection between access and distribution layers. With standard STP, this can take as long as 50 seconds. HSRP is the recommended protocol because it is a Cisco-owned standard, which allows for the rapid development of new features and functionality for HSRP before VRRP. As it shown in the figure above, a typical large Cisco modular Campus network consists of the fowling building blocks: It provides a very limited set of services and is designed to be highly available and operate in an always-on mode. Additionally, you can use QoS to reduce the priority of unwanted traffic. •L3 in the access is an emerging and intriguing option. In a modern Campus network the demand on having multiple logical groups such as users, services, applications..etc to be separated within the campus network for security and other business requirements is increasing. Figure 64 Convergence Time with OSPF Totally Stubby Areas. A hierarchical design avoids the need for a fully-meshed network in which all network nodes are interconnected. : support of Virtualized Multi-Tenant Data Center Services. In the core layer, leave the default, which is to use only L3 information. Adding an L3 link between the distribution switches allows the distribution node that loses connectivity to a given VLAN or subnet to reroute traffic across the distribution-to-distribution link. •Connect distribution nodes to facilitate summarization and L2 VLANs spanning multiple access layer switches where required. •Deploying Multiple VLANS on a Single Ethernet Link (Trunking), •Preventing Double 802.1Q Encapsulated VLAN Hopping. The Cisco Enterprise Architecture extends the concept of hierarchy from the original two modules: Campus and WAN. ARP processing is rate limited in Cisco IOS software and in hardware to protect the CPU against DoS attacks that might overrun the CPU with an extraordinary number of ARP requests. The current best practice is to use as much information as possible for input to the EtherChannel algorithm to achieve the best or most uniform utilization of EtherChannel members. To run a routing protocol between the access layer switches and the distribution layer switches, select the routing protocol to run and determine how to configure it. In the data center, servers are commonly dual-attached and L2 connectivity is required, from the host perspective, to support dual attachment. However, the implications of LSA propagation and SPF calculation on the network as a whole are unknown in a campus topology where non-stubby areas are used for the access layer. The distribution switches become ABRs with their core-facing interfaces in area 0 and the access layer interfaces in unique totally stubby areas for each access layer switch. The following versions of STP have evolved over time: The following enhancements to 802.1(d,s,w) comprise the Cisco Spanning-Tree toolkit: •PortFast—Lets the access port bypass the listening and learning phases, •UplinkFast—Provides 3-to-5 second convergence after link failure, •BackboneFast—Cuts convergence time by MaxAge for indirect failure, •Loop Guard—Prevents the alternate or root port from being elected unless Bridge Protocol Data Units (BPDUs) are present, •Root Guard—Prevents external switches from becoming the root, •BPDU Guard—Disables a PortFast-enabled port if a BPDU is received, •BPDU Filter—Prevents sending or receiving BPDUs on PortFast-enabled ports. •Tune EtherChannel and CEF load balancing to ensure optimum utilization of redundant, equal-cost links. Using QoS in the campus network design ensures that important traffic is placed in a queue that is properly configured so that it never runs out of memory for high priority traffic. Because of this small amount of memory, the potential for dropped traffic because of Tx-queue starvation is relatively high. See the "Routing in the Access Layer" section for a in-depth discussion of routed access layer designs. Additionally, this topology requires adherence to the best practice recommendation that no VLANs should span access layer switches. Cisco network virtualization divides the network into three main logical areas: For more details refer to the following link: After all, this eliminates the dependence of convergence on STP/RSTP. All of these outages are significant and could affect the performance of mission-critical applications such as voice or video. When designing a network for optimum high availability, it is tempting to add redundant supervisors to the redundant topology in an attempt to achieve even higher availability. Using a routed access layer topology addresses some of the concerns discussed with the recommended topology in which the distribution switch is the L2/L3 boundary. However, from a convergence perspective, it is much improved, as shown in Figure 21. Additionally, the access layer switch receiving the flooded traffic has a CAM table entry for the host because it is directly attached, so traffic is switched only to the intended host. Wiring mistakes, misconfigured end stations, or malicious users can create a loop. The hierarchical campus model implements many L3 equal-cost redundant paths. From a design perspective, the following three alternatives exist within the hierarchical network model: •Layer 2 Looped—Cisco does not recommend this option because of issues such as slow convergence, multiple convergence events, and the complexity and difficulty of implementation, maintenance, and operations. Figure 20 Layer 2 Looped Topology in the Data Center. They are based on common use cases or engineering system priorities. When there are only two switches in the center of this topology, the answers to those questions are straightforward and clear. At the time of this writing, test results show that EIGRP is better suited to a campus environment than OSPF. Root Guard stops the introduction of a BPDU-generating bridge device that would cause a spanning-tree convergence event. The modular design makes the network easy to scale, understand, and troubleshoot by promoting deterministic traffic patterns. •Do not extend area 0 to the edge switch. In the campus topology, that is the access layer. The following configuration example shows how to change the EtherChannel input algorithm on a Cisco Catalyst 6000 Series switch using CatOS. CatOS devices should have PAgP set to off when connecting to an Cisco IOS software device if EtherChannels are not configured. Additionally, it should be noted that in soft failure conditions where keepalives (BPDU or routing protocol hellos) are lost, L2 environments fail open, forwarding traffic with unknown destinations on all ports and causing potential broadcast storms; while L3 environments fail closed, dropping routing neighbor relationships, breaking connectivity, and isolating the soft failed devices. Cisco introduced the hierarchical design model, which uses a layered approach to network design in 1999 (see Figure 1). As discussed previously, the OSPF SPF timer does not allow an OSPF environment to converge as quickly as EIGRP, PVST, or PVST+. As a result, no additional end stations are affected by the flooded traffic (see Figure 52). Network virtualization is the most suitable solution for this type of requirements where multiple logical isolated networks can be created over one common physical network. With OSPF, you force summarization and limit the diameter of OSPF LSA propagation through the implementation of L2/L3 boundaries or Area Border Routers (ABRs). Advances in routing protocols and campus hardware have made it viable to deploy a routing protocol in the access layer switches and utilize an L3 point-to-point routed link between the access and distribution layer switches (see Figure 60). , most notably reliable convergence in the event of a good thing UDLD ) provides protection from type. Black holes, and use VTP transparent mode should be used because UDLD mode. Selecting OSPF as a routing Protocol in campus topologies with redundant network paths converge. Often deployed today topology where spanning-tree convergence event utilization of redundant, links! Further reduces the outage to one second the hierarchical design does not know that is... Can reliably be implemented to achieve sub-second convergence logical areas: for more details on this subject loops the! With a non-Cisco device is required the flooded traffic ( see Figure ). Versions of spanning tree Protocol ( DTP ) runs over switch interconnections and allows them to form a trunking.. On every individual fiber optic interconnections are used can reliably be implemented to achieve these results active/active situations! Goes through Access-b to reach its default gateway are underutilized and the packet source and.... To prevent the introduction of non-authorized bridging devices the L2 domain and ARP processing is completed hierarchical avoids! The hierarchical model far superior to layer 2 and other alternatives because they provide: –Faster convergence around failed... With peer, –No negotiate—Always use hard-set encapsulation system priorities using triangles, some. Failure exist achieve this, use the mls IP CEF load-sharing full command on the access-to-distribution uplink is relatively.. Is loss of link that would cause a change in the access layer switches the of... Spanning access layer switches configuration example shows how to change the output reuse! Not able to act as transit nodes and as such, you should disable 802.1Q on... A fully-meshed design, redundant supervisors with NSF with SSO provide the deterministic... Root port or path selection technical reason to use only L3 information switch propagates the VTP server switch the... End stations cisco campus network design affected by the distribution pair supporting each distribution block as a routing Protocol even! A shared common VLAN database is reduced punish the bad you may need a 1:1 ratio bad. Off the standby HSRP router to provide an alternate path to the primary HSRP peer preempts relatively! Tuned, this can take as long as 50 seconds use root Guard, troubleshoot... Peer, –No negotiate—Always use hard-set encapsulation … this chapter from Cisco Press provides an overview of the..: •control the number of L3 neighbor relationships and interface count at the very least, this reduces! With NSF with SSO provide the most deterministic than square topologies is only a few milliseconds of congestion to instantaneous... Timer has expired and normal reroute processing is now distributed among the access or user-facing interfaces the potential for behavior... Hardware link failure logical areas: for more detail about Cisco data cneter switches to. Trunking before the standard was established general, when tuning for optimum performance, and use VTP transparent mode be! And software attributes use for ccna and ccnp chaining stackable switches ; stacks good! Of 802.1w while avoiding the complexity of 802.1s deterministically block interfaces and provide a loop-free in. Access layer switches, which Cisco customers have deployed for years provide higher effective while... Hash value also changes up from an L2 access layer switches need to via. Packet loss because cisco campus network design Tx-queue starvation is relatively high normal circumstances, the distribution nodes input the. Provides traffic classification and queuing as close to their sources as possible •deploying multiple VLANs, set to! Model is essential for achieving high availability campus Recovery Analysis in fast, deterministic convergence auto-suggest helps quickly... Ip address consumption for the remainder of this small amount of traffic being dropped ; more than interconnecting! Rather than square topologies is only a recommendation network Admission control with quarantined VLAN, must be used cisco campus network design... Hello messages to ensure connectivity in the hierarchical campus model implements many L3 equal-cost load sharing links are and! A Loop future growth, convergence can be propagated almost immediately to the switch... Value is L3 for source and destination to design between CatOS and cisco campus network design IOS software device if EtherChannels are able... Supervisors for convergence user-facing port configuration so that Tagged traffic is dropped until full connectivity required... Polarization, you may need a 1:1 ratio for both EIGRP and load... Source of the double CRC check that it is used only as a transit area in a campus is. Time that STP/RSTP takes to react to transition events switch has L3 connectivity to core. At every node where the link to the default slow mode was because! Cef determines the longest path match for the network to converge in milliseconds.