Browsed by
Category: Cisco

CCIE Data: Lab Blueprint 1.1c Implementing Port Channels

CCIE Data: Lab Blueprint 1.1c Implementing Port Channels

CCIE Data Center Lab Blueprint

1.1c Implementing Port Channels


ConfigBytes #2

Port Channels

A port channel bundles physical links into a channel group to create a single logical link that provides the aggregate bandwidth of up to 16 physical links. If a member port within a port channel fails, the traffic previously carried over the failed link switches to the remaining member ports within the port channel.

  • F and M series line card port members cannot be mixed into a port-channel.
  • On a single switch, the port-channel compatibility parameters (SPEED,DUPLEX,ETC) must be the same among all the port-channel members on the physical switch.
  • Use port-channels for resiliency and aggregation of throughput.
  • 8 member links per port-channel prior to 5.1
  • NXOS 5.1> 16
  •  member links
  • L2 & L3 port-channels available on NXOS
  • Port-channel interface ID range 1-4096
  • Configuration changes made to logical port-channel interface is inherited by the individual member interfaces.
  • You can use static port channels, with no associated aggregation protocol, for a simplified configuration. For more flexibility, you can use LACP. When you use LACP, the link passes protocol packets. You cannot configure LACP on shared interfaces.
  • PAgP is NOT supported on NXOS
  • The port channel is operationally up when at least one of the member ports is up and that port’s status is channeling. The port channel is operationally down when all member ports are operationally down.
Note After a Layer 2/3 port becomes part of a port channel, all configurations must be done on the port channel; you can no longer apply configurations to individual port-channel members. you must apply the configuration to the entire port channel.

2015-04-06 08.14.44 am

Compatibility Requirements

When you add an interface to a channel group, the software checks certain interface attributes to ensure that the interface is compatible with the channel group. For example, you cannot add a Layer 3 interface to a Layer 2 channel group. The Cisco NX-OS software also checks a number of operational attributes for an interface before allowing that interface to participate in the port-channel aggregation.

The compatibility check includes the following operational attributes:

  • (Link) speed capability
  • Access VLAN
  • Allowed VLAN list
  • Check rate mode
  • Duplex capability
  • Duplex configuration
  • Flow-control capability
  • Flow-control configuration
  • Layer 3 ports—Cannot have subinterfaces
  • MTU size
  • Media type, either copper or fiber
  • Module Type
  • Network layer
  • Port mode
  • SPAN—Cannot be a SPAN source or a destination port
  • Speed configuration
  • Storm control
  • Tagged or untagged
  • Trunk native VLAN

Use the show port-channel compatibility-parameters command to see the full list of compatibility checks that the Cisco NX-OS uses.


You can only add interfaces configured with the channel mode set to on to static port channels, and you can only add interfaces configured with the channel mode as active or passive to port channels that are running LACP. You can configure these attributes on an individual member port. If you configure a member port with an incompatible attribute, the software suspends that port in the port channel.


Alternatively, you can force ports with incompatible parameters to join the port channel if the following parameters are the same:

  • (Link) speed capability
  • Speed configuration
  • Duplex capability
  • Duplex configuration
  • Flow-control capability
  • Flow-control configuration


Port Channel Load Balancing

  • Port channels provide load balancing by default
  • Port-channel load balancing uses L2 (MAC), L3 (IP), or L4 (port) to select the link
  • SRC or DST or both SRC and DST
  • Per switch (global) or per module. Per module takes precedence over per switch
  • L3 default is SRC/DST IP address
  • L2/non-IP default is SRC/DST MAC address
  • 6.0(1) for F series line card L2 load balancing
  • Must be in the default VDC to configure

You can configure load balancing either by the entire system or by specific modules, regardless of the VDC. The port-channel loadbalancing is a global setting across all VDCs.

If the ingress traffic is Multiprotocol Label Switching (MPLS) traffic, the software looks under the labels for the IP address on the packet.

The load-balancing algorithms that use port channels do not apply to multicast traffic. Regardless of the load-balancing algorithm you have configured, multicast traffic uses the following methods for load balancing with port channels:

  • Multicast traffic with Layer 4 information—Source IP address, source port, destination IP address, destination port
  • Multicast traffic without Layer 4 information—Source IP address, destination IP address
  • Non-IP multicast traffic—Source MAC address, destination MAC address
Note Devices that run Cisco IOS can optimize the behavior of the member ports. ASICs if a failure of a single member occurred if you enter the port-channel hash-distribution command. The Cisco Nexus 7000 Series device performs this optimization by default and does not require or support this command.

Cisco NX-OS Release 6.1(3) supports a new Result Bundle Hash (RBH) mode to improve load balancing on port-channel members on Cisco Nexus 7000 M Series I/O XL modules and on F Series modules. With the new RBH modulo mode, the RBH result is based on the actual count of port-channel members.



2015-04-06 08.15.47 am



  • Feature disabled by default. Must be enable feature first
  • Up to 16 active interfaces with 5.1>
  • Active 8, 8 Standby before 5.1
  •  Modes are active, passive, or ON (static port-channel NO LACP)
  • ON mode or static port channels is the DEFAULT mode

Both the passive and active modes allow LACP to negotiate between ports to determine if they can form a port channel based on criteria such as the port speed and the trunking state.


The passive mode is useful when you do not know whether the remote system, or partner, supports LACP.


Ports can form an LACP port channel when they are in different LACP modes if the modes are compatible as in the following examples:


  • A port in active mode can form a port channel successfully with another port that is in active mode.
  • A port in active mode can form a port channel with another port in passive mode.
  • A port in passive mode cannot form a port channel with another port that is also in passive mode, because neither port will initiate negotiation.
  • A port in on mode is not running LACP and cannot form a port channel with another port that is in active or passive mode.


LACP System ID is the combination of the LACP System Priority and MAC Address. Value of system priority is 1-32,768. Lower priority value = higher system priority. 1 being the highest priority.


Port Priority values are from 1-65535. Port priority + port number (interface ID) = LACP Port ID

Lower PortID value = higher priority to be chosen for forwarding/active vs. standby links. Default port priority is 32,768


Prerequisites for Port Channeling

Port channeling has the following prerequisites:

  • You must be logged onto the device.
  • If necessary, install the Advanced Services license and enter the desired VDC.
  • All ports in the channel group must be in the same VDC.
  • All ports for a single port channel must be either Layer 2 or Layer 3 ports.
  • All ports for a single port channel must meet the compatibility requirements. See the “Compatibility Requirements” section for more information about the compatibility requirements.
  • You must configure load balancing from the default VDC.

Guidelines and Limitations

Port channeling has the following configuration guidelines and limitations:

  • The LACP port-channel minimum links and maxbundle feature is not supported for host interface port channels.
  • You must enable LACP before you can use that feature.
  • You can configure multiple port channels on a device.
  • Do not put shared and dedicated ports into the same port channel. (See “Configuring Basic Interface Parameters,” for information about shared and dedicated ports.)
  • For Layer 2 port channels, ports with different STP port path costs can form a port channel if they are compatibly configured with each other. See the “Compatibility Requirements” section for more information about the compatibility requirements.
  • In STP, the port-channel cost is based on the aggregated bandwidth of the port members.
  • After you configure a port channel, the configuration that you apply to the port channel interface affects the port channel member ports. The configuration that you apply to the member ports affects only the member port where you apply the configuration.
  • LACP does not support half-duplex mode. Half-duplex ports in LACP port channels are put in the suspended state.
  • You must remove the port-security information from a port before you can add that port to a port channel. Similarly, you cannot apply the port-security configuration to a port that is a member of a channel group.
  • Do not configure ports that belong to a port channel group as private VLAN ports. While a port is part of the private VLAN configuration, the port channel configuration becomes inactive.
  • Channel member ports cannot be a source or destination SPAN port.
  • You cannot configure the ports from an F1 and an M1 series linecard in the same port channel because the ports will fail to meet the compatibility requirements.
  • You cannot configure the ports from an M1 and M2 series linecard in the same port channel.
  • You cannot configure the ports from an F2e and an F3 series linecard in the same port channel because the ports will fail to meet the compatibility requirements.
  • Beginning with Cisco NX-OS Release 5.1, you can bundle up to 16 active links into a port channel on the F1 series linecard.
  • F1 Series modules do not support load balancing of non-IP traffic based on a MAC address. If ports on an F1 Series module are used in a port channel and non-IP traffic is sent over the port channel, Layer 2 traffic might get out of order.
  • Only F Series and the XL type of M Series modules support the RBH modulo mode.


Feature History for Configuring Port Channels

Feature Name Release Feature Information
Display policy errors on interfaces and VLANs 6.2(2) Added the show interface status error policy command.
Prevent traffic-drop during bi-directional flow on F2 or F2e modules 6.2(2) Added the asymmetric keyword to port-channel load-balance command to improve load balancing across port channels.
Result Bundle Hash load balancing 6.1(3) Support for the RBH modulo mode to improve load balancing across port channels.
Minimum links for FEX fabric port channel 6.1(3) This feature was introduced.
Port channels hash distribution 6.1(1) Support for port channel hash distribution fixed and adaptive mode.
Load-balancing supports F2 modules 6.0(1) Added support for F2 modules on load-balancing across port channels.
Port channels 5.2(1) Support increased to 528 port channels.
Minimum links and Maxbundle for LACP 5.1(1) This feature was introduced.
Port channels 4.2(1) Support increased to 256 port channels.
Port channels 4.0(1) This feature was introduced.


Example Lab Question and Configuration


Port Channel Task

Assuming that more links will be added later, with the desire for minimal traffic disruption (LACP), configure the following:

Configure trunking on port channel 100 from N7K1 to UCS FI-A, and ensure that the same port channel number is used later from the UCS side.


interface Ethernet1/22


  switchport mode trunk

  switchport trunk allowed vlan 100,200,300,400,500

  channel-group 100 mode active (LACP)

  no shutdown


Configure trunking on port channel 200 from N7K1 to UCS FI-B, and ensure that the same port channel number is used later from the UCS side.


interface Ethernet1/24


  switchport mode trunk

  switchport trunk allowed vlan 100,200,300,400,500

  channel-group 200 mode active (LACP)

  no shutdown


Ensure that both of these port channels transition immediately to a state of

forwarding traffic.

“Int port-channel 100” & “Int port-channel 200”

“spanning-tree port type edge trunk”


Ensure that the N7K1 is the primary device in LACP negotiation. Ensure that the hashing algorithm takes L3 and L4 for both source and destination into account.

“lacp system-priority 1” Lower system priority value = higher priority


“port-channel load-balance src-dst ip-l4port”


Trunk only previously created VLANs 100,200,300,400,500 southbound from N7K1 to both FIs.


Verify with “Show port-channel summary”




Cisco Smart Install

Cisco Smart Install

This is my first post in a new series called “Config Bytes”.

My objective is simple. Take a technology that I’m working on with a customer and post the data points.


A global company asked me if there was an easy way to provision switches for rapid deployment. They are somewhat limited on networking personal and this would save the team some time if they could automate the staging of switches before deployment . The basic requirements were a standardized image depending on the platform and initial config for access switches. I had two viable solutions to match these requirements 1) Prime Infrastructure Plug & Play 2) Smart Install

2015-03-24 10.56.32 am

Smart Install:

Since the launch of the 3850/3650 access layer switches, we had slides that mentioned all the value add features of the Catalyst line. One of those bullet points was smart install and I remember this for the 3750x as well. At the end of 2014, we put out an updated configuration guide for smart install. I used this as a basis for design and configuration.

You can read up on all the details, but let me summarize a few key points.

  • Smart Install is a plug-and-play configuration and image-management feature that provides zero-touch deployment (ZTD) for new switches. You can ship a switch to a location, place it in the network and power it on with no configuration required on the device.
  • Two roles for the switch infrastructure “clients” & “director”
  • Director can be an multilayer switch or router
  • Clients connect to director and pull down image and config without any intervention (ZTD)
  • If a client switch was already deployed, you must “wr erase” and reload without a startup-config for smart install to work. Out of the box, no intervention required.
  • If using an L3 switch for director the smart install “vstack” VLAN must be up or the director can fallback to a client role. Just make sure the VLAN has at lease one access port up/up if using that SVI for the director.
  • TFTP and DHCP services are required, however they can co-reside on the director. This is how I configured it in the example inline.
  • Make sure your director device has plenty of flash memory to store the images and configs. If you have many different PIDs, your going to need more flash. I found that 2GB on the 3650/4500x was suffice for my customer.
  • Be patient while the image is loaded to the client. This process takes time (sometimes up to an hour).
  • I found that using the .tar format for the images worked the best. I’m not even sure if the .bin format is supported.
  • If you want to verify the supported clients on the director use this command “show stack group built-in ?”

Table A-1 Supported Switches

Switch  Can be Director?  Can be Client? 
Catalyst 6500 Supervisor Engine 2T-10GE Yes No
Catalyst 4500 Supervisor Engine, 6E, 6LE, 7E, 7LE Yes No
Catalyst 3850 Yes Yes
Catalyst 3750-X Yes Yes
Catalyst 3750-E Yes Yes
Catalyst 3750 Yes Yes
Catalyst 3650 Yes Yes
Catalyst 3560-X Yes Yes
Catalyst 3560-E Yes Yes
Catalyst 3560-C No Yes
Catalyst 3560 Yes Yes
Catalyst 2960-S No Yes
Catalyst 2960-SF No Yes
Catalyst 2960-C No Yes
Catalyst 2960-P No Yes
Catalyst 2960 No Yes
Catalyst 2975 No Yes
IE 2000 Yes Yes
IE 3000 Yes Yes
IE 3010 Yes Yes
SM-ES2 SKUs No Yes
SM-ES3 SKUs No Yes
NME-16ES-1G-P No Yes
SM-X-ES3 SKUs Yes Yes

Table A-2 Supported Routers 

Router  Can be Director?  Can be Client? 
Cisco 3900 Series Integrated Services Routers G2 Yes No
Cisco 2900 Series Integrated Services Routers G2 Yes No
Cisco 1900 Series Integrated Services Routers G2 Yes No
Cisco 3800 Series Integrated Services Routers Yes No
Cisco 2800 Series Integrated Services Routers Yes No
Cisco 1800 Series Integrated Services Routers Yes No

Table A-3 Minimum Software Releases for Directors and Clients

Directors  Minimum Software Release 
Catalyst 6500 Supervisor Engine 2T-10GE Cisco IOS Release 15.1(1)SY
Catalyst 4500 Supervisor Engine 7E and 7LE Cisco IOS Release XE 3.4SG
Catalyst 4500 Supervisor Engine 6K and 6LE Cisco IOS Release 15.1(2)SG
Catalyst 3850 Cisco IOS Release 3.2(0)SE
Catalyst 3650 Cisco IOS Release 3.3(0)SE
Cisco 3900, 2900, and 1900 Series Integrated Services Routers G2 Cisco IOS Release 15.1(3)T
Cisco 3800, 2800, and 1800 Series Integrated Services Routers Cisco IOS Release 15.1(3)T
Catalyst 3750-E, 3750, 3560-E, and 3560 Switches Cisco IOS Release 12.2(55)SE
Catalyst 3750-X and 3560-X Switches Cisco IOS Release 12.2(55)SE
SM-X-ES3 SKUs Cisco IOS Release 15.0(2)EJ

Table A-4 Minimum Software Releases for Clients

Smart-Install Capable Clients1 Minimum Software Release 
Catalyst 3750-E, 3750, 3560-E, and 3560 Switches Cisco IOS Release 12.2(52)SE
Catalyst 3750-X and 3560-X Switches Cisco IOS Release 12.2(53)SE2
Catalyst 3560-C Compact Switches Cisco IOS Release 12.2(55)EX
Catalyst 2960 and 2975 Switches Cisco IOS Release 12.2(52)SE
Catalyst 2960-S Switches Cisco IOS Release 12.2(53)SE1
Catalyst 2960-C Compact Switches Cisco IOS Release 12.2(55)EX1
Catalyst 2960-SF Cisco IOS Release 15.0(2)SE
Catalyst 2960- P Cisco IOS Release 15.2(2)SE
IE 2000 Cisco IOS Release 15.2(2)SE
IE 3000 Cisco IOS Release 15.2(2)SE
IE 3010 Cisco IOS Release 15.2(2)SE
SM-ES3 SKUs, NME-16ES-1G-P Cisco IOS Release 12.2(52)SE
SM-ES2 SKUs Cisco IOS Release 12.2(53)SE1
SM-X-ES3 SKUs Cisco IOS Release 15.0(2)EJ

2015-03-24 10.58.00 am

Configuration Example:

n3tArk_3850#sh run | s vstack

description SmartInstall_vstack_lan
description smart_install_vstack_mgmt
vstack group custom 2960c product-id
image flash:c2960c405-universalk9-tar.152-3.E.tar
config flash:smartinstall_config_2960c.txt
match WS-C2960C-12PC-L
vstack dhcp-localserver smart_install
vstack director
vstack basic

n3tArk_3850#sh run int vlan 1

interface Vlan1
description smart_install_vstack_mgmt
ip address

n3tArk_3850#sh run | s tftp

ip tftp source-interface Vlan777
tftp-server client_cfg.txt
tftp-server flash:smartinstall_config_2960c.txt
tftp-server flash:c2960c405-universalk9-tar.152-3.E.tar
tftp-server flash:2960c-imagelist.txt

n3tArk_3850#sh vstack status
SmartInstall: ENABLED

2015-03-24 10.43.20 am

n3tArk_3850#sh vstack download-status
SmartInstall: ENABLED

2015-03-24 10.44.18 am


That’s pretty much it! Here is a link to a YouTube video I created to show how easy this is to get up and running.

Hope this was helpful. Please let feedback/comments in the section if I missed any key points or you want me to elaborate more on something specific.












VIRL is HERE along with a new logo.

Dec 1st (aka Cyber Monday) brings us many good deals, including $50 off (virl50 at checkout) the $199 personal edition price.

If you have not seen my previous posts on CML, basically VIRL is the same as CML without TAC support and limited scale (15 nodes). If you don’t want to read through my previous posts, I’ll summarize inline.


What is VIRL?

VIRL enables users to rapidly design, configure and simulate network topologies. The VIRL virtualization framework provides a platform for high-fidelity network simulations that can be used for hands-on training, education, testing and development.

  • VIRL provides the ability to design network topologies with a GUI
  • VIRL Personal Edition provides IOSv, IOS XRv, CSR1000v and NX OSv!
  • You can integrate real network environments with your virtual network simulations


More information about VIRL

  1. VIRL website:
  2. VIRL Community Support:
  3. Pricing:
    • $199.99 for VIRL Personal Edition Annual Subscription License
    • $79.99 for VIRL Personal Edition Academic Version (students & teachers)  Annual Subscription License
  4. Other promos: First 25 purchasers of Personal Edition and the Academic Version will get free VIRL t-shirts


Verify that your PC or laptop meets the following minimum requirements:

• Host system must be able to access the Internet periodically

• Four CPU cores and 8GB of DRAM – more resources allows for larger simulations

• Intel VT-x / EPT or AMD-V / RVI virtualization extensions present and enabled in the BIOS

• 50GB of free disk space for installation

You must purchase and install one of the following supported Hypervisors in order to run Cisco VIRL.:

• VMware Fusion Pro v5.02 or later (including v6.x or v7.x)

• VMware Workstation v8.04 or later (including v9.x and 10.x)

• VMware Player v5.02 or later (including v6.x)

• ESXi 5.1 / 5.5 using the vSphere Client: ESXi 5.1U2 (Build 1483097) or ESXi 5.5U1 (Build 1623387)

These Hypervisors are not included as part of Cisco VIRL and must be purchased separately.

Cisco Modeling Labs 1.0: First Impressions & Getting Started

Cisco Modeling Labs 1.0: First Impressions & Getting Started

2014-09-16 01.30.37 pm

When a Legend becomes Real

I’m still pinching myself. Last week I delivered my very first Cisco Modeling Labs (CML) 1.0 demo to a customer. Overall, they were pretty darn excited, however there are some things that we need to address to make it a GREAT fit for their specific testing/validation environment.

Let’s take a step back and talk high level about CML for a moment. CML is the Cisco TAC supported variant of VIRL. The FCS date for CML 1.0 was 08.11.14. Almost a month later and several hours behind the wheel, I can say it was totally worth the wait. If your looking for a deep dive into the architecture behind the scenes, check out my previous blog post on the subject.

Let’s start with some of the most important aspects of CML to set expectations accordingly.

  • CML is NOT an emulator. The CML images are compiled specifically for the virtual machine environment (KVM). This is how you can scale to 150-200 nodes. It’s actual IOS/XR/XE/NX-OS code optimized for the VM. I was a huge fan of GNS/Dynamips, but the scale always left something to be desired. This is one of the major issues with emulation, PERFORMANCE.
  • CML WILL NOT validate ASICs, line cards, or any other hardware specific functionality/behavior. If your getting CML for this reason, it will NOT be a good representation.
  • CML is GREAT for config verification and migration/functionality testing. For example going from single IPv4 stack to dual stack, testing PfR configs, IGP configs, route policies, etc…
  • CML will also be GREAT for testing new code and features. The BU’s are committed to updating the CML images. For example: My IOSv image is 15.4(2)T1 which is pretty recent.  “IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.4(2)T1”
  • CML is GREAT for troubleshooting problems in an isolated environment.
  • CML allows you to integrate the virtual simulated environment with the physical lab network.
  • CML images available TODAY/09.16.14 are IOSv (included with your 15 node base license), IOS XR, and IOS XE in the form of CSR1000v. There is also a Linux server image for hosts.
  • CML team recommends UCS C220 M3 server or C460 M2, but you can really bring your own hardware for the host. ESXi 5.0, 5.1 or 5.5 is REQUIRED. Check out this URL for the data sheet and requirements. 
  • There is NO cloud/hosted offering of CML.
  • Be sure to check out the CML Q&A for anything I may have missed.

Craig Brown (TME): Cisco Modeling Labs Overview

Getting Started (see inline for ordering info)

  1. Download the install guide
  2. Setup your ESXi host
  3. Download the CML OVA
  4. Deploy the CML OVA
  5. Run through the “First Time” scripts on the Ubuntu guest
  6. Install the necessary license keys
  7. Add any additional images (IOS-XRv, CSR1000v, linux server)
  8. Download the CML client (OSX or Windows) from http://IP_OF_CML_SERVER/download
  9. Connect to the CML server
  10. Design, Build, Visualize, Simulate

This is a really just an overview. Your gonna want to go through the install guide and ensure your following the requirements and recommendations. I’ll be posting an instructional video on youtube shortly with a step-by-step guide on how to get started.


  • Only GigE virtual interfaces are supported currently. No serial interfaces or 10G/40G
  • Additional images (IOS-XRv, CSR1000v, etc) must be purchased separately. Only IOSv is included with the base license
  • Modeling of traffic patterns (traffic flow creation) are slated for the CML 1.1 release

Thoughts and Closing

In closing, I hope your as excited as I am about CML. It’s been a long time coming and I’m really glad the CML team took the time to get this right. I see many applications for CML in my personal journey. Let’s start with my home lab. I’m blessed to have access to Cisco hardware, but my lab gets HOT and my electric bill goes through the roof. I’ll use CML to validate customer configs, design and test IWAN/PfR configs, CCIE DC studies (NX-OSv image), EEM applet validation, and routing configs. Hopefully this saves me from the hundreds of dollars in electric to run a cat6500 and nexus 3k’s at home. 🙂

I used GNS3/Dynamips and IOU/IOL for many years. I will just say this, CML blows them away. I love GNS, but my problem has also been two fold. Scale and relevance. With regards to relevance, I was running the 7200 image and old IOS code. It’s just not current enough and emulated platforms suffer when it comes to performance. IOU/IOL is internal to Cisco only.

If your wondering about VIRL personal edition, my understanding is we’ll eventually release this to Cisco DEVNET. I just don’t have any committed date (update Dec 1st, 2014) at this point in time. This is going to be great for those studying for Cisco certifications from the CCNA to CCIE level.

If your interested in a 30 day trial of CML, reach out to your Cisco account team.

I hope you found this post informative and helpful. If you have any suggestions on how I can best demonstrate CML, please leave feedback. I’m going to talk to the CML team and see if they plan on conducting a WISP lab at Cisco Live next year. If not, I’ll be hosting one. It’s that good. EVERYONE needs to see it.

UPDATE: I’m told by one of the TME’s that CML will be demoed at Cisco Live, Cancun in Nov. 

Ordering Information

2014-09-16 02.13.06 pm

Next Wave of UCS Innovation

Next Wave of UCS Innovation

Today was a BIG day for us at Cisco. We announced our next wave of UCS products and continue building our data center innovation superhighway. Did we announce one product? NO! We announced four major UCS products today at #UCSGRANDSLAM and it was AWESOME! I knew about this stuff for months, but had to keep quite. As you can imagine, I was at the point of imploding because I just wanted to share this info with EVERYONE. Here is a quick recap of the UCS portfolio expansion announced today.

  • UCS Mini provides the full power of Cisco Unified Computing in a smaller, all-in-one solution that is simple, easy to manage, yet expandable. Great for IoT/IoElocal processing (Fog) and ROBO customers. 

2014-09-04 03.38.09 pm

  • UCS M-Series Modular Servers for Online Content Providers and Cloud Service Providers and for distributed applications in Industrial High Performance Computing (HPC) and Enterprise Grid. What about dedicated hosting and cloud services?


  • Cisco UCS C3160 Rack Server is a modular, capacity-optimized solution ideal for distributed data analytics, unstructured data repositories and media streaming and transcoding. I have one customer looking at this now for vSAN. 

2014-09-04 03.39.28 pm

  • Cisco M4 Generation UCS Rack and Blade Servers are armed with the latest processing power providing increased performance, efficiency and computing density. Intel Haswell architecture, E5 v3. 

2014-09-04 03.40.40 pm

All that said, I’m ecstatic about today’s announcement and can’t wait to hear from our customers on the challenges that can be overcome with these latest additions to the UCS family. I think about five short years ago when naysayers said Cisco had NO PLACE IN THE SERVER MARKET. They were WRONG! We are #1 in the US and #2 worldwide in the x86 blade server market. I’m confident we’ll be the #1 server vendor worldwide in no time at all.

UCS | Powering Applications at Every Scale

As soon as the video of today’s announcement is posted, I’ll link it here. Stay tuned!




Virtual Internet Routing Lab/Cisco Modeling Lab:

UPDATE (08.07.2014):

Cisco Modeling Labs 1.0 Corporate Edition
Available August 11, 2014

This is an excerpt from an email one of my colleagues received today.

“We are very excited to announce that Cisco Modeling Labs 1.0 Corporate Edition is expected to ship on Monday, August 11th (if this changes we’ll let you know).

As you know, Cisco Modeling Labs 1.0 Corporate Edition is a game changing product with powerful virtualization features that provide corporations and service providers around the world with agility, flexibility and cost savings.

Product information can be found at the following locations:

Thank you again for your patience and continued interest in Cisco Modeling Labs 1.0 Corporate Edition.

The CML Team”


I wanted to take a few moments and give an update on CML/VIRL. I have had many inquires from my clients about CML/VIRL, it just makes sense to summarize these conversations and post something for those of us that can’t wait to get our hands on the first customer shipping (FCS) of CML/VIRL. ***IMPORTANT*** CML is the TAC supported version of VIRL. Just keep this in mind when we get into the “When” section of this post.


First off what the heck is CML/VIRL? CML started off as a project called Virtual Internet Routing Lab (VIRL) and is a graphical front-end to virtualized networking devices. Hold the phone! That sounds like IOU/IOU-WEB or GNS/Dynamips… What makes CML/VIRL better? Glad you asked. With GNS3/Dynamips your emulating the ASIC/CP-CPU hardware and running the actual IOS image on that emulated node. Each node is emulated and in the case of GNS3/Dynamips your choices are limited to older hardware such as 2600 and 7200 series routers. The images supported are only the monolithic IOS images and not the newer IOS-XE images found on newer routers such as the ASR1K and 4451x. I have ALWAYS had GNS3/Dynamips on my laptops as a quick and dirty syntax validation tool and for small scenario recreates (BGP peering configs, ACL validation, etc…). It’s especially useful in offline environments like studying for the CCIE R&S on a coast-to-coast flight. But, alas all was not good. My hardware choices were limited and the topologies were small because if they grew too big, my MacBook would become a personal space heater. This brings me to my biggest complaint about Dynamips, performance. Because the complete hardware is emulated to accommodate the original IOS image, it’s as slow as a Smart FourTwo (slowest 0-60 car). With CLM/VIRL each node is a virtual appliance that runs images designed for x86. The hypervisor is KVM/QEMU, Openstack is the orchestration, VM maestro is the graphical topology edition tool, and AutoNetKit is used for network configuration and rendering. This allows your lab/topology to scale much better then Dynamips or IOU, have better performance, and introduce other appliances into the environment such as a linux jumpbox or any other appliances we decide to support (no commitments here, just theory). 2014-07-09 12.23.51 am   2014-07-09 12.42.18 am Now for a dose of sad news, L2 appliances are not supported in CML/VIRL. Perhaps in the future this may be different, but for now it’s L3 only. You do get the vswitch within KVM, so it’s not a total bust. In fact, this is a critical component as connecting your lab devices together and connecting the virtual world to the physical world. Also, no serial interface support, Ethernet only. Again, perhaps this will change down the road…


What virtual appliances will be supported? 1) IOS-XE: VM CSR1000V 2) IOS-XR: VM XRVR 3) NX-OS: VM vNXOS 4) IOS: VM vIOS 5) Servers/3rd party appliances Host OS is ubuntu server v12.04.2 2014-07-09 12.24.06 am


This is my number one question from clients. I personally first saw VIRL back at Cisco Live US 2013 in Orlando, FL. They had a demo setup just outside the WISP labs. That year I hosted my own WISP lab (Nexus 3548 Algoboost technology) and got to spend a decent amount of time playing with the beta and talking to the team. Keep this key factor in mind. There are two VIRL platforms.

1) Individual customers

2) Corporate customers

For individual customers the target is July 30th, 2014 TBD and will be available to ANY customer registered on Devnet. This is community supported. For more information on ‘Devnet’, please take a look at For corporate customers joining the “dev-innovate” program. VIRL will be included in the software bundle. For corporate customer looking for a TAC supported VIRL, this is the CML product and the target date is July 15th August 11th, 2014. ***Disclaimer*** Target dates are subject to change 


Because your tired of getting burns from your laptop after running a 14 node simulation in GNS/Dynamips or you don’t want to deal with getting the right image for IOU/IOL (Cisco employees of course) <GRIN> In all seriousness, I personally have been looking for something more realistic/serious for a test/dev environment. There are many times where customers ask for input on designs and I want to validate a theory via syntax before making a recommendation. I have done this for over 7 years with either real hardware (sometimes VERY expensive) or dynamips, albeit to a much smaller scale and with those limited node selection. My other major driver for VIRL is one of my customers is an ASR 9K shop and I don’t know IOS-XR that well. What better way to learn and save on my home lab electricity bill? Here are some of the “official” use cases.

  • Learn or provide training on new IOS versions or releases without the cost of purchasing, deploying, and maintaining expensive hardware
  • Stage and / or develop device configurations ahead of actual deployments
  • Test new software capabilities without impacting actual networks or hardware
  • Evaluate changes to network architectures or configurations – what-if scenarios
  • Trouble-shoot or diagnose control- or management-plane issues without scheduling network maintenance windows
  • Create and connect virtual instances of new hardware or solutions to real, existing networks to evaluate their impact, performance, or behavior

Each of these activities – prior to VIRL – required expensive hardware resources that were both static and costly.  VIRL on the other hand allows complete flexibility in terms of the architectures that can be created, limited only by the compute resources (which are significantly less costly than network hardware) that can be dedicated for use.


Deploy the Ubuntu VIRL OVA of course… Just kidding! While getting started really is that easy, getting everything setup and configured is a little more involved. There will also a bare metal installer. I’ll be updating this section shortly when more public information is available. The resource requirements are pretty high for laptop/individual deployments. For example XRv requires 1.3GB of RAM (real or swap). Keep this in mind as it will limit how many VMs can be run. Real world customer deployments will be based on properly sized UCS servers. Mac (OSX 10.7+) /PC (Windows 7) minimum requirements: 8-16GB RAM (4-8GB for VIRL) and 20GB of disk space 2014-07-09 12.41.25 am


VIRL enables customers to support many critical missions – designing, training, release-testing, configuration-staging, and others – without the expense of buying and staging real network hardware.  Networks of any complexity can be created and tested using the same software that will run on the real routing platforms.

The cost of network hardware required for training, testing, certification, pre-deployment, and other non-production activities can be a significant burden for customers – from the lone individual studying for CCIE to the largest of SPs.  VIRL, by providing the ability to deploy large, multi-OS virtual networks on comparatively inexpensive compute platforms, can significantly reduce both capital expenses and the expenses – both monetary and time-wise – associated with deploying hardware for non-production activities.

If you found this post to be helpful, please leave feedback.


2014-07-09 12.43.27 am

CCIE R/S v5: Everything’s Gonna be Alright

CCIE R/S v5: Everything’s Gonna be Alright


It’s been roughly five months since I passed the v4 CCIE R/S and I’m starting to hear potential CCIE R/S candidates freaking out about the upcoming changes. I know this feeling all too well, because like many of you I started on v3 and passed on v4. I will never sugar coat this, it’s a royal pain in the arse when the blueprint gets revised, especially if you have been studying (really studying) the current blueprint. That being said, I generally love what I see with the v5 blueprint and believe it’s best for the program to evolve and stay relevant. Let’s take a look at these changes and I’ll do my best to summarize what I know so far.

Six main pillars for written and lab.

1.0: Network Principles
2.0: L2 Technologies
3.0: L3 Technologies
4.0: VPN Technologies
5.0: Infrastructure Security
6.0: Infrastructure Services

Why do I like this? Well they really just consolidated the 11 topics from v4. Since we are not only dealing with MPLS VPN, but also IPSEC VPN/DMVPN it makes sense to group these together under “VPN technologies”. IPv4/IPv6/L3 multicast/routing protocols are now grouped under “L3 Technologies” and “Network Principles” is really only applicable to the written (in the “real world” this is a prerequisite) and completely new to the blueprint. “Infrastructure Security” will cover technologies such as router and switch security features, but also PKI/crypto. Finally, “Infrastructure Services”. Expect things like management, QoS, services, optimization, etc here. I would imagine since they are moving LAN QoS OFF the lab and into the written, things like SRR/WRR/RSVP are GONE. Thank goodness!!!

Not so bad right? Well let’s talk more about what was taken off the written and lab.

Topics Removed from the CCIE RS v4.0 Exam:

• Flexlink, ISL, Layer 2 Protocol Tunneling
• Frame-Relay (LFI, FR Traffic Shaping)
• IOS Firewall and IPS

If your like me your excited to see things like WCCP, IOS FW, and RSVP go bye bye. I did scratch my head on one of those subjects perhaps it’s because I spent so much time on the technology, Frame Relay. The only reason I say this is because many carriers are still using frame relay as the encapsulation for MPLS VPN solutions on TDM transport. This is going away, but it’s still out there in production just at a much lesser degree then 5-10 years ago. Still, I’m sure many are happy to see my good old friend Frame Relay put out to pasture.

Let’s continue with the subjects moved to the written, but removed from the lab.

Topics Moved from the CCIE RS v4.0 Lab exam to the CCIE RS v5.0 Written Exam:
• Describe IPv6 Multicast
• Describe RIPv6 (RIPng)
• Describe IPv6 Tunneling Techniques
• Describe Device Security using IOS AAA with TACACS+ and RADIUS
• Describe 802.1x
• Describe Layer 2 QoS
• Identify Performance Routing (PfR)

Oh man, there are so many on this list that I’m happy to see go to the written. Where do I begin… PfR!
PfR could be an exam of it’s own. If you don’t believe me go and configure a complex policy with multiple probes and get back to me with a verdict. Very happy to see 802.1x and v6 multicast move to the written as well. I kind of liked the ipv6 tunneling stuff on the lab, but that’s just me. GOODBYE RIPng, sorry nobody ever used you.

Now the fun begins. Here are topics that were added to the written, but not in the lab,

Topics Added to the CCIE Routing and Switching v5.0 Written Exam:
• Describe basic software architecture differences between IOS and IOS XE
• Identify Cisco Express Forwarding Concepts
• Explain General Network Challenges
• Explain IP, TCP and UDP Operations
• Describe Chassis Virtualization and Aggregation Technologies
• Explain PIM Snooping
• Describe WAN Rate-based Ethernet Circuits
• Describe BGP Fast Convergence Features
• ISIS (for IPv4 and IPv6)
• Describe Basic Layer 2 VPN – Wireline
• Describe Basic L2VPN – LAN Services
• Describe GET VPN
• Describe IPv6 Network Address Translation

ISIS? OK, so FabricPath and OTV leverage ISIS under the covers, but really adding it back to the written? Poor ISIS, your hot then your not, then your hot again. One word comes to my mind “pong”.
I like the IOS vs IOS XE, PIM snooping, switch virtualization, BDF/BGP, and CEF topics. GET VPN? Well, I guess someone is using this out there for it to be put on the written exam. Overall these additions to the written seem to allow Cisco to vet out candidates for the lab a little better and bring the curriculum a little more current.

“The decisions regarding which topics should be added, moved or retired were based on feedback received from key industry Subject Matter Experts (SME). These decisions reflect the evolution of the expectations of a candidate performing on the job role.”

I will agree with that quote. It’s from the learning@cisco exam update for v5.

Topics Added to the CCIE Routing and Switching v5.0 Written and Lab Exams:

• Use IOS Troubleshooting Tools
• Apply Troubleshooting Methodologies
• Interpret Packet Capture
• Implement and Troubleshoot Bidirectional Forwarding Detection
• Implement EIGRP (multi-address) Named Mode
• Implement, Troubleshoot and Optimize EIGRP and OSPF Convergence and Scalability
• Implement and Troubleshoot DMVPN (single hub)
• Implement and Troubleshoot IPsec with pre-shared key
• Implement and Troubleshoot IPv6 First Hop Security

Alright here is where things start to get interesting. For the most part these subjects make sense and don’t seem “crazy”. But, things like IPsec/DMVPN and IPv6 First Hop Security are going to raise some eyebrows. I cannot agree more with putting in DMVPN/IPSEC. For one thing it’s relevant and in almost all my clients networks. Second with Cisco pitching iWAN as transport independent (DMVPN), intelligent path control (PfR), optimizing (WAAS), and security (CWS) you can see where this is going.
Having expert level knowledge in VPN, BDF/BGP, troubleshooting, and packet capture interpretation is only going to make you a better engineer in the long run.

Finally, and I saved the best for last. The format of the lab. Please don’t shoot the messenger, but keep in mind that the CCIE is the most prestigious certification in our industry. It’s the top of the summit, the best of the best, unrivaled, CCIE is the #1.

In order to maintain that level of prestige the certification program for CCIE has to continue to evolve and mature. I remember thinking of quitting when v3 changed to v4, but then I pressed on and I’m so happy that I did because it was the hardest I have done from an education perspective, but it’s also the most rewarding.

OK, enough stalling. Here is the skinny. The lab format will consists of the following modules.

1: Troubleshooting (TS)
2: Diagnostic (DIAG)
3: Configuration (CFG)

CFG and TS are using virtual devices (IOU). This is nothing new for v4 TS, but for CFG this is new. The content delivery system will be similar to the web-based system for v4. My thoughts are that since the v4 TS supported large topologies, you may see this on v5 CFG now. More realistic, but also more overwhelming in my opinion. DIAG has no devices and I’ll get more into this later.

Here is the flow: TS(2hrs/variable) > DIAG (30 min/fixed) > CFG (5.5 hours/variable)

Now here is the cool part. Your TS section will allow up to 2.5 hours, but the extra 30 minutes will be deducted from the CFG section. If you spend less time on TS, you get that time in CFG. Pretty awesome as I always needed more time on TS myself.

2014-03-03 03.16.14 pm

Let’s talk turkey on this DIAG section as I’m sure many of you are curious like I was. “No devices” what gives man?

From Cisco directly…

“A new exam module called “diagnostic module” has been added and will focus on the skills required to properly diagnose network issues. The time for this new lab module is fixed to 30 minutes, no more or no less.”

Use this link for all the details on the DIAG module, but here is my summary. It’s a multiple-choice (drag and drop as well) section that will test your troubleshooting and analytic skills. It’s not open-ended questions (praise the Lord!!!), so there will be the “RIGHT” answers. Perhaps this is CEQ vs. OEQ on v4.
I’m not going to embellish here, this would be my biggest point of contention if I was taking the v5 lab for the same reasons I had with OEQ on v4. They are already testing you on theory (written) and troubleshooting (TS section), is this really necessary as it will be weighted on the overall grade. If you do terrible on this section, but passed TS and CFG you will not pass the lab. It’s extra stress that I don’t feel is necessary. Just my 2c that’s all. All that said, just like when I took the v4 and it had OEQ, you want to be the best? SUCK IT UP and DO IT!

2014-03-03 03.15.42 pm

Were almost at the home stretch. Let me summarize a few key things here.

The exam should be the ROUTING and switching exam because clearly L2 technologies are not as prevalent as L3. Perhaps moving CFG to IOU has something to do with it, but it’s clear that your going to want to really study up on L3 this time around. It’s nice from a focus perspective. Here is the breakdown which is awesome.

2014-03-03 03.23.36 pm
Exam Number: The exam number has changed from 350-001 to 400-101 and the written format is the same as v4.

Lab Gear: The recommendation for lab gear is ISR G2 2900 w/ 15.3T and 3560x with 15.0SE (IP Services). I don’t have a handle on how many are required. I think this may be a challenge because they are able to create large CFG topologies in IOU. Perhaps VIRL when it comes out.

Lab Dates: The CCIE Routing and Switching (R&S) Written and Lab exams are being revised from v4.0 to v5.0. The last day to test for both the Written and Lab v4.0 exams will be June 3, 2014. The CCIE R&S Written and Lab exams v5.0 will be available for testing on June 4, 2014.

Words of Encouragement: Sure things have changed and some of these changes are overwhelming right now, but I love the fact that routing is back in vogue and they removed some of the older “noise” from the exams. I never agreed with PfR on the CFG section and absolutely hated ZBF on v4 lab. GOOD RIDDANCE! INE and IPexpert already seemed to have good materiel in the works for v5 and #CLUS 2014 will have some practice labs for v5. Obtaining the CCIE is and always will be the pinnacle of my career certification goals. While the road was challenging and a little bumpy, I would not have it any other way. In fact, I’m gearing up for #2 with Data Center later this year.

If it was easy, everyone would be a CCIE. Just keep that in mind as you embark on your own journey and NEVER GIVE UP!

I leave you with a song I think was made for CCIE’s in training. ENJOY!

“It’s gonna take time, a whole lot of precious time, it’s going to take patience and time to do it right child.”
“It’s gonna take money, a whole lot of spending money, it’s going to take plenty of money, to do it right”
“And this time I know it’s for real, The feelings that I feel, I know if I put my mind to it, I know that I really can do it”

-George Harrison
Song: I got my mind set on you 

Man, that song was really made for CCIE candidates!


Follow me @Cisco_East

Follow me @Cisco_East

Just wanted to post something quick with reference to my twitter feed. I now use this as my primary means of communicating technology news.

CCIE #40755 (Routing & Switching)

CCIE #40755 (Routing & Switching)

“It’s gonna take time, a whole lot of precious time, it’s going to take patience and time to do it right child.”
“It’s gonna take money, a whole lot of spending money, it’s going to take plenty of money, to do it right”

-George Harrison
Song: I got my mind set on you 

I’m pretty sure George had the ladies on his mind and NOT the CCIE when he wrote that song. I can tell you no other lyrics resonate as strong as these when it comes to my personal journey of becoming inducted into the League of Extraordinary Engineers. Yes my friends, after 5+ LONG years, I’m officially in da club. My number is 40755 and oh boy does it feel AWESOME.

Because this journey was very difficult, I would go as far to say it’s the most difficult educational challenge I committed myself to, it’s only right that I share my story with other CCIE candidates to instill hope and encouragement. If it was easy, everyone would be a CCIE. Just keep that in mind as you embark on your own journey.

And so the story begins in 2008 when I passed the CCIE R&S written and only had a small window to take the v3 lab. This was sometime in september if I recall correctly. I was naive in thinking this is going to be cake, I mean how hard could this lab really be? I was thinking that I may only need 1-2 attempts, but I should have it done by the end of the year no problem. Well my first lab was v3 (lab guide printed on REAL paper in binder) and I actually did pretty good. My major issues were managing the clock and weakness on certain on security related services. Other than that it was a noble attempt. This gave me confidence and when I went to reschedule I realized something awful. The blueprint changed and there were no more seats left for the v3 lab. Now hear comes the madness, I was offered a “free” beta lab for the v4 which I accepted the challenge. Let’s just say that after taking the v4 beta, I was humbled in a the most extreme way. Now begins a radical format change (changes) to the lab. Open ended questions, troubleshooting, removal of open ended questions. I tried very hard to adapt to these changes, but as a poor test taker to begin with it was very challenging to say the least.

I was working at a small ISP in Central, PA at the time of this endeavor. God opened up a great door of opportunity in August of 2010 and I jumped in feet first… Where did I go??? CISCO!!!

While this major transition is occurring we’re also expecting our third child. I started on August 1st and Leo was born on August 28th. Man life was crazy and through all this I was sticking to my studies. I forget the details, but since my CCIE written was first passed in 2008, I had to take the written again before I could schedule another lab. I did this december of 2010 and would actually wait a full year before taking the v4 exam again. My third attempt was in Nov of 2011, this is where it gets interesting. I took the lab in San Jose instead of RTP this time. I flew out of Philadelphia airport and my laptop was stolen out of my checked in luggage. The TSA agent even left one of those “inspected by TSA” tickets in the bag. It was a surgical strike as only my laptop and power cable were removed from the bag. All my study notes were on that laptop… Needless to say, this was one heck of a trip. I did not pass, but did OK. The troubleshooting section was VERY tough.

Now pay attention because this is where I made the biggest mistake. I took almost a full year before my next attempt. NEVER DO THIS!!! If you can manage it, keep coming back every 30-60 days if possible. No more than 90 days. Things just got so busy between life and work that I waited yet ANOTHER year before diving back. By this time RTP had a new proctor (David) and let me tell you all this. He is by far my favorite proctor. David constantly encouraged me and drove me to keep coming back ASAP. With his recommendation and such a strong support system behind me I was able to pass after my 3rd consecutive attempt. It feels great to have my life back and know I can focus on the most important thing that was neglected… My family. While my wife and children supported me through this endeavor, there is no doubt that it took it’s toll on all of us. I could not have done this without the support of my family, friends, and colleagues. THANK YOU!!!

Passing lab experience:

September 28th, 2013

I drove down to RTP, NC from Central PA early Friday morning. My stomach was bothering me the night before probably due to nerves. I get so sick just thinking about the exam that I’m miserable every time I went to building 3. I get to RTP at about 3pm on Friday and ate a bland meal at Chipotle in Morrisville. I went back to the hotel room and practiced INE labs and reviewed my TS notes. My weak areas are still services because there are so many and being an expert in all of them is impossible (at least for me), but there are some that I take pride in my knowledge like EEM and multicast. Here’s the worst part. I could NOT sleep. I think I may of had 45min – 1hour, but that’s it. No matter what I tried I could not fall asleep. In addition, my stomach is a wreak. I drink half a bottle of pepto in hopes of relief. It did not come… Now for those of you who know me. I don’t drink or smoke. Heck eating some spicy foods is about as risky of a move that I make when it comes to what goes in my body. I NEVER drank anything like red bull or monster in my life. Those of you know know me would probably say that I’m wired to begin with. Why the heck would I even need something like that in the first place. Well this morning I did and my buddy John told me it helped him get through the lab the prior week before. So I drove to sheetz early in the morning and bought a red bull and start bucks energy drink. I settled on the Starbucks and drank the whole can. It was tasty, but what the heck is 80mg of caffeine going to do to me? I’ll tell you what it did. I became Bevis aka cornholio. I was so wired within 30 minutes of drinking that I forgot I was even tired. When I got to Building 3 we all went in and I began right away. Thanks to the power of caffeine, I was typing at like 150 WPM. Hit some major roadblocks in TS, but the energy infusion was too powerful an ally for TS to overcome. I felt good based on my results that Starbucks and I conquered TS. OK, well perhaps the Holy Spirit and me because there were some miraculous things that happened in the last 15-20 minutes.

I don’t even waste time, I jump right into configuration and heck I don’t think I even used the bathroom up to this point. No time for potty breaks. I get my configuration and my smile is ear to ear after reading though it. Let’s just say this, it was a test that jives with my skills. I felt good about the objective this config had set before me. I felt like I was running in auto pilot mode. My typing is loud and fast and I’m starting to feel bad because none of the other candidates were using ear plugs. I must have sounded like an old school author with his typewriter. By lunch I’m done with all L2/L3 and started on some of the services. Best time I had yet. Lunch is quick and I get back to it. By 1:30, I’m done with everything I could possibly configure. I take the next 45 minutes for verification, config backups, and reload. I’m pretty sure at a little after 2pm, I ended the lab. My heart was still racing, but something strange happened to my body. My guess is all the caffeine wore off as well as the adrenaline and I was crashing. I actually went into the break room and sat in the chair for a quick power nap. David stopped by and we talked a little about the lab. I felt really good about it and told him “If I don’t pass it this time, your might see a grown man crying”. To which he replies, “that’s nothing new”. Now comes the worst part… WAITING. I grab some food and head back to the hotel room. My intention was to eat and sleep, but again I could not fall asleep. My body and mind are a complete disaster. I’m waiting for this email with the results and it probably won’t be till tomorrow I find out if I did it. So, I do something that I have not really done in the last 5 years. Enjoy life’s simple pleasures. I go to the local movie theater and see Riddick. It was OK, but no pitch black. By this time you would think sleep was inevitable right? WRONG! I can’t sleep one wink. I get in the shower at 3:30am and check out of the hotel by 4am. I’m on the road heading back to PA. I keep checking my email every chance I get, still nothing. I stop in VA for some rest and decided to check my email. THIS IS IT! I have a message. The anticipation is killing me, do I even want to look at this now… I did and this is what I got!

  •  Your CCIE status is Certified ( CCIE# 40755 )
  • Your next CCIE Recertification due by September 28, 2015

I notify everyone via FB, Twitter, text, IM, calls, you name it. Then I crash in the car only to wake up at like 10am. My excitement level at this point is sky high. I can’t contain myself when talking to people on the phone. I’m thinking about all the things I wanted to do when I passed. Get a custom tag with my number, finally buy the pinball machine I have talked about for years, but the most important thing was this… Reconnect with my wife and family. When I reflected on my attitude, especially when studying for each lab attempt it was like I was a non-existent husband/father. So, it’s with great happiness and peace that I enjoy life again and return back home both physically and mentally.

In closing, I leave you candidates to be with the following wisdom.

1) Be prepared to make great sacrifices on this journey

2) Never give up

3) While it’s one of the most challenges journeys you can embark on, it’s also the most rewarding

4) Never give up

5)  Always keep in perspective that all your hard work will make you a better engineer regardless if you pass or not

6) Never give up

7) If you need a boost, drink some serious caffeine before taking the lab.


I want to again thank God, my family, friends, colleagues, INE, for the support and encouragement that was essential for my success. Oh! one more thing…

“And this time I know it’s for real, The feelings that I feel, I know if I put my mind to it, I know that I really can do it”

Man, that song was really made for CCIE candidates.


CCIE Studies: Performance Routing PfR/OER

CCIE Studies: Performance Routing PfR/OER


Hey fellow CCIE’s candidates and networking geeks. Today I want to step deep into the realm of PfR or Performance Routing. First let’s go back in time to the predecessor, Optimized Edge Routing or OER. As crazy as this sounds, OER came out in 2006 with IOS 12.3 . So, technically before all this SDN fanfare, Cisco actually decoupled the control (part of it at least) and data plane with OER/PfR back in the dizay.


OER/PfR was created to help with a major issue that plagues many mid-market customers even to this day, proper load sharing and/or balancing on the edge of the network. Who wants to have redundant Internet connections, possibly even with diverse providers and have one of those connection sit there idle until something blows up? The short answer, pretty much nobody. Your paying for that circuit, you should be using it. Well, Shaun why not just use BGP? Well that’s a great question! You sure could and advertise part of your networks off one connection and the remaining networks off the other connection. That would achieve a level of load sharing inbound to the enterprise. Traffic egressing out of the enterprise could also be split to share the two connections. Sometimes the issue with BGP peering is the complexity and requirements. When I worked at the SP, a class C (/24) was the longest prefix that you could advertise. I heard it’s now a /23, but that has not been confirmed. Working with ARIN for a direct assignment of two IPv4 /24’s will be an exercise in patience. Remember we are running out of IPv4 space, perhaps you could get some IPv6 block for half price… J/K All that said, it can be a pain in the you know what to make this happen and not all companies have the resources to manage that type of edge peering agreement with the providers.

Well that’s where OER/PfR comes into play. Let’s keep this simple because OER/PfR can be quite a deep subject. Rather than base forwarding decisions on destination and lowest cost metric, why not take a path’s characteristics into consideration such as jitter, delay, utilization, load distribution, packet loss/health, or even MOS score? That’s the power of OER/PfR!!!

This is right from

“PfR can also improve application availability by dynamically routing around network problems like black holes and brownouts that traditional IP routing may not detect. In addition, the intelligent load balancing capability of PfR can optimize path selection based on link use or circuit pricing.”

So, what did we do without BGP or OER/PfR? Typically, static routes with a floating static route for the redundant link using IP SLA/objecting for state monitoring (far end reachability). Again we are paying for something we can’t use. To quote Brian Dennis from INE. “It’s something we always accepted, like STP. You paying for something you can’t use”. The good news, you don’t need to live in that world any more. We have evolved with technologies like Fabric Path/TRILL, vPC, OER/PfR, SDN. Man, it’s a good time to be into networking!

Let’s think about some use cases: Internet connection load sharing/balancing, application specific traffic steering based on performance (latency), loss/delay sensitive hosted IP telephony traffic, leverage burstable based circuits, etc…

In summary, PfR allows the network to intelligently choose link resources as needed to reduce operational costs. Sounds like a sales pitch right? Well I am a Cisco SE after all, it’s in my DNA plus I found that diddy in one of the PfR FAQs.

OK, now that you have an good background on the origins of OER/PfR, let’s talk about the major difference between OER and PfR. In short, OER was destination prefix based and PfR expanded the capabilities to include route control on a per application basis.

Let’s also get one major thing out of the way first before we drill into the specifics. With a holistic view of the EDGE network your able to accomplish this level of traffic engineering on a per application level. If there is something wrong within the PfR network devices the traffic will FALL BACK to old school forwarding. Got that? No catastrophic failure where the routers are sticking their hands up screaming for help.


OK, let’s talk a little about the components required for a PfR edge network.

***IOS 15.1+ minimum recommended for production network***

Versioning: Major versions must match! If running 12.4(T) the version is 2.x. Is running IOS 15 the version is 3.x. It’s OK to have say a 2.1 and a 2.2, but not a 2.x and a 3.x version, this is NOT supported. 

Border Router (BR): In the data plane of the edge network, monitors prefixes and reports back to MC. 
Master Controller (MC):
 Centralized control plane for central processoring and database for statistics collection. 
1x Internal Interface-
BRs ONLY peer with each other over internal interfaces (directly connected or via tunnel). Also used between BR and MC.
2x External Interfaces- OER/PfR expects traffic to flow between internal and external interfaces.
Route Control: Parent Route REQUIRED! This explanation is right from the Cisco FAQ.

A parent route is a route that is equal to, or less specific than, the destination prefix of the traffic class being optimized by Performance Routing. The parent route should have a route through the Performance Routing external interfaces. All routes for the parent prefix are called parent routes. For Performance Routing to control a traffic class on a Performance Routing external interface, the parent route must exist on the Performance Routing external interface. BGP and Static routes qualify as Performance Routing parent routes. In Cisco IOS Release 12.4(24)T and later releases, any route in RIB, with an equal or less specific mask than the traffic class, will qualify as a parent route.

For any route that PfR modifies or controls (BGP, Static, PIRO, EIGRP, PBR), having a Parent prefix in the routing table eliminates the possibility of a routing loop occurring. This is naturally a good thing to prevent in routed networks.

Now, since I’m an active CCIE candidate I’m gonna say this, IOS 12.4(T) has bugs with PfR. For one, the command operative syntax is still “OER” and certain functionality just seems downright broken. My lab consists of real 3560’s and ISR routers, so it’s not like I’m using emulation/GNS/dynamips and that’s my issue. I cannot stress enough, if doing a POC in a non-PROD environment feel free to use IOS 12.4(T). In a “real world” production environment, never settle for less than 15.1. ASR 1K requires IOS XE 2.6 or higher for PfR support.

Hardware Platform Support: ISR G1(RIP), G2, ASR, 7600, Cat6500, and 7200’s (RIP)
Classic IOS Feature Set Required: SP Services/Advance IP/Enterprise/Advance Enterprise
Universal IOS Image: Data Package required


Pfr faq fig3.jpg

I was going to use a complex CCIE sample config, but there are so many good examples of PfR already on the Cisco PfR Wiki.

Instead, let me concentrate on the basic requirements starting with the border router.

BR Config: 

key chain PFR
 key 1
  key-string PFR

oer border
 local Loopback0
 master key-chain PFR

ip route Serial1/2 (PARENT ROUTE)
ip route Serial1/1 (PARENT ROUTE)

MC Config: 

oer master
border key-chain PFR
interface Serial1/2 external
interface Serial1/1 external
interface Serial1/0 internal
periodic-interval 0
monitor-period 1
mode route control
resolve utilization priority 1 variance 10
no resolve delay
no resolve range


Granted, this is the most basic form of route control, but it will inject a route for the monitored prefix based on interface throughput utilization. I believe the default is 75% utilized.

Here are some useful commands to monitor/troubleshoot PfR.

“show pfr/oer master”

Conn Status: SUCCESS, PORT: 3949
Version: 2.2
Number of Border routers: 1
Number of Exits: 2
Number of monitored prefixes: 1 (max 5000)
Max prefixes: total 5000 learn 2500
Prefix count: total 1, learn 1, cfg 0
PBR Requirements met
Nbar Status: Inactive

Border Status UP/DOWN AuthFail Version ACTIVE UP 03:29:17 0 2.2

Global Settings:
max-range-utilization percent 20 recv 0
mode route metric bgp local-pref 5000
mode route metric static tag 5000
trace probe delay 1000
exit holddown time 60 secs, time remaining 0

Default Policy Settings:
backoff 300 3000 300
delay relative 50
holddown 300
periodic 0
probe frequency 56
number of jitter probe packets 100
mode route control
mode monitor both
mode select-exit good
loss relative 10
jitter threshold 20
mos threshold 3.60 percent 30
unreachable relative 50
resolve utilization priority 1 variance 10

Learn Settings:
current state : STARTED
time remaining in current state : 115 seconds
no delay
no inside bgp
no protocol
monitor-period 1
periodic-interval 0
aggregation-type prefix-length 24
prefixes 100
expire after time 720

“show pfr/oer master border detail” 

Border Status UP/DOWN AuthFail Version8.8.8.8 ACTIVE UP 03:31:46 0 2.2

External Capacity Max BW BW Used Load Status Exit Id
Interface (kbps) (kbps) (kbps) (%)
——— ——– —— ——- ——- —— ——
Se1/2 Tx 1544 1158 0 0 UP 2
Rx 1544 0 0
Se1/1 Tx 1544 1158 0 0 UP 1
Rx 1544 0 0

“show ip cache flow”

IP packet size distribution (25713 total packets):
1-32 64 96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .040 .000 .200 .000 .001 .000 .000 .000 .000 .000 .000 .000 .000 .000

512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.003 .000 .007 .000 .743 .000 .000 .000 .000 .000 .000

IP Flow Switching Cache, 4456704 bytes
2 active, 65534 inactive, 1007 added
16475 ager polls, 0 flow alloc failures
Active flows timeout in 1 minutes
Inactive flows timeout in 15 seconds
IP Sub Flow Cache, 533256 bytes
2 active, 16382 inactive, 1151 added, 1007 added to flow
0 alloc failures, 0 force free
1 chunk, 1 chunk added
last clearing of statistics never
Protocol Total Flows Packets Bytes Packets Active(Sec) Idle(Sec)
——– Flows /Sec /Flow /Pkt /Sec /Flow /Flow
TCP-Telnet 10 0.0 256 144 0.1 19.5 6.9
TCP-other 59 0.0 68 110 0.2 9.0 2.3
ICMP 13 0.0 1470 1500 1.3 52.3 3.5
Total: 82 0.0 313 1146 1.8 17.1 3.1

SrcIf SrcIPaddress DstIf DstIPaddress Pr SrcP DstP Pkts

“show pfr/oer master traffic-class”

OER Prefix Statistics:
Pas – Passive, Act – Active, S – Short term, L – Long term, Dly – Delay (ms),
P – Percentage below threshold, Jit – Jitter (ms),
MOS – Mean Opinion Score
Los – Packet Loss (packets-per-million), Un – Unreachable (flows-per-million),
E – Egress, I – Ingress, Bw – Bandwidth (kbps), N – Not applicable
U – unknown, * – uncontrolled, + – control more specific, @ – active probe all
# – Prefix monitor mode is Special, & – Blackholed Prefix
% – Force Next-Hop, ^ – Prefix is denied

DstPrefix Appl_ID Dscp Prot SrcPort DstPort SrcPrefix
Flags State Time CurrBR CurrI/F Protocol
PasSDly PasLDly PasSUn PasLUn PasSLos PasLLos EBw IBw
ActSDly ActLDly ActSUn ActLUn ActSJit ActPMOS ActSLos ActLLos
——————————————————————————– N defa N N N N
U U 0 0 0 0 0 0
U U 0 0 N N N N

“show oer border routes static”

Flags: C – Controlled by oer, X – Path is excluded from control,
E – The control is exact, N – The control is non-exact

Flags Network Parent Tag
CE 5000


Well folks, that’s all the steam I have left after pouring out my heart on PfR/OER. I hope this post was informative. Please drop me a line if you have any questions or I was not clear on any of my points. I appreciate any and all feedback. In my mind, Cisco gave us a glimpse into the future of networking way back in 2006. With data center technologies evolving on a daily basis, it’s only a matter of time before there is an MC for the enterprise network rather than just the edge. Heck Google is doing that already with 25% of all the Internet traffic TODAY! Until next time, keep those blinky lights flashing.