Archive

Archive for June, 2012

OSPF: Anatomy of a routing protocol

June 24, 2012 5 comments

To become a CCIE you need detailed knowledge of OSPF. Most candidates will have read the TCP/IP Volumes by Doyle and maybe even the RFC written by J Moy.

This book is also written by J Moy and it is basically the bible on OSPF. The great thing about this book is that is like a less dry version of the RFC and it explains the design decisions in OSPF.

Why did they choose to make it link state instead of distance vector? RIP had issues with large networks and the large updates being sent and your network diameter was limited due to the hop count limit. Why did they choose to run it over IP instead of making it a link layer protocol or run it over UDP or TCP? This is also explained in the book.

Creating a protocol takes a lot of work and making it interoperable is a challenge. Moy describes how they did interoperability tests where the first test was hosted at Proteon. It was a common occurence to see developers from several companies leaning over a competitor’s shoulder, trying to fix a bug!

I’m still reading the book but it has been great so far. I you get a Safari subscription you can read all the books you like. I recently did and I can really recommend it.

Categories: CCIE, OSPF Tags: , ,

Cisco Live – Great resource for CCIE studies

June 20, 2012 2 comments

Hey guys,

I’m at home enjoying the time with my family which as you have seen has just grown with a new member 🙂 Cisco Live US was just held in San Diego and now the sessions have been uploaded. They used to cost money but now they are free! It’s a too good resource to not use it. It has sessions for routing&switching, NMS, voice, video, service provider and basically everything you can think of. It even has CCIE sessions with advice and what to expect at the lab. Go to www.ciscolive365.com and create an account and you will have access. You will also have access to earlier events.

It’s a girl!

June 15, 2012 13 comments

Hi everyone.

Just home to pack some things. Yesterday at 14.31 me and my fiancee got a beautiful baby girl weighing 3720 grams and 50 cm long. I’ll be back in a couple of days.

Categories: Announcement Tags:

More on SSM

June 9, 2012 4 comments

As you’ve noticed I’ve been studying SSM and what better way to learn than to blog about it. I recently got a Safari subscription which has been great so far. I’ve been reading some in the book Interdomain Multicast Routing: Practical Juniper Networks and Cisco Systems Solutions which has been great so far.

We are still using the same topology and now we will look a bit more in detail what is happening.

R1 will be the source, sending traffic from its loopback. R3 will be the client running IGMPv3 on its upstream interface to R2. As explained in previous post I am doing this to simulate an end host otherwise I would configure it on R3 downstream interface and then it would sen a PIM Join upstream.

To run SSM we need IGMPv3 or use some form of mapping as described in previous post. It is important to note though that IGMPv3 is not specific for SSM. With SSM a (S,G) pair is described as a channel. Instead of join/leave it is now called subscribe and unsubscribe.

So the first thing that happens is that the client (computer or STB) sends IGMPv3 membership report to the destination IP 224.0.0.22. This is the IP used for IGMPv3. This is how the packet looks in Wireshark.

The destination IP is 224.0.0.22 which corresponds to the multicast MAC 01-00-5E-00-00-16. 16 in hex is 22 in decimal.

We clearly see that it is version 3 and the type is Membership Report 0x22. Number of group records show how many groups are being joined.

Then the actual group record is shown (225.0.0.1) and the type is Allow New Sources. The number of sources is 1. And then we see the channel (S,G) that is joined.

Then R2 sends a PIM Join towards the source.

We can see that it is a (S,G) join. The SPT is built.

R2 will send general IGMPv3 queries to see if there are still any receivers connected to the LAN segment.

The query is sent to all multicast hosts (224.0.0.1) and if still receiving the multicast the host will reply with a report.

The type is Membership Query (0x11). The Max Response Time is 10 seconds which is the time that the host has to reply within.

We can see in this report that the record type is Mode is include (1) compared to Allow New Sources when the first report was sent.

Now R3 unsubscribes to the channel and the IGMP report is used once again.

The type is now Block Old Sources (6).

After this has been sent the IGMP querier (router) has to make sure that there are no other subscribers to the channel so it sends out a channel specific query.

If noone responds to this the router will send a PIM Prune upstream as can be seen here.

Finally. How can we see which router is the IGMP querier? Use the show ip igmp interface command.

R2#show ip igmp interface fa0/0
FastEthernet0/0 is up, line protocol is up
  Internet address is 23.23.23.2/24
  IGMP is enabled on interface
  Current IGMP host version is 3
  Current IGMP router version is 3
  IGMP query interval is 60 seconds
  IGMP querier timeout is 120 seconds
  IGMP max query response time is 10 seconds
  Last member query count is 2
  Last member query response interval is 1000 ms
  Inbound IGMP access group is not set
  IGMP activity: 2 joins, 1 leaves
  Multicast routing is enabled on interface
  Multicast TTL threshold is 0
  Multicast designated router (DR) is 23.23.23.2 (this system)
  IGMP querying router is 23.23.23.2 (this system)
  Multicast groups joined by this system (number of users):
      224.0.1.40(1)

We can see some interesting things here. We can see which router is the designated router and IGMP querier. By default the IGMP querier is the router with the lowest IP and the DR is the one with highest IP. DR can be affected by chancing the DR priority. We can also see which timers are used for query interval and max response time among other timers.

So by now you should have a good grasp of SSM. It does not have a lot of moving parts which is nice.

Categories: CCIE, Multicast Tags: , , ,

Multicast – SSM mapping

June 8, 2012 5 comments

This is a followup post to the first one on SSM. The topology is still the same.

If you want to find it in the documentation it is found in the IGMP configuration guide

I guess the reason to place it under IGMP is that SSM requires IGMPv3. To find SSM mapping go to Products-> Cisco IOS and NX-OS Software-> Cisco IOS-> Cisco IOS Software Release 12.4 Family-> Cisco IOS Software Releases 12.4T-> Configure-> Configuration Guides-> IP Multicast Configuration Guide Library, Cisco IOS Release 12.4T-> IP Multicast: IGMP Configuration Guide, Cisco IOS Release 12.4T-> SSM mapping

So why would we use SSM mapping in the first place? IGMPv3 is not supported everywhere yet. Maybe the Set Top Box (STB) is not supporting IGMPv3 but your ISP wants to support SSM. Then some transition mechanism must be used. There are a few options available like IGMPv3 lite, URD and SSM mapping. IGMPv3 lite is daemon running on the host supporting a subset of IGMPv3 until proper IGMPv3 has been implemented. With URD a router intercepts the URL requests from the user and the router joins the multicast stream to the correct source even though the user is not sending IGMPv3 reports. This requires that the multicast group and source is coded into the web page with links to the multicast streams.

SSM mapping takes IGMPv2 reports and convert them to IGMPv3. We can either use a DNS server and query it for sources or use static mappings as I will explain here. Static mapping is done on the Last Hop Router (LHR) and it is fairly simple. This is how we configure it.

R2(config)#access-list 2 permit 225.0.0.100
R2(config)#ip igmp ssm-map enable
R2(config)#ip igmp ssm-map static 2 1.1.1.1
R2(config)#no ip igmp ssm-map query dns

The config is pretty self explanatory. First we create an access-list that defines the groups to be used for SSM mapping. Then we enable SSM mapping. Then we tie together the ACL with the sources that are allowed to send to those groups. Now we need to verify the mapping. First we take a look at R2 with show ip igmp ssm-mapping.

R2#show ip igmp ssm-mapping
SSM Mapping  : Enabled
DNS Lookup   : Disabled
Mcast domain : in-addr.arpa
Name servers : 255.255.255.255

Looks good so far. We will use R3 to simulate a client joining to 225.0.0.100 via IGMPv2. We will debug IGMP to see the report coming in. R3 will join the group via the igmp join-group command. One thing is important to note here. Usually we configure ip igmp-join group on downstream interface to simulate LAN segment and then PIM Join is sent upstream. In this case we want only IGMP join to be sent so therefore we configure the igmp join-group on the upstream interface. Also no PIM should be enabled. This makes the router act as a pure host and not do any multicast routing. What would happen otherwise is that the router will have RPF failures when the source is sending traffic because for traffic not in SSM mode a RPF lookup is done against the RP. Since no RP is configured the RPF would fail, as a workaround we can configure a static RP even though it isn’t really used it would make the RPF check pass.

R3(config)int fa0/0
R3(config-if)#ip igmp join-group 225.0.0.100

This is the debug output from R3.

IGMP(0): Send v2 Report for 225.0.0.100 on FastEthernet0/0

We can clearly see that IGMPv2 report was sent. Now we go to R2 to see if it is converting the IGMPv2 join to IGMPv3.

IGMP(0): Received v2 Report on FastEthernet0/0 from 23.23.23.3 for 225.0.0.100
IGMP(0): Convert IGMPv2 report (*, 225.0.0.100) to IGMPv3 with 1 source(s) using STATIC

It is clear that the conversion is taking place. We look in the MRIB as well.

R2# 
sh ip mroute | be \(
(*, 224.0.1.40), 03:18:48/00:02:54, RP 0.0.0.0, flags: DCL
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    FastEthernet0/0, Forward/Sparse, 03:18:48/00:02:54
    Serial0/0, Forward/Sparse, 03:18:48/00:02:44

(1.1.1.1, 225.0.0.100), 03:18:26/00:02:57, flags: sTI
  Incoming interface: Serial0/0, RPF nbr 12.12.12.1
  Outgoing interface list:
    FastEthernet0/0, Forward/Sparse, 03:18:26/00:02:57

We see that we now have (S,G) joins in R2! As a final step we will also verify in R1.

sh ip mroute | be \(
(*, 224.0.1.40), 03:20:44/stopped, RP 0.0.0.0, flags: DCL
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    Serial0/1, Forward/Sparse, 03:20:44/00:00:49

(*, 225.0.0.100), 03:20:43/stopped, RP 0.0.0.0, flags: SP
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list: Null

(1.1.1.1, 225.0.0.100), 00:01:01/00:02:28, flags: T
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    Serial0/0, Forward/Sparse, 00:01:01/00:03:27

Now the ping should be successful.

R1#ping
Protocol [ip]:
Target IP address: 225.0.0.100
Repeat count [1]: 5
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Interface [All]: serial0/0
Time to live [255]:
Source address: 1.1.1.1
Type of service [0]:
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 225.0.0.100, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1

Reply to request 0 from 23.23.23.3, 16 ms
Reply to request 1 from 23.23.23.3, 16 ms
Reply to request 2 from 23.23.23.3, 16 ms
Reply to request 3 from 23.23.23.3, 16 ms
Reply to request 4 from 23.23.23.3, 16 ms

So the important thing here is to make R3 act as a pure host otherwise it will not work. This is a bit overkill for verification but I just wanted to show how it could be done.

Categories: CCIE, Multicast Tags: , , , ,

The history of Ethernet – DIX vs 802.3

June 6, 2012 14 comments

I’m planning to do a post on BPDUs sent by Cisco switches and analyze why they are sent. To fully understand the coming post first we need to understand the different versions of Ethernet. There is more than one version? Yes, there is although mainly one is used for all communication.

Most people will know that Robert Metcalfe was one of the inventors of Ethernet. Robert was working for Xerox back then. Digital, Intel and Xerox worked together on standardizing Ethernet. This is why it is often referred to as a DIX frame. The DIX version 1 standard was published in 1980 and the version used today is version 2. This is why we refer to Ethernet II or Ethernet version 2. The DIX version is the frame type that is most often used.

IEEE was also working on standardizing Ethernet. They began working on it in February 1980 and that is why the standard is called 802 where 802.3 is the Ethernet standard. We refer to it as Ethernet even though when IEEE released their standard it was called “IEEE 802.3 Carrier Sense Multiple Access with Collision Detection (CSMA/CD)
Access Method and Physical Layer Specifications”. So here we see the term CSMA/CD for the first time.

I’m not here to give you a history lesson but instead explain the frame types and briefly discuss the fields in them. We start with the DIX frame or Ethernet II frame. This is the frame that is most commonly used today. It looks like this.

The preamble is a pattern of alternating ones and zeroes and ending with two ones. When this pattern is received it is known that anything that comes after this pattern is the actual frame.

The source and destination MAC is used for switching based on the MAC.

The EtherType field specifies that upper level protocol. Some of the most well known ones are:

0x0800 – IP
0x8100 – 802.1Q tagged frame
0x0806 – ARP
0x86DD – IPv6

After that follow the actual payload which should be between 46 – 1500 bytes in size.

In the end there is a Frame Checking Sequence (FCS) which is used to check the validity of the frame. If the CRC check fails the frame is dropped.

In total the frame will be maximum 1514 bytes or 1518 if counting the FCS.

When it comes to 802.3 Ethernet there are actually two frame formats. One is 802.3 with 802.2 LLC SAP header. It looks like this.

This was the original version from the IEEE. Many of the fields are the same. Let’s look at those that are not.

The preamble is now divided in preamble and Start Frame Delimiter (SFD) but the function is the same.

The length field is used to indicate how many bytes of data are following this field before the FCS. It can also be used to distinguish between DIX frame and 802.3 frame as for DIX the values in this field will be higher e.g. 0x806 for ARP. If this value is greater than 1536 (0x600 Hex) then it is a DIX frame and the value is an Ethertype value.

Then we have some interesting values called DSAP, SSAP and Control. SAP stands for Service Access Point, the S and D in SSAP and DSAP stands for source and destination.

They have a similar function as the Ethertype. The SAP is used to distinguish between different data exchanges on the same station. The SSAP indicates from which service the LLC data unit was sent and the DSAP indicates the service to which the LLC data unit is being sent. IP has a SAP of 6 and 802.1D (STP) has a SAP of 42. It would be very strange to have a different SSAP and DSAP so these values should be the same. IP to IP would be SSAP of 06 and DSAP of 06. One bit (LSB) in the DSAP is used to indicate if it is a group address or an individual address. If it is set to zero it refers to an individual address going to a Local SAP (LSAP). One bit in the SSAP (LSB) indicates if it is a command or response packet. That leaves us with 128 possible different SAPs for SSAP and DSAP.

The contol field is used to select if communication should be connection-less or connection-oriented. Usually error recovery and flow control are performed by higher level services such as TCP.

The IEEE had problems to address all the layer 3 processes due to the short DSAP and SSAP fields in the header. This is why they introduced a new frame format called Subnetwork Access Protocol (SNAP). Basically this header is using the type field found in the DIX header. If the SSAP and DSAP is set to 0xAA and the Control field is set to 0x03 then SNAP encapsulation will follow. SNAP has a five byte extension to the standard 802.2 LLC header and it consists of a 3 byte OUI and a two byte Type field.

From a vendor perspective this is good because then they can have an OUI and then create their own types to use. If we look at PVST+ BPDUs from a Cisco device we will see that they are SNAP encapsulated where the organization code is Cisco (0x00000c) and the PID is PVSTP+ (0x010b). CDP is also using SNAP and it has a PID of CDP (0x0200). I will talk more about BPDUs and STP in a following post but first I wanted to provide the background on the Ethernet frame types used.

In summary there are three different Ethernet frame types used. DIX frame, also called Ethernet II, IEEE 802.3 with LLC and IEEE 802.3 with SNAP encapsulation. There are others out there as well but these are the three major ones and the DIX one is by far the most common one.

Categories: CCIE, Ethernet Tags: , , , , , ,

Source Specific Multicast (SSM) and IGMP filtering

June 4, 2012 16 comments

Regular multicast is known as Any Source Multicast (ASM). It is based on a many to many
model where the source can be anyone and only the group is known. For some applications
like stock trading exchange this is a good choice but for IPTV usage it makes more
sense to use SSM as it will scale better when there is no need for a RP.

ASM builds a shared tree (RPT) from the receiver to the RP and a
Shortest Path Tree (SPT) from the sender to the RP. Everything must pass through the RP
until switching over to the SPT building a tree directly from receiver to sender.
The RPT uses a (*,G) entry and the SPT uses a (S,G) entry in the MRIB.

SSM uses no RP, instead it uses IGMP version 3 to signal what channel (source) it wants
to join for a group. IGMPv3 can use INCLUDE messages that specify that only these
sources are allowed or they can use EXCLUDE to allow all sources except for these ones.

SSM has the IP range 232.0.0.0/8 allocated and it is the default range in IOS but we can
also use SSM for other IP ranges. If we do we need to specify that with an ACL.

SSM can be enabled on all routers that should work in SSM mode but it is only
really needed on the routers that have receivers connected since that is the place
where the behavior is really changed. Instead of sending a (*,G) join to the RP
the Last Hop Router (LHR) sends a (S,G) join directly to the source.

This is the topology we are using.

It is really simple. R1 is acting as a multicast source and R2 will both simulate a client
and do filtering. R3 will simulate an end host. R1 will source the traffic from its loopback.
OSPF has been enabled on all relevant interfaces.

We will start by enabling SSM for the range 225.0.0.0/24 on R2.

R2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R2(config)#access-list 1 permit 225.0.0.0 0.0.0.255
R2(config)#ip pim ssm range 1

R2 will now use SSM behavior for the 225.0.0.0/24 range. R2 will join the group 225.0.0.1.
We will debug IGMP and PIM to follow everything that happens. When using igmp join-group
on an interface the router simulates IGMP report coming in on that interface. We will see
later why this is important. So first we enable debugging to the buffer.
Also we must enable multicast routing and enable PIM sparse-mode on the relevant interfaces.

R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#ip multicast-routing
R1(config)#int s0/0
R1(config-if)#ip pim sparse-mode
R1(config-if)#do debug ip pim
PIM debugging is on
R1(config-if)#
R2(config)#ip multicast-routing
R2(config)#int s0/0
R2(config-if)#ip pim sparse-mode
R2(config-if)#int f0/0
R2(config-if)#ip pim sparse-mode
R2(config-if)#ip igmp version 3
R2(config-if)#
*Mar  1 00:18:37.595: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 23.23.23.2 on interface FastEthernet0/0
R2(config-if)#do debug ip igmp
IGMP debugging is on
R2(config-if)#do debug ip pim
PIM debugging is on

Then we join the group on the Fa0/0 interface and look at what happens.

R2(config)#int f0/0
R2(config-if)#ip igmp join-group 225.0.0.1 source 1.1.1.1

We take a look at the log.

IGMP(0): Received v3 Report for 1 group on FastEthernet0/0 from 23.23.23.2
IGMP(0): Received Group record for group 225.0.0.1, mode 5 from 23.23.23.2 for 1 sources
IGMP(0): Updating expiration time on (1.1.1.1,225.0.0.1) to 180 secs
IGMP(0): Setting source flags 4 on (1.1.1.1,225.0.0.1)
IGMP(0): MRT Add/Update FastEthernet0/0 for (1.1.1.1,225.0.0.1) by 0
PIM(0): Insert (1.1.1.1,225.0.0.1) join in nbr 12.12.12.1's queue
IGMP(0): MRT Add/Update FastEthernet0/0 for (1.1.1.1,225.0.0.1) by 4
PIM(0): Building Join/Prune packet for nbr 12.12.12.1
PIM(0): Adding v2 (1.1.1.1/32, 225.0.0.1), S-bit Join
PIM(0): Send v2 join/prune to 12.12.12.1 (Serial0/0)
IGMP(0): Building v3 Report on FastEthernet0/0
IGMP(0): Add Group Record for 225.0.0.1, type 5
IGMP(0): Add Source Record 1.1.1.1
IGMP(0): Add Group Record for 225.0.0.1, type 6

R2 is receiving an IGMP report (created by itself) and then it generates a PIM join and
sends it to R1. We look how R1 is receiving it.

PIM(0): Received v2 Join/Prune on Serial0/0 from 12.12.12.2, to us
PIM(0): Join-list: (1.1.1.1/32, 225.0.0.1), S-bit set
PIM(0): RPF Lookup failed for 1.1.1.1
PIM(0): Add Serial0/0/12.12.12.2 to (1.1.1.1, 225.0.0.1), Forward state, by PIM SG Join

Then we verify by looking at the mroute table and by pinging.

R1#sh ip mroute 225.0.0.1 | be \(
(*, 225.0.0.1), 00:09:42/stopped, RP 0.0.0.0, flags: SP
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list: Null

(1.1.1.1, 225.0.0.1), 00:01:49/00:01:40, flags: T
  Incoming interface: Null, RPF nbr 0.0.0.0
  Outgoing interface list:
    Serial0/0, Forward/Sparse, 00:01:49/00:02:39

Now we do a regular ping which should fail since we are not sourcing traffic from the loopback.

R1#ping 225.0.0.1 re 3

Type escape sequence to abort.
Sending 3, 100-byte ICMP Echos to 225.0.0.1, timeout is 2 seconds:
...

This is expected and what is good about SSM is that it makes sending to groups from any
source more difficult which is good from a security perspective.

Now we do an extended ping and source from the loopback.

R1#ping
Protocol [ip]:
Target IP address: 225.0.0.1
Repeat count [1]: 5
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Interface [All]: serial0/0
Time to live [255]:
Source address: 1.1.1.1
Type of service [0]:
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 225.0.0.1, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1

Reply to request 0 from 12.12.12.2, 52 ms
Reply to request 1 from 12.12.12.2, 48 ms
Reply to request 2 from 12.12.12.2, 48 ms
Reply to request 3 from 12.12.12.2, 36 ms
Reply to request 4 from 12.12.12.2, 40 ms

So our SSM is working and we didn’t even have to enable it on R1! What if we have
clients not supporting IGMPv3? Then we could do SSM mapping. I could do that in
another post if there is interest for it. For now lets look at filtering. If we
were using ASM then we use a standard ACL and match which multicast groups are
allowed to send joins for. The joins would be (*,G) which is the same as
host 0.0.0.0 in an ACL.

To filter SSM we use an extended ACL where the source in the extended ACL
is the multicast source and the destination is which group to match. We will
create an ACL permitting 1.1.1.1 as source for the groups 225.0.0.1, 225.0.0.2
and 225.0.0.3. Anything else will be denied which we will see by debugging IGMP.
When we are doing filtering it is important to rembember that the IGMP report
generated by the router itself (igmp join-group) will also be subject to the ACL
so make sure to include that.

R2(config)#ip access-list extended IGMP_FILTER
R2(config-ext-nacl)#permit igmp host 1.1.1.1 host 225.0.0.1
R2(config-ext-nacl)#permit igmp host 1.1.1.1 host 225.0.0.2
R2(config-ext-nacl)#permit igmp host 1.1.1.1 host 225.0.0.3
R2(config-ext-nacl)#deny igmp any any
R2(config-ext-nacl)#int f0/0
R2(config-if)#ip igmp access-group IGMP_FILTER

Now we make R3 join a group not allowed and look at the debug output on R2.

R3(config)#int f0/0
R3(config-if)#ip igmp version 3
R3(config-if)#ip igmp join-group 225.0.0.10 source 1.1.1.1

This is from the log on R2.

IGMP(0): Received v3 Report for 1 group on FastEthernet0/0 from 23.23.23.3
IGMP(*): Source: 1.1.1.1, Group 225.0.0.10 access denied on FastEthernet0/0
R2#sh ip access-lists IGMP_FILTER
Extended IP access list IGMP_FILTER
    10 permit igmp host 1.1.1.1 host 225.0.0.1 (6 matches)
    20 permit igmp host 1.1.1.1 host 225.0.0.2
    30 permit igmp host 1.1.1.1 host 225.0.0.3
    40 deny igmp any any (7 matches)

As we can see that group is not allowed so the IGMP join will not make it through.
SSM can be very useful and it is not difficult to setup. In fact it is mostly
easier than ASM to setup.

Categories: CCIE, Multicast Tags: , , , ,