Archive

Archive for January, 2012

Route redistribution – filtering and mitigating loops

January 30, 2012 6 comments

This post is about route redistribution and the different filtering techniques
we have available in our toolbelt. This post requires that you have a basic
understanding of route redistribution. For some good posts look at Petr
Lapukhovs posts at INE.

First lets define what is route redistribution? Generally we will use route
redistribution when multiple routing protocols are running in the network or
multiple instances of the same routing protocol is running. This can be due
to mergers, acquisitions or a fork lift upgrade. Maybe network is running
OSPF but is migrating to EIGRP or vice versa. We can also have redistribution
of connected and static routes.

What are some of the issues we can run into with route redistribution? We can
have routing loops in the network. These may not always be visible right away.
We can see them when doing a traceroute or when using debug ip routing.

We can also have issues with route feedback. Route feedback is when a redistributed
route gets redistributed back into the same protocol from which it originated. This
can lead to suboptimal routing or routing loops.

How do we define how “believable” a prefix is? First we must know that only the
same prefixes will be compared. 162.14.1.0/24 and 162.14.1.0/25 are not the same
prefixes. Longer match always wins! If we are comparing the two same prefixes
from different routing protocols then the AD will determine which one is the
best. Lower AD wins. If the AD for some reason is the same then the default AD
is the tie breaker.

Routes that are external should be less trusted than internal routes. EIGRP does
this by default by setting AD to 170 for external routes but 90 for internal. If
the other protocols did the same then we would not have issues with routing loops
at all. OSPF uses the same AD for internal and external prefixes (110) but we
can modify the AD for external routes if we want to. RIP does not have this
possibility.

Routing loops generally occur when we have redistribution between a protocol with
higher AD to a protocol with lower AD. This means that RIP is most often involved
in loops since it has the highest AD and we can’t define an external AD.

Lets look at a topology where loops can occur. This image is hand drawn and something
I do when doing labs to try to spot potential issues. I use a different color for
different protocols and draw arrows where redistribution occurs.

We are doing mutual redistribution on R1 and R2. The issue here is that we will
have suboptimal routing. You can download topology and configs here.

Look at the show ip route and traceroute from R1 to R2’s loopback.

R1 is going the whole way round even though it has a direct route via RIP to
R2. This is of course due to RIP having a higher AD than OSPF. Lets look at
a few different ways of fixing this. If this was the CCIE lab you would do nothing
unless it was asked of you to provide optimal routing. We don’t have a loop so it’s
not really a big issue at the moment. The issue here is that OSPF is not setting
a higher AD on it’s external prefixes. So we will have to do this manually.

So we set the AD to something higher than RIP, 121 in this case. Now we take
the direct path. Remember that AD is a local setting so this would have to be done
on all routers choosing suboptimal path.

We could also lower the AD of RIP. Either we do it for all routes or for a selection
of routes. Here we will select the routes with an ACL. We can set the distance for
all route sources or for a specific one based on the IP. Here we only have one so
we don’t really care, we will match on 0.0.0.0 255.255.255.255.

So now the AD is 109 for the RIP route which beats OSPF of 110. This would of course
also be have to be done on all routers with suboptimal path.

This is another way of doing it. We are setting a distance of 255 for the RIP routes
when they are entering as OSPF routes. 255 is not a valid distance for installing to
RIB so RIP routes will be preferred.

We can also use a distribute-list to control what routes get installed via OSPF.
Since OSPF is a link state protocl the LSA will of course propagate to other
routers.

As you can see the route is still present in the OSPF database but it does not
get installed into the RIB.

There is also a more fancy way of using distribute-lists. We can tie them to
a routing-protocol and define what is allowed to go out from that protocol
into the routing protocol that we are currently configuring. We will configure
R2 so that RIP routes are not allowed to be redistributed into OSPF. This will
kill any redundancy in the network.

So we go to the config mode of the routing protocol we are redistributing into.
Then we define with the distribute-list what is allowed to go out from other
protocols into the one we are now configuring. This is an effective way of
filtering when we have a lot of redistribution going on. In a small scenario
like this it does not make much sense but it’s very handy in large scenarios.

There is still one tool left and that is the route-map. The route-map is the
most flexible and scalable solution of all. We can choose what prefixes get
redistributed with an access-list or prefix-list. Lets try that first.

Here we matched prefixes with a prefix-list. The prefix-list has a deny for the
loopback of R2 and permits anything else. The route-map only has a permit statement,
deny in prefix-list and permit in route-map means that the prefix does not match
and moves to the next statement which is an implicit deny. The permit matches the
permit of the route-map and allows anything else.

This is an example of route feedback.

R5 is the only redistribution point. The issue here is that the routes that R3
learns from R1 and R2 via RIP will arrive at R5. R5 then redistributes into
OSPF. R3 will receive these LSA’s and find that this path is better due to a
lower AD. R3 will then install this route. R3 stops announcing that route via
RIP. R5 looses its route via RIP and can’t redistribute it into OSPF, so it
stops announcing it via OSPF. R3 installs the RIP route again and the fun
has just begun. Debug ip routing will show this procedure repeat over and over.

For our final tool we need a more complicated scenario to make full use of it.
First lets take a look at the topology. Download configs and topology here.

We have mutual redistribution on R3 and R5. The issue here is that we are
redistributing into RIP with a seed metric of 1. R3 sees R1’s loopback via
RIP with a metric of 2. R3 redistributes this information into OSPF. R5 learns
this information via OSPF and then redistributes into RIP with a metric of 1.
R3 now has two possible paths to R1 loopback. One with a metric of 1 and one
with a metric of two. Of course the lower metric wins. This means that R5
points towards R3 via R4 and R3 points to R5. Ladies and gentlemen, we have
a routing loop. There is definately a risk of loops when doing mutual redistribution.
When I redistribute something into RIP I usually set a quite high seed metric like 7.
This lowers the risk of loops because the RIP metric internally should be lower unless
it’s a very large network.

The probably best way to filter redistribution is to use route tagging. We set
a tag in a route-map and then base our filters on this. Sometimes it can be difficult
knowing where a route originated and from which protocol it came. If we set good tags
we can see both just by looking at the tag. I usually set a tag like 390, that means
that router 3 originated the route and it came from EIGRP. A tag of 4120 would mean
that it was a RIP route from R4 to begin with. Now lets try this technique as well.

We will set a tag of 3120 on R3 and also deny routes with a tag of 5110 on R3. This
is used to prevent R3 from taking OSPF routes from R5 received over RIP and then
redistributing them back into OSPF.

This is what the filtering looks like on R5.

Here we deny routes with a tag of 3120 and set our own tag of 5110.

Note that there is a risk that the loop still remains. R3 is announcing
routes via RIP natively to R5. We have a chicken and egg problem here.
This is the scenario before tagging. R3 redistributes RIP into OSPF. R5 receives
the routes and install them via OSPF since AD is lower than RIP. R5 then redistributes
these back into RIP. R3 sees the lower metric via R5 and installs this route in the
RIB. We now have a loop.

We start implementing tagging. R3 tags RIP routes going into OSPF
and denies any prefixes that R5 has already redistributed from OSPF to RIP.
R5 denies routes from R3 with a tag of 3120 from going back into RIP and sets
its own tag of 5110. There is a risk that R5 sees the best path via RIP to R3
and then announces an OSPF route which R3 installs. We need to make sure that
R5 sees the best path via OSPF to have a stable network. This can be done by
clearing routing table and shutting down links. To make sure this does not
happen we should also tag RIP routes going into OSPF on R5.

So you see that redistribution can be very complicated and there are a lot of
tools available. Try to check what routes are native to which routing protocol
and make sure that these are preferred in that domain. You can use distance
or other tools to make sure this happens.

In the real lab there could be hidden bombs that you need to detect to have a stable
topology. Probably you won’t be that restricted what you could do but there could
be some restrictions. So it’s good to have as many tools as possible. If all else
fails, do what it takes to get connectivity. Use distribute-lists or whatever
to have a stable topology. Yes, you will loose points but you can definately
not finish the lab if you don’t have a stable topology.

I hope this post has been informative and if you want to give med feedback
post in the comments section.

OSPF magic – Make interarea routes become intraarea

January 26, 2012 3 comments

This is a follow up post to my last OSPF post about repairing area 0.
In the comments section Ray asked me what we can do if we have a
scenario where we have another area that is discontigous. In this
example we are using area 1. Area 1 is used everywhere but between
R1 and R4 is the backbone area. The routes will therefore be
interarea. What if we were told to make these routes appear as
intra area? First look at the topology. Download .net file and
initial and final configs here.

First we start by confirming that routes received on R1
are coming in as interarea routes.

Yes they are. This is expected behaviour. So what can we do to
make them appear as intraarea? Using a virtual link does not help
since it always belongs to area 0. We could use the tunnel technique
that was used in the previous post. Let’s try that. Same procedure as
last time. Source tunnel from physical interface. Use IP unnumbered
from an interface in area 1.

The tunnel is up and we have a new adjacency.

The IP address is 0.0.0.0/0 and located in area 1. Lets look at the
router LSA for area 1.

The address is 0.0.0.16 just as the last time. This should be 10 according
to the SNMP MIB but as one of my friends Patrick pointed out 16 in decimal
is 0x10 in hex. Maybe it is encoded in hex?

So now lets check the routing table.

Problem solved. Routes are now received as intraarea. So there you have it,
another OSPF problem solved.

Repairing a discontigous area 0 in OSPF

January 25, 2012 11 comments

First start by looking at the diagram above. This is a scenario I ran into while
doing my mock lab. It is pretty obvious that we will have issues with OSPF since area 0
is discontigous. You should be able to spot this by just looking at the diagram.

So what are the tools we have to repair discontigous areas? The most common and
straightforward is to use a virtual-link. However in this scenario it is not allowed.
First lets look what issues we will have in this topology. I have prepared the
topology and initial configs with OSPF for you which can be downloaded here, final configs are also included.

We start by looking at the routing table of R1.

We can only see the loopback of R2. Why is that? R1 is an ABR. It has an
interface in area 0 and in another area. It will not install interarea routes
received on a non backbone interface.

This is the database on R1. We can see that all IA routes are indeed in
the database but they are not being installed into the RIB. To test our
thesis from the last OSPF post, lets shutdown Fa0/0 on R1. This should in
theory mean that R1 is no longer an ABR and that it can install routes
received on a non backbone area.

It works. So our knowledge of OSPF seems good. So this is an option we
can use, although we would not have reachability to R2’s loopback. The techniques
I show here would generate no points for the OSPF section but having reachability
in IGP is critial so if you can’t do something the right way, cheat!
What I mean by that is that if you don’t have reachability you will loose points
later in BGP, MPLS, multicast and maybe even IPv6 and security section. So you
see a lot of points are at stake and if we can’t solve a task the right way do it
some other way. Use static routes if you must!

We were not allowed to use a virtual-link but lets try that anyway to
see that we can install all the routes.

Yes, now we have reachability. We would loose 2 or 3 points probably for doing
this but could potentially save us 10-15 points later. I ended up having to do
this on my mock since I had problems configuring the solution below.

So if we can’t use a virtual link, where does that leave us? We need a link
between R1 and R4 that is located in area 0. How about a tunnel? Yes, that is
definately an option. However we are not allowed to introduce any new
IP addresses into the network. How do we solve that? IP unnumbered, we have to
borrow an IP from another interface. Lets setup the tunnel.

We source the tunnel from a physical interface and the destination is the other
side of the FR connection. Why source from the physical interface? We don’t have
reachability to loopbacks yet since our IGP is broken, therefore we have to use
the physical address. Why unnumbered from the Ethernet interfaces? They are located
in area 0. We must have an IP that is located in area 0 to repair the discontigous network.

So now we have an adjacency both over serial0/0 and over the tunnel. The tunnel
interface is point-to-point and has a very high cost. Notice that the IP for
the tunnel is 0.0.0.0/0.

Now look at the routing table of R1.

We can see that prefixes are reached via 155.1.34.4. This is the IP of Fa0/0
on R4 which was derived from the IP unnumbered. So what is interesting here
is that the adjacency between R1 and R4 is not in the same subnet, so we
are bypassing that check. We can also see that we need to recurse routing
lookups since the routes are pointing at 155.1.34.4.

This route is reached via the tunnel. If we look at the OSPF database on R1
we see a router LSA for R4 and there is the P2P link.

Router Interface address is 0.0.0.16. This is interesting, normally this is
the IP address of the interface. We are now using unnumbered. I’m not sure
why it is choosing 16. RFC says the following:

“For unnumbered point-to-pointlinks, the Link Data field should be set to
the unnumbered interface’s MIB-II [Ref8] ifIndex value.

If we look at the SNMP MIB, must enable SNMP first.

So this value should be 10. Seems like Cisco is not really following
the RFC (RFC2328) here. We can enable that behaviour with interface-id snmp-if-index.

Now we have the expected value of 10.

You can read more at Cisco.

So this should give you a look into OSPF. I have also attached a packet capture so that you can look at the LS Update going over the tunnel interface. We can see that the source of the
packet is 155.1.14.4 which is R4 IP addess and the destination is 155.1.14.1.
In there we have the other IP packet which has a source of 155.1.34.4 and a destination
of 224.0.0.5.

First mock lab done (Cisco 360)

January 20, 2012 3 comments

So I had my first run at a mock lab (Cisco 360). It was not a great score by any means. I got 59% on the TS and 49% on the config. Still, it does show that I’m not THAT far away. The lab was rated as advanced and there were a lot of tricky stuff. Some tasks I solved but not in the way I should have done.

I learned a lot of leassons. How to read the tasks and how to solve tasks in TS without breaking requirements. I spent a lot of time on some EIGRP config because I misread the task. The task was to do unequal load balance but I thought I wasn’t allowed to change the delay of the interface which I was.

Very good practice indeed. Will try another mock in a couple of weeks.

50000 views!

January 17, 2012 1 comment

The site has now hit 50000 views. I did not really expect this when I started the blog. Thanks to all my readers 🙂

I’m doing my final prep for the CCIE. Vol1 has be finished. Right now I’m focusing on full scale labs and troubleshooting labs. Tonight I have 2 TS labs planned and on friday I’m taking a Cisco360 mock lab. I’ve heard that they are high quality and close to the experience of the real exam. I hope so because they cost a pretty penny.

I’ll post my results when done, hopefully they won’t be too terrible but I’m not expecting a really good result either.

How well do you really know OSPF?

January 15, 2012 3 comments

OSPF is by far the most complicated IGP on the RS lab if we go in to the details of the protocol and the loop prevention mechanisms. While doing some labs this week I ran in to some interesting issues. This post will describe different scenarios and test your understanding of OSPF. I will show a diagram and ask a question, try to answer it and when you want to know the answer then the text is hidden below. You can see the text by marking it since it’s written with a white font.

Lets begin. We will start with an issue I ran into while doing an ASET lab (Cisco partner labs). Look at the following diagram.

R5 is a PE in a MPLS VPN backbone. R5 is running OSPF in a VRF towards the CE R4. However R5 will not install interarea routes received from R4, why? Look at the following output.

R5#sh ip ospf data sum 1.1.1.3

OSPF Router with ID (1.1.1.5) (Process ID 100)

OSPF Router with ID (5.5.5.5) (Process ID 1)

Summary Net Link States (Area 51)

Routing Bit Set on this LSA
LS age: 1386
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 1.1.1.3 (summary Network Number)
Advertising Router: 1.1.1.4
LS Seq Number: 80000003
Checksum: 0xED3
Length: 28
Network Mask: /32
TOS: 0 Metric: 76

R5 is part of the MPLS super backbone (area0) so it is an ABR. ABRs will not install interarea routes that are received over a non backbone area. If the area between R4 and R5 was area 0 then there would be no issues but since we are using area 51 we have to use a workaround. We enable the capability vrf-lite under the ospf process which disables the loop checks of the PE

Now lets define some of the concepts of OSPF and test your understanding. Look at the diagram.

Is R3 an ABR?

No, the definition is that an ABR is a router with an UP interface in the backbone area and at least another area.

Will R3 generate IA routes for Area 1 and 2?

No, it is not an ABR so it will not generate any IA routes.

Now we create a loopback on R3 and put it in OSPF area 0. No OSPF adjacencies can of course be built over this interface. Will R3 generate any summary routes now?

Yes, even though R3 is not really connected to Area 0 it has an UP interface in area 0 so it will announce itself as an ABR and generate summary routes.

R2#sh ip ospf border-routers

OSPF Process 1 internal Routing Table

Codes: i – Intra-area route, I – Inter-area route

i 1.1.1.1 [1] via 155.1.12.1, FastEthernet0/1, ABR, Area 1, SPF 7
i 3.3.3.3 [1] via 155.1.23.3, FastEthernet0/0, ABR, Area 1, SPF 7

Summary Net Link States (Area 1)

Link ID ADV Router Age Seq# Checksum
33.33.33.33 3.3.3.3 13 0x80000001 0x004566
155.1.16.0 1.1.1.1 1958 0x8000000D 0x00B9C5
155.1.34.0 1.1.1.1 939 0x80000006 0x00155D
155.1.34.0 3.3.3.3 19 0x80000001 0x00CEA2
155.1.35.0 1.1.1.1 942 0x80000006 0x00145C
155.1.35.0 3.3.3.3 21 0x80000001 0x00C3AC
155.1.45.0 1.1.1.1 942 0x80000006 0x009BCB
155.1.45.0 3.3.3.3 21 0x80000001 0x005F06
155.1.46.0 1.1.1.1 942 0x80000006 0x0086E0

If we traceroute the IP of R5 this is the path.

R1#traceroute 155.1.35.5

Type escape sequence to abort.
Tracing the route to 155.1.35.5

1 155.1.16.6 140 msec 76 msec 48 msec
2 155.1.46.4 92 msec 64 msec 64 msec
3 155.1.45.5 184 msec * 108 msec

Why is this and how can we make the path go directly from R1 to R3 to R5? Look at the OSPF database.

Summary Net Link States (Area 1)

Link ID ADV Router Age Seq# Checksum
33.33.33.33 3.3.3.3 534 0x80000001 0x004566
155.1.16.0 1.1.1.1 437 0x8000000E 0x00B7C6
155.1.34.0 1.1.1.1 1459 0x80000006 0x00155D
155.1.34.0 3.3.3.3 542 0x80000001 0x00CEA2
155.1.35.0 1.1.1.1 1463 0x80000006 0x00145C
155.1.35.0 3.3.3.3 543 0x80000001 0x00C3AC
155.1.45.0 1.1.1.1 1463 0x80000006 0x009BCB
155.1.45.0 3.3.3.3 543 0x80000001 0x005F06
155.1.46.0 1.1.1.1 1463 0x80000006 0x0086E0

R3 is announcing the summary but it is not being used by R1. Why?

R1 and R4 have full adjacencies via Area 0 so they will not accept summary routes received on a non backbone interface. To fix this we need a full adjacency between R1 and R3 which is created with a virtual link.

R3(config-router)#area 1 virtual-link 1.1.1.1
R1(config-router)#area 1 virtual-link 3.3.3.3

R1#sh ip ospf data sum 155.1.35.0

OSPF Router with ID (1.1.1.1) (Process ID 1)

Summary Net Link States (Area 0)

Routing Bit Set on this LSA
LS age: 10 (DoNotAge)
Options: (No TOS-capability, DC, Upward)
LS Type: Summary Links(Network)
Link State ID: 155.1.35.0 (summary Network Number)
Advertising Router: 3.3.3.3
LS Seq Number: 80000001
Checksum: 0xC3AC
Length: 28
Network Mask: /24
TOS: 0 Metric: 1

R1#traceroute 155.1.35.5

Type escape sequence to abort.
Tracing the route to 155.1.35.5

1 155.1.13.3 64 msec 40 msec 44 msec
2 155.1.35.5 80 msec * 64 msec

R1 is now using the direct path since R3 is now a real ABR. However as R3 now has an adjacency to Area 0 it will no longer accept summary routes from R4. If we want to fix this another virtual link is needed.

I hope this post gave you some things to think about regarding OSPF. This post was mostly inspired by the great document from Petr Lapukhov

Categories: CCIE, OSPF Tags: , , ,