Administrators and engineers often live in an environment where we don't get everything we want. We may strongly recommend the purchase of some equipment, software, or circuits, and even come up with the business case to back our recommendations, but in the end; it’s not our money or our decision. So sometimes, or all the time, depending on your environment our your disposition, we have to come up with a solution that is a little more complicated to make the best of what we have. This also can be rewarding because we get a challenge.
In this scenario, you had a single circuit to your ISP and were running low on bandwidth. A new requirement was coming and you had to add a new router, R3 to your network, and a new subnet, 212.15.80.192/27, of users and server behind it. You had told your boss, that he would need more bandwidth and recommended upgrading the existing circuit to accommodate the additional traffic of the new subnet and a redundant circuit of the same size. Disregarding your advice, he ordered a new circuit, and insisted you dedicate the new subnet to the circuit while existing networks traverse the old circuit and you allow both networks across that either link for redundancy.
Rather than getting upset, because he didn't listen to your advice, get excited, because you get to try something a little different. First let’s take a look at the diagram below and talk about your network before adding router R3 to the topology.
In this topology you were using BGP up to your ISP and advertised the entire IP space that you owned, the 212.15.80.0/24 network. The ISP gave you only a partial route, which means, you did not receive the entire Internet's routing table, and what would you need that for? You only had one link. So, you took one reliable network from the ISP, the 12.0.0.0/8 network, and filtered out all other BGP advertisements. And you set that network to be your default network in the router, so your gateway of last resort was to always route to the 12.0.0.0/8 network, this is a fairly standard practice and you can usually call your ISP and they will tell you what network is best to use as a default network, or it can be found in your contract with the ISP.
On the inside of your network, you are using EIGRP to R2 and you’re redistributing your EIGRP routes into BGP. To create the /24 advertisement to the ISP, you do and aggregate-address statement in BGP, which says basically that as long as you know a route to something in the /24 network, create and advertisement for the whole network because it belongs to you.
Your configuration before you start looks like this:
!Existing Config
hostname R1
!
ip cef
!
interface FastEthernet0/0
description ***LINK_TO_R2***
ip address 212.15.80.1 255.255.255.252
duplex auto
speed auto
!
interface Serial0/0
description ***CIRCUIT_1***
ip address 12.63.78.2 255.255.255.252
clock rate 2000000
!
router eigrp 1
passive-interface default
no passive-interface FastEthernet0/0
network 212.15.80.0
no auto-summary
!
router bgp 3999
no synchronization
bgp log-neighbor-changes
aggregate-address 212.15.80.0 255.255.255.0
redistribute eigrp 1
neighbor 12.63.78.1 remote-as 456
no auto-summary
!
ip default-network 12.0.0.0
!
!
end
Now we have a starting point. My goal in this demonstration is to make it as simple as possible, so there are several different variations of this configuration you can do; often when manipulating BGP, you end up using route-maps, but I decided to show how to do BGP route-manipulation without route-maps and instead perform the majority of the route selection by using the most specific route.
What do I mean by that? Well, we already identified that we have 3 subnets in our 212.15.80.0/24 network that need to be routed to the Internet, the 212.15.80.32/27, the 212.15.80.64/27, and new the 212.15.80.192/27. So how I plan to do this is to advertise the 212.15.80.0/24 to the ISP out of both links, but advertise the old two out of the old link with a 212.15.80.0/25 advertisement and the new network out the new link. This will allow traffic returning to your old subnets to go over the old link and the traffic returning to the new subnet to go over the new link because the ISP router to choose the /27 routes over the /24 routes because they are more specific. This should handle your incoming/return traffic. So for you outgoing traffic, you will first set your old circuit as primary by assigning a higher weight to it.
To do these steps, you will apply the following changes:
First, use ACLs as filters for what advertisements you will send out each link.
ip access-list extended CIRCUIT1-OUT
permit ip host 212.15.80.0 host 255.255.255.0
permit ip host 212.15.80.0 host 255.255.255.128
ip access-list extended CIRCUIT2-OUT
permit ip host 212.15.80.0 host 255.255.255.0
permit ip host 212.15.80.192 host 255.255.255.224
Then configure BGP, to first enable the new connection, then use the new filters, aggregate the .32/27 and .64/27subnets into a .0/25, and set the weight of the neighbor so the old link will remain primary for most Internet bound traffic.
router bgp 3999
!enable the new connection
neighbor 12.78.45.49 remote-as 456
!use the new filters for outbound advertisements
neighbor 12.63.78.1 distribute-list CIRCUIT1-OUT out
neighbor 12.78.45.49 distribute-list CIRCUIT2-OUT out
!aggregate .32/27 and .64/27 to .0/25
aggregate-address 212.15.80.0 255.255.255.128
!keep the old link as primary for Internet bound traffic
neighbor 12.63.78.1 weight 200
neighbor 12.78.45.49 weight 100
With this complete, go ahead and enable the new circuit.
interface Serial0/1
description ***CIRCUIT_2***
ip address 12.78.45.50 255.255.255.252
no shutdown
If everything goes well, you should have a BGP neighbor adjacency with 12.78.45.49, which you are not using yet. You can see the status of a neighbor with, "show ip bgp summary." If the state has a word like, "idle" or "active" the neighbor adjacency has not formed, but if a number is shown, it has.
R1#sho ip bgp sum
BGP router identifier 212.15.80.5, local AS number 3999
BGP table version is 10, main routing table version 10
9 network entries using 1080 bytes of memory
10 path entries using 520 bytes of memory
6/5 BGP path/bestpath attribute entries using 744 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
Bitfield cache entries: current 3 (at peak 3) using 96 bytes of memory
BGP using 2464 total bytes of memory
BGP activity 9/0 prefixes, 10/0 paths, scan interval 60 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
12.63.78.1 4 456 179 177 10 0 0 02:53:46 4
12.78.45.49 4 456 179 178 10 0 0 02:53:48 4
And you can check to see that our gateway of last resort is set to the old neighbor, 12.63.78.1.
R1#sho ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is 12.63.78.1 to network 12.0.0.0
B 216.99.15.0/24 [20/0] via 12.63.78.1, 00:00:32
212.15.80.0/24 is variably subnetted, 7 subnets, 4 masks
C 212.15.80.0/30 is directly connected, FastEthernet0/0
B 212.15.80.0/25 [200/0] via 0.0.0.0, 16:03:13, Null0
B 212.15.80.0/24 [200/0] via 0.0.0.0, 16:03:13, Null0
D 212.15.80.32/27 [90/307200] via 212.15.80.2, 16:04:01, FastEthernet0/0
D 212.15.80.64/27 [90/284160] via 212.15.80.2, 16:04:01, FastEthernet0/0
* 12.0.0.0/8 is variably subnetted, 4 subnets, 3 masks
B* 12.0.0.0/8 [20/0] via 12.63.78.1, 16:03:15
C 12.78.45.48/30 is directly connected, Serial0/1
C 12.63.78.0/30 is directly connected, Serial0/0
B 12.128.0.0/9 [20/0] via 12.63.78.1, 00:00:34
B 148.78.0.0/16 [20/0] via 12.63.78.1, 00:00:34
At this point, return traffic should come back to you using its respective circuits, but outbound traffic is preferred out only the old circuit. How can you send traffic destined to the same places (Internet) over a separate circuit based upon its source when normal routing is sent based on its destination?
Your answer is a policy based route.
Such an example is as follows:
You would first define which traffic will be routed differently with an ACL.
ip access-list extended LINK2_PREFERRED
permit ip 212.15.80.192 0.0.0.31 any
Then you would create a route-map that defines the traffic and the action to take.
route-map PUSH_OVER_LINK2 permit 10
match ip address LINK2_PREFERRED
set ip next-hop 12.78.45.49
And finally you would assign that policy to the traffic on the interface it would enter the router.
interface FastEthernet0/1
description ***LINK_TO_R3***
ip policy route-map PUSH_OVER_LINK2
But this is not what you are going to do. You are going to use an extra technique to make sure that the new link is still a feasible next hop. A very easy way is based on what you have already done. Right now, you are using the default network of 12.0.0.0/8, and you are basing your 0.0.0.0/0 route upon that. What you can do is base the 0.0.0.0/0 based upon the policy based route to another potential default network. In this example, you look at your BGP routes and see the 12.128.0.0/9 and ask the ISP if this is a good 'extra' default network.
To perform this you will first need to filter out the BGP advertisements from the ISP, so that on the old link you will only receive the 12.0.0.0/8, and on the new link, you will receive the 12.0.0.0/8 and the 12.128.0.0/9.
ip access-list extended CIRCUIT1-IN
permit ip host 12.0.0.0 host 255.0.0.0
ip access-list extended CIRCUIT2-IN
permit ip host 12.0.0.0 host 255.0.0.0
permit ip host 12.128.0.0 host 255.128.0.0
router bgp 3999
neighbor 12.63.78.1 distribute-list CIRCUIT1-IN in
neighbor 12.78.45.49 distribute-list CIRCUIT2-IN in
After some time, or a clear neighbor soft, your routing table will change:
R1#clear ip bgp * soft in
R1#sho ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is 12.63.78.1 to network 12.0.0.0
212.15.80.0/24 is variably subnetted, 7 subnets, 4 masks
C 212.15.80.0/30 is directly connected, FastEthernet0/0
B 212.15.80.0/25 [200/0] via 0.0.0.0, 14:11:23, Null0
B 212.15.80.0/24 [200/0] via 0.0.0.0, 14:11:23, Null0
D 212.15.80.32/27 [90/307200] via 212.15.80.2, 14:12:09, FastEthernet0/0
D 212.15.80.64/27 [90/284160] via 212.15.80.2, 14:12:11, FastEthernet0/0
* 12.0.0.0/8 is variably subnetted, 4 subnets, 3 masks
B* 12.0.0.0/8 [20/0] via 12.63.78.1, 14:11:25
C 12.78.45.48/30 is directly connected, Serial0/1
C 12.63.78.0/30 is directly connected, Serial0/0
B 12.128.0.0/9 [20/0] via 12.78.45.49, 14:11:25
You now receive the 12.128.0.0/9 route only if you have a BGP connection to your ISP on the new circuit. Now you track the condition of the route using a track statement.
track 1 ip route 12.128.0.0 255.128.0.0 reachability
If the connection is up, the track will look like this:
R1#sho track
Track 1
IP route 12.128.0.0 255.128.0.0 reachability
Reachability is Up (BGP)
2 changes, last change 02:54:26
First-hop interface is Serial0/1
If it is down, it will look like this:
R1#sho track
Track 1
IP route 12.128.0.0 255.128.0.0 reachability
Reachability is Down (no route)
3 changes, last change 00:00:01
First-hop interface is unknown
Now you can create the policy-based route, the way you wanted it, where it ensures the new link is up.
ip access-list extended LINK2_PREFERRED
permit ip 212.15.80.192 0.0.0.31 any
route-map PUSH_OVER_LINK2 permit 10
match ip address LINK2_PREFERRED
set ip next-hop verify-availability 12.78.45.49 1 track 1
interface FastEthernet0/1
description ***LINK_TO_R3***
ip policy route-map PUSH_OVER_LINK2
And enable the link to R3.
interface FastEthernet0/1
description ***LINK_TO_R3***
ip address 212.15.80.5 255.255.255.252
no shutdown
router eigrp 1
no passive-interface FastEthernet0/1
When R3 is configured and forms an EIGRP neighbor adjacency we should be ready to do a little testing. First you should ensure that you are sending the right routes to the ISP on each link.
R1#sho ip bgp nei 12.63.78.1 advertised-routes
BGP table version is 10, local router ID is 212.15.80.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 212.15.80.0/25 0.0.0.0 32768 i
*> 212.15.80.0 0.0.0.0 32768 i
Total number of prefixes 2
R1#sho ip bgp nei 12.78.45.49 advertised-routes
BGP table version is 10, local router ID is 212.15.80.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 212.15.80.0 0.0.0.0 32768 i
*> 212.15.80.192/27 212.15.80.6 307200 32768 ?
Total number of prefixes 2
Just as planned, right? Now check to see if your policy based route is going as planned with a couple of traceroutes, one from R2, the normal path, and one from R3, for the policy based route. (In this example, 8.8.8.8 is a loopback address created at the ISP router.)
R2#traceroute 8.8.8.8 source fa0/1
Type escape sequence to abort.
Tracing the route to 8.8.8.8
1 212.15.80.1 32 msec 20 msec 44 msec
2 12.63.78.1 64 msec * 64 msec
R2#
R3#traceroute 8.8.8.8 source fa0/1
Type escape sequence to abort.
Tracing the route to 8.8.8.8
1 212.15.80.5 72 msec 16 msec 48 msec
2 12.78.45.49 64 msec * 40 msec
They both have a different 2nd hop, which is through the expected circuit.
Here is a traceroute from the ISP, demonstrating the return traffic is correct:
ISP#traceroute 212.15.80.193
Type escape sequence to abort.
Tracing the route to 212.15.80.193
1 12.78.45.50 20 msec 40 msec 20 msec
2 212.15.80.6 [AS 3999] 24 msec * 68 msec
ISP#traceroute 212.15.80.33
Type escape sequence to abort.
Tracing the route to 212.15.80.33
1 12.63.78.2 12 msec 16 msec 24 msec
2 212.15.80.2 [AS 3999] 56 msec * 8 msec
Your final configuration should look something like the one below. Now maybe with the money your boss saved in circuits, he can give it to you for a raise. But, he probably won't. As for this example, it can be used and expanded in other ways. Often, it may be desirable to dedicate the traffic to a specific protocol. This can often be done with NAT, where you NAT the traffic of a specific type to a pool of IP addresses, allowing you to advertise that pool over the link the same way.
!New config
hostname R1
!
ip cef
!
track 1 ip route 12.128.0.0 255.128.0.0 reachability
!
interface FastEthernet0/0
description ***LINK_TO_R2***
ip address 212.15.80.1 255.255.255.252
duplex auto
speed auto
!
interface Serial0/0
description ***CIRCUIT_1***
ip address 12.63.78.2 255.255.255.252
clock rate 2000000
!
interface FastEthernet0/1
description ***LINK_TO_R3***
ip address 212.15.80.5 255.255.255.252
ip policy route-map PUSH_OVER_LINK2
duplex auto
speed auto
!
interface Serial0/1
description ***CIRCUIT_2***
ip address 12.78.45.50 255.255.255.252
clock rate 2000000
!
router eigrp 1
passive-interface default
no passive-interface FastEthernet0/0
no passive-interface FastEthernet0/1
network 212.15.80.0
no auto-summary
!
router bgp 3999
no synchronization
bgp log-neighbor-changes
aggregate-address 212.15.80.0 255.255.255.0
aggregate-address 212.15.80.0 255.255.255.128
redistribute eigrp 1
neighbor 12.63.78.1 remote-as 456
neighbor 12.63.78.1 weight 200
neighbor 12.63.78.1 distribute-list CIRCUIT1-IN in
neighbor 12.63.78.1 distribute-list CIRCUIT1-OUT out
neighbor 12.78.45.49 remote-as 456
neighbor 12.78.45.49 weight 100
neighbor 12.78.45.49 distribute-list CIRCUIT2-IN in
neighbor 12.78.45.49 distribute-list CIRCUIT2-OUT out
no auto-summary
!
ip default-network 12.0.0.0
!
ip access-list extended CIRCUIT1-IN
permit ip host 12.0.0.0 host 255.0.0.0
ip access-list extended CIRCUIT1-OUT
permit ip host 212.15.80.0 host 255.255.255.0
permit ip host 212.15.80.0 host 255.255.255.128
ip access-list extended CIRCUIT2-IN
permit ip host 12.0.0.0 host 255.0.0.0
permit ip host 12.128.0.0 host 255.128.0.0
ip access-list extended CIRCUIT2-OUT
permit ip host 212.15.80.0 host 255.255.255.0
permit ip host 212.15.80.192 host 255.255.255.224
ip access-list extended LINK2_PREFERRED
permit ip 212.15.80.192 0.0.0.31 any
!
!
!
!
route-map PUSH_OVER_LINK2 permit 10
match ip address LINK2_PREFERRED
set ip next-hop verify-availability 12.78.45.49 1 track 1
!
!
end
Note: You may have to work with your ISP to accept smaller sized network advertisements. It may be beneficial to explain they only need to advertise the /24 out to the Internet and accept the smaller networks only within their topology.
