MPLS: Inter–AS Option C : Route Reflector Issues
This is in continuation from the previous post MPLS: Inter-AS VPNs: Option C: Multi-hop peering with remote PE
· Sometimes there might be a situation, where the remote neighbor is peering with our RR, rather than the PE.
· In such cases, let’s see what problems will arise???
· Let’s assume P2 is RR for AS 200
· Now, PE1 is peering with P2 instead of PE2.
· So, I need to advertise P2’s loopback also into BGP so that PE1 can reach P2.
ASBR2(config)#router bgp 200
ASBR2(config-router)#network 3.3.3.3 mask 255.255.255.255
ASBR2(config-router)#
ASBR1(config)#do show run | i prefix-list PE2-LOOP
!
ip prefix-list PE2-LOOP seq 5 permit 4.4.4.4/32
!
ASBR1(config)#ip prefix-list PE2-LOOP seq 10 permit 3.3.3.3/32
ASBR1(config)#
PE1#show ip route 3.3.3.3
Routing entry for 3.3.3.3/32
Known via "ospf 1", distance 110, metric 1
Tag 200, type extern 2, forward metric 128
Last update from 100.1.12.2 on Serial1/1, 00:01:52 ago
Routing Descriptor Blocks:
* 100.1.12.2, from 7.7.7.7, 00:01:52 ago, via Serial1/1
Route metric is 1, traffic share count is 1
Route tag 200
PE1#ping 3.3.3.3 source 1.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 76/112/152 ms
PE1#
P2(config-router)#do ping 1.1.1.1 source 3.3.3.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
Packet sent with a source address of 3.3.3.3
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 92/128/156 ms
P2(config-router)#
Configuring EBGP VPN4 between PE1 and P2 (which is RR for AS 200):
PE1(config)#do show run | i neighbor
bgp log-neighbor-changes
neighbor 4.4.4.4 remote-as 200
neighbor 4.4.4.4 ebgp-multihop 255
neighbor 4.4.4.4 update-source Loopback0
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended
PE1(config)#router bgp 100
PE1(config-router)#no neighbor 4.4.4.4 remote-as 200
PE1(config-router)#
*Jun 22 13:34:58.757: %BGP-5-ADJCHANGE: neighbor 4.4.4.4 Down Neighbor deleted
PE1(config-router)#neighbor 3.3.3.3 remote-as 200
PE1(config-router)#neighbor 3.3.3.3 ebgp-multihop 255
PE1(config-router)#neighbor 3.3.3.3 update-source Loopback0
PE1(config-router)#address vpnv4 uni
PE1(config-router-af)#neighbor 3.3.3.3 activate
PE1(config-router-af)#
PE1(config-router-af)#
P2(config-router-af)#do show run | begin router bgp 200
router bgp 200
bgp router-id 3.3.3.3
no bgp default ipv4-unicast
no bgp default route-target filter
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 100
neighbor 1.1.1.1 ebgp-multihop 255
neighbor 1.1.1.1 update-source Loopback0
neighbor 4.4.4.4 remote-as 200
neighbor 4.4.4.4 update-source Loopback0
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community extended
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended
exit-address-family
!
PE2(config)#do show run | i neighbor
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 100
neighbor 1.1.1.1 ebgp-multihop 255
neighbor 1.1.1.1 update-source Loopback0
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community extended
PE2(config)#router bgp 200
PE2(config-router)#no neighbor 1.1.1.1 remote-as 100
PE2(config-router)#neighbor 3.3.3.3 remote-as 200
PE2(config-router)#neighbor 3.3.3.3 update-source loopback 0
PE2(config-router)#add vpnv4 uni
PE2(config-router-af)#neighbor 3.3.3.3 activate
PE2(config-router-af)#
PE2(config-router-af)#
PE1#show ip bgp vpnv4 all
BGP table version is 15, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:1 (default for vrf VPN_A)
*> 10.1.15.0/24 0.0.0.0 0 32768 ?
*> 10.1.46.0/24 3.3.3.3 0 200 ?
*> 11.11.11.11/32 10.1.15.5 65 32768 ?
*> 12.12.12.12/32 3.3.3.3 0 200 ?
PE1#
P2(config-router)#do show ip bgp vpnv4 all
BGP table version is 5, local router ID is 3.3.3.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:1
*> 10.1.15.0/24 1.1.1.1 0 0 100 ?
*>i10.1.46.0/24 4.4.4.4 0 100 0 ?
*> 11.11.11.11/32 1.1.1.1 65 0 100 ?
*>i12.12.12.12/32 4.4.4.4 65 100 0 ?
P2(config-router)#
PE2#show ip bgp vpnv4 all
BGP table version is 15, local router ID is 4.4.4.4
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:1 (default for vrf VPN_A)
*>i10.1.15.0/24 1.1.1.1 0 100 0 100 ?
*> 10.1.46.0/24 0.0.0.0 0 32768 ?
*>i11.11.11.11/32 1.1.1.1 65 100 0 100 ?
*> 12.12.12.12/32 10.1.46.6 65 32768 ?
PE2#
Observations:
· In PE2, next-hop for the routes advertised by PE1, is PE1’s loopback 1.1.1.1 as Route-Reflector P2 didn’t change the next-hop address (EBGP à IBGP : next-hop will not be changed)
· In PE1, next-hop for the routes advertised by PE2, is RR’s loopback 3.3.3.3 as P2 changes the next-hop while passing IBGP update to EBGP neighbor.
CE1#traceroute 12.12.12.12
Type escape sequence to abort.
Tracing the route to 12.12.12.12
1 10.1.15.1 84 msec 12 msec 32 msec
2 100.1.12.2 [MPLS: Labels 19/20 Exp 0] 212 msec 204 msec 236 msec
3 100.1.27.7 [MPLS: Labels 21/20 Exp 0] 192 msec 228 msec 248 msec
4 100.1.78.8 [MPLS: Labels 17/20 Exp 0] 252 msec 188 msec 184 msec
5 100.1.38.3 [MPLS: Label 20 Exp 0] 188 msec 292 msec 236 msec
6 10.1.46.4 [MPLS: Label 21 Exp 0] 156 msec 172 msec 136 msec
7 10.1.46.6 268 msec * 244 msec
CE1#
Observation:
· CE1 – PE1 – P1 – ASBR1 – ASBR2 – P2 – PE2 – CE2
· This seems to be fine
· But what If I have another path to reach PE2 from ASBR2 (Just the control information is received from the Route Reflector and the actual traffic need not go through Route Reflector)
Let me add one more link between ASBR2 – PE2
ASBR2 : Ser1/2 – ASBR2 : Ser1/2
PE2(config-if)#do show run int ser1/2
Building configuration...
Current configuration : 88 bytes
!
interface Serial1/2
ip address 100.1.48.4 255.255.255.0
serial restart-delay 0
end
PE2(config-if)#
ASBR2(config-if)#do show run int ser1/2
Building configuration...
Current configuration : 88 bytes
!
interface Serial1/2
ip address 100.1.48.8 255.255.255.0
serial restart-delay 0
end
ASBR2(config-if)#
ASBR2(config-if)#int ser1/0
ASBR2(config-if)#ip ospf cost 60000
ASBR2(config-if)#router ospf 1
ASBR2(config-router)#network 100.1.48.8 0.0.0.0 area 0
ASBR2(config-router)#
PE2(config-if)#router ospf 1
PE2(config-router)#net
PE2(config-router)#network 100.1.48.4 0.0.0.0 area 0
PE2(config-router)#
ASBR2(config-router)#do show ip route ospf
100.0.0.0/8 is variably subnetted, 5 subnets, 2 masks
O 100.1.34.0/24 [110/128] via 100.1.48.4, 00:00:01, Serial1/2
3.0.0.0/32 is subnetted, 1 subnets
O 3.3.3.3 [110/129] via 100.1.48.4, 00:00:01, Serial1/2
4.0.0.0/32 is subnetted, 1 subnets
O 4.4.4.4 [110/65] via 100.1.48.4, 00:00:01, Serial1/2
ASBR2(config-router)#
· Now, ASBR2 is going through PE2 to reach P2 (which is RR)
Let’s see the traceroute now
CE1#
CE1#traceroute 12.12.12.12
Type escape sequence to abort.
Tracing the route to 12.12.12.12
1 10.1.15.1 72 msec 32 msec 28 msec
2 100.1.12.2 [MPLS: Labels 19/20 Exp 0] 192 msec 200 msec 220 msec
3 100.1.27.7 [MPLS: Labels 21/20 Exp 0] 188 msec 144 msec 200 msec
4 100.1.78.8 [MPLS: Labels 17/20 Exp 0] 188 msec 244 msec 160 msec
5 100.1.48.4 [MPLS: Labels 17/20 Exp 0] 184 msec 204 msec 220 msec
6 100.1.34.3 [MPLS: Label 20 Exp 0] 244 msec 268 msec 192 msec
7 10.1.46.4 [MPLS: Label 21 Exp 0] 188 msec 184 msec 172 msec
8 10.1.46.6 204 msec * 268 msec
CE1#
Observations:
· Traffic went through one unnecessary loop between PE2 – P2(RR)
· Actually traffic should go to PE2, which is the original last-hop for 12.12.12.12/32, But RR has changed the next-hop to it-self while forwarding it to PE1.
· That’s why PE1 is sending the data first to P2 (RR) and then RR is sending the data to PE2, which is the actual last-hop for 12.12.12.12/32
· I have just one redundant connection, but think about multiple paths that can exist in the Service Provider Network.
· If something goes wrong, troubleshooting would be a night mare!
· Solution: change the settings on P2(RR), so that it won’t change the next-hop value while forwarding the update to PE1.
P2(config-router)#router bgp 200
P2(config-router)#address-family vpnv4 unicast
P2(config-router-af)#neighbor 1.1.1.1 next-hop-?
next-hop-self next-hop-unchanged
P2(config-router-af)#neighbor 1.1.1.1 next-hop-unchanged
P2(config-router-af)#
PE1#show ip bgp vpnv4 all
BGP table version is 17, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:1 (default for vrf VPN_A)
*> 10.1.15.0/24 0.0.0.0 0 32768 ?
*> 10.1.46.0/24 4.4.4.4 0 200 ?
*> 11.11.11.11/32 10.1.15.5 65 32768 ?
*> 12.12.12.12/32 4.4.4.4 0 200 ?
PE1#
CE1#traceroute 12.12.12.12
Type escape sequence to abort.
Tracing the route to 12.12.12.12
1 10.1.15.1 68 msec 80 msec 0 msec
2 100.1.12.2 [MPLS: Labels 18/21 Exp 0] 208 msec 168 msec 204 msec
3 100.1.27.7 [MPLS: Labels 20/21 Exp 0] 156 msec 184 msec 200 msec
4 100.1.78.8 [MPLS: Labels 18/21 Exp 0] 208 msec 220 msec 156 msec
5 10.1.46.4 [MPLS: Label 21 Exp 0] 104 msec 200 msec 108 msec
6 10.1.46.6 204 msec * 160 msec
CE1#
Observation:
· Now, we observe that the extra looping is not happening and the traffic is directly going to the last-hop PE2.
· Router Reflector need not come in the actual data path, it exist only in the control path.
· It is very essential to remember the above mentioned point while designing the Route Reflector in any network.
