How to Build a Layer 3 MPLS VPN in Cisco Modeling Labs (IOS XR)

Overview

In this lab, we will build a Layer 3 MPLS VPN allowing customer edge (CE) routers to send private traffic across our MPLS Core. We will provide end-to-end connectivity between customer sites while maintaining route separation through VRFs. This lab will use the Cisco IOS XRd (IOS XR) image for all PE and P routers. To learn how to build a Layer 3 VPN with IOS XE, see How to Build a Layer 3 MPLS VPN in Cisco Modeling Labs (IOS XE).

You can download the lab.yaml file and router configuration files to get started with here.

Scenario

Customer A and Customer B both have 2 sites in different cities. Both customers have requested private Layer 3 VPNs between their respective offices. Neither customer will be able to see each other’s routes, despite connecting to the same provider edge equipment.

Method

The provider edge (PE) routers will peer via OSPF with the customer edge (CE) routers and use VRFs to keep the routing tables separate. They will then use MP-BGP to share the customers routes with each other via VPNv4. The customer routes and traffic will transit the core via MPLS, and the provider (P) routers will not see customer routes or IP Addresses, only transport labels. The end result is that CE-1A and CE-2A can communicate with each other, and CE-1B and CE-2B can as well. However, Customer A’s routers will not be able to communicate with Customer B’s routers or see their routes and vice versa.

Equipment

• Customer Edge Routers (IOSv)
  • CE-1A
  • CE-1B
  • CE-2A
  • CE-2B
• Provider Edge Routers (IOS XRd)
  • PE-1
  • PE-2
• Provider Core Routers (IOS XRd)
  • P1
  • P2

Step 1 – Adding the Customer Routers

We will add four IOSv images to the topology and connect them to the provider core. CE-1A and CE-1B will connect to PE-1 and CE-2A and CE-2B will connect to PE-2. We will configure them with IP addresses on the interfaces connecting to the service provider as well as loopback addresses. I’m using /30 subnets for the CE-PE links. Because customer routes will be segregated by VRFs, it is safe to use overlapping loopback addresses as long as they are unique within the customer network.

Here’s the configuration on CE-1A:

hostname CE-1A
!
ip domain-name nickdoeslabs.com
!
interface gigabitEthernet0/0
  no shutdown
  ip address 10.0.0.1 255.255.255.252
  description link to PE-1 
!
interface Loopback 0
  no shutdown
  ip address 1.1.1.1 255.255.255.255

The resulting topology looks like this:

With our CE routers configured we can now configure the customer-facing interfaces on the PE routers.

Step 2 – Basic Connectivity to the Provider Edge Routers

On the PE routers we will assign IP addresses on the links facing the customer routers. These will be assigned as follows:

We will also create our VRFs and assign the appropriate interfaces to them. VRFs will provide a virtual routing table, keeping customer routes and traffic isolated and private. VRFs are Layer 3 tools that function similarly to VLANs at Layer 2. Layer 3 traffic and routes cannot cross between VRFs by default.

Our VRF assignments will be:

For each PE router we will use the following configurations:

!!!! PE-1 !!!!!
config
!
VRF CUSTOMER_A
VRF CUSTOMER_B
commit
!
int gi0/0/0/1
  ip address 10.0.0.2 255.255.255.252
  no shutdown
  vrf CUSTOMER_A
!
int gi0/0/0/3
  ip address 10.0.0.6 255.255.255.252
  no shutdown
  vrf CUSTOMER_B
!
commit

!!!! PE-2 !!!!
config
!
VRF CUSTOMER_A
VRF CUSTOMER_B
commit
!
int gi0/0/0/1
  ip address 10.0.0.10 255.255.255.252
  no shutdown
  vrf CUSTOMER_A
!
int gi0/0/0/3
  ip address 10.0.0.14 255.255.255.252
  no shutdown
  vrf CUSTOMER_B
!
commit

At this point both PE routers can ping their directly connected customer routers:

This is a good time to take note of the isolation that VRF’s provide. We can only ping each customer within it’s respective VRF. Pinging between VRF’s will fail. In this example, because the 10.0.0.1 route belongs to the CUSTOMER_A VRF, we cannot ping it from the CUSTOMER_B VRF:

You can also see that we have separate routing tables for each customer:

Next we will set up OSPF so that the CE routers can share their routes with the PE routers.

Step 3 – Configuring OSPF (CE-PE)

We need a way for the customers to share their routes with the service provider. While this can be done with any routing protocol, I’m going to use OSPF since it is popular in enterprise networking.

The routers will all be in OSPF area 0 and the only networks the customers are advertising is their loopbacks and their WAN interfaces. The router IDs have to be unique so that the PE routers don’t flag them as duplicate router IDs.

!!!CE-1A!!!
router ospf 1
  router-id 10.10.10.10
  network 1.1.1.1 0.0.0.0 area 0
  network 10.0.0.0 0.0.0.3 area 0
  passive-interface Lo0

!!!CE-1B!!!
router ospf 1
  router-id 11.11.11.11
  network 1.1.1.1 0.0.0.0 area 0
  network 10.0.0.4 0.0.0.3 area 0
  passive-interface Lo0

!!!CE-2A!!!
router ospf 1
  router-id 12.12.12.12
  network 2.2.2.2 0.0.0.0 area 0
  network 10.0.0.8 0.0.0.3 area 0
  passive-interface Lo0

!!!CE-2B!!!
router ospf 1
  router-id 13.13.13.13
  network 2.2.2.2 0.0.0.0 area 0
  network 10.0.0.12 0.0.0.3 area 0
  passive-interface Lo0

Next we will configure the provider edge routers. For these we will need to configure a separate process within each customer’s VRF to keep the processes separate. The PE routers can use the same Router ID to identify themselves for each process.

!!! PE-1 !!!
config
!
router ospf CUSTOMER_A
  router-id 1.1.1.1
  vrf CUSTOMER_A 
  area 0
  int gi0/0/0/1
!!
router ospf CUSTOMER_B
  router-id 1.1.1.1
  vrf CUSTOMER_B
  area 0 
  int gi0/0/0/3
!!
commit
end

!!! PE-2 !!!
config
!
router ospf CUSTOMER_A
  router-id 3.3.3.3
  vrf CUSTOMER_A 
  area 0
  int gi0/0/0/1
!!
router ospf CUSTOMER_B
  router-id 3.3.3.3
  vrf CUSTOMER_B
  area 0 
  int gi0/0/0/3
!!
commit
end

At this point we can run show ip route vrf CUSTOMER_* and confirm that we have learned the customer’s loopback routes via OSPF in each VRF.

Now we are ready to share these routes between our PE routers. We will do this with BGP via VPNv4.

Step 3 – Configuring BGP (PE-PE)

At this point both PE routers know about the customer routes advertised by their directly connected OSPF neighbor (CE Router). Our goal is to get PE-1 and PE-2 to exchange those routes with each other. BGP is perfect for this for two reasons. First, BGP allows remote peering which means we can peer the two PE routers even though they could be in different cities, or even countries. As long as we have Layer 3 connectivity between them (we do through IS-IS), we can make them BGP neighbors. More importantly, BGP is the only routing protocol that supports VPNv4 (via MP-BGP) which will allow us to keep our customers routes separate from each other while they pass through the provider core.

We’ll start by configuring the peering between PE-1 and PE-2 with the following commands:

!!! PE-1 !!!
config
!
router bgp 500
  address-family vpnv4 unicast
  neighbor 3.3.3.3 
    address-family vpnv4 unicast
    remote-as 500
    update-source Lo0
!
commit
!
end

!!! PE-2 !!!
config
!
router bgp 500
  address-family vpnv4 unicast
  neighbor 1.1.1.1 
    address-family vpnv4 unicast
    remote-as 500
    update-source Lo0
!
commit
!
end

Once this is done we can verify the neighborship using the command show bgp vpnv4 unicast neighbors brief:

Now we just need to pass the customer routes learned via OSPF into BGP and vice versa. But before we do that we need to add unique route distinguishers and route targets to our VRFs.

Step 4 – Configuring Route Distinguishers and Route Targets

Route distinguishers and route targets are components of VPNv4 which help keep customer routes and traffic segregated. The route distinguisher is a unique label which is added to customer routes to distinguish them from each other. This allows for what we call overlapping routes, customers using the same IPs. Because the IP is encapsulated in a route distinguisher, it doesn’t cause conflicts. Route targets tell the PE router which VRF to import the customer route to. I discuss these concepts in detail in this post: Provider Networking – Route Distinguishers vs Route Targets.

To configure the route distinguishers and route targets we will use the following commands on both PE routers. The Import route-target tells the router which routes to bring in to the VRF. The export route target tells the router what to label them as they leave the VRF. Also note that we are using address-family ipv4 since the customer routes getting put in the VRF will be ipv4. This is not to be confused with vpnv4 which we are using to transport them.

Note: It is recommended to use a unique route distinguisher for each PE. A good format is router-id:customer-id. This isn’t necessary in this lab because it is a basic example.

config
!
vrf CUSTOMER_A
 rd 500:1
 address-family ipv4 unicast
  import route-target
   100:100
  !
  export route-target
   100:100
  !
!
vrf CUSTOMER_B
 rd 500:2
 address-family ipv4 unicast 
  import route-target
   200:200
  !
  export route-target
   200:200
  !
!
commit
!
end

Step 5 – Redistributing Routes

This final part is pretty straightforward. We have 2 routing protocols running. OSPF knows the path to each customer router but not the path through the provider network. BGP knows the path through the provider network but not to the customer routers. We are going to redistribute routes from each OSPF process into BGP, separating them by VRF. We are also going to do the reverse, and redistribute routes from BGP into OSPF.

Running these commands on both routers will accomplish this:

config
router bgp 500
  vrf CUSTOMER_A
    address-family ipv4 unicast
      redistribute ospf CUSTOMER_A
!
  vrf CUSTOMER_B
    address-family ipv4 unicast
      redistribute ospf CUSTOMER_B
!
router ospf CUSTOMER_A
  address-family ipv4 unicast
    vrf CUSTOMER_A
      redistribute bgp 500
!
router ospf CUSTOMER_B
  address-family ipv4 unicast
    vrf CUSTOMER_B  
     redistribute bgp 500
!
commit
end

At this point we can verify that our PE routers now see the routes from the other side of the VPN by running show ip route vrf [VRF NAME]:

As we can see each PE router has learned the directly connected customer loopback through OSPF, and the remote customer loopback (and WAN subnet) through BGP. The routes have been segregated into separate routing tables that are private to each VRF.

Step 6 – Verification

First we are going to go to each CE router and make sure we can ping the remote CE router through the VPN.

Pings are successful between the CE routers across the VPN. We can also check the routing tables on each CE router and we will see the remote routes marked as OSPF Inter Area since they were redistributed into OSPF.

We can also take a peek behind the curtain and see the VPNv4 table on our PE routers with the command show ip bgp vpnv4 unicast:

Here we can see that the customer routes are separated by their route distinguishers which is what really makes the magic happen.

I want to demonstrate one more thing since it is the key to understanding the Layer 3 MPLS VPN data plane. I’m going to run the command show ip bgp vpnv4 unicast labels:

Here, in the far right column we can see the actual MPLS labels that are being shared automatically via MP-BGP. These labels will encapsulate the customer data, keeping it segregated as it crosses the MPLS core. For example when Customer A wants to send some data to 2.2.2.2, the router will prepend it with the MPLS label 24005 before sending it out. It will then get a standard MPLS Transport label as well which will allow it to cross the core.

Conclusion

And that’s it! We have successfully configured private layer 3 VPN services for two customers and verified that they are working properly. We used VPNv4 (via MP-BGP), OSPF, VRFs, Route Distinguishers, and Route Targets to accomplish this, on top of our existing core that is running MPLS-LDP and IS-IS.

If you’d like to download the lab.yaml file for this lab as well as the router configurations, they are available for free here.