Using apply-groups to insert policy in a predictable manner….

Recently I’ve been solving some interesting problems around how routing policy can be implemented. What I’ve been trying to do is have a way that enables me to quickly insert a policy at the start of a set of routing-policies on a BGP neighbor to stop sharing routes.

For those who are not aware, you can implement multiple policies in a chain on a BGP session. You do this like so;

[edit protocols bgp group R2]
[email protected]# show 
export [ policy-1 policy-2 policy-3 ];
neighbor 192.168.3.1 {
    peer-as 2;
}

In this configuration, all policies will be evaluated sequentially. Policy with matching terms will take action as defined in the policy on the routes matched. Routes will continue to be processed by subsequent terms in each policy until a terminating action is hit. A “terminating action” is something like an “accept” or “reject” action (once you hit one of these you use the actions gathered so far including the terminating action, then stop processing)

Let us say for the sake of this test that the policies we have applied to our peer do 2x as-path prepends, add a MED value, then accept all routes;

[edit policy-options]
[email protected]# show 
policy-statement policy-1 {
    then as-path-prepend "4 4";
}
policy-statement policy-2 {
    then {
        metric 400;
    }
}
policy-statement policy-3 {
    then accept;
}

This would result in the following routes being sent from this router (note the MED and as-path prepends);

[email protected]# run show route advertising-protocol bgp 192.168.3.1    

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
  Prefix		  Nexthop	       MED     Lclpref    AS path
* 172.16.172.0/24         Self                 400                4 4 [4] I
* 192.168.3.0/24          Self                 400                4 4 [4] I
* 192.168.6.0/24          Self                 400                4 4 [4] I

In the scenario I described at the start, the aim was to be able to easily (with one line of configuration) insert a policy at the start of the policy-set that rejected all routes. However, as described above, this peer might have existing policies that allow routes to be sent.

Assume we say that I have already created the following policy;

[edit policy-options]
+   policy-statement policy-reject {
+       then reject;
+   }

If I quickly activated this by adding it as an export policy, it would appear as the last policy on the policy-chain;

[email protected]# set protocols bgp group R2 export policy-reject 

[edit]
[email protected]# show | compare 
[edit protocols bgp group R2]
-    export [ policy-1 policy-2 policy-3 ];
+    export [ policy-1 policy-2 policy-3 policy-reject ];

As we can see below, this does not reject any of the routes, because the reject action is processed after a terminating accept action in policy-3;

[email protected]# run show route advertising-protocol bgp 192.168.3.1              

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
  Prefix		  Nexthop	       MED     Lclpref    AS path
* 172.16.172.0/24         Self                 400                4 4 [4] I
* 192.168.3.0/24          Self                 400                4 4 [4] I
* 192.168.6.0/24          Self                 400                4 4 [4] I

In order to implement this in the manner I want with this way of doing it, I would have to do the following to achieve it;

[edit protocols bgp group R2]
[email protected]# delete export 
[email protected]# set export policy-reject
[email protected]# set export policy-1
[email protected]# set export policy-2
[email protected]# set export policy-3

You could shorten this by using the “insert” functionality, however you are still requiring a re-ordering of policies as opposed to a one liner to activate this.

The trick I have come up to solve this is quite cool. I am adding a term to an existing policy with an apply-group;

[email protected]# show | compare 
[edit]
+ groups {
+     policy-reject {
+         policy-options {
+             policy-statement policy-1 {
+                 term reject-term {
+                     then reject;
+                 }
+             }
+         }
+     }
+ }

When we activate this apply-group, the term gets inserted into policy-1, which is already ordered to be applied prior to the terminating accept action in policy-3;

[email protected]# show | compare 
[edit]
+ apply-groups policy-reject;

We can see here that it is implemented in policy-1;

[edit policy-options policy-statement policy-1]
[email protected]# show | display inheritance 
##
## 'reject-term' was inherited from group 'policy-reject'
##
term reject-term {
    ##
    ## 'then' was inherited from group 'policy-reject'
    ## 'reject' was inherited from group 'policy-reject'
    ##
    then reject;
}
then as-path-prepend "4 4";

As policy-1 is already the first policy to be processed, this ensures that without having to re-order our existing policies we can insert a reject policy at the start. We can now check the result;

[email protected]# run show route advertising-protocol bgp 192.168.3.1    

[edit]
[email protected]#

And it is working!

You might ask why I did not just apply a new policy in an apply-group instead of a new term? The reason for this is that new policies are added at the end of the policy chain, while a new term in a pre-existing policy uses the existing place that policy has in the policy-chain.

For my purposes, this has enabled me to give out a one-line command which a large group of people can use to disable the advertisement of certain routes at certain times. As with everything in Junos though, there are many possible uses for this set of functionality. Hope this helps!

BGP local-as – as-path manipulation options

BGP local-as is a handy feature which allows you to pretend to be a member of a different ASN for the purposes of peering with another ASN. This is pretty handy for migration purposes – i.e. if ISP A buys ISP B and wants to migrate all of ISP B’s customers to peer with ISP A without having to get them to change their peering configuration. This feature is very simple to configure, and most who have worked for any length of time in a service provider will be familiar with it, however many will be googling if they have to implement any of the as-path manipulation features offered within it! In this blog post, I’m going to go through each of these as-path manipulation options in some detail.

This blog post is based on a lab of 4 routers (each in its own ASN) laid out in the following manner;
BGP local-as as-path lab

The base configuration of the lab we shall be working on can be found here; local-as-blog-lab.pdf

Before we get into this, it’s worth noting that i’m configuring all my BGP parameters in the “neighbor” hierarchy out of sheer force of habit – but I could equally be configuring the peer-as & local-as in the “group” hierarchy.

As noted in the diagram, 1.1.1.1/32 is advertised into BGP from R1, and 6.6.6.6/32 from R6. These are the only routes that are being advertised via BGP, and are there to show us what happens in each direction when we configure local-as with the various options offered to us within this feature.

Right, so first up, lets say that R2 thinks that R4’s ASN is actually AS9999. We’ll firstly configure this on R2;

[edit protocols bgp group R4 neighbor 192.168.3.2]
-     peer-as 4;
+     peer-as 9999;

Now we will find that this BGP session isn’t doing so well at coming up;
On R2;

[email protected]# run show bgp summary | match 9999           
192.168.3.2            9999          0          2       0       0          14 Active

Okay, so now we need to turn on local-as on R4 for the BGP session facing R2 in order that R4 pretends to be inside AS9999;

[edit protocols bgp group R2 neighbor 192.168.3.1]
+      local-as 9999;

And it’s now up and looking good – R4 is pretending for the sakes of the session to R2 that it is in AS9999;

[email protected]# run show bgp summary | match 9999    
192.168.3.2            9999          5          5       0       0          48 1/1/1/0              0/0/0/0

Okay, now we’ll have a look at how the routes look in either direction. Firstly lets examine 1.1.1.1/32 as received on R6;

[email protected]# run show route 1.1.1.1 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1/32         *[BGP/170] 00:02:29, localpref 100
                      AS path: 4 9999 2 1 I
                    > to 192.168.6.1 via ge-0/0/1.0

And now for 6.6.6.6/32 on R1;

[email protected]# run show route 6.6.6.6 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

6.6.6.6/32         *[BGP/170] 00:03:06, localpref 100
                      AS path: 2 9999 4 6 I
                    > to 192.168.1.2 via ge-0/0/1.0

We can see that as9999 now appears as an additional ASN in the as-path in both directions through the R2-R4 BGP peering. This is important to note – while we would expect it to be added as routes are learned by R2 from R4, R4 is also adding it to routes it learns to keep the as-path consistent in both directions.

Now lets have a look at the options available to us for the local-as feature;

[email protected]# set protocols bgp group R2 neighbor 192.168.3.1 local-as ?
Possible completions:
  <as_num>              Autonomous system number in plain number or 'higher 16bits'.'Lower 16 bits' (asdot notation) format
  alias                Treat this AS as an alias to the system AS
  loops                Maximum number of times this AS can be in an AS path (1..10)
  no-prepend-global-as  Do not prepend global autonomous-system number in advertised paths
  private              Hide this local AS in paths learned from this peering

Before we go any further, it’s worth noting that you cannot configure both the private and alias options on the same neighbor/group.

Loops is not going to be covered in in this post, but at a high level this is a way to influence allow X number of as-path loops in the as-path.

The private option instructs the router with local-as configured to stop adding the configured “local-as” to routes it learns from the peer with local-as configured against it.

Lets configure the private option on this on R4 now;

[edit protocols bgp group R2 neighbor 192.168.3.1 local-as]
+      private;

As expected, there is no change on the 6.6.6.6/32 route from R1, which R4 is learning from R2;

[email protected]# run show route 6.6.6.6    

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

6.6.6.6/32         *[BGP/170] 00:00:02, localpref 100
                      AS path: 2 9999 4 6 I
                    > to 192.168.1.2 via ge-0/0/1.0

However, R6 now sees 1.1.1.1/32 without as9999 in the as-path – R4 is no longer adding this to the as-path as it receives the route from R2;

[email protected]# run show route 1.1.1.1 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1/32         *[BGP/170] 00:02:29, localpref 100
                      AS path: 4 2 1 I
                    > to 192.168.6.1 via ge-0/0/1.0

We’ll now roll this back ready to move on to the next option;

[email protected]# show | compare 
[edit protocols bgp group R2 neighbor 192.168.3.1 local-as]
-      private;

The next option we will look at is the alias feature. This behaves the same way as private for routes learned from the peer that local-as is configured on – it does not add the local-as to the path, but as normal just adds the local system ASN as normal. However the difference is that when sending routes to the neighbor, the router with local-as configured will omit the local system ASN, placing just the configured local-as in the as-path (instead of the default behaviour which is to insert both the local system and configured local-as in the path).

Given that there is nothing like seeing it for yourself, we will now configure this option on R4 and take a peek;

[edit protocols bgp group R2 neighbor 192.168.3.1 local-as]
+      alias;

Firstly we inspect 1.1.1.1/32 (learned by R4 from R2 and viewed on R6) which we can see looks as if there was no local-as configured on R4 at all – the as-path is as normal. This is the same as in the private option;

[email protected]# run show route 1.1.1.1 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1/32         *[BGP/170] 00:00:47, localpref 100
                      AS path: 4 2 1 I
                    > to 192.168.6.1 via ge-0/0/1.0

Next, we’ll have a look at 6.6.6.6/32 (sent by R4 to R2 and viewed on R1) which we can see has AS9999 but not AS4 in the path;

[email protected]# run show route 6.6.6.6    

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

6.6.6.6/32         *[BGP/170] 00:00:56, localpref 100
                      AS path: 2 9999 6 I
                    > to 192.168.1.2 via ge-0/0/1.0

Before we move on, lets quickly remove the alias option so that we can see what happens when we configure the next feature;

[edit protocols bgp group R2 neighbor 192.168.3.1 local-as]
-      alias;

The final option available to us is one called no-prepend-global-as. While private and alias cannot be configured at the same time as each other (as alias is the functionality of private plus one further change to the as-path), no-prepend-global-as can be configured in conjunction with either of these other options. The no-prepend-global-as essentially implements only the function that is present in alias but not private – this is that when sending routes to the neighbor, the router with local-as configured will omit the local system ASN, placing just the configured local-as in the as-path (instead of the default behaviour which is to insert both the local system and configured local-as in the path).

We’ll now configure this;

edit protocols bgp group R2 neighbor 192.168.3.1 local-as]
+      no-prepend-global-as;

And then we’ll have a look. Like alias we can see that the configured local-as on R4 is in the as-path as seen by R1 (learned from R4 by R2), but not the R4 system AS;

[email protected]# run show route 6.6.6.6 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

6.6.6.6/32         *[BGP/170] 00:05:53, localpref 100
                      AS path: 2 9999 6 I
                    > to 192.168.1.2 via ge-0/0/1.0

However, we can see that unlike alias, it does not modify the default behaviour on routes learned from R2 by R4 (and we have both the R4 system ASN + the configured local-as;

[email protected]# run show route 1.1.1.1 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1.1.1.1/32         *[BGP/170] 00:06:19, localpref 100
                      AS path: 4 9999 2 1 I
                    > to 192.168.6.1 via ge-0/0/1.0

Local-as is a really useful feature – generally for the purposes of migrations. Some of the as-path manipulation options can be invaluable in complex/odd peering scenarios where there is a requirement to disguise what is actually happening on the network – be it a superficial requirement or the prevention of a an apparent loop being signified in the as-path.

I hope this article helps clarify the various options for you!

Subpolicies

A useful feature I’ve been playing with is the ability to use “sub” policies within JUNOS policies for the purposes of matching. This is quite useful for me in some of the work I’ve been doing, as I’ve managed to shrink ~300 lines of policy to ~120 lines by implementing this as much as possible.

Essentially the way it works is that you can use a policy as a from criterion in another policy. i.e.

[edit policy-options policy-statement peerXXX-bgp-export]
[email protected]# show 
term default {
    from policy subpolicy_match_default;
    then reject;
}

[edit policy-options policy-statement subpolicy_match_default]
[email protected]# show 
term v4-default {
    from {
        family inet;
        route-filter 0.0.0.0/0 exact;
    }
    then accept;
}
term v6-default {
    from {
        family inet6;
        route-filter 0::/0 exact;
    }
    then accept;
}
then reject;

What will happen is that everything that is accepted by subpolicy_match_default is a MATCH for the purposes of term default in policy peerXXX-bgp-export. Everything that is rejected by subpolicy_match_default is NO MATCH for the purposes of this. So in this case both the v4 and v6 default routes will be accepted by the sub-policy which will cause the main policy to reject them for export to peerXXX (as they match the term in the main policy which then rejects them), while all other routes will be rejected (as they do not match the term in the main policy which is doing the reject (given that routes rejected by the sub policy are not a match for the purposes of the main policy).

A fairly trivial feature probably, but hellishly useful when configuring lots of similar policies to be able to refer to what is effectively a set of repetitively used subroutines within your policy….

BGP “allow”

A useful feature which I’ve struck (and am in fact currently waiting for a midnight change window to implement on one of our route reflectors) is the BGP “allow” feature in JUNOS. What this does is allows you to specify a netblock for RPD to accept incoming BGP connections from (in a passive mode) and stand up neighbor relationships to any connecting devices – without having to specifically configure neighbors for each device.

The configuration for this quite simple – in place of a neighbor statement we do this;

set protocols bgp group ABC allow 1.1.1.0/24

The one caveat I’ve struck so far with this is that for some reason JUNOS will not allow you to do MD5 auth on any BGP group with this feature enabled;

[email protected]# set protocols bgp group ABC authentication-key beer   

{master}[edit]
[email protected]# commit check 
re0: 
[edit protocols bgp group ABC]
  'allow'
    May not be configured with authentication-key
error: configuration check-out failed: (statements constraint check failed)

{master}[edit]

I find this irritating, as I would consider it ever so slightly less secure to dynamically allow connections on say your route-reflectors from the rest of your network. Having said this, you should only be using this feature for IP ranges which you have a strong control over (i.e. know that nobody can spoof being inside), so this should not be an issue. But it is an irritating downside to an otherwise awesome feature.

I’ve so far struck a couple of interesting uses for this. The first is to use it on route-reflectors – when you’re deploying 10 MPLS PEs in a month it gets a little tiring constantly logging onto your route-reflectors and configuring up more and more BGP sessions to new devices. Call me lazy – but I see this as a quite cool way to automate this problem away.

The other use was potentially slightly more interesting. A friend was configuring a CDN type network with a bunch of content servers, and was after a way for VMs to be dynamically built and fired up on a particular vlan, then to start advertising anycast routes via a BGP session with no configuration on the routers (this making automating it easier for him). He successfully used this feature to achieve this.

The final thing I would note is that when using this feature, the Juniper isn’t going to perform exactly the same as it would when you have explicitly configured BGP neighbors as far as SNMP traps – as it’s not going to know about the peers until they initiate a connection, so if you are relying on traps to monitor sessions this may be something to watch out for.