Dr Ahriakin's Singalong JNCIE-Sec Blog

[zoidberg]

I knew it was in there somewhere. Just hard to remember which without firing up some traffic

Do you have just the 1 SPC? It has been a long time since I looked at that command, but I thought the values differed per SPU. I thought it worked that looking at the stats for the CP shows the number of new sessions created globally, as the CP knows all. Then each flow SPU will report the number of new sessions being created on them. In the case of a single SPC, where you have one SPU as CP and one SPU as flow, the number on the CP and the flow SPU would look the same. If you had another SPU, the cps reported per SPC should be roughly the CP cps / 3 flow SPU.

[zoidberg]

Alright. Time to stop being a slacker and fire up my test boxes to make sure I remember what I am talking about

[zoidberg]

Ok. Here are my quick and dirty results.


So this one seems to show the current number of sessions per SPU, not a cps. Though, you could do the math to measure that value between the time intervals.

> show security monitoring performance session node 0 fpc 3 pic 1    
node0:
--------------------------------------------------------------------------
fpc  3  pic  1
Last 60 seconds:
 0:  414719   1:  415095   2:  412903   3:  413290   4:  411118   5:  411459
 6:  409203   7:  409499   8:  407340   9:  407568  10:  405340  11:  405414
12:  403188  13:  403362  14:  401148  15:  401238  16:  399024  17:  399001

And the same command on the CP gave me nothing.

> show security monitoring performance session node 0 fpc 3 pic 0    
node0:
--------------------------------------------------------------------------
fpc  3  pic  0
Last 60 seconds:
 0:       0   1:       0   2:       0   3:       0   4:       0   5:       0
 6:       0   7:       0   8:       0   9:       0  10:       0  11:       0
12:       0  13:       0  14:       0  15:       0  16:       0  17:       0

Not sure what this is telling me.

> show security monitoring performance spu node 0 fpc 3 pic 0    
node0:
--------------------------------------------------------------------------
fpc  3  pic  0
Last 60 seconds:
 0:   5   1:   8   2:   6   3:   9   4:   6   5:   6
 6:   6   7:   9   8:   6   9:   9  10:   6  11:   9
12:   6  13:   9  14:   6  15:   9  16:   6  17:   9

> show security monitoring performance spu node 0 fpc 3 pic 1        
node0:
--------------------------------------------------------------------------
fpc  3  pic  1
Last 60 seconds:
 0:   0   1:   0   2:   0   3:   2   4:   0   5:   3
 6:   2   7:   4   8:   3   9:   3  10:   3  11:   4

And this one gave me nothing. Great. Love it when features disappear in new code :S

> show security monitoring fpc 3 node 0 | match Second 
Total Session Creation Per Second (for last 96 seconds on average):    0
IPv4  Session Creation Per Second (for last 96 seconds on average):    0
IPv6  Session Creation Per Second (for last 96 seconds on average):    0
Total Session Creation Per Second (for last 96 seconds on average):    0
IPv4  Session Creation Per Second (for last 96 seconds on average):    0
IPv6  Session Creation Per Second (for last 96 seconds on average):    0

And snmp seemed closest. It was higher than the cps being sent from my test box, but I'm assuming this is including session close rate as well. Or, it may be an average value over 30, 60, 90 seconds.

> show snmp mib walk jnxJsNodeSessionCreationPerSecond    
jnxJsNodeSessionCreationPerSecond.0 = 0
jnxJsNodeSessionCreationPerSecond.1 = 47650

[Ahriakin]

I tested it on a 5800 with 7 SPCs, all 13 flow SPUs show the same values (allowing for minor traffic variance between commands) that match to the SNMP side. So yup it is definitely at least trying to show the global CPS value under each SPU. This was on 11.4R6.5.

I think your show commands above were giving zeroes as from the SNMP output it looks like Node1 was active on that cluster.

[zoidberg]

Looking at it now, node1 is active for rg1. I was pretty sure I was on node0 for rg0 and rg1 when I played with this yesterday. Oh well. Need more sleep and more caffeine I guess

[Ahriakin]

I've done that a few times on maintenance windows, log in bleary eyed, then have a second or 2 of panic thinking there's an outage...then blink and check the other node...oh...

[zoidberg]

Ya, one of those weeks. I was too distracted and annoyed that my op scripts got wiped off my box that I neglected to do a simple show chassis cluster status, so I assumed my lab was the way I left it. Always a mistake, too many people know the password haha.

I get that panic from my customer now and then. They log into node1, do a show security flow session, and discover that all the sessions show 0 bytes sent and received, and then they freak out. Need to point out that those are the node0 backup sessions, and even though you're logged into node1, that command dumps all sessions starting with node0, unless you tell it otherwise.

7 SPCs? Very nice. Always cool to bump into other people with loaded 5800s... it's way more fun than Aldur and his rack of 210s Hehe

Do you do dual-control links? I haven't played with those on the 5800s yet, only on my 3600s. Though I may have enough lab gear now where I can get away with borrowing some REs for some play time. Just curious how well that works on the 5800s and if you've run into any interesting challenges with it.

[Ahriakin]

And getting ready to bump some to 9 SPCs (damn I wish I could do a Spinal-Tap '11' but then there'd be none of those pesky IOC things in there). Moooaaarrr Powarrrr !!!!!!!
I haven't played with dual links, since it's not going into split-brain unless the fabric-link fails also that'd be planning for triple failure ( I like direct runs so there's no common point of failure for either link except the cables themselves).

[Aldur]

7 SPCs? Very nice. Always cool to bump into other people with loaded 5800s... it's way more fun than Aldur and his rack of 210s Hehe

Awww, now you've gone done and hurt my feelings And by the way it's a rack of 100s, sheesh

But seriously now, I am kind of jealous, we typically use the branch stuff in Ed services so I don't get to play with the big bad boxes.

[zoidberg]

Moooaaarrr Powarrrr !!!!!!!

Oh ya, there is much more power coming soon I wish I could brag about my lab and really make Aldur feel like he's missing out (hehe), but I'm not certain what is and isn't public yet...

[zoidberg]

I haven't played with dual links, since it's not going into split-brain unless the fabric-link fails also that'd be planning for triple failure ( I like direct runs so there's no common point of failure for either link except the cables themselves).

Ditto. That's why I haven't played with them on the 5800s.

I've had challenges with the 3600 CRMs and dual-ctl-links, but there's a firmware update available that seems to be taking care of those headaches.

[Ahriakin]

Indeed, I have the new performance matrix and just put in for a quote for some of those nice new toys . Our network changes at an insane rate and we're always bugging our SE for the next big thing so we went through your 2013 roadmap end of last year..can't wait to get my hands on some.

[zoidberg]

they're shiny ... and heavy! haha. it's pretty impressive what they can do. looking forward to the network upgrades this year.

[Ahriakin]

Today was a mixed bag. I decided to focus on a set scenario rather than individual areas. The goal, OSPF peering internally between 1 standalone SRX and a cluster, that segment using a std. RETH on the cluster. The opposite side using standalone interfaces peering with 2 different routers via BGP to simulate multiple discrete WAN links. Then to try and use pre-empt, filtering and IP-monitoring to have the RETH follow the active BGP path from the standalones. I got as far as the base topology and peering but then realised I couldn't do IP-Monitoring on my small branch boxes in 11.1, so upgraded to 11.4 again before calling it quits for the day...I figured since I'd gone that far I should see it through even if it meant blurring software revision boundaries.

I think if I set RG1 to pre-empt (RETH side) for Node0, BGP to favour the peer on that chassis aswell then set IP-Monitoring to check Peer-A's WAN link address it should kinda work. Like I said I'm trying avoid Z-path forwarding if possible and have stateful redundancy 'follow the route'.

Anyway good practice as I haven't worked with Dynamic routing or IP-Monitoring on these before, hell I was straining to try and remember how to set it up on the Cisco routers I was using as BGP peers

[Ahriakin]

Busy busy day....tired tired Ahriakin. Methinks I overdid it a bit. I spent the afternoon working on the scenario above, focusing more on the routing side as it turns out since by my Cisco and Juniper route-fu is weak, but I got multiple OSPF and BGP instances with filtering working across it correctly. I didn't get the IP-Monitoring side working yet, but had to stop right as I got to it since I had the next InetZero lab session to do. Soo, a bite of food and on to the lab workbook.
Today was UTM, my weakest area by far and one I was dreading since I never use it and skimmed through just enough to pass it on the JNCIS-Sec. It was a actually a really good session. The lab itself was clear and well written and pretty challenging, with the end tasks having you revisit some of the earlier policies as you added more features and the like. Again terminal access was flawless, no crashes or loss of console connectivity that can happen with different rack vendors. I had a quick read of the O'Reilly security book chapter on it beforehand and then dove in. This is definitely an area where the actual configuration is MUCH easier than the texts make it seem. Yes there are a ton of options but as long as you don't get over zealous in your Stanza navigation it's very intuitive. Jumping section by section and then simply using 'set ?' can get you through a lot...but slowly....that's the key when deciding to spend more time on something or not, I know I could take a good stab a this in the lab but it would take me much longer than I could afford relying on working it out as I go. So even though it now worries me much less I will definitely be revisiting this one again soon.

Sooo, 3 hours left on my session but I'm calling it a day. I need to remind the missus who the strange man sitting in here is.

[Ahriakin]

Actually it looks like my little scenario would be better off using Conditional Route Advertizement, came across that one today on the 11.4 Security Configuration Guide, and that's what I shall be playing with tonight. Basically you use the inside Reth active status (well really which node the next-hop is active on) as a Conditional parameter that is then applied into your route export policy. So with 2 Standalone Wan links you set the redistribution policy for Wan1 to match your list only if those routes are active on Node0, block it otherwise, reverse for Wan2...at least that's what I think it's meant to do, have to play around with it to be sure . It's designed to avoid Z-path forwarding which is exactly what I want.

[Ahriakin]

Conditional Router Advertisement worked like a charm...in one direction. I can withdraw and re-advertise the LAN routes in sync with it's RETH failover across the each Node's WAN links but I haven't found a way yet to pull or re-prioritize the peers advertisement to the SRX at the same time, so it can still be asymmetric. The problem is the process has to track a RETH, so with 2 standalones on the WAN side I can't do the same in reverse. I tried tricking the system by making both WAN links the only interfaces in 2 new RETHs but that has it's own issues when it comes to actual failover. Anyway I'll have to think over this during the day and maybe take one more stab at it tonight before giving up.

[Ahriakin]

Frustrating night. During the day I decided to give up on the routing side, traffic will have to possibly be asymmetric. So I went to test Inter-VRF NAT as described in many examples....and it doesn't work. Every piece of documentation I could find describes using a Static-NAT set with "from instance". The NAT itself works, the problem is that since it's a Static NAT it takes place before the route-lookup, so when it gets to that stage it's still looking on the Ingress VRF and just loops back into the ingress network. I checked the logs from a flow trace and verified this behaviour. I spent a lot of time troubleshooting that part and got nowhere. So I took a different approach and configured opposing Source and Destination-NAT pairs for the overlapped addresses, I just made sure to set the Destination-NAT pools in the opposing VRF rather than the ingress in some blind attempt to force some sort of logical slingshot across their tables. And it worked. I've copied out my configs and some traces to look over tomorrow when I have time, I'm still not 100% that I did the InterVrf 100% right, or that I'm doing this correctly. Tired as hell at the moment so it'll have to wait.


EDIT: If anyone has gotten inter-vrf NAT working for overlapped space PLEASE chime in

[Ahriakin]

In case you're interested here are some of the details. I'm using the RFC6598 range of 100.64.0.0/10 for the NAT range on both sides (/11 per side).

Inter-VRF Static NAT
Routes on both sides are pretty similar (in particular both have 10.30.30.0/24 with different hops).

[edit security nat static]
            rule-set ACME-NAT {
                from routing-instance LAN;
                rule DOUBLE-NAT-ACME {
                    match {
                        destination-address 100.96.30.30/32;
                    }
                    then {
                        static-nat {
                            prefix {
                                10.30.30.30/32;
                            }
                        }
                    }
                }
            }
            rule-set LAN-NAT {
                from routing-instance ACME;
                rule DOUBLE-NAT-LAN {
                    match {
                        destination-address 100.64.10.202/32;
                    }
                    then {
                        static-nat {
                            prefix {
                                10.10.10.202/32;
                            }
                        }
                    }
                }
            }
        }
    }

Failure Flow Trace

Jan 15 21:30:05 21:30:05.520366:CID-1:RT:flow_initiate_first_path: first pak no session
Jan 15 21:30:05 21:30:05.520366:CID-1:RT:  flow find session returns error.
Jan 15 21:30:05 21:30:05.520366:CID-1:RT: ----- flow_process_pkt rc 0x7 (fp rc -1)
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:<10.10.10.202/250->100.96.30.30/2793;1> matched filter ICMP:
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:packet [84] ipid = 51601, @4091a31a
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:---- flow_process_pkt: (thd 1): flow_ctxt type 14, common flag 0x0, mbuf 0x4091a100, rtbl_idx = 5
Jan 15 21:30:06 21:30:06.536392:CID-1:RT: flow process pak fast ifl 69 in_ifp fe-0/0/0.0
## Initial connection
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  fe-0/0/0.0:10.10.10.202->100.96.30.30, icmp, (8/0)
Jan 15 21:30:06 21:30:06.536392:CID-1:RT: find flow: table 0x42687228, hash 15454(0xffff), sa 10.10.10.202, da 100.96.30.30, sp 250, dp 2793, proto 1, tok 20486
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  no session found, start first path. in_tunnel - 0, from_cp_flag - 0
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  flow_first_create_session
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:Installing pending sess (758) in ager
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:First path alloc and instl pending session, natp=0x44768170, id=758
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  flow_first_in_dst_nat: in <fe-0/0/0.0>, out <N/A> dst_adr 100.96.30.30, sp 250, dp 2793
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  chose interface fe-0/0/0.0 as incoming nat if.
## Static happens as expected, 100.96.30.30 xlated to 10.30.30.30
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:flow_first_rule_dst_xlate: packet 10.10.10.202->100.96.30.30 nsp2 0.0.0.0->10.30.30.30.
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:flow_first_routing: vr_id 5, call flow_route_lookup(): src_ip 10.10.10.202, x_dst_ip 10.30.30.30, in ifp fe-0/0/0.0, out ifp N/A sp 250, dp 2793, ip_proto 1, tos 0
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:Doing DESTINATION addr route-lookup
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:flow_rt_lkup in VR-id: 5
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:flow_rt_lkup: Found route entry 0x0x45bc02e0,nh id 0x264, out if 0x47
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:flow_rt_lkup: nh word 0x2640011
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:flow_ipv4_rt_lkup success 10.30.30.30, iifl 0x45, oifl 0x47
## Route-lookup happens on Ingress VRF, tries to push the connection back out the same Zone
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  routed (x_dst_ip 10.30.30.30) from LAN (fe-0/0/0.0 in 0) to fe-1/0/0.0, Next-hop: 10.10.15.20
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:flow_first_policy_search: policy search from zone LAN-> zone LAN (0x114,0xfa0ae9,0xae9)
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  app 0, timeout 60s, curr ageout 60s
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  packet dropped, denied by policy
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  denied by policy default-policy-00(2), dropping pkt
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:  packet dropped,  policy deny.
Jan 15 21:30:06 21:30:06.536392:CID-1:RT:set_nat_invalid: natp:id 758, flag 3c5f

[zoidberg]

Not sure if this is quite the inter-vr nat you are looking for... but I've done something similar.

Traffic comes in VR-A, gets filter based forwarded to VR-B. Nat rule matches from zone VR-A-zone to zone VR-B-zone. VR-B has it's route to send traffic out it's egress interface. And return traffic into VR-B has routes direct traffic to next-table VR-A.inet.0.

A weird oddity/bug in 10.x caused a problem that sounds similar to what you are describing. The route lookup done during sessions establishment was actually being done in inet.0, not in VR-B.inet.0 like you would expect, and not even in VR-A.inet.0 where the traffic ingressed. So, as a workaround, I believe I needed to install a matching route in inet.0 saying next-table VR-B.inet.0 (I think that's how I did it). I thought this bug was fixed somewhere in 10.4 however, and I'm pretty sure you're using a 11.1 or 11.4.

[zoidberg]

What do your routing tables look like?

[Ahriakin]

And the Source+destination NAT pairs that appear to work.


From LAN Source to ACME Host
Session ID: 6087, Policy name: Permit-ANY/4, State: Forward, Timeout: 2, Valid
In: 10.10.10.202/1 --> 100.96.30.30/3109;icmp, If: fe-0/0/0.0, Pkts: 1, Bytes: 84
Out: 100.96.30.30/3109 --> 10.10.10.202/1;icmp, If: fe-1/0/1.0, Pkts: 0, Bytes: 0


FROM ACME Source to LAN Host
Session ID: 3203, Policy name: Permit-ANY/5, State: Active, Timeout: 2, Valid
In: 10.30.30.30/5621 --> 100.64.10.202/6812;icmp, If: fe-1/0/1.0, Pkts: 0, Bytes: 0
Out: 10.10.10.202/6812 --> 100.96.30.30/47572;icmp, If: fe-0/0/0.0, Pkts: 1, Bytes: 100

[edit security nat]
    nat {
        source {
            pool ACME-NAT {
                routing-instance {
                    LAN;
                }
                address {
                    100.96.30.30/32;
                }
                port inactive: no-translation;
            }
            pool LAN-NAT {
                routing-instance {
                    ACME;
                }
                address {
                    100.64.10.202/32;
                }
                port inactive: no-translation;
            }
            rule-set SOURCE-ACME-LAN {
                from zone ACME;
                to zone LAN;
                rule SNAT-1 {
                    match {
                        source-address 10.30.30.30/32;
                    }
                    then {
                        source-nat {
                            pool {
                                ACME-NAT;
                            }
                        }
                    }
                }
            }
            rule-set SOURCE-LAN-ACME {
                from zone LAN;
                to zone ACME;
                rule SNAT-2 {
                    match {
                        source-address 10.10.10.202/32;
                    }
                    then {
                        source-nat {
                            pool {
                                LAN-NAT;
                            }
                        }
                    }
                }
            }
        }
        destination {
            pool LAN {
                routing-instance {
                    LAN;
                }
                address 10.10.10.202/32;
            }
            pool ACME {
                routing-instance {
                    ACME;
                }
                address 10.30.30.30/32;
            }
            rule-set DEST-NAT-ACME {
                from zone LAN;
                rule DEST-NAT-ACME-LAN {
                    match {
                        destination-address 100.96.30.30/32;
                    }
                    then {
                        destination-nat pool ACME;
                    }
                }
            }
            rule-set DEST-NAT-LAN {
                from zone ACME;
                rule DEST-NAT-LAN-ACME {
                    match {
                        destination-address 100.64.10.202/11;
                    }
                    then {
                        destination-nat pool LAN;
                    }
                }
            }
        }

Flow Trace (edited for brevity)

Jan 15 22:21:50 22:21:52.293174:CID-2:RT:  fe-0/0/0.0:10.10.10.202->100.96.30.30, icmp, (8/0)
Jan 15 22:21:50 22:21:52.293174:CID-2:RT: find flow: table 0x42686e08, hash 12619(0xffff), sa 10.10.10.202, da 100.96.30.30, sp 0, dp 2967, proto 1, tok 20486 
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:  no session found, start first path. in_tunnel - 0, from_cp_flag - 0
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:  flow_first_create_session
<snip>
## Destination NAT happens as expected
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:  chose interface fe-0/0/0.0 as incoming nat if.
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:flow_first_rule_dst_xlate: DST xlate: 100.96.30.30(2967) to 10.30.30.30(2967), rule/pool id 1/32769
## Correct opposing VRF is chosen for the route lookup
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:flow_first_routing: vr_id 4, call flow_route_lookup(): src_ip 10.10.10.202, x_dst_ip 10.30.30.30, in ifp fe-0/0/0.0, out ifp N/A sp 0, dp 2967, ip_proto 1, tos 0
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:Doing DESTINATION addr route-lookup
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:flow_rt_lkup in VR-id: 4
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:flow_rt_lkup: Found route entry 0x0x45baeac0,nh id 0x267, out if 0x48
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:flow_rt_lkup: nh word 0xc0010 
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:flow_ipv4_rt_lkup success 10.30.30.30, iifl 0x45, oifl 0x48 
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:  routed (x_dst_ip 10.30.30.30) from LAN (fe-0/0/0.0 in 0) to fe-1/0/1.0, Next-hop: 10.30.226.22
<snip>
## Source NAT happens as expected
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:flow_first_src_xlate: src nat returns status: 1, rule/pool id: 2/32773, pst_nat: False.
Jan 15 22:21:50 22:21:52.293174:CID-2:RT:  dip id = 5/1, 10.10.10.202/0->100.64.10.202/20979 protocol 58
<mega snip>

[Ahriakin]

And finally routing tables (Sorry Mr. Z you're right I should have posted them first ).

LAN to ACME Firewall

Routing tables for each Vrf, test host of 10.30.30.30 existing in both with different next hops

ACME VRF
ACME.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
Restart Complete
+ = Active Route, - = Last Active, * = Both

10.0.0.0/8 *[BGP/170] 01:19:15, MED 0, localpref 100
AS path: 2600 I
> to 10.20.126.21 via fe-0/0/1.0
10.20.20.0/24 *[BGP/170] 01:19:15, MED 0, localpref 100
AS path: 2600 I
> to 10.20.126.21 via fe-0/0/1.0
10.20.126.0/24 *[Direct/0] 04:29:01
> via fe-0/0/1.0
10.20.126.200/32 *[Local/0] 04:29:01
Local via fe-0/0/1.0
10.30.30.0/24 *[BGP/170] 01:19:32, MED 0, localpref 100
AS path: 2600 I
> to 10.30.226.22 via fe-1/0/1.0
10.30.226.0/24 *[Direct/0] 04:29:01
> via fe-1/0/1.0
10.30.226.201/32 *[Local/0] 04:29:01
Local via fe-1/0/1.0

LAN VRF
LAN.inet.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[OSPF/150] 04:17:29, metric 0, tag 0
> to 10.10.10.202 via fe-0/0/0.0
10.0.0.0/8 *[OSPF/150] 04:04:25, metric 0, tag 0
> to 10.10.10.202 via fe-0/0/0.0
10.10.10.0/24 *[Direct/0] 04:29:01
> via fe-0/0/0.0
10.10.10.200/32 *[Local/0] 04:29:01
Local via fe-0/0/0.0
10.10.15.0/24 *[Direct/0] 04:29:01
> via fe-1/0/0.0
10.10.15.201/32 *[Local/0] 04:29:01
Local via fe-1/0/0.0
10.20.20.0/24 *[OSPF/150] 04:08:50, metric 20, tag 0
> to 10.10.15.20 via fe-1/0/0.0
10.30.30.0/24 *[OSPF/150] 03:46:47, metric 20, tag 0
> to 10.10.15.20 via fe-1/0/0.0
10.100.23.0/24 *[OSPF/10] 04:28:43, metric 2
> to 10.10.10.202 via fe-0/0/0.0
10.202.202.0/24 *[OSPF/150] 04:04:25, metric 0, tag 0
> to 10.10.10.202 via fe-0/0/0.0
172.16.12.0/24 *[OSPF/150] 01:21:22, metric 20, tag 0
> to 10.10.15.20 via fe-1/0/0.0
192.168.1.0/24 *[OSPF/150] 04:17:29, metric 0, tag 0
> to 10.10.10.202 via fe-0/0/0.0
202.202.202.202/32 *[OSPF/10] 04:28:43, metric 1
> to 10.10.10.202 via fe-0/0/0.0
224.0.0.5/32 *[OSPF/10] 04:29:04, m


Routing instances

routing-instances {
    ACME {
        instance-type virtual-router;
        interface fe-0/0/1.0;
        interface fe-1/0/1.0;
        routing-options {
            graceful-restart {
                restart-duration 120;
            }
            inactive: instance-import LAN-to-ACME;
        }
        protocols {
            bgp {
                local-as 199;
                graceful-restart {
                    stale-routes-time 10;
                }
                group WAN1 {
                    peer-as 2600;
                    neighbor 10.20.126.21;
                }
                group WAN2 {
                    peer-as 2600;
                    neighbor 10.30.226.22;
                }
            }
        }
    }
    LAN {
        instance-type virtual-router;
        interface fe-0/0/0.0;
        interface fe-1/0/0.0;
        routing-options {
            inactive: instance-import ACME-to-LAN;
        }
        protocols {
            ospf {
                graceful-restart {
                    restart-duration 10;
                    notify-duration 10;
                }
                area 0.0.0.0 {
                    interface fe-0/0/0.0;
                    interface fe-1/0/0.0;
                }
            }
        }
    }
}

## Need this since there is nothing inet.0 currently, mgmt is local.
routing-options {
    max-interface-supported 0;
}

[Ahriakin]

Thanks for looking at it.

I couldn't use next-table for some reason as it would not accept the VRFs as valid tables, which is pretty odd since I've done similar before on the 5800s. Even though it exists I've been told by multiple Juniper folks to try not to use it as it can cause some odd results.

FBF....now that I should have tried. It may be the missing link since all that seems to be missing is forcing the session to process primarily in the opposing VRF. Even though it's kind've a sledge-hammer/walnut approach at least that way I can make it use the routing-instance I want. Unfortunately this is all on my home lab and I still haven't setup remote access to it. Maybe I can jimmy up something here, minus the backend devices since all i really want at this stage is to get the inter-vrf working without the multiple source+dest nat complications.

[zoidberg]

Hmmm, not sure why next-table wouldn't work. You need to specify vrname.inet.0, not just vrname. Also, Junos does prevent you from doing two-way next-tables.... so vr-b cannot have a route saying next-table vr-a.inet.0, at the same time as having vr-a with a route saying next-table vr-b.inet.0. The commit will fail as it does not permit two VRs to point to each other with static routes like that, even if those two routes are not overlapping and will not cause a routing loop, Junos still blocks it.