вторник, 21 апреля 2015 г.

ARP flux

Понадобилось мне на одном сервере CentOS 6 терминировать два ip-адреса из одной подсети, да так, чтобы у каждого ip был свой MAC и трафик ходил симметрично, т.е. на какой интерфейс пришёл с него же и ушёл.

Не такая простая задача, как кажется на первый взгляд - стек tcp/ip в linux следует
weak host model.



C назначением ip-адресов проблем нет:

# ip addr show dev eth1
4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:14:5e:cd:44:85 brd ff:ff:ff:ff:ff:ff
    inet 192.168.253.85/28 brd 192.168.253.95 scope global eth1
    inet6 fe80::214:5eff:fecd:4485/64 scope link
       valid_lft forever preferred_lft forever
# ip addr show dev eth2
5: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:14:5e:cd:44:86 brd ff:ff:ff:ff:ff:ff
    inet 192.168.253.86/28 brd 192.168.253.95 scope global eth2
    inet6 fe80::214:5eff:fecd:4486/64 scope link
       valid_lft forever preferred_lft forever


ARP победил следующим образом:
# это, на самом деле, значения по-умолчанию 
sysctl net.ipv4.conf.eth1.arp_filter=0
sysctl net.ipv4.conf.eth2.arp_filter=0
 

#Define different modes for sending replies in response to
#    received ARP requests that resolve local target IP addresses:
#    0 - (default): reply for any local target IP address, #configured
#    on any interface
#    1 - reply only if the target IP address is local address
#    configured on the incoming interface
#    2 - reply only if the target IP address is local address
#    configured on the incoming interface and both with the
#    sender's IP address are part from same subnet on this interface
sysctl net.ipv4.conf.eth1.arp_ignore=2
sysctl net.ipv4.conf.eth2.arp_ignore=2
 

#Define different restriction levels for announcing the local
#    source IP address from IP packets in ARP requests sent on
#    interface:
#    0 - (default) Use any local address, configured on any #interface
#    1 - Try to avoid local addresses that are not in the target's
#    subnet for this interface. This mode is useful when target
#    hosts reachable via this interface require the source IP
#    address in ARP requests to be part of their logical network
#    configured on the receiving interface. When we generate the
#    request we will check all our subnets that include the
#    target IP and will preserve the source address if it is from
#    such subnet. If there is no such subnet we select source
#    address according to the rules for level 2.
#    2 - Always use the best local address for this target.
#    In this mode we ignore the source address in the IP packet
#    and try to select local address that we prefer for talks with
#    the target host. Such local address is selected by looking
#    for primary IP addresses on all our subnets on the outgoing
#    interface that include the target IP address. If no suitable
#    local address is found we select the first local address
#    we have on the outgoing interface or on all other interfaces,
#    with the hope we will receive reply for our request and
#    even sometimes no matter the source IP address we announce.


sysctl net.ipv4.conf.eth1.arp_announce=2sysctl net.ipv4.conf.eth2.arp_announce=2
 

ARP заработал.
# ip nei sh dev eth1
192.168.253.92 lladdr 00:90:1a:a3:9c:77 REACHABLE
192.168.253.90 lladdr 00:90:1a:a4:f7:69 REACHABLE
192.168.253.91 lladdr 00:90:1a:a4:25:cd REACHABLE
 

# ip nei sh dev eth2
192.168.253.90 lladdr 00:90:1a:a4:f7:69 REACHABLE
192.168.253.91 lladdr 00:90:1a:a4:25:cd REACHABLE
192.168.253.92 lladdr 00:90:1a:a3:9c:77 REACHABLE


Запросы приходят, сервер отвечает, но трафик с сервера возвращается через один интерфейс, несмотря на то, что на сервере прописаны одинаковые маршруты через оба интерфейса. На linux есть очень продвинутый policy routing. В итоге получилось вот что:

Добавляем в /etc/iproute2/rt_tables таблицу для каждого интерфейса:
11      eth1
12      eth2


# common routes
ip route add 172.31.64.0/19 via 192.168.253.90 dev eth1 table eth1
ip route add 172.31.96.0/20 via 192.168.253.91 dev eth1 table eth1
ip route add 172.31.112.0/20 via 192.168.253.92 dev eth1 table eth1

ip route add 192.168.253.80/28 dev eth1 src 192.168.253.85
ip rule add from 192.168.253.85 table eth1
ip rule add fwmark 0x85/0x85 lookup eth1
 


# common routes
ip route add 172.31.64.0/19 via 192.168.253.90 dev eth2 table eth2
ip route add 172.31.96.0/20 via 192.168.253.91 dev eth2 table eth2
ip route add 172.31.112.0/20 via 192.168.253.92 dev eth2 table eth2

ip route add 192.168.253.80/28 dev eth2 src 192.168.253.86
ip rule add from 192.168.253.86 table eth2
ip rule add fwmark 0x86/0x86 lookup eth2
 

Пришлось полностью отключить uRPF для данных интерфейсов:
# no rpf check
sysctl net.ipv4.conf.eth1.rp_filter=0
sysctl net.ipv4.conf.eth2.rp_filter=0



пятница, 9 января 2015 г.

FreeBSD anycast DNS resolver

Пример anycast DNS resolver'а для FreeBSD.

В качестве рекурсивного DNS-сервера, как и раньше, используем unbound. А вот для установки BGP-сессии - openbgpd из проекта OpenBSD, конечно, "ортодоксам" доступна и quagga. В перспективе стоит присмотреться к exabgp.

И openbgpd и unbound есть в портах, так что с установкой проблем быть не должно. Я предпочитаю собирать из исходников, чтобы можно было самому указать необходимые опции:
cd /usr/ports/net/openbgpd && make config install clean
cd /usr/ports/dns/unbound && make config install clean


Сетевые интерфейсы:
  • bge1 - стык с маршрутизатором на "серых" адресах, исключительно для BGP.
  • lo1 - реальные адреса, на которые сервер принимает запросы от клиентов и с которого шлёт запросы в Internet. 
# cat /etc/rc.conf

cloned_interfaces="lo1"
ifconfig_bge1="inet 192.168.250.102 netmask 255.255.255.252 mtu 9000"

# resolver DNS source
ifconfig_lo1="inet 1.0.128.12 netmask 255.255.255.255"
# resolver DNS anycast
ifconfig_lo1_alias0="inet 1.0.128.10 netmask 255.255.255.255"
# resolver DNS anycast
ifconfig_lo1_alias1="inet 1.0.128.14 netmask 255.255.255.255"

# routing
defaultrouter="192.168.250.101"



# firewall
pf_enable="YES"
pflog_enable="YES"


Разрешаем запуск установленных демонов:

# cat /etc/rc.conf.local
#
# recursive DNS
unbound_enable="YES"

# for anycast DNS
openbgpd_enable="YES"


Настраиваем firewall, куда ж без него в наше время.
# cat /etc/pf.conf
ext_if="lo1"
bgp_if="bge1"

set loginterface $bgp_if
set limit { states 1000000, frags 15000, src-nodes 500000 }

table <clients> const {
    # тут нужно добавить "свои" адреса
    192.168.0.0/16
    172.16.0.0/12
    10.0.0.0/8
    100.64.0.0/10
}

set skip on lo

scrub in





# "хитрый" NAT чтобы все программы, которые "лезут" в Интернет использовали реальный адрес на lo1

nat on $bgp_if inet from ($bgp_if:0) to !($bgp_if:network) -> ($ext_if:0)

block in log
pass out

pass in on $bgp_if inet proto tcp to ($bgp_if) port { ssh }
pass in on $bgp_if inet proto tcp from <clients> to ($ext_if) port { ssh, domain }


# в случае udp принимать запрос от клиента без создания записи в таблице состояний 
pass in  quick on $bgp_if inet proto udp from <clients> to ($ext_if) port { domain } no state
pass out quick on $bgp_if inet proto udp from ($ext_if) port { domain } to <clients> no state

pass in on $bgp_if inet  proto icmp  from any to { ($bgp_if) ($ext_if) } icmp-type  { unreach, squench, echoreq, timex, paramprob }


Для openbgpd у нас предельно лаконичный конфиг:

# cat /usr/local/etc/bgpd.conf
AS 65001
router-id 192.168.250.102

fib-update no # фактически не используем bgp-маршруты, нам хватит и маршрута по умолчанию из rc.conf

log updates



# эти три маршрута мы будем анонсировать

network 1.0.128.12/32
network 1.0.128.10/32
network 1.0.128.14/32
 

group ROUTER01 {
 remote-as 65004
 announce self

 neighbor 192.168.250.101 {
  descr router4

  local-address 192.168.250.102
 }
}



Проверить, что BGP-сессия поднялась можно командой:
# bgpctl show ip bgp summary
Neighbor                   AS    MsgRcvd    MsgSent  OutQ Up/Down  State/PrfRcvd
router4                   65004      41772      40082     0 01w6d22h      1



В конфиге unbound'а указываем на каких адресах слушать,  а с какого отправлять запросы в Интернет:


# cat /usr/local/etc/unbound/unbound.conf
server:
 # specify the interfaces to answer queries from by ip-address.
        interface: 1.0.128.10
        interface: 1.0.128.14

 # specify the interfaces to send outgoing queries to authoritative
 # Specify every interface on a 'outgoing-interface:' line.
        outgoing-interface: 1.0.128.12

CentOS anycast dns-resolvers

Исторически сложилось, что ISP сообщает клиенту два IP-адреса в качестве настроек DNS. Многие ОС и протоколы, например IPCP из PPP, и не поддерживают больше, а реальность такова, что у крупных операторов два сервера не способны обслуживать все клиентские запросы. А если география предоставления услуг охватывает страны или континенты, то два сервера вообще не вариант.

Для преодоления этой ситуации можно использовать достаточно простой подход, который используется для корневых DNS-серверов - anycast DNS. Этот подход можно использовать и для других протоколов работающих по принципу запрос-ответ без сохранения состояния, например, для NTP.

Этот конкретный пример будет для CentOS6 (unbound, quagga) и Juniper MX, но есть рабочие серверы на FreeBSD.

 

Начнём с CentOS:

Сетевые интерфейсы.

eth0 

"смотрит" в сторону MX'a. Экономим ipv4 - bgp будем поднимать на "серых" адресах 10.1.0.48/29.

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=6A:02:85:D0:E2:01
BOOTPROTO=none
ONBOOT="yes"
TYPE=Ethernet
IPADDR=10.1.0.50
NETMASK=255.255.255.248
GATEWAY=10.1.0.49
DEFROUTE=yes
ONBOOT=yes
IPV6INIT=yes
IPV6ADDR=1234:5800:0:100::a/125
IPV6_DEFAULTGW=1234:5800:0:100::9
IPV6_DEFROUTE=yes
NAME="eth0"
USERCTL=no


loopbacks:

Вот тут и будут наши реальные адреса: два, на которые мы принимаем запросы от клиентов (1.0.128.10  и 1.0.128.14) и один (1.0.128.8), с которого мы будем отправлять запросы в "мир".

# cat /etc/sysconfig/network-scripts/ifcfg-lo
DEVICE=lo
IPADDR=127.0.0.1
NETMASK=255.0.0.0
NETWORK=127.0.0.0
BROADCAST=127.255.255.255
ONBOOT=yes
IPV6INIT=yes
IPV6ADDR=1234:5800::1/128
NAME=loopback


# cat /etc/sysconfig/network-scripts/ifcfg-lo:0
DEVICE=lo:0
ONPARENT=yes
IPADDR=1.0.128.8
NETMASK=255.255.255.255
ONBOOT=yes
NAME="DNS requests source"



# cat /etc/sysconfig/network-scripts/ifcfg-lo:1
DEVICE=lo:1
ONPARENT=yes
IPADDR=1.0.128.10
NETMASK=255.255.255.255
ONBOOT=yes
NAME="resolver#1"


# cat /etc/sysconfig/network-scripts/ifcfg-lo:2
DEVICE=lo:2
ONPARENT=yes
IPADDR=1.0.128.14
NETMASK=255.255.255.255
ONBOOT=yes
NAME="resolver#2"
 

BGP

Используем BGP, так как он легко фильтруется, поддерживает неравновесное распределение нагрузки и т.д.. Анонсируем нашему маршрутизатору все три ipv4-адреса. С ipv6 есть небольшая хитрость - next-hop global.


Quagga:

# cat /etc/quagga/bgpd.conf
!
! Zebra configuration saved from vty
!   2014/05/12 08:55:50
!
hostname ns-resolver00
password zebra
enable password quagga
log file /var/log/quagga/zebra.log
!
router bgp 65001
 bgp router-id 1.0.128.8
 bgp log-neighbor-changes
 bgp graceful-restart
 network 1.0.128.8/32
 network 1.0.128.10/32
 network 1.0.128.14/32
 neighbor 10.1.0.49 remote-as 65004
 neighbor 10.1.0.49 description "router4"
 neighbor 10.1.0.49 route-map RM-DNS_RESOLVER out
 neighbor 1234:5800:0:100::9 remote-as 65004

 neighbor 1234:5800:0:100::9 description "router6"
 no neighbor 1234:5800:0:100::9 activate
!
 address-family ipv6
 network 1234:5800::1/128
 neighbor 1234:5800:0:100::9 activate
 neighbor 1234:5800:0:100::9 route-map RM-DNS_RESOLVER6 out
 exit-address-family
!
access-list 99 permit 1.0.128.10
access-list 99 permit 1.0.128.14
access-list 99 permit 1.0.128.8
!
ipv6 prefix-list PL-DNS_RESOLVER6 seq 5 permit 1234:5800::1/128
!
route-map RM-DNS_RESOLVER permit 10
 match ip address 99
!
route-map RM-DNS_RESOLVER6 permit 10
 match ipv6 address prefix-list PL-DNS_RESOLVER6
 set ipv6 next-hop global 1234:5800:0:100::a
!

Unbound

Явно указываем, на какие адреса принимаем запросы, а с какого отправляем в "мир". Аналогичные опции есть у всех современных DNS-рекурсоров.

server:
 # specify the interfaces to answer queries from by ip-address.
        interface: 1.0.128.10
        interface: 1.0.128.14
        interface: 1234:5800::1

        
        interface-automatic: no

 # specify the interfaces to send outgoing queries to authoritative
 # server from by ip-address. If none, the default (all) interface
 # is used. Specify every interface on a 'outgoing-interface:' line.
        outgoing-interface: 1.0.128.8
        outgoing-interface: 1234:5800:0:100::a


Cron

На случай отказа поместим периодическую проверку работоспособности рекурсора в cron:

#minute hour    mday    month   wday    command
PATH=/usr/bin:/bin:/usr/sbin:/sbin
#
@reboot                                 echo '0' > /var/tmp/dns-check
*/1     *       *       *       *       service unbound status >/dev/null || service unbound restart
*/2     *       *       *       *       /var/adm/check_dns.sh
 

# cat /var/adm/check_dns.sh
#!/bin/sh

export PATH=/sbin:/usr/sbin:/usr/local/sbin:/bin:/usr/bin:/usr/local/bin

MAIL_RECIPIENT="NOC@bigisp.domain"
DATE=$(date "+%Y-%m-%d %H:%m")
FFILE="/var/tmp/dns-check"

STATUS=$(head -n1 "${FFILE}")


drill
bigisp.domain > /dev/null
ST=$?
if [ "${ST}" != "0" ]; then
    if [ "${STATUS}" -ge 5 ]; then
        echo "Immediate actions for resolver are required!!!" | mail -s "Alert from resolver [server stopped]" ${MAIL_RECIPIENT}
    elif [ "${STATUS}" = "5" ]; then
        echo "Shutdown resolver at ${DATE}" | mail -s "Alert from resolver [Critical Error]" ${MAIL_RECIPIENT}
        vtysh -E -c "conf t
router bgp 65001
no network 1.0.128.10/32
no network 1.0.128.14/32
no network 1234:5800::1/128
end
clear ip bgp 10.1.0.49 out"
    else
        echo "Restart unbound at ${DATE} because 'drill
bigisp.domain @resolver' returned: ${ST}" | mail -s "Alert from resolver [unbound restarted]" ${MAIL_RECIPIENT}
        service unbound restart
    fi
    STATUS=$(( ${STATUS}+1 ))
    echo "${STATUS}" > "${FFILE}"
else
    echo "0" > "${FFILE}"
fi



iptables:

Используем iptables для защиты сервера. Небольшой трюк с NOTRACK чтобы не переполнить таблицу состояний, в остальном всё стандартно. Используем SNAT чтобы программы, например, wget или yum, которые не умеют привязываться к адресу loopback'а имели доступ в Internet.

cat /etc/sysconfig/iptables
# Generated by iptables-save v1.4.7 on Thu Sep 18 20:17:49 2014
*raw
:PREROUTING ACCEPT [288965:24355290]
:OUTPUT ACCEPT [290702:22209038]
-A PREROUTING -p udp -m udp --sport 53 -j NOTRACK
-A PREROUTING -p udp -m udp --dport 53 -j NOTRACK
-A OUTPUT -p udp -m udp --sport 53 -j NOTRACK
-A OUTPUT -p udp -m udp --dport 53 -j NOTRACK
COMMIT
# Completed on Thu Sep 18 20:17:49 2014
# Generated by iptables-save v1.4.7 on Thu Sep 18 20:17:49 2014
*nat
:PREROUTING ACCEPT [889:58520]
:POSTROUTING ACCEPT [46:2760]
:OUTPUT ACCEPT [46:2760]
-A POSTROUTING -s 10.1.0.48/29 ! -d 10.1.0.48/29 -o eth0 -j SNAT --to-source
1.0.128.8
COMMIT
# Completed on Thu Sep 18 20:17:49 2014
# Generated by iptables-save v1.4.7 on Thu Sep 18 20:17:49 2014
*mangle
:PREROUTING ACCEPT [288971:24355694]
:INPUT ACCEPT [288971:24355694]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [290707:22209502]
:POSTROUTING ACCEPT [290707:22209502]
COMMIT
# Completed on Thu Sep 18 20:17:49 2014
# Generated by iptables-save v1.4.7 on Thu Sep 18 20:17:49 2014
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [263101:20083880]
-A INPUT -p udp -m udp --sport 53 -j ACCEPT
-A INPUT -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -m state --state INVALID -j DROP
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -s 10.1.0.49/32 -p tcp -m state --state NEW -m tcp -m multiport --dports 22,179 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
-A OUTPUT -p udp -m udp --dport 53 -j ACCEPT
COMMIT
# Completed on Thu Sep 18 20:17:49 2014

JunOS:

Сетевой интерфейс:

DNS-cерверы включены в virtual chassis из нескольких коммутаторов juniper ex4200, поэтому у нас тут LACP-транк.

interfaces {

 ae3 {
        description "ae48 ex4200";
        vlan-tagging;
        mtu 9192;
        encapsulation flexible-ethernet-services;
        aggregated-ether-options {
            minimum-links 1;
            lacp {
                active;
                periodic fast;
            }
        }

        unit 53 {
            description SERVERS-DNS-RESOLVERs;
            vlan-id 53;
            family inet {
                no-redirects;
                policer {
                    arp POL-ARP;
                }
                address 10.1.0.49/29;
            }
            family inet6 {
                address 1234:5800:0:100::9/125;
            }
        }

    }

}

BGP


С BGP всё просто, но опять же есть небольшой трюк с ipv6.

protocols {

    bgp {

        group dns4 {
            type external;
            passive;
            import rm-dns-in;
            family inet {
                unicast {
                }
            }
            export default-only;
            peer-as 65001;
            multipath;

            neighbor 10.1.0.50 {
                description resolver0;
            }
            neighbor 192.168.250.102 {
                description resolver1;
            }
            neighbor 10.1.0.52 {
                description resolver2;
            }
            neighbor 10.1.0.53 {
                description resolver3;
            }
        }

        group dns6 {
            type external;
            passive;
            import rm-dns6-in;
            family inet6 {
                unicast;
            }
            export default6-only;
            peer-as 65001;

            neighbor 1234:5800:0:100::a {
                description resolver0;
            }
            neighbor 1234:5800:0:100::b {
                description resolver1;
            }
            neighbor 1234:5800:0:100::c {
                description resolver2;
            }
            neighbor 1234:5800:0:100::d {
                description resolver3;
            }
        }

    }

}

policy-options {

    policy-statement rm-dns-in {
        term dns {
            from {
                route-filter 1.0.128.8/29 upto /32;
            }
            then accept;
        }
        term deny {
            then reject;
        }
    }



    policy-statement rm-dns6-in {
        term dns {
            from {
                family inet6;
                route-filter 1234:5800::0/128 exact;
                route-filter 1234:5800::1/128 exact;
                route-filter fec0:000:0000:ffff::1/128 exact;
            }
            then {
                local-preference 400;
                accept;
            }
        }
        term deny {
            then reject;
        }
    }

}
 

среда, 26 ноября 2014 г.

sham link между junos и ios



Конфигурация.

Cisco:

ip vrf clientOSPF

 rd 65500:308

 route-target export 65500:308

 route-target import 65500:308

!

interface Loopback308

 ip vrf forwarding clientOSPF

 ip address 192.168.253.103 255.255.255.255

 no ip redirects

 no ip proxy-arp

 ip mtu 1500

 ip ospf 308 area 0

!

interface TenGigabitEthernet3/4.308

 description "[client-id]"

 encapsulation dot1Q 308

 ip vrf forwarding clientOSPF

 ip address 192.168.2.246 255.255.255.248

 no ip redirects

 no ip proxy-arp

 ip mtu 1500

 ip ospf authentication message-digest

 ip ospf message-digest-key 1 md5 7 XXXXXXXXXX

 ip ospf priority 250

 ip ospf lls disable

 ip ospf 308 area 0

 no cdp enable

!

router ospf 308 vrf clientOSPF

 router-id 192.168.253.103

 event-log size 1000

 log-adjacency-changes

 area 0 sham-link 192.168.253.103 192.168.253.109

 redistribute connected

 redistribute bgp 65500 metric-type 1 subnets route-map RM-clientOSPF-bgp2ospf

 default-information originate

!

router bgp 65500

 bgp router-id 172.16.128.103

 bgp log-neighbor-changes

!

 address-family ipv4 vrf clientOSPF

  no synchronization

  redistribute static

  redistribute connected

  redistribute ospf 308 vrf clientOSPF match internal external 1 external 2

 exit-address-family

 !

route-map RM-clientOSPF-bgp2ospf permit 10

 match ip address 12

 set tag 65500

!

access-list 12 deny   192.168.253.103

access-list 12 deny   192.168.253.109

access-list 12 permit any

!

 Juniper MX:

set interfaces lo0 unit 308 description "[CLIENTOSPF]"

set interfaces lo0 unit 308 family inet no-redirects

set interfaces lo0 unit 308 family inet address 192.168.253.109/32



set interfaces xe-2/0/1 unit 308 vlan-id 308

set interfaces xe-2/0/1 unit 308 family inet mtu 1500

set interfaces xe-2/0/1 unit 308 family inet no-redirects

set interfaces xe-2/0/1 unit 308 family inet policer arp POL-ARP

set interfaces xe-2/0/1 unit 308 family inet address 192.168.2.237/30



set routing-instances CLIENTOSPF instance-type vrf

set routing-instances CLIENTOSPF interface xe-2/0/1.308

set routing-instances CLIENTOSPF interface lo0.308

set routing-instances CLIENTOSPF route-distinguisher 65500:308

set routing-instances CLIENTOSPF vrf-import IMP-CLIENTOSPF-VRF

set routing-instances CLIENTOSPF vrf-export EXP-CLIENTOSPF-VRF

set routing-instances CLIENTOSPF vrf-target target:65500:308

set routing-instances CLIENTOSPF vrf-table-label

set routing-instances CLIENTOSPF protocols ospf export CLIENTOSPF-BGP->OSPF

set routing-instances CLIENTOSPF protocols ospf sham-link local 192.168.253.109

set routing-instances CLIENTOSPF protocols ospf area 0.0.0.0 sham-link-remote 192.168.253.103 metric 1

set routing-instances CLIENTOSPF protocols ospf area 0.0.0.0 interface xe-2/0/1.308 metric 10

set routing-instances CLIENTOSPF protocols ospf area 0.0.0.0 interface xe-2/0/1.308 authentication md5 1 key "$9$-XXXXXXXXXX"

set routing-instances CLIENTOSPF protocols ospf area 0.0.0.0 interface lo0.308 passive



set policy-options policy-statement CLIENTOSPF-BGP->OSPF term loopbacks from route-filter 192.168.253.0/24 upto /32

set policy-options policy-statement CLIENTOSPF-BGP->OSPF term loopbacks then reject

set policy-options policy-statement CLIENTOSPF-BGP->OSPF term vrf then tag 65500

set policy-options policy-statement CLIENTOSPF-BGP->OSPF term vrf then external type 1

set policy-options policy-statement CLIENTOSPF-BGP->OSPF term vrf then accept



set policy-options policy-statement IMP-CLIENTOSPF-VRF term vrf from protocol bgp

set policy-options policy-statement IMP-CLIENTOSPF-VRF term vrf from community COMMUNITY_CLIENTOSPF_TARGET

set policy-options policy-statement IMP-CLIENTOSPF-VRF term vrf then accept

set policy-options policy-statement IMP-CLIENTOSPF-VRF term last then reject



set policy-options policy-statement EXP-CLIENTOSPF-VRF term vrf from protocol direct

set policy-options policy-statement EXP-CLIENTOSPF-VRF term vrf from protocol ospf

set policy-options policy-statement EXP-CLIENTOSPF-VRF term vrf then community add COMMUNITY_CLIENTOSPF_TARGET

set policy-options policy-statement EXP-CLIENTOSPF-VRF term vrf then accept

set policy-options policy-statement EXP-CLIENTOSPF-VRF term last then reject



set policy-options community COMMUNITY_CLIENTOSPF_TARGET members target:65500:308


Проверка.

На cisco:

c7604-jas#sh ip ospf 308 sham-links

Sham Link OSPF_SL6 to address 192.168.253.109 is up

Area 0 source address 192.168.253.103

  Run as demand circuit

  DoNotAge LSA not allowed (Number of DCbitless LSA is 7). Cost of using 1 State POINT_TO_POINT,

  Timer intervals configured, Hello 10, Dead 40, Wait 40,

    Hello due in 00:00:07

    Adjacency State FULL (Hello suppressed)

    Index 1/1, retransmission queue length 0, number of retransmission 2

    First 0x0(0)/0x0(0) Next 0x0(0)/0x0(0)

    Last retransmission scan length is 1, maximum is 1

    Last retransmission scan time is 0 msec, maximum is 0 msec


Маршрут на loopback не должен быть "виден" в OSPF: 

c7604-jas#sh ip ro vrf clientOSPF 192.168.253.109

 Routing Table: clientOSPF

Routing entry for 192.168.253.109/32

  Known via "bgp 65500", distance 200, metric 0, type internal

  Redistributing via ospf 308

  Last update from 172.16.128.109 11:25:15 ago

  Routing Descriptor Blocks:

  * 172.16.128.109 (default), from 172.16.128.106, 11:25:15 ago

      Route metric is 0, traffic share count is 1

      AS Hops 0

      MPLS label: 35

      MPLS Flags: MPLS Required


А вот клиентский маршрут наоборот, должен быть "виден" через OSPF:

c7604-jas#sh ip ro vrf clientOSPF 192.168.2.237

 Routing Table: clientOSPF

Routing entry for 192.168.2.236/30

  Known via "ospf 308", distance 110, metric 11, type intra area

  Redistributing via bgp 65500

  Advertised by bgp 65500 match internal external 1 & 2

  Last update from 172.16.128.109 11:24:53 ago

  Routing Descriptor Blocks:

  * 172.16.128.109 (default), from 192.168.2.253, 11:24:53 ago

      Route metric is 11, traffic share count is 1

      MPLS label: 35

      MPLS Flags: MPLS Required


На juniper:


user@mx240> show ospf interface instance CLIENTOSPF

Interface           State   Area            DR ID           BDR ID          Nbrs

lo0.308             DRother 0.0.0.0         0.0.0.0         0.0.0.0            0

shamlink.0          PtToPt  0.0.0.0         0.0.0.0         0.0.0.0            1

xe-2/0/1.308        BDR     0.0.0.0         192.168.2.253   192.168.253.109    1



user@mx240> show ospf neighbor instance CLIENTOSPF

Address          Interface              State     ID               Pri  Dead

192.168.253.103  shamlink.0             Full      192.168.253.103    1     -

192.168.2.238    xe-2/0/1.308           Full      192.168.2.253      1    34


Важное замечание:

Note: In Junos OS Release 9.6 and later, an OSPFv2 sham link is installed in the routing table as a hidden route. Additionally, a BGP route is not exported to OSPFv2 if a corresponding OSPF sham link is available.

Именно поэтому:


user@mx240> show route table CLIENTOSPF.inet.0



CLIENTOSPF.inet.0: 41 destinations, 68 routes (41 active, 0 holddown, 27 hidden)

+ = Active Route, - = Last Active, * = Both



0.0.0.0/0          *[OSPF/150] 11:34:35, metric 61, tag 0

                    > to 192.168.2.238 via xe-2/0/1.308

10.35.17.0/24      *[OSPF/10] 11:34:35, metric 11

                    > to 192.168.2.238 via xe-2/0/1.308

169.254.8.0/24     *[OSPF/10] 11:34:35, metric 11

                    > to 192.168.2.238 via xe-2/0/1.308

192.168.2.128/29   *[BGP/170] 11:23:58, MED 102, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 44

192.168.2.136/29   *[BGP/170] 11:23:58, MED 101, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 306

192.168.2.140/30   *[BGP/170] 11:23:58, MED 0, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 16, Push 209(top)

192.168.2.144/30   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                      to 172.16.128.69 via xe-2/1/1.0, Push 24, Push 207(top)

                    > to 172.16.128.65 via xe-2/2/0.0, Push 24, Push 160(top)

192.168.2.148/30   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 583, Push 209(top)

192.168.2.152/30   *[BGP/170] 11:23:58, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 24, Push 207(top)

                      to 172.16.128.65 via xe-2/2/0.0, Push 24, Push 160(top)

192.168.2.156/30   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 24, Push 207(top)

                      to 172.16.128.65 via xe-2/2/0.0, Push 24, Push 160(top)

192.168.2.160/30   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                      to 172.16.128.69 via xe-2/1/1.0, Push 516, Push 202(top)

                    > to 172.16.128.65 via xe-2/2/0.0, Push 516, Push 155(top)

192.168.2.164/30   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 146, Push 209(top)

192.168.2.168/30   *[BGP/170] 11:23:58, MED 0, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 16, Push 209(top)

192.168.2.172/30   *[BGP/170] 11:23:58, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 24, Push 207(top)

                      to 172.16.128.65 via xe-2/2/0.0, Push 24, Push 160(top)

192.168.2.176/28   *[BGP/170] 11:23:58, MED 0, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 16, Push 209(top)

192.168.2.192/28   *[BGP/170] 11:23:58, MED 0, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                      to 172.16.128.69 via xe-2/1/1.0, Push 16, Push 202(top)

                    > to 172.16.128.65 via xe-2/2/0.0, Push 16, Push 155(top)

192.168.2.232/30   *[BGP/170] 11:23:58, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 23, Push 206(top)

192.168.2.236/30   *[Direct/0] 11:34:55

                    > via xe-2/0/1.308

192.168.2.237/32   *[Local/0] 11:34:55

                      Local via xe-2/0/1.308

192.168.2.240/29   *[BGP/170] 11:23:58, MED 0, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 38

192.168.2.252/30   *[OSPF/10] 11:34:35, metric 11

                    > to 192.168.2.238 via xe-2/0/1.308

192.168.3.192/26   *[BGP/170] 11:23:58, MED 51, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 146

192.168.4.224/27   *[BGP/170] 11:23:58, MED 51, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 23, Push 206(top)

192.168.6.224/27   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                      to 172.16.128.69 via xe-2/1/1.0, Push 537, Push 202(top)

                    > to 172.16.128.65 via xe-2/2/0.0, Push 537, Push 155(top)

192.168.7.224/27   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 794, Push 209(top)

192.168.8.224/27   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 361, Push 209(top)

192.168.9.224/27   *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 24, Push 207(top)

                      to 172.16.128.65 via xe-2/2/0.0, Push 24, Push 160(top)

192.168.10.224/27  *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 24, Push 207(top)

                      to 172.16.128.65 via xe-2/2/0.0, Push 24, Push 160(top)

192.168.12.224/27  *[BGP/170] 11:23:58, MED 51, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 375, Push 209(top)

192.168.21.224/27  *[BGP/170] 11:23:58, MED 51, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 23, Push 206(top)

192.168.22.224/27  *[BGP/170] 11:23:58, MED 51, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 247

192.168.24.224/27  *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 160, Push 202(top)

                      to 172.16.128.65 via xe-2/2/0.0, Push 160, Push 155(top)

192.168.25.224/27  *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 598, Push 209(top)

192.168.26.224/27  *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 437, Push 209(top)

192.168.27.224/27  *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 24, Push 207(top)

                      to 172.16.128.65 via xe-2/2/0.0, Push 24, Push 160(top)

192.168.28.224/27  *[BGP/170] 11:23:58, MED 21, localpref 100, from 172.16.128.106

                      AS path: I, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 24, Push 207(top)

                      to 172.16.128.65 via xe-2/2/0.0, Push 24, Push 160(top)

192.168.32.0/24    *[OSPF/150] 11:34:35, metric 60, tag 0

                    > to 192.168.2.238 via xe-2/0/1.308

192.168.33.0/24    *[OSPF/150] 11:34:35, metric 60, tag 0

                    > to 192.168.2.238 via xe-2/0/1.308

192.168.253.103/32 *[BGP/170] 11:23:58, MED 0, localpref 100, from 172.16.128.106

                      AS path: ?, validation-state: unverified

                    > to 172.16.128.69 via xe-2/1/1.0, Push 38

192.168.253.109/32 *[Direct/0] 11:34:55

                    > via lo0.308

224.0.0.5/32       *[OSPF/10] 11:34:55, metric 1

                      MultiRecv

  
А вот эти самые hidden: 


user@mx240> show ospf route instance CLIENTOSPF

Topology default Route Table:



Prefix             Path  Route      NH       Metric NextHop       Nexthop

                   Type  Type       Type            Interface     Address/LSP

10.10.10.9         Intra Area/AS BR IP          102 shamlink.0

192.168.2.243      Intra AS BR      IP            2 shamlink.0

192.168.2.253      Intra AS BR      IP           10 xe-2/0/1.308  192.168.2.238

192.168.253.103    Intra Area/AS BR IP            1 shamlink.0

0.0.0.0/0          Ext1  Network    IP           61 xe-2/0/1.308  192.168.2.238

10.35.17.0/24      Intra Network    IP           11 xe-2/0/1.308  192.168.2.238

169.254.8.0/24     Intra Network    IP           11 xe-2/0/1.308  192.168.2.238

192.168.2.128/29   Intra Network    IP          103 shamlink.0

192.168.2.136/29   Intra Network    IP          102 shamlink.0

192.168.2.144/30   Ext1  Network    IP           22 shamlink.0

192.168.2.148/30   Ext1  Network    IP           22 shamlink.0

192.168.2.152/30   Ext1  Network    IP            2 shamlink.0

192.168.2.156/30   Ext1  Network    IP           22 shamlink.0

192.168.2.160/30   Ext1  Network    IP           22 shamlink.0

192.168.2.164/30   Ext1  Network    IP           22 shamlink.0

192.168.2.172/30   Ext1  Network    IP            2 shamlink.0

192.168.2.232/30   Ext1  Network    IP            2 shamlink.0

192.168.2.236/30   Intra Network    IP           10 xe-2/0/1.308

192.168.2.240/29   Intra Network    IP            2 shamlink.0

192.168.2.252/30   Intra Network    IP           11 xe-2/0/1.308  192.168.2.238

192.168.3.192/26   Ext1  Network    IP           52 shamlink.0

192.168.4.224/27   Ext1  Network    IP           52 shamlink.0

192.168.6.224/27   Ext1  Network    IP           22 shamlink.0

192.168.7.224/27   Ext1  Network    IP           22 shamlink.0

192.168.8.224/27   Ext1  Network    IP           22 shamlink.0

192.168.9.224/27   Ext1  Network    IP           22 shamlink.0

192.168.10.224/27  Ext1  Network    IP           22 shamlink.0

192.168.12.224/27  Ext1  Network    IP           52 shamlink.0

192.168.21.224/27  Ext1  Network    IP           52 shamlink.0

192.168.22.224/27  Ext1  Network    IP           52 shamlink.0

192.168.24.224/27  Ext1  Network    IP           22 shamlink.0

192.168.25.224/27  Ext1  Network    IP           22 shamlink.0

192.168.26.224/27  Ext1  Network    IP           22 shamlink.0

192.168.27.224/27  Ext1  Network    IP           22 shamlink.0

192.168.28.224/27  Ext1  Network    IP           22 shamlink.0

192.168.32.0/24    Ext1  Network    IP           60 xe-2/0/1.308  192.168.2.238

192.168.33.0/24    Ext1  Network    IP           60 xe-2/0/1.308  192.168.2.238

192.168.253.103/32 Intra Network    IP            2 shamlink.0


В этом гаджете обнаружена ошибка