Custom FreeBSD Ucarp Setup

Custom FreeBSD UCARP Setup | 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗

This entry is sponsored by fme AG company – and this article aspires to share about interesting HA setup on node01a and node01b VMs.

The interesting case here is that the 10.0.0.x/24 is limited in one of the clients – most IP(s) are taken. With normal carp(4) installation we would need 3 IP(s) from that network – two IP(s) for each of the two nodes and an additional HA IP for the service. To overcome that we will use 3 IPs(2) from different 192.168.0.x/24 network and only one IP from the limited 10.0.0.x/24 network. The net/ucarp port that we will use here has only one disadvantage that we can overcome easily – it does not do anything to MAC addresses of the network interfaces – and knowing how the network switches work – they may ‘redirect’ traffic to ‘bad’ interface with ‘bad’ MAC address – so we will also use something like HA MAC concept and move that HA MAC with the HA IP.

List of these Bhyve VMs below.

bhyve # vm list | grep -e NAME -e node01 NAME DATASTORE LOADER CPU MEMORY VNC AUTO STATE node01a default uefi 2 4G - No Running (64537) node01b default uefi 2 4G - No Running (60781)

Below You will find summary of IPs(s) and MAC(s) addresses.

The net/ucarp provides the ‘UP’ and ‘DOWN’ scripts – I just copied them to new /usr/local/etc/ucarp/ dir and added some content to them.

node0a # pkg info -l ucarp ucarp-1.5.2.20171201: /usr/local/etc/rc.d/ucarp /usr/local/sbin/ucarp /usr/local/sbin/ucarp-down /usr/local/sbin/ucarp-up /usr/local/share/licenses/ucarp-1.5.2.20171201/ISC /usr/local/share/licenses/ucarp-1.5.2.20171201/LICENSE /usr/local/share/licenses/ucarp-1.5.2.20171201/catalog.mk

node0a # cp /usr/local/sbin/ucarp-up /usr/local/etc/ucarp/100-up.sh

node0a # cp /usr/local/sbin/ucarp-down /usr/local/etc/ucarp/100-down.sh

The node01a config /etc/rc.conf file related to UCARP config is below – all the other options are default and standardized – I also added hostname and networking information for clarity.

hostname="node01a.local" ifconfig_vtnet0="inet 192.168.0.11/24" ifconfig_vtnet1="up" ucarp_enable="YES" ucarp_preempt="YES" ucarp_shutdown="YES" ucarp_src="192.168.0.11" ucarp_addr="192.168.0.100" ucarp_if="vtnet0" ucarp_pass="S3CUREcarp" ucarp_vhid="100" ucarp_upscript="/usr/local/etc/ucarp/100-up.sh" ucarp_downscript="/usr/local/etc/ucarp/100-down.sh" The node01b config /etc/rc.conf file related to UCARP config below.

hostname="node01b.local" ifconfig_vtnet0="inet 192.168.0.12/24" ifconfig_vtnet1="up" ucarp_enable="YES" ucarp_preempt="YES" ucarp_shutdown="YES" ucarp_src="192.168.0.12" ucarp_addr="192.168.0.100" ucarp_if="vtnet0" ucarp_pass="S3CUREcarp" ucarp_vhid="100" ucarp_upscript="/usr/local/etc/ucarp/100-up.sh" ucarp_downscript="/usr/local/etc/ucarp/100-down.sh" Now – the UCARP needs the ‘UP’ and ‘DOWN’ scripts – the ‘UP’ scripts are the same on both nodes but the ‘DOWN’ scripts are different because they bring back the ‘local’ MAC after stepping down from the ‘master’ role. The

The /usr/local/etc/ucarp/100-up.sh for both node01a and node01b hosts below.

#! /bin/sh

if [ -z "${1}" -o -z "${2}" ] then cat /dev/null

if [ ! -z "${3}" ] then /sbin/ifconfig "${3}" -alias "${2}" fi

/sbin/ifconfig "${1}" alias "${2}" netmask 255.255.255.255

# HA IP AND HA MAC /sbin/ifconfig vtnet1 ether 58:9c:fc:ff:01:ff /sbin/ifconfig vtnet1 inet 10.0.0.100/24 up /sbin/route add default 10.0.0.1 /sbin/route change default 10.0.0.1

The /usr/local/etc/ucarp/100-down.sh for node01a node below.

#! /bin/sh

if [ -z "${1}" -o -z "${2}" ] then cat /dev/null

/sbin/ifconfig "${1}" -alias "${2}"

# HA IP AND HA MAC - BRING BACK node01a MAC /sbin/ifconfig vtnet1 ether 58:9c:fc:ff:01:aa /sbin/ifconfig vtnet1 delete 10.0.0.100

The /usr/local/etc/ucarp/100-down.sh for node01b node.

#! /bin/sh

if [ -z "${1}" -o -z "${2}" ] then cat /dev/null

/sbin/ifconfig "${1}" -alias "${2}"

# HA IP AND HA MAC - BRING BACK node01b MAC /sbin/ifconfig vtnet1 ether 58:9c:fc:ff:01:bb /sbin/ifconfig vtnet1 delete 10.0.0.100

To move the HA IP from the ‘master’ to the ‘backup’ is to execute this command on the ‘master’ host:

node01a # service ucarp restart

This is how ‘master’ networking looks like – its on node01b host this time:

node01b # netstat -Win -f inet you have mail Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll vtnet0 - 192.168.0.0/24 192.168.0.12 168965 - - 263946 - - vtnet0 - 192.168.0.0/32 192.168.0.100 0 - - 0 - - vtnet1 - 10.0.0.0/27 10.0.0.100 9496832 - - 11104229 - - lo0 - 127.0.0.0/8 127.0.0.1 7071008 - - 7071036 - -

node01b # netstat -Wrn -f inet Routing tables

Internet: Destination Gateway Flags Nhop# Mtu Netif Expire default 10.0.0.1 UGS 22 1500 vtnet1 127.0.0.1...

Custom FreeBSD Ucarp Setup

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

NHS apologises and admits Palantir have access to identifiable patient data

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills