CARP on FreeBSD 10.0-CURRENT

Comments Off on CARP on FreeBSD 10.0-CURRENT

I’ve been experimenting with carp(4) on FreeBSD/i386 10.0-CURRENT and FreeBSD/amd64 10.0-CURRENT for the past year or so. carp(4) is no longer a pseudo-interface, but rather accessible on every conceivable interface.

To regain reasonable speed it’s necessary to disable the debugging techniques normally employed in 10.0-CURRENT with a kernel configuration file like this one:

include GENERIC
ident CARP
nooptions DEADLKRES
nooptions INVARIANTS
nooptions INVARIANT_SUPPORT
nooptions WITNESS
nooptions WITNESS_SKIPSPIN
device carp

The kernel configuration file is appropriately named CARP and placed in the two directories /usr/src/sys/i386/conf and /usr/src/sys/amd64/conf.

Don’t forget to add a line like this one to the /etc/make.conf file:

KERNCONF=CARP

On both nodes, the /etc/hosts file has this contents:

::1                     localhost localhost.example.net
127.0.0.1               localhost localhost.example.net

192.0.2.2               dhcp01.example.net dhcp01
2001:db8:1:2::2         dhcp01.example.net dhcp01
192.0.2.3               dhcp02.example.net dhcp02
2001:db8:1:2::3         dhcp02.example.net dhcp02
192.0.2.4               dhcp.example.net   dhcp
2001:db8:1:2::4         dhcp.example.net   dhcp

On the master node, the /etc/rc.conf file has this contents:

background_fsck="NO"
hostname="dhcp01.example.net"
log_in_vain="1"
icmp_log_redirect="YES"
#ifconfig_vtnet1="DHCP"
#ifconfig_vtnet0="DHCP"
ifconfig_vtnet0="inet 192.0.2.2 netmask 255.255.255.0"
ifconfig_vtnet0_ipv6="inet6 2001:db8:1:2::2 prefixlen 64"
ifconfig_vtnet0_alias0="inet 192.0.2.4 netmask 255.255.255.0 vhid 1 pass dettekanikkedu advbase 1 advskew 0"
ifconfig_vtnet0_alias1="inet6 2001:db8:1:2::4 prefixlen 64 vhid 1"
syslogd_flags="-8ccssvv"
inetd_enable="YES"
inetd_flags="-lwW -C 60 -a dhcp.example.net"
#sshd_enable="YES"
#ntpdate_enable="YES"
ntpdate_hosts="ntp.example.net"
#ntpd_enable="YES"
defaultrouter="192.0.2.1"
#netwait_enable="YES"
netwait_ip="192.0.2.1"
netwait_if="vtnet0"
ipv6_defaultrouter="2001:db8:1:2::1"
ipv6_default_interface="vtnet0"
ip6addrctl_policy="ipv6_prefer"
keymap="norwegian.iso.kbd"
keyrate="fast"
keybell="off"
font8x16="iso-8x16"
font8x14="iso-8x14"
font8x8="iso-8x8"
moused_enable="YES"
dumpdev="AUTO"
sendmail_enable="NONE"

On the slave node, the /etc/rc.conf file has this contents:

background_fsck="NO"
hostname="dhcp02.example.net"
log_in_vain="1"
icmp_log_redirect="YES"
#ifconfig_vtnet1="DHCP"
#ifconfig_vtnet0="DHCP"
ifconfig_vtnet0="inet 192.0.2.3 netmask 255.255.255.0"
ifconfig_vtnet0_ipv6="inet6 2001:db8:1:2::3 prefixlen 64"
ifconfig_vtnet0_alias0="inet 192.0.2.4 netmask 255.255.255.0 vhid 1 pass dettekanikkedu advbase 1 advskew 10"
ifconfig_vtnet0_alias1="inet6 2001:db8:1:2::4 prefixlen 64 vhid 1"
syslogd_flags="-8ccssvv"
inetd_enable="YES"
inetd_flags="-lwW -C 60 -a dhcp.example.net"
#sshd_enable="YES"
#ntpdate_enable="YES"
ntpdate_hosts="ntp.example.net"
#ntpd_enable="YES"
defaultrouter="192.0.2.1"
#netwait_enable="YES"
netwait_ip="192.0.2.1"
netwait_if="vtnet0"
ipv6_defaultrouter="2001:db8:1:2::1"
ipv6_default_interface="vtnet0"
ip6addrctl_policy="ipv6_prefer"
keymap="norwegian.iso.kbd"
keyrate="fast"
keybell="off"
font8x16="iso-8x16"
font8x14="iso-8x14"
font8x8="iso-8x8"
moused_enable="YES"
dumpdev="AUTO"
sendmail_enable="NONE"

You also need to add this line to the /etc/sysctl.conf file on both nodes:

net.inet.carp.preempt=1

You may see console/log messages such as these ones when each node attempts to become the master or the backup:

ifa_add_loopback_route: insertion failed: 17
ifa_add_loopback_route: deletion failed: 48

I’m not entirely sure what this means.

A few more things needs to be ironed out:

  1. Ensuring pxed responds with the common 192.0.2.4 IPv4 address.
  2. Ensuring tftpd through inetd responds with the common 192.0.2.4 IPv4 address.
  3. Ensuring dhcpd on both nodes gets notified by the other node on which addresses are assigned to which clients, even on subnets with dynamic allocation. And yes, ensuring dhcpd responds with the common 192.0.2.4 IPv4 address.
  4. All other services should use the node specific addresses, be it 192.0.2.2 and 2001:db8:1:2::2, or 192.0.2.3 and 2001:db8:1:2::3.
  5. The switch must tolerate the 192.0.2.4 IPv4 address and the 2001:db8:1:2::4 IPv6 address being moved between the switchports used by the master and the backup nodes.
  6. Is it better to use the service addresses as the primary ones on both nodes, and letting the node specific addresses be the alias addresses?