Alternative ZFS layout

I’ve been experimenting with FreeBSD 10.0-CURRENT and an alternative ZFS layout for more than a year. The idea is to have a simple way of switching between boot environments. Thus, you have the opportunity of reverting to a previous boot environment without resorting to snapshots and rollbacks as you would with a sole boot environment. At the same time we avoid using the immediate top-level ZFS filesystem.

Here’s the alternative ZFS layout. It’s inspired by (Open)Solaris and other sources, so it’s certainly not a work entirely of my own.

In the above case the bootfs zpool property would be set to
zroot/ROOT/20130623-r252101.

In a real system, /home should probably reside on a separate zpool, as should the filesystems for any database system.

Is it wise to have the Subversion global revision number for the FreeBSD base repository embedded in the name of each boot environment? Which of the following would you prefer?

1. A date, as in 20130718?
2. The Subversion global revision number, as in r253445?
3. Both a date and the Subversion global revision number, as in 20130718-r253445?
4. Both the Subversion global revision number and a date, as in r253445-20130718?
5. The Subversion last changed revision number, as in r253445?
6. Both a date and the Subversion last changed revision number, as in 20130718-r253445?
7. Both the Subversion last changed revision number and a date, as in r253445-20130718?
8. None of the above?

I have begun appreciating the third option, but maybe it’s even wiser to use the Subversion last changed revision number, i.e. the sixth option.

Upgrading the system would be as follows. Note that zroot/ROOT/yyyymmdd-rxxxxxx represents the current boot environment, while zroot/ROOT/YYYYMMDD-rXXXXXX represents the clone, with zroot/ROOT/yyyymmdd-rxxxxxx@pre-YYYYMMDD-rXXXXXX being its origin.

1. svn up /usr/src
2. rm -Rf /usr/obj/*
3. cd /usr/src
4. make -j 12 -DNO_CLEAN buildworld buildkernel
5. zfs snapshot zroot/ROOT/yyyymmdd-rxxxxxx@pre-YYYYMMDD-rXXXXXX
6. zfs clone -o mountpoint=/YYYYMMDD \
zroot/ROOT/yyyymmdd-rxxxxxx@pre-YYYYMMDD-rXXXXXX zroot/ROOT/YYYYMMDD-rXXXXXX
7. make DESTDIR=/YYYYMMDD installworld installkernel
8. zfs inherit mountpoint zroot/ROOT/YYYYMMDD-rXXXXXX
9. zpool set bootfs=zroot/ROOT/YYYYMMDD-rXXXXXX zroot
10. rmdir /YYYYMMDD
11. Reboot to single user mode.
12. mergemaster -p
13. mergemaster -Fi
14. cd /usr/src
15. make delete-old
16. make delete-old-libs
17. Optionally rebuild sysutils/lsof and/or emulators/virtualbox-ose-additions to account for changes in the kernel internals.
18. Optionally promote the new clone to be independent of its origin:
    zfs promote zroot/ROOT/YYYYMMDD-rXXXXXX
    Note the snapshot used for the clone’s origin has been moved to be a part of the clone, i.e. zroot/ROOT/YYYYMMDD-rXXXXXX@pre-YYYYMMDD-rXXXXXX. The snapshot is still linked to the previous boot environment.
19. Optionally destroy older boot environments to conserve disk space.
20. Optionally destroy the snapshots for the current boot environment, if warranted and possible.
21. Optionally update the boot blocks, particularly if you intend to upgrade the zpools and/or filesystems.
22. Optionally upgrade the zpools and/or filesystems.
23. Reboot to multi user mode.

Should the need for booting an older boot environment arise, say when you are unable to set the bootfs zpool property using the conventional method, you can accomplish this by interrupting the boot loader, typically by hitting the escape key when the boot menu is shown. Use the lszfs poolname command to inspect the available filesystems. Usually, lszfs zroot/ROOT should give an idea of the available boot environments. Change the currdev variable to the desired boot environment:

OK unload
OK set currdev="zfs:zroot/ROOT/yyyymmdd-rxxxxxx:"
OK load kernel
OK load opensolaris
OK load zfs
OK boot

Ensure you begin the currdev string with zfs: and terminate the string with a colon, :. Note that setting currdev does not change the bootfs zpool property.

Andriy Gapon’s talk entitled “Practical ZFS For A Common FreeBSD User” at KyivBSD, 2012, put me right on track for considering the use of snapshots and clones. Here’s a picture from the session.

After having used the recipe above, I came to realise the boot times keeps increasing. It turns out all the snapshots are being transferred to the newest BE. At some point it just becomes unbearable, and you need to cut through the crap by creating a fresh BE without any attachments to the prior ones. (I really do miss the old behaviour of the zfs promote command.) To ease the migration from the previous BE to the new one, I began using this procedure:

1. svn up /usr/src
2. rm -Rf /usr/obj/*
3. cd /usr/src
4. make -j 12 -DNO_CLEAN buildworld buildkernel
5. zfs create -o mountpoint=/YYYYMMDD \
zroot/ROOT/YYYYMMDD-rXXXXXX
6. export DESTDIR=/YYYYMMDD
7. make installworld installkernel
8. tar cf - /.bash* /.cshrc /.inputrc /.profile /.rnd /.shrc \
/COPYRIGHT /boot/device.hints /boot/loader.conf /boot/modules \
/boot/zfs/zpool.cache /entropy /etc /media /root | \
(cd ${DESTDIR}; tar xvvf -)
9. ln -s usr/compat ${DESTDIR}/compat
10. ln -s /usr/local/bin/perl5.16.3 ${DESTDIR}/usr/bin/perl
11. ln -s /usr/local/bin/perl5.16.3 ${DESTDIR}/usr/bin/perl5
12. zfs inherit mountpoint zroot/ROOT/YYYYMMDD-rXXXXXX
13. zpool set bootfs=zroot/ROOT/YYYYMMDD-rXXXXXX zroot
14. rmdir /YYYYMMDD
15. Reboot to single user mode.
16. mergemaster -p
17. mergemaster -Fi
18. cd /usr/src
19. make delete-old
20. make delete-old-libs
21. Optionally rebuild sysutils/lsof and/or emulators/virtualbox-ose-additions to account for changes in the kernel internals.
22. Optionally destroy older boot environments to conserve disk space.
23. Optionally update the boot blocks, particularly if you intend to upgrade the zpools and/or filesystems.
24. Optionally upgrade the zpools and/or filesystems.
25. Reboot to multi user mode.

You might need to adjust the paths accumulated by the first tar command in step 8, and the symlinks in steps 9, 10, and 11.

For production systems one should consider naming the zpools using this as a template:

${hostname}_${pool_role}_g${generation_number}

E.g.:

enterprise_zroot_g06
enterprise_zdata_g06
vico_sensor_data_g01
voyager_omega_g01
yamato_rpool_g02
yamato_section_31_g02

This way each zpool is given a unique name and may easily be imported to other systems, including its own successor and possibly even one of its predecessors. This is handy if and when you prepare new (root) pools for an existing system.

Using two pools the layout would become this:

Here’s a shell script to create the ZFS layout:

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${ROOTPOOL}" ]; then
  echo "$0: you must set the environment variable ROOTPOOL to the name of your root pool" >/dev/stderr;
  echo "$0: e.g. export ROOTPOOL=zroot" >/dev/stderr;
  exit 69;
fi

if ! zpool list "${ROOTPOOL}" >/dev/null 2>/dev/null; then
  echo "$0: root pool ${ROOTPOOL} does not exist" >/dev/stderr;
  echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${ROOTPOOL} ..." >/dev/stderr;
  exit 69;
fi

if [ -z "${DATAPOOL}" ]; then
  echo "$0: you must set the environment variable DATAPOOL to the name of your data pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=zdata" >/dev/stderr;
  echo "$0: your data pool can be the same as your root pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=\${ROOTPOOL}" >/dev/stderr;
  exit 69;
fi

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  if ! zpool list "${DATAPOOL}" >/dev/null 2>/dev/null; then
    echo "$0: data pool ${DATAPOOL} does not exist" >/dev/stderr;
    echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${DATAPOOL} ..." >/dev/stderr;
    exit 69;
  fi;
fi

if [ -z "${RELEASEDATE}" ]; then
  echo "$0: you must set the environment variable RELEASEDATE to the date of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEDATE=20130602" >/dev/stderr;
  exit 69;
fi

if [ -z "${RELEASEREV}" ]; then
  echo "$0: you must set the environment variable RELEASEREV to the Subversion revision number of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEREV=251259" >/dev/stderr;
  exit 69;
fi

# It is assumed the pools were created with -O mountpoint=legacy.

zfs create ${ROOTPOOL}/ROOT
zfs create ${ROOTPOOL}/ROOT/${RELEASEDATE}-r${RELEASEREV}

zpool set bootfs=${ROOTPOOL}/ROOT/${RELEASEDATE}-r${RELEASEREV} ${ROOTPOOL}

zfs create ${ROOTPOOL}/tmp
zfs create ${ROOTPOOL}/usr
zfs create ${ROOTPOOL}/usr/compat
zfs create ${ROOTPOOL}/usr/compat/linux
zfs create ${ROOTPOOL}/usr/local
zfs create ${ROOTPOOL}/usr/local/certs
zfs create ${ROOTPOOL}/usr/local/etc
zfs create ${ROOTPOOL}/usr/local/etc/namedb
zfs create ${ROOTPOOL}/usr/obj
zfs create ${ROOTPOOL}/usr/ports
zfs create ${ROOTPOOL}/usr/ports/distfiles
zfs create ${ROOTPOOL}/usr/ports/packages
zfs create ${ROOTPOOL}/usr/ports/workdirs
zfs create ${ROOTPOOL}/usr/src
zfs create ${ROOTPOOL}/var
zfs create ${ROOTPOOL}/var/backups
zfs create ${ROOTPOOL}/var/crash
zfs create ${ROOTPOOL}/var/db
zfs create ${ROOTPOOL}/var/db/pkg
zfs create ${ROOTPOOL}/var/db/ports
zfs create ${ROOTPOOL}/var/empty
zfs create ${ROOTPOOL}/var/log
zfs create ${ROOTPOOL}/var/mail
zfs create ${ROOTPOOL}/var/named
zfs create ${ROOTPOOL}/var/run
zfs create ${ROOTPOOL}/var/spool
zfs create ${ROOTPOOL}/var/tmp
zfs create ${ROOTPOOL}/var/unbound

zfs create ${DATAPOOL}/home

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs create ${DATAPOOL}/usr;
  zfs create ${DATAPOOL}/usr/local;
fi
zfs create ${DATAPOOL}/usr/local/pgsql
zfs create ${DATAPOOL}/usr/local/www

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs create ${DATAPOOL}/var;
  zfs create ${DATAPOOL}/var/db;
  zfs create ${DATAPOOL}/var/spool;
fi
zfs create ${DATAPOOL}/var/db/mysql
zfs create ${DATAPOOL}/var/spool/ftp

zfs set atime=off ${DATAPOOL}/var/spool/ftp;

zfs set compression=gzip-9 ${ROOTPOOL}/tmp
zfs set compression=gzip-9 ${ROOTPOOL}/usr/local/certs
zfs set compression=gzip-9 ${ROOTPOOL}/usr/local/etc
zfs set compression=gzip-9 ${DATAPOOL}/usr/local/www
zfs set compression=gzip-9 ${ROOTPOOL}/usr/ports
zfs set compression=gzip-9 ${ROOTPOOL}/usr/src
zfs set compression=gzip-9 ${ROOTPOOL}/var/db/pkg
zfs set compression=gzip-9 ${ROOTPOOL}/var/db/ports
zfs set compression=gzip-9 ${ROOTPOOL}/var/log
zfs set compression=gzip-9 ${ROOTPOOL}/var/mail
zfs set compression=gzip-9 ${ROOTPOOL}/var/named
zfs set compression=gzip-9 ${ROOTPOOL}/var/tmp
zfs set compression=gzip-9 ${ROOTPOOL}/var/unbound

zfs set compression=off ${ROOTPOOL}/usr/ports/distfiles
zfs set compression=off ${ROOTPOOL}/usr/ports/packages
zfs set compression=off ${ROOTPOOL}/usr/ports/workdirs

zfs set exec=off ${ROOTPOOL}/usr/local/etc/namedb
zfs set exec=off ${ROOTPOOL}/usr/ports/distfiles
zfs set exec=off ${ROOTPOOL}/usr/ports/packages
zfs set exec=off ${ROOTPOOL}/usr/src
zfs set exec=off ${ROOTPOOL}/var/backups
zfs set exec=off ${ROOTPOOL}/var/crash
zfs set exec=off ${ROOTPOOL}/var/db
zfs set exec=off ${ROOTPOOL}/var/empty
zfs set exec=off ${ROOTPOOL}/var/log
zfs set exec=off ${ROOTPOOL}/var/mail
zfs set exec=off ${ROOTPOOL}/var/named
zfs set exec=off ${ROOTPOOL}/var/run
zfs set exec=off ${ROOTPOOL}/var/spool
zfs set exec=off ${ROOTPOOL}/var/unbound

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
zfs set exec=off ${DATAPOOL}/var/spool;
fi

zfs set exec=on ${ROOTPOOL}/var/db/pkg
zfs set exec=on ${ROOTPOOL}/var/db/ports

zfs set setuid=off ${ROOTPOOL}/tmp
zfs set setuid=off ${DATAPOOL}/home
zfs set setuid=off ${ROOTPOOL}/usr/local/etc/namedb
zfs set setuid=off ${ROOTPOOL}/usr/ports
zfs set setuid=off ${ROOTPOOL}/usr/src
zfs set setuid=off ${ROOTPOOL}/var/backups
zfs set setuid=off ${ROOTPOOL}/var/crash
zfs set setuid=off ${ROOTPOOL}/var/db
zfs set setuid=off ${ROOTPOOL}/var/empty
zfs set setuid=off ${ROOTPOOL}/var/log
zfs set setuid=off ${ROOTPOOL}/var/mail
zfs set setuid=off ${ROOTPOOL}/var/named
zfs set setuid=off ${ROOTPOOL}/var/run
zfs set setuid=off ${ROOTPOOL}/var/spool
zfs set setuid=off ${DATAPOOL}/var/spool
zfs set setuid=off ${ROOTPOOL}/var/tmp
zfs set setuid=off ${ROOTPOOL}/var/unbound

zfs set recordsize=8K  ${DATAPOOL}/usr/local/pgsql
zfs set recordsize=16K ${DATAPOOL}/var/db/mysql

Here’s a script for setting the temporary mountpoints rooted at ${DESTDIR}:

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${ROOTPOOL}" ]; then
  echo "$0: you must set the environment variable ROOTPOOL to the name of your root pool" >/dev/stderr;
  echo "$0: e.g. export ROOTPOOL=zroot" >/dev/stderr;
  exit 69;
fi

if ! zpool list "${ROOTPOOL}" >/dev/null 2>/dev/null; then
  echo "$0: root pool ${ROOTPOOL} does not exist" >/dev/stderr;
  echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${ROOTPOOL} ..." >/dev/stderr;
  exit 69;
fi

if [ -z "${DATAPOOL}" ]; then
  echo "$0: you must set the environment variable DATAPOOL to the name of your data pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=zdata" >/dev/stderr;
  echo "$0: your data pool can be the same as your root pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=\${ROOTPOOL}" >/dev/stderr;
  exit 69;
fi

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  if ! zpool list "${DATAPOOL}" >/dev/null 2>/dev/null; then
    echo "$0: data pool ${DATAPOOL} does not exist" >/dev/stderr;
    echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${DATAPOOL} ..." >/dev/stderr;
    exit 69;
  fi;
fi

if [ -z "${RELEASEDATE}" ]; then
  echo "$0: you must set the environment variable RELEASEDATE to the date of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEDATE=20130602" >/dev/stderr;
  exit 69;
fi

if [ -z "${RELEASEREV}" ]; then
  echo "$0: you must set the environment variable RELEASEREV to the Subversion revision number of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEREV=251259" >/dev/stderr;
  exit 69;
fi

if [ -z "${DESTDIR}" ]; then
  echo "$0: you must set the environment variable DESTDIR to the destination directory" >/dev/stderr;
  echo "$0: e.g. export DESTDIR=/tmp/zroot" >/dev/stderr;
  exit 69;
fi

if [ ! -d "${DESTDIR}" ]; then
  mkdir -p "${DESTDIR}";

  if [ ! -d "${DESTDIR}" ]; then
    echo "$0: unable to create destination directory ${DESTDIR}" >/dev/stderr;
    exit 69;
  fi;
fi

cd /

zfs set mountpoint=${DESTDIR}                  ${ROOTPOOL}/ROOT/${RELEASEDATE}-r${RELEASEREV}
zfs set mountpoint=${DESTDIR}/home             ${DATAPOOL}/home
zfs set mountpoint=${DESTDIR}/tmp              ${ROOTPOOL}/tmp
zfs set mountpoint=${DESTDIR}/usr/compat/linux ${ROOTPOOL}/usr/compat/linux
zfs set mountpoint=${DESTDIR}/usr/local        ${ROOTPOOL}/usr/local
zfs set mountpoint=${DESTDIR}/usr/obj          ${ROOTPOOL}/usr/obj
zfs set mountpoint=${DESTDIR}/usr/ports        ${ROOTPOOL}/usr/ports
zfs set mountpoint=${DESTDIR}/usr/src          ${ROOTPOOL}/usr/src
zfs set mountpoint=${DESTDIR}/var              ${ROOTPOOL}/var

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs set mountpoint=${DESTDIR}/usr/local/pgsql ${DATAPOOL}/usr/local/pgsql;
  zfs set mountpoint=${DESTDIR}/usr/local/www   ${DATAPOOL}/usr/local/www;
  zfs set mountpoint=${DESTDIR}/var/db/mysql    ${DATAPOOL}/var/db/mysql;
  zfs set mountpoint=${DESTDIR}/var/spool/ftp   ${DATAPOOL}/var/spool/ftp;
fi

mkdir ${DESTDIR}/proc
mkdir ${DESTDIR}/usr/compat/linux/proc
chmod 1777 ${DESTDIR}/tmp
chmod 1777 ${DESTDIR}/var/tmp

mkdir ${DESTDIR}/media
mkdir ${DESTDIR}/media/cdrom
mkdir ${DESTDIR}/media/dvdrom

ln -s usr/compat ${DESTDIR}/compat

Pick one of these two shell scripts to extract a base system into ${DESTDIR}:

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${DESTDIR}" ]; then
  echo "$0: you must set the environment variable DESTDIR to the destination directory" >/dev/stderr;
  echo "$0: e.g. export DESTDIR=/tmp/zroot" >/dev/stderr;
  exit 69;
fi

if [ ! -d "${DESTDIR}" ]; then
  echo "$0: destination directory ${DESTDIR} does not exist" >/dev/stderr;
  echo "$0: maybe mkdir -p ${DESTDIR} needs to be run" >/dev/stderr;
  exit 69;
fi

if ! zfs get mountpoint "${DESTDIR}" >/dev/null 2>/dev/null; then
  echo "$0: destination directory ${DESTDIR} is not a ZFS filesystem" >/dev/stderr;
  exit 69;
fi

cd /dist/8.*/base

echo "$0: Please answer y to the following question!"
./install.sh
# Answer y

cd ../games
./install.sh

cd ../info
./install.sh

cd ../lib32 && ./install.sh

cd ../manpages
./install.sh

cd ../kernels
./install.sh generic

cd ${DESTDIR}/boot
rmdir kernel
mv GENERIC kernel

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${DESTDIR}" ]; then
  echo "$0: you must set the environment variable DESTDIR to the destination directory" >/dev/stderr;
  echo "$0: e.g. export DESTDIR=/tmp/zroot" >/dev/stderr;
  exit 69;
fi

if [ ! -d "${DESTDIR}" ]; then
  echo "$0: destination directory ${DESTDIR} does not exist" >/dev/stderr;
  echo "$0: maybe mkdir -p ${DESTDIR} needs to be run" >/dev/stderr;
  exit 69;
fi

if ! zfs get mountpoint "${DESTDIR}" >/dev/null 2>/dev/null; then
  echo "$0: destination directory ${DESTDIR} is not a ZFS filesystem" >/dev/stderr;
  exit 69;
fi

cd ${DESTDIR}

tar xvvf /usr/freebsd-dist/kernel.txz
tar xvvf /usr/freebsd-dist/base.txz
tar xvvf /usr/freebsd-dist/lib32.txz
tar xvvf /usr/freebsd-dist/doc.txz
tar xvvf /usr/freebsd-dist/games.txz

Here’s a script for setting the final mountpoints:

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${ROOTPOOL}" ]; then
  echo "$0: you must set the environment variable ROOTPOOL to the name of your root pool" >/dev/stderr;
  echo "$0: e.g. export ROOTPOOL=zroot" >/dev/stderr;
  exit 69;
fi

if ! zpool list "${ROOTPOOL}" >/dev/null 2>/dev/null; then
  echo "$0: root pool ${ROOTPOOL} does not exist" >/dev/stderr;
  echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${ROOTPOOL} ..." >/dev/stderr;
  exit 69;
fi

if [ -z "${DATAPOOL}" ]; then
  echo "$0: you must set the environment variable DATAPOOL to the name of your data pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=zdata" >/dev/stderr;
  echo "$0: your data pool can be the same as your root pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=\${ROOTPOOL}" >/dev/stderr;
  exit 69;
fi

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  if ! zpool list "${DATAPOOL}" >/dev/null 2>/dev/null; then
    echo "$0: data pool ${DATAPOOL} does not exist" >/dev/stderr;
    echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${DATAPOOL} ..." >/dev/stderr;
    exit 69;
  fi;
fi

if [ -z "${RELEASEDATE}" ]; then
  echo "$0: you must set the environment variable RELEASEDATE to the date of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEDATE=20130602" >/dev/stderr;
  exit 69;
fi

if [ -z "${RELEASEREV}" ]; then
  echo "$0: you must set the environment variable RELEASEREV to the Subversion revision number of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEREV=251259" >/dev/stderr;
  exit 69;
fi

cd /

zfs set readonly=on ${ROOTPOOL}/var/empty

zfs unmount -a

zfs inherit mountpoint ${ROOTPOOL}/ROOT/${RELEASEDATE}-r${RELEASEREV}

zfs set mountpoint=/home             ${DATAPOOL}/home
zfs set mountpoint=/tmp              ${ROOTPOOL}/tmp
zfs set mountpoint=/usr/compat/linux ${ROOTPOOL}/usr/compat/linux
zfs set mountpoint=/usr/local        ${ROOTPOOL}/usr/local
zfs set mountpoint=/usr/obj          ${ROOTPOOL}/usr/obj
zfs set mountpoint=/usr/ports        ${ROOTPOOL}/usr/ports
zfs set mountpoint=/usr/src          ${ROOTPOOL}/usr/src
zfs set mountpoint=/var              ${ROOTPOOL}/var

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs set mountpoint=/usr/local/pgsql ${DATAPOOL}/usr/local/pgsql;
  zfs set mountpoint=/usr/local/www   ${DATAPOOL}/usr/local/www;
  zfs set mountpoint=/var/db/mysql    ${DATAPOOL}/var/db/mysql;
  zfs set mountpoint=/var/spool/ftp   ${DATAPOOL}/var/spool/ftp;
fi

zfs unmount -a

The five shell scripts are available at this URL: http://ximalas.info/~trond/create-zfs/.

Shell scripts with some canmount=off experiments are available at this URL: http://ximalas.info/~trond/create-zfs/canmount/.

Here’s a shell script to create the ZFS layout:

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${ROOTPOOL}" ]; then
  echo "$0: you must set the environment variable ROOTPOOL to the name of your root pool" >/dev/stderr;
  echo "$0: e.g. export ROOTPOOL=zroot" >/dev/stderr;
  exit 69;
fi

if ! zpool list "${ROOTPOOL}" >/dev/null 2>/dev/null; then
  echo "$0: root pool ${ROOTPOOL} does not exist" >/dev/stderr;
  echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${ROOTPOOL} ..." >/dev/stderr;
  exit 69;
fi

if [ -z "${DATAPOOL}" ]; then
  echo "$0: you must set the environment variable DATAPOOL to the name of your data pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=zdata" >/dev/stderr;
  echo "$0: your data pool can be the same as your root pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=\${ROOTPOOL}" >/dev/stderr;
  exit 69;
fi

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  if ! zpool list "${DATAPOOL}" >/dev/null 2>/dev/null; then
    echo "$0: data pool ${DATAPOOL} does not exist" >/dev/stderr;
    echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${DATAPOOL} ..." >/dev/stderr;
    exit 69;
  fi;
fi

if [ -z "${RELEASEDATE}" ]; then
  echo "$0: you must set the environment variable RELEASEDATE to the date of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEDATE=20130602" >/dev/stderr;
  exit 69;
fi

if [ -z "${RELEASEREV}" ]; then
  echo "$0: you must set the environment variable RELEASEREV to the Subversion revision number of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEREV=251259" >/dev/stderr;
  exit 69;
fi

# It is assumed the pools were created with -O mountpoint=legacy.

zfs create ${ROOTPOOL}/do-not-destroy
zfs set reservation=1G ${ROOTPOOL}/do-not-destroy

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs create ${DATAPOOL}/do-not-destroy;
  zfs set reservation=1G ${DATAPOOL}/do-not-destroy;
fi

zfs create ${ROOTPOOL}/ROOT
zfs create ${ROOTPOOL}/ROOT/${RELEASEDATE}-r${RELEASEREV}

zpool set bootfs=${ROOTPOOL}/ROOT/${RELEASEDATE}-r${RELEASEREV} ${ROOTPOOL}

zfs create ${ROOTPOOL}/media
zfs create ${ROOTPOOL}/nfs
zfs create ${ROOTPOOL}/tmp

zfs create ${ROOTPOOL}/usr
zfs set canmount=off ${ROOTPOOL}/usr

zfs create ${ROOTPOOL}/usr/compat
zfs create ${ROOTPOOL}/usr/compat/linux
zfs create ${ROOTPOOL}/usr/local
zfs create ${ROOTPOOL}/usr/local/certs
zfs create ${ROOTPOOL}/usr/local/etc
zfs create ${ROOTPOOL}/usr/local/etc/namedb
zfs create ${ROOTPOOL}/usr/obj
zfs create ${ROOTPOOL}/usr/ports
zfs create ${ROOTPOOL}/usr/ports/distfiles
zfs create ${ROOTPOOL}/usr/ports/local
zfs create ${ROOTPOOL}/usr/ports/packages
zfs create ${ROOTPOOL}/usr/ports/workdirs
zfs create ${ROOTPOOL}/usr/src
zfs create ${ROOTPOOL}/var
zfs create ${ROOTPOOL}/var/backups
zfs create ${ROOTPOOL}/var/crash
zfs create ${ROOTPOOL}/var/db
zfs create ${ROOTPOOL}/var/db/pkg
zfs create ${ROOTPOOL}/var/db/ports
zfs create ${ROOTPOOL}/var/empty
zfs create ${ROOTPOOL}/var/log
zfs create ${ROOTPOOL}/var/mail
zfs create ${ROOTPOOL}/var/named
zfs create ${ROOTPOOL}/var/run
zfs create ${ROOTPOOL}/var/spool
zfs create ${ROOTPOOL}/var/tmp
zfs create ${ROOTPOOL}/var/unbound

zfs create ${DATAPOOL}/home

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs create ${DATAPOOL}/usr;
  zfs set canmount=off ${DATAPOOL}/usr;

  zfs create ${DATAPOOL}/usr/local;
  zfs set canmount=off ${DATAPOOL}/usr/local;
fi

zfs create ${DATAPOOL}/usr/local/pgsql
zfs create ${DATAPOOL}/usr/local/www

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs create ${DATAPOOL}/var;
  zfs set canmount=off ${DATAPOOL}/var;

  zfs create ${DATAPOOL}/var/db;
  zfs set canmount=off ${DATAPOOL}/var/db;

  zfs create ${DATAPOOL}/var/spool;
  zfs set canmount=off ${DATAPOOL}/var/spool;
fi

zfs create ${DATAPOOL}/var/db/mysql
zfs create ${DATAPOOL}/var/spool/ftp

zfs set atime=off ${DATAPOOL}/var/spool/ftp;

zfs set compression=gzip-9 ${ROOTPOOL}/tmp
zfs set compression=gzip-9 ${ROOTPOOL}/usr/local/certs
zfs set compression=gzip-9 ${ROOTPOOL}/usr/local/etc
zfs set compression=gzip-9 ${DATAPOOL}/usr/local/www
zfs set compression=gzip-9 ${ROOTPOOL}/usr/ports
zfs set compression=gzip-9 ${ROOTPOOL}/usr/src
zfs set compression=gzip-9 ${ROOTPOOL}/var/db/pkg
zfs set compression=gzip-9 ${ROOTPOOL}/var/db/ports
zfs set compression=gzip-9 ${ROOTPOOL}/var/log
zfs set compression=gzip-9 ${ROOTPOOL}/var/mail
zfs set compression=gzip-9 ${ROOTPOOL}/var/named
zfs set compression=gzip-9 ${ROOTPOOL}/var/tmp
zfs set compression=gzip-9 ${ROOTPOOL}/var/unbound

zfs set compression=off ${ROOTPOOL}/usr/ports/distfiles
zfs set compression=off ${ROOTPOOL}/usr/ports/packages
zfs set compression=off ${ROOTPOOL}/usr/ports/workdirs

zfs set exec=off ${ROOTPOOL}/usr/local/etc/namedb
zfs set exec=off ${ROOTPOOL}/usr/ports/distfiles
zfs set exec=off ${ROOTPOOL}/usr/ports/packages
zfs set exec=off ${ROOTPOOL}/usr/src
zfs set exec=off ${ROOTPOOL}/var/backups
zfs set exec=off ${ROOTPOOL}/var/crash
zfs set exec=off ${ROOTPOOL}/var/db
zfs set exec=off ${ROOTPOOL}/var/empty
zfs set exec=off ${ROOTPOOL}/var/log
zfs set exec=off ${ROOTPOOL}/var/mail
zfs set exec=off ${ROOTPOOL}/var/named
zfs set exec=off ${ROOTPOOL}/var/run
zfs set exec=off ${ROOTPOOL}/var/spool
zfs set exec=off ${ROOTPOOL}/var/unbound

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
zfs set exec=off ${DATAPOOL}/var/spool;
fi

zfs set exec=on ${ROOTPOOL}/var/db/pkg
zfs set exec=on ${ROOTPOOL}/var/db/ports

zfs set setuid=off ${ROOTPOOL}/tmp
zfs set setuid=off ${DATAPOOL}/home
zfs set setuid=off ${ROOTPOOL}/usr/local/etc/namedb
zfs set setuid=off ${ROOTPOOL}/usr/ports
zfs set setuid=off ${ROOTPOOL}/usr/src
zfs set setuid=off ${ROOTPOOL}/var/backups
zfs set setuid=off ${ROOTPOOL}/var/crash
zfs set setuid=off ${ROOTPOOL}/var/db
zfs set setuid=off ${ROOTPOOL}/var/empty
zfs set setuid=off ${ROOTPOOL}/var/log
zfs set setuid=off ${ROOTPOOL}/var/mail
zfs set setuid=off ${ROOTPOOL}/var/named
zfs set setuid=off ${ROOTPOOL}/var/run
zfs set setuid=off ${ROOTPOOL}/var/spool
zfs set setuid=off ${ROOTPOOL}/var/tmp
zfs set setuid=off ${ROOTPOOL}/var/unbound

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
zfs set setuid=off ${DATAPOOL}/var/spool;
fi

zfs set recordsize=8K  ${DATAPOOL}/usr/local/pgsql
zfs set recordsize=16K ${DATAPOOL}/var/db/mysql

Here’s a script for setting the temporary mountpoints rooted at ${DESTDIR}:

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${ROOTPOOL}" ]; then
  echo "$0: you must set the environment variable ROOTPOOL to the name of your root pool" >/dev/stderr;
  echo "$0: e.g. export ROOTPOOL=zroot" >/dev/stderr;
  exit 69;
fi

if ! zpool list "${ROOTPOOL}" >/dev/null 2>/dev/null; then
  echo "$0: root pool ${ROOTPOOL} does not exist" >/dev/stderr;
  echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${ROOTPOOL} ..." >/dev/stderr;
  exit 69;
fi

if [ -z "${DATAPOOL}" ]; then
  echo "$0: you must set the environment variable DATAPOOL to the name of your data pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=zdata" >/dev/stderr;
  echo "$0: your data pool can be the same as your root pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=\${ROOTPOOL}" >/dev/stderr;
  exit 69;
fi

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  if ! zpool list "${DATAPOOL}" >/dev/null 2>/dev/null; then
    echo "$0: data pool ${DATAPOOL} does not exist" >/dev/stderr;
    echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${DATAPOOL} ..." >/dev/stderr;
    exit 69;
  fi;
fi

if [ -z "${RELEASEDATE}" ]; then
  echo "$0: you must set the environment variable RELEASEDATE to the date of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEDATE=20130602" >/dev/stderr;
  exit 69;
fi

if [ -z "${RELEASEREV}" ]; then
  echo "$0: you must set the environment variable RELEASEREV to the Subversion revision number of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEREV=251259" >/dev/stderr;
  exit 69;
fi

if [ -z "${DESTDIR}" ]; then
  echo "$0: you must set the environment variable DESTDIR to the destination directory" >/dev/stderr;
  echo "$0: e.g. export DESTDIR=/tmp/zroot" >/dev/stderr;
  exit 69;
fi

if [ ! -d "${DESTDIR}" ]; then
  mkdir -p "${DESTDIR}";

  if [ ! -d "${DESTDIR}" ]; then
    echo "$0: unable to create destination directory ${DESTDIR}" >/dev/stderr;
    exit 69;
  fi;
fi

cd /

zfs set mountpoint=${DESTDIR}       ${ROOTPOOL}/ROOT/${RELEASEDATE}-r${RELEASEREV}
zfs set mountpoint=${DESTDIR}/home  ${DATAPOOL}/home
zfs set mountpoint=${DESTDIR}/media ${ROOTPOOL}/media
zfs set mountpoint=${DESTDIR}/nfs   ${ROOTPOOL}/nfs
zfs set mountpoint=${DESTDIR}/tmp   ${ROOTPOOL}/tmp
zfs set mountpoint=${DESTDIR}/usr   ${ROOTPOOL}/usr
zfs set mountpoint=${DESTDIR}/var   ${ROOTPOOL}/var

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs set mountpoint=${DESTDIR}/usr ${DATAPOOL}/usr;
  zfs set mountpoint=${DESTDIR}/var ${DATAPOOL}/var;
fi

mkdir ${DESTDIR}/proc
mkdir ${DESTDIR}/usr/compat/linux/proc
chmod 1777 ${DESTDIR}/tmp
chmod 1777 ${DESTDIR}/var/tmp

mkdir ${DESTDIR}/media/a
mkdir ${DESTDIR}/media/cdrom
mkdir ${DESTDIR}/media/dvdrom
mkdir ${DESTDIR}/media/dosimage
mkdir ${DESTDIR}/media/floppy
mkdir ${DESTDIR}/media/isoimage
mkdir ${DESTDIR}/media/udfimage
mkdir ${DESTDIR}/media/ufsimage

ln -s usr/compat ${DESTDIR}/compat

Pick one of these two shell scripts to extract a base system into ${DESTDIR}:

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${DESTDIR}" ]; then
  echo "$0: you must set the environment variable DESTDIR to the destination directory" >/dev/stderr;
  echo "$0: e.g. export DESTDIR=/tmp/zroot" >/dev/stderr;
  exit 69;
fi

if [ ! -d "${DESTDIR}" ]; then
  echo "$0: destination directory ${DESTDIR} does not exist" >/dev/stderr;
  echo "$0: maybe mkdir -p ${DESTDIR} needs to be run" >/dev/stderr;
  exit 69;
fi

if ! zfs get mountpoint "${DESTDIR}" >/dev/null 2>/dev/null; then
  echo "$0: destination directory ${DESTDIR} is not a ZFS filesystem" >/dev/stderr;
  exit 69;
fi

cd /dist/8.*/base

echo "$0: Please answer y to the following question!"
./install.sh
# Answer y

cd ../games
./install.sh

cd ../info
./install.sh

cd ../lib32 && ./install.sh

cd ../manpages
./install.sh

cd ../kernels
./install.sh generic

cd ${DESTDIR}/boot
rmdir kernel
mv GENERIC kernel

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${DESTDIR}" ]; then
  echo "$0: you must set the environment variable DESTDIR to the destination directory" >/dev/stderr;
  echo "$0: e.g. export DESTDIR=/tmp/zroot" >/dev/stderr;
  exit 69;
fi

if [ ! -d "${DESTDIR}" ]; then
  echo "$0: destination directory ${DESTDIR} does not exist" >/dev/stderr;
  echo "$0: maybe mkdir -p ${DESTDIR} needs to be run" >/dev/stderr;
  exit 69;
fi

if ! zfs get mountpoint "${DESTDIR}" >/dev/null 2>/dev/null; then
  echo "$0: destination directory ${DESTDIR} is not a ZFS filesystem" >/dev/stderr;
  exit 69;
fi

cd ${DESTDIR}

tar xvvf /usr/freebsd-dist/kernel.txz
tar xvvf /usr/freebsd-dist/base.txz
tar xvvf /usr/freebsd-dist/lib32.txz
tar xvvf /usr/freebsd-dist/doc.txz
tar xvvf /usr/freebsd-dist/games.txz

Here’s a script for setting the final mountpoints:

#!/bin/sh

# Copyright © 2013, Trond Endrestøl <Trond.Endrestol@ximalas.info>
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

if [ -z "${ROOTPOOL}" ]; then
  echo "$0: you must set the environment variable ROOTPOOL to the name of your root pool" >/dev/stderr;
  echo "$0: e.g. export ROOTPOOL=zroot" >/dev/stderr;
  exit 69;
fi

if ! zpool list "${ROOTPOOL}" >/dev/null 2>/dev/null; then
  echo "$0: root pool ${ROOTPOOL} does not exist" >/dev/stderr;
  echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${ROOTPOOL} ..." >/dev/stderr;
  exit 69;
fi

if [ -z "${DATAPOOL}" ]; then
  echo "$0: you must set the environment variable DATAPOOL to the name of your data pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=zdata" >/dev/stderr;
  echo "$0: your data pool can be the same as your root pool" >/dev/stderr;
  echo "$0: e.g. export DATAPOOL=\${ROOTPOOL}" >/dev/stderr;
  exit 69;
fi

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  if ! zpool list "${DATAPOOL}" >/dev/null 2>/dev/null; then
    echo "$0: data pool ${DATAPOOL} does not exist" >/dev/stderr;
    echo "$0: try: zpool create -o cachefile=/tmp/zpool.cache -O mountpoint=legacy ${DATAPOOL} ..." >/dev/stderr;
    exit 69;
  fi;
fi

if [ -z "${RELEASEDATE}" ]; then
  echo "$0: you must set the environment variable RELEASEDATE to the date of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEDATE=20130602" >/dev/stderr;
  exit 69;
fi

if [ -z "${RELEASEREV}" ]; then
  echo "$0: you must set the environment variable RELEASEREV to the Subversion revision number of the release you are installing" >/dev/stderr;
  echo "$0: e.g. export RELEASEREV=251259" >/dev/stderr;
  exit 69;
fi

cd /

zfs set readonly=on ${ROOTPOOL}/var/empty

zfs unmount -a

zfs inherit mountpoint ${ROOTPOOL}/ROOT/${RELEASEDATE}-r${RELEASEREV}

zfs set mountpoint=/home  ${DATAPOOL}/home
zfs set mountpoint=/media ${ROOTPOOL}/media
zfs set mountpoint=/nfs   ${ROOTPOOL}/nfs
zfs set mountpoint=/tmp   ${ROOTPOOL}/tmp
zfs set mountpoint=/usr   ${ROOTPOOL}/usr
zfs set mountpoint=/var   ${ROOTPOOL}/var

if [ "${DATAPOOL}" != "${ROOTPOOL}" ]; then
  zfs set mountpoint=/usr ${DATAPOOL}/usr;
  zfs set mountpoint=/var ${DATAPOOL}/var;
fi

zfs unmount -a

Constructive comments are always welcome.