Migrating FreeBSD from i386 to amd64

Comments Off on Migrating FreeBSD from i386 to amd64

I have (successfully) attempted to migrate a running i386 stable/9 system into a running amd64 stable/9 system, and attempted to migrate a running i386 stable/10 system into a running amd64 stable/10 system, only to see if these tasks are in fact feasible. The results speaks for themselves, given the boundary conditions outlined below.

If your system has a separate root filesystem of less than 1 GiB, then I suggest you consider scrapping the 32-bit system altogether, and install a fresh 64-bit system. If you wish to continue having a separate root filesystem, make that filesystem no smaller than 2 GiB to accommodate future expansion. You may gain additional free space by deleting /boot/kernel*/*.symbols prior to installing a new kernel. If this isn’t good enough, then you should limit the modules installed using the MODULES_OVERRIDE directive in your kernel configuration file. (When 11.0 comes out, all kernel symbol files will live in /usr/lib/debug/boot/kernel, relieving the stress on the root filesystem.)

The boundary conditions are:

  • base built from the appropriate stable branch of the source code tree,
  • amd64 snapshot for the appropriate stable branch to speed up parts of the transition,
  • the system being able to run with the GENERIC kernel until a new custom kernel is optionally installed,
  • if your system needs a custom kernel to function properly, one can be precompiled on a spare 64-bit system and transferred to the subject at the right moment,
  • ports built from the ports collection with ports-mgmt/portmaster as a vital instrument, and
  • the ports listed below.

All 32-bit test systems carried a small, but representable arsenal of ports:

  1. ports-mgmt/pkg,
  2. ports-mgmt/dialog4ports,
  3. ports-mgmt/portmaster,
  4. ports-mgmt/portupgrade,
  5. devel/subversion,
  6. misc/mc,
  7. shells/bash,
  8. sysutils/parallel,
  9. sysutils/smartmontools,
  10. and of course, the build and run-time dependencies of the above.

This selection of ports does not rule out any corner cases, but it should serve as a good indicator. Please let me know of any ports that deserve special attention.

Remember these things:

  1. Have good backups, in plural, of everything precious.
  2. Have good dumps of any DBMSs.
    • MySQL, PostgreSQL, etc, might not be as architecture-neutral as SQLite used for the package database.
  3. This is an onsite job, although lights-out subsystems might be an acceptable substitute.
  4. If possible, try to replicate your setup and perform multiple tests using:
    1. spare hardware, and/or
    2. virtual machines, possibly with the aid of snapshots, and/or
    3. ZFS with the aid of snapshots, and possibly with the aid of clones.

Make sure all your installed ports are up-to-date while they are 32-bit native executables/libraries. Also, make sure the source tree is up-to-date. Don’t synchronise /usr/ports again until all ports have been converted to 64-bit native executables/libraries, unless something goes wrong, or you decide to remove all installed ports and begin anew. Never underestimate the luxury of having your own metaport specifying what you want installed on your systems. Use x11/xorg/Makefile and its associated files as templates for your own local/blah-blah-blah metaport.

The SQLite database /var/db/pkg/local.sqlite seems to be architecture-neutral. To be on the safe side, make sure the backup file, /var/backups/pkg.sql.xz, is current by running /usr/local/etc/periodic/daily/411.pkg-backup. See section 4.4.7 (Restoring the Package Database) in https://www.freebsd.org/doc/handbook/pkgng-intro.html on how to restore this SQLite database.

While I’m an avid fan of ports-mgmt/portupgrade, using portmaster is a true saint for completing the migration. If you don’t have ports-mgmt/portmaster installed at this stage, do so now.

Other than following the old instructions, it can be timesaving to upgrade, or maybe even downgrade, the running system to the revision of one of the available snapshots, boot from the amd64 snapshot, CD or memory stick, and carefully extract the files from the base, kernel, lib32, and, optionally, the games distributions, overwriting their i386 counterparts. I guess an amd64 release corresponding to your running i386 release can be used when you don’t follow the stable branches. I haven’t (yet) tested this hypothesis.

The directories /boot, /etc, /root, and /var must be exempted when extracting the base distribution, otherwise you risk losing valueable files. See below for further details.

If you can’t use the amd64 GENERIC kernel, then use a spare system, physical or virtual, to build a custom kernel. Install the custom kernel and its modules in place of the amd64 GENERIC kernel at the appropriate step below.

Assuming you boot off an amd64 installation media, select Shell, and adjust these commands to suit your system. In particular, at steps 2 and 10, ensure you mount and dismount all filesystems necessary to complete the transition:

mkdir /tmp/ROOT
mount /dev/gpt/root0 /tmp/ROOT
cd /tmp/ROOT
chflags -R 0 *
tar xvvf /usr/freebsd-dist/base.txz --exclude ./boot --exclude ./etc --exclude ./root --exclude ./var
tar xvvf /usr/freebsd-dist/kernel.txz
tar xvvf /usr/freebsd-dist/lib32.txz
tar xvvf /usr/freebsd-dist/games.txz # optional
cd /
umount /tmp/ROOT
reboot # into single user mode off the harddrive

Next, ports-mgmt/pkg should be recompiled separately before all the other installed ports. Here comes portmaster to the rescue:

portmaster -f -D -v pkg-
portmaster -a -f -D -v -x pkg-

To make sure nothing is missing, getting the chflags attributes back into shape, and to ensure an otherwise up-to-date base system, you should:

  1. Blow away /usr/obj/*,
  2. Review any custom kernel configuration files (remember these configuration files now resides in /usr/src/sys/amd64/conf),
  3. Compile both world and kernel,
  4. Install the kernel,
  5. Reboot into single user mode,
  6. Run mergemaster -p,
  7. Install the new world,
  8. Run mergemaster -Fi,
  9. Run make -D BATCH_DELETE_OLD_FILES delete-old, and
  10. Run make -D BATCH_DELETE_OLD_FILES delete-old-libs.

After a long, long wait, and with nothing gone wrong, you should have successfully migrated your system from i386 to amd64.