The FreeBSD Project is slowly moving away from Subversion to Git. This poses questions like how do I do a checkout, and how do I get the equivalent of Subversion’s revision numbers for use in scripts and filenames? This is not the official documentation, merely some notes for myself as I tag along.

Note, any mistakes below are my own. Please, leave a comment or send me an email if you think I’m out of my mind or worse.

stable/11 and stable/12

Users of stable/11 and stable/12 can continue as before, and make the transition at their own pace.

Transition schedule

Here’s the tentative schedule. https://git.freebsd.org/src.git won’t exist until December 22, give or take. https://git.freebsd.org/ports.git won’t exist until March 2021.

Draft of the FreeBSD Git transition documents

Here’s a draft of the FreeBSD Git transition documents by Warner Losh and contributors.

Browsing the repos and commit logs on the web

Head over to https://cgit.freebsd.org/ as https://svnweb.freebsd.org/ will eventually disappear.

Git, Subversion, and ViewVC

You need to set WITH_SUBVERSION_VER=LTS in /etc/make.conf and in similar files for Poudriere or Synth if you have devel/viewvc installed.

Global configuration file ~/.gitconfig

[filter "lfs"]
	smudge = git-lfs smudge -- %f
[user]
	name = Trond Endrestøl
	email = trond.endrestol@ximalas.info
[core]
	editor = mcedit
[alias]
	gnlog = log --graph --pretty=format:'%Cred%h %C(green)%t %Creset %C(red)%ad %Creset-%C(yellow)%d%Creset %s %n      %N %-GG' --date=short
[pull]
	ff = only

Note, this file contains leading tabs, just like a Makefile. The tabs are probably not essential.

Cloning and checking out the 13.0-CURRENT branch

git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/src.git freebsd-src

This will download the entire repo, check out 13.0-CURRENT aka main, and make it possible to switch to other branches such as stable/12. All files end up in a directory named freebsd-src.

Cloning and checking out the 13.0-CURRENT branch on my laptop

zfs create zroot/usr/src-git
git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' --depth 1 https://git.freebsd.org/src.git /usr/src-git

This should limit the initial download to the absolute minimum. When I’m ready to make the transition from Subversion to Git, I can type these commands:

zfs rename zroot/usr/src     zroot/usr/src-svn
zfs rename zroot/usr/src-git zroot/usr/src

Cloning and checking out the stable/12 branch

git clone -o freebsd -b stable/12 --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/src.git freebsd-src-stable-12

This will download the entire repo, check out the stable/12 branch, and make it possible to switch to other branches such as stable/13. All files end up in a directory named freebsd-src-stable-12.

Cloning and checking out the ports head branch

git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/ports.git freebsd-ports

This will download the entire repo, check out the main branch, and make it possible to switch to other branches. All files end up in a directory named freebsd-ports.

Updating the working copies

cd freebsd-src           && git pull --ff-only -q
cd freebsd-src-stable-12 && git pull --ff-only -q
cd freebsd-ports         && git pull --ff-only -q

Switching from stable/12 to stable/13

cd freebsd-src-stable-12 && git checkout stable/13
cd .. && mv freebsd-src-stable-12 freebsd-src-stable-13

Switching back to CURRENT

cd freebsd-src-stable-13 && git checkout main
cd .. && mv freebsd-src-stable-13 freebsd-src-main

Git’s equivalent of revision numbers

I need to distinguish each ZFS boot environment not only by date and time, but also by some unique identifier. Subversion’s global revision number has up to now served such a purpose.

CURRENT_TIMESTAMP=`date +%Y%m%d-%H%M%S`
SVN_REVISION=`svn info --show-item revision /usr/src`
NEW_SNAPSHOT="${CURRENT_TIMESTAMP}-r${SVN_REVISION}"
CURRENT_BE=`df -t zfs / | tail -n 1 | awk '{print $1}'`
BE_ROOT=`dirname ${CURRENT_BE}`
NEW_BE="${BE_ROOT}/${NEW_SNAPSHOT}"
zfs snapshot ${CURRENT_BE}@${NEW_SNAPSHOT}
zfs clone -o mountpoint=/mnt ${CURRENT_BE}@${NEW_SNAPSHOT} ${NEW_BE}

I use this construct based on commit 8d405efd73d3991fe1647f91a2b7c9989dd5f18f to remain compatible with the current kernel identification:

CURRENT_TIMESTAMP=`date +%Y%m%d-%H%M%S`
GIT_COMMIT_COUNT=`git -C /usr/src rev-list --count HEAD`
GIT_SHORT_COMMITHASH=`git -C /usr/src rev-parse --verify --short=12 HEAD`
NEW_SNAPSHOT="${CURRENT_TIMESTAMP}-c${GIT_COMMIT_COUNT}-g${GIT_SHORT_COMMITHASH}"
CURRENT_BE=`df -t zfs / | tail -n 1 | awk '{print $1}'`
BE_ROOT=`dirname ${CURRENT_BE}`
NEW_BE="${BE_ROOT}/${NEW_SNAPSHOT}"
zfs snapshot ${CURRENT_BE}@${NEW_SNAPSHOT}
zfs clone -o mountpoint=/mnt ${CURRENT_BE}@${NEW_SNAPSHOT} ${NEW_BE}

The two Git commands above give you a count of the available commits and a short identifier of the latest commit. Admittedly, the commit hashes don’t sort as well as Subversion’s revision numbers and they are a bit difficult to read.

Empty $FreeBSD$ keywords and mergemaster(8)

With empty $FreeBSD$ keywords, mergemaster(8) have nothing to compare other than the actual file contents. Previously, users of mergemaster -Fp and mergemaster -Fi only merged files when the $FreeBSD$ keyword differed. Now, we find ourselves constantly merging files such as /etc/master.passwd and /etc/group. I urge the FreeBSD project to either fix mergemaster(8) or begin manually assigning values to the $FreeBSD$ keyword for files belonging in /etc. A PR is in the works, PR 252132. See also PR 206866.

Handling local changes

Kernel configuration files and local metaports are important to me. I should probably cease using CVS for my own stuff, and let Git handle things for me. I’m sure it will be a mistake not to keep extra copies of such files elsewhere.

Consider running these commands as appropriate:

  • git stash
  • git stash list
  • git stash apply
  • git pull --autostash --ff-only -v

A more advanced case using my laptop as an example, would be:

git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' --depth 1 https://git.freebsd.org/src.git /usr/src
cd /usr/src
git checkout -b local
cp -p /usr/src-svn/sys/amd64/conf/E590T sys/amd64/conf/E590T
cp -p /usr/src-svn/sys/amd64/conf/ZFS   sys/amd64/conf/ZFS
git add sys/amd64/conf/E590T sys/amd64/conf/ZFS
git commit

Ensure your cwd is /usr/src and repeat the last two commands after editing the kernel configuration file. git commit -a will never pick up new files. Get into the habit of (re)adding new and modified files prior to commit.

To update this source tree, run:

cd /usr/src
git checkout main
git pull --no-edit --no-ff
git rebase main local

I imagine a similar approach can be used for the ports tree and my local metaports. Can the command sequence above be forced to run non-interactive, e.g. when run from cron(8)? Any error messages would be sent as mail, ideally leaving no processes behind, and hopefully force a human to take manual action.

Creating a mirror of the doc repo

zfs create -o mountpoint=/git enterprise_zdata/git
zfs create enterprise_zdata/git/git.ximalas.info
zfs create enterprise_zdata/git/git.ximalas.info/freebsd
zfs create enterprise_zdata/git/git.ximalas.info/freebsd/doc
zfs create enterprise_zdata/git/git.ximalas.info/freebsd/ports
zfs create enterprise_zdata/git/git.ximalas.info/freebsd/src
git clone --mirror https://git.freebsd.org/doc.git /git/git.ximalas.info/freebsd/doc
git clone --mirror https://git.freebsd.org/src.git /git/git.ximalas.info/freebsd/src

Would git -C /git/git.ximalas.info/freebsd/doc fetch --all -q be the right and sole command for keeping the doc mirror up to date?

I wonder if this can be a starting point for running git daemon from inetd(8). It will be crucial to deny push and other destructive actions, only clone and pull actions should be allowed. Maybe by running git daemon as the user nobody or git_daemon, nothing bad can happen as long as the files are owned by root:wheel, and thus only the “other” access rights applies, usually r-x (dirs) and r-- (files).

/etc/inetd.conf has been amended with:

#git	stream	tcp46	nowait	nobody	/usr/local/bin/git	git daemon --inetd --verbose --export-all --interpolated-path=/git/%H%D /git/git.ximalas.info/freebsd/doc /git/git.ximalas.info/freebsd/ports /git/git.ximalas.info/freebsd/src

The URLs will be:

  • git://git.ximalas.info/freebsd/doc
  • git://git.ximalas.info/freebsd/ports
  • git://git.ximalas.info/freebsd/src

This service won’t go live until the ports repo is finalized sometime in early 2021, probably in March. I don’t encourage you to use my Git service, but you can use my setup as a template for your own private mirror. The usual disclaimers apply.

Browsing the commit logs locally

Don’t forget to install devel/tig as it’s a beautiful ncurses based tool for browsing the Git trees, the commit logs, the diffs, etc.

devel/tig showing commit list and a commit log entry

devel/tig showing commit list and a commit log entry

devel/tig showing commit list and a diff below the commit log entry

devel/tig showing commit list and a diff below the commit log entry

wide screen view of devel/tig showing commit list and a commit log entry

wide screen view of devel/tig showing commit list and a commit log entry

Hit the h key to get help. Hit the q key to close the help window. Hit the q key to return to the previous view, and eventually all the way back to the command prompt. Hit the Q key to exit immediately.

Open questions

  • Should I set up a private Git mirror of FreeBSD like I did when we used CVS/CVSup and later Subversion?
    • The aim is to minimize network traffic from the Internet, by having one computer serve the other computers on the same LAN.
    • I employ a dedicated builder for both the src and the ports trees, but the bullet point above remain valid. One-off experiments comes to mind.
    • If so, figure out how.
  • How do we ensure our working copies are up to date when some vulnerability is announced?
    • Subversion’s (global) revision numbers only increment and this is a huge improvement over the per file revision numbers of RCS & CVS.
    • Git’s commit identifiers are not sequential.
    • Will the FreeBSD Project continue to refer to Subversion revision numbers synthesized from the Git commit identifiers?
      • Is this even possible?

Error message while installing devel/git 2.29.2

[5/9] Reinstalling git-2.29.2...
===> Creating groups.
Using existing group 'git_daemon'.
===> Creating users
Using existing user 'git_daemon'.
[5/9] Extracting git-2.29.2: 100%
pkg: Failed to execute lua script: [string "shell_path = pkg.prefixed_path("libexec/git-c..."]:7: attempt to index a nil value (global 'shell')
pkg: lua script failed

Huh? A Lua script failed to run. This happened on stable/12 amd64 r368505 with the ports tree at r558227.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>