All FreeBSD systems under my care got upgraded last Friday to fix some NTP bugs. That upgrade introduced a new bug in the kernel. The bug first appeared at r298004 in base/head, and later at r298134 in base/stable/10. The i386-based systems were more notably affected than the amd64-based systems, as the former typically has less memory than the latter.

Thanks to Mark Johnston, the bug was identified as a missing call to xpt_free_ccb() in sys/cam/cam_periph.c. Applying the patch below, which is also due to Mark Johnston, solved the problem. The patch is applicable to both base/stable/10 and base/head until proper commits exists.

base/head was updated at r298977, and I’m sure stable/10 will follow in a matter of days, hopefully not weeks.

Without pointing fingers at anyone, this bug shows how important it is to free allocated, but unneeded memory before the pointer is lost, particularly in long-running software such as an operating system kernel.

I find it strange that the kernel simply freezes when running out of memory, instead of detecting the condition and call on panic(), which will usually reboot the poor system.

Index: sys/cam/cam_periph.c
===================================================================
--- sys/cam/cam_periph.c        (revision 298974)
+++ sys/cam/cam_periph.c        (working copy)
@@ -1876,6 +1876,7 @@

                if (cgd->ccb_h.status == CAM_REQ_CMP)
                        sbuf_bcat(&sb, cgd->serial_num, cgd->serial_num_len);
+               xpt_free_ccb((union ccb *)cgd);
        }
        sbuf_printf(&sb, "\" ");
        sbuf_printf(&sb, "cam_status=\"0x%x\" ", ccb->ccb_h.status);