Raspberry Pi 4B Guix System Network Boot

Configuration Management with Guix

I've been working on a project involving Near-Field Communication (NFC). Most of the software is written in Clojure, along with some C and ARM Assembly. The application environment is a Raspberry Pi (4b) and an STM32 interfacing with a PN532 and PN5180. Guix is being used to build and manage the configuration of the environment.

The PN532 emulates various types of ISO/IEC 14443 devices, and the PN5180 reads the emulated data. The STM32 coordinates with these devices via I2C/SPI, and the Pi provides a tiny environment for building and deploying code to the STM32.

Initially, the application environment was highly constrained and consisted of a pi cluster with no microSD cards or attached storage. Additionally, the only means of interfacing with the Pis was over UART via their GPIO pins. The Restrictions no longer exist, but I found the process of building a Guix system for this setting interesting, so I figure I'd go over the setup for posterity.

note about Guix

Guix is a GNU Project, and as such, it is built with the Linux-libre, a variant of the Linux kernel free from all proprietary binary blobs. Due to this, certain devices will not work with it (e.g. a Raspberry Pi using a broadcom chip-set).

The nonguix channel provides packages that are unavailable in official channels due to their conflicting values. This channel can be used to build non-free Guix systems (hence nonguix). Discussion of nonguix is prohibited in official guix correspondence. Consider this post anathema.

Building and the System

The storage server builds the kernel, initrd, and /gnu/store and places them in an TFTP/NFS share. The Linux kernel will happily mount an NFS share, but the Raspberry Pi firmware is only capable of using tftp, so both methods are used to share the same directory (/mnt/storage).

guix system init --no-bootloader --system=aarch64-linux rpi-pxe.scm /mnt/storage

The kernel image and initrd are served with atftpd. In a simpler network topology the built-in tftp server provided by dnsmasq may be good enough, but in my case it struggled with packet loss.

(define atftpd-service
  (shepherd-service
   (provision '(tftp atftpd))
   (requirement '(networking zfs-import))
   (start #~(make-forkexec-constructor
             (list #$(file-append atftp "/sbin/atftpd")
                   "--daemon"
                   "--no-fork"
                   "--no-multicast"
                   "--verbose=5"
                   "--logfile" "/var/log/atftpd.shepherd.log"
                   "/mnt/storage")))
   (stop #~(make-kill-destructor))))

Rasbperry Pi Configuration Files

Firmware files for the various Raspberry Pi models can be found in this repository. Had a microSD card been an option to me at the time, I would have made use of this blob-free firmware distribution, or u-boot. In my case, any free space I could spare took priority.

config.txt

arm_64bit=1
enable_uart=1
uart_2ndstage=1
dtoverlay=disable-bt
kernel=Image
initramfs initrd followkernel
device_tree=bcm2711-rpi-4-b.dtb

Guix Configuration

Kernel Configuration

The kernel is built from the downstream kernel source provided by the Raspberry Pi foundation. This variant of the kernel is heavily patched to make it work with all of the devices on-board the Raspberry Pi. That of course means proprietary binary blobs are a part of the picture.

(define-public linux-raspberrypi
  (package
    (inherit linux)
    (name "linux-raspberrypi")
    (version "6.18.y")
    (source
     (origin
       (method git-fetch)
       (uri (git-reference
             (url "https://github.com/raspberrypi/linux")
             (commit "f3ee12aca79d77b3e6708b85ca04bb1deb1f973b")))
       (file-name (git-file-name name version))
       (sha256
        (base32 "0maqwwr2ndm4d1wxmm8k0qx5xrgprggxkfz107f83iw1m03xcg4s"))))
    (synopsis "Official Raspberry Pi Linux Kernel")
    (description "Downstream kernel built by the Raspberry Pi foundation.")))

Since the environment is very particular, the kernel is built with some additional configuration:

  • The Pis relied on DHCP for their ip, hostname, and tftp server location (CONFIG_PI_PNP, CONFIG_IP_PNP_DHCP)
  • The Pi kernel needs the ability to attach an NFS mount to itself. I allowed multiple versions of NFS simply because I didn't want to fight with permission issues. (CONFIG_NFS_V*)
  • Since the NFS root is read-only, the kernel needs the ability to store state in an overlay (CONFIG_OVERLAY_FS)
  • In the near future M.2 SATA SSDs attached via USB3.0 would be made available (CONFIG_USB*, CONFIG_BLK_DEV_SD, CONFIG_SCSI).
(define-public linux-raspberrypi-net
  (customize-linux
   #:name "linux-raspberrypi-net"
   #:linux linux-raspberrypi
   #:defconfig "bcm2711_defconfig"
   #:configs
   '("CONFIG_IP_PNP=y"
     "CONFIG_IP_PNP_DHCP=y"
     "CONFIG_ROOT_NFS=y"
     "CONFIG_NFS_FS=y"
     "CONFIG_NFS_V3=y"
     "CONFIG_NFS_V4=y"
     "CONFIG_NFS_V4_1=y"
     "CONFIG_NFS_V4_2=y"
     "CONFIG_OVERLAY_FS=y"
     "CONFIG_USB_UAS=y"
     "CONFIG_USB_STORAGE=y"
     "CONFIG_BLK_DEV_SD=y"
     "CONFIG_SCSI=y"
     "CONFIG_USB_XHCI_HCD=y"
     "CONFIG_USB_XHCI_PCI=y"
     "CONFIG_USB_XHCI_PLATFORM=y")))
...
    (kernel-arguments
     '("earlycon=pl011,0xfe201000"
       "console=ttyAMA0,115200"
       "loglevel=7"
       "ip=dhcp"
       "genet.force_reneg=y"
       "modprobe.blacklist=brcmfmac,brcmutil"
       "ro"
       "rootwait"
       "--"
       "--volatile-root"))

The earlycon parameter is telling the kernel which serial driver to use, and where it's address in memory resides. The genet.force_reneg=y tells the Broadcom GENET ethernet driver to renegotiate it's connection, which was added after I found that the Pi struggled to stay on the network after switching from the firmware to the kernel. Bluetooth is disabled simply because other peripherals were occupying the GPIO pins affected by those drivers.

File System

Here is a snippet of the configuration from the storage server. The Pi mounts /storage as /mnt is configured with fsid=0.

(service nfs-service-type
         (nfs-configuration
          (idmap-domain "localdomain")
          (nfs-versions '("4.2" "4.1" "4.0" "3"))
          (nfsd-udp? #t)
          (exports
           '(("/mnt" "192.168.0.0/16(rw,insecure,sync,no_subtree_check,crossmnt,fsid=0)")
             ...
             ("/mnt/storage" "192.168.0.0/16(ro,insecure,sync,no_subtree_check,no_root_squash)")))))

Since the guix system built on the NAS was stored at /mnt/storage, when a Pi mounted this directory as it's root (/), the /gnu/store became available. I'm included extra information here to help the Pi easily find and connect to the NAS, but in better network conditions the extra options aren't needed.

(file-systems
 (list (file-system
        (mount-point "/")
        (device "192.168.3.2:/storage")
        (type "nfs")
        (options "vers=4.2,ro,tcp,hard,nolock,timeo=600,retrans=5,nconnect=4,addr=192.168.3.2")
        (check? #f))

        %pseudo-terminal-file-system
            %immutable-store))

Serial Console Connection

All of the Pis were attached via UART to a waveshare device that makes them accessible via a serial connection. This agetty configuration ensures the service presents the same kind of connection that was originally passed to the kernel.

(modify-services %base-services
                 (delete static-networking-service-type)
                 (agetty-service-type
                  config =>
                  (agetty-configuration
                   (inherit config)
                   (tty "ttyAMA0")
                   (baud-rate "115200")
                   (term "vt100")
                   (flow-control? #t)
                   (extra-options '("-L")))))))

Parting Thoughts

While the project itself is very interesting, the experience of building a kernel for the Raspberry Pi has soured me on the platform. Left out of this post are the many hurdles I had to jupm over to arrive at a working environment.

For anyone considering Guix for embedded development my input would be this: Guix is incredible for embedded development, and there are single-board computers out there that are much more deserving of its presence. Given the option, I would replace the Raspberry Pi 4B's in this project with an Olimex, or BeagleBoard device.

Full Configuration

A copy of the full configuration can be found here.