Booting CM4 from SSD drive! For industrial applications, separate the system area from the data area.

Introduction.

The Raspberry Pi 5 (hereafter referred to as Pi 5) has a PCIe connection port on the main unit, allowing NVMe-connected SSD drives to be used like a standard PC. In fact, the use of PCIe has been available since Raspberry Pi 4 (hereafter referred to as Pi 4), although it requires an expansion HAT.
Compute Module 4 (hereafter referred to as CM4) is similar to Pi 4.

Here we will show you how to mount an NVMe-connected SSD drive in the industrial Raspi “PL-R4” and treat it as a boot drive.

Advantages of SSD drive activation

For industrial Raspi, where robustness is important, it will also be necessary to convert the file system to ROM to avoid system corruption in the event of an unexpected power shutdown.
ROM stands for Read Only Memory.

In a typical Raspberry Pi, the OS is basically put on a microSD card, which makes it difficult to separate it from the data area.

The CM4 used for industrial applications is based on eMMC boot. Since an unused microSD card can also be used, there is an advantage of using the microSD card as a separate data area even if the file system in the eMMC is converted to ROM.

Even though eMMC is faster than a microSD card, it is no match for an SSD drive. If you need faster processing, you can use the SSD drive as a boot drive instead of the eMMC, and you can use the microSD card for another area in the same way.

Another disadvantage is that microSD cards have a higher chance of corruption. eMMC and SSD drives are superior in this regard as well. Furthermore, the advantage of starting up on an SSD drive is significant due to its higher speed.

The industrial Raspi “PL-R4” used is, designed so that an NVMe-connected SSD drive can be mounted on the main unit. NVMe-connected SSD drive can be installed in the main body.
Please refer to the following procedure as it is a bit complicated.

Edit and write boot loader to boot SSD

The CM4 boot loader cannot be rewritten independently, as is the case with Pi 4.

Prepare a commercially available Pi 4, separate from the main body of the industrial Raspi “PL-R4”, which contains CM4. (You can use another PL-R4 main unit, but we will use Pi 4 this time.
Build and use “rpiboot” tool software for writing from source.

The Pi 4 used for writing was pre-set up to allow SSH connections and remote control via VNC.

Other than that, you will need a USB cable to write CM4, because you will need to connect the Pi 4 to the port for writing with a USB cable to feed the bootloader.

What you need:

  • PL-R4 main unit
  • Pi 4 set
  • USB type-C to type-A cable
  • rpiboot” built from source
  • microHDMI cable
  • display

Note that the cable to be used is somewhat special, with the CM4 side having a USB-C type connector shape and the Pi 4 having a USB-A type.
If you search for “USB-C to A”, you will find the relevant product.

USB-C to A cable example
USB-C to A cable example
Left: Pi 4 (USB Type-A), Right: PL-R4 (USB-Type-C)

CM4 cannot update EEPROM independently

For a typical Pi 4, you can use the rpi-eeprom-updatecommand.
However, it cannot be executed on CM4.

sudo rpi-eeprom-update
rpi-eeprom-update is not enabled by default on CM4.
The recommended method for flashing the EEPROM is rpiboot.
See: https://github.com/raspberrypi/usbboot/blob/master/Readme.md
Run with -h for more information.

It’s a bit cumbersome, but in a way it makes EEPROM mistakes less likely.

From this point onward, we will execute on the Pi 4 side, which is prepared separately.

Editing and replacing the bootloader (bootloader)

If you want to change to SSD drive booting with NVMe connection, you can use the sudo rpi-eeprom-configcommand for commercial Pi 5, etc. CM4 is basically the same, but you have to replace the entire rewritten boot loader.

The tool software “rpiboot” is used to send files to CM4. The files are physically sent to the CM4 connected by a USB cable.

The flow of the procedure is as follows.

Reassemble a separately prepared Pi 4 with the option to boot from the SSD drive for a specific version of the boot loader.
The resulting boot loader can be written to CM4 using “rpiboot” software and replaced, which is the same as adding the option.

Boot Loader Version

Before I started I checked the version on the PL-R4 side, it was 2023-01-11. The file name is “pieprom-2023-01-11.bin”.

vcgencmd bootloader_version
2023/01/11 17:40:52
version 8ba17717fbcedd4c3b6d4bce7e50c7af4155cba9 (release)
timestamp 1673458852
update-time 0
capabilities 0x0000007f

In this case, we used the latest “pieprom-2024-10-21.bin” at the time of writing.
In stable and latest, “pieprom-2024-10-21.bin” was the latest. Please read below, substituting the year and date for the different files used.

Check the update history of each firmware file and select the desired version.

rpi-eeprom: h ttps://github.com/raspberrypi/rpi-eeprom/blob/master/releases.md

Using rpiboot

rpiboot is prepared by building from source as recommended by the official.
git clone as follows. Worked in the home directory.

sudo apt install git libusb-1.0-0-dev pkg-config build-essential
git clone --recurse-submodules --shallow-submodules --depth=1 https://github.com/raspberrypi/usbboot
cd usbboot
make
cd recovery

We will now work within /usbboot/recovary.

Firmware Location

After running the git clone, the firmware files were extracted in the following directory in Pi 4 (bookworm).

/lib/firmware/raspberrypi/bootloader-2711/stable
/lib/firmware/raspberrypi/bootloader-2711/latest
/lib/firmware/raspberrypi/bootloader-2711/default

Apparently since bookwrom the directory structure has split into bootloader-2711and bootloader-2712. 2712 is for Pi 5. CM4 will have 2711 to choose from.

*Location in bullseye is at /lib/firmware/raspberrypi/bootloader/.

Extract the conf file from the boot loader and append it

First, extract the configuration file from the boot loader.

rpi-eeprom-config /lib/firmware/raspberrypi/bootloader-2711/latest/pieeprom-2024-10-21.bin -o boot.conf

This will rewrite the original empty boot.conf in the recovary directory. Edit it.

sudo nano boot.conf

Append BOOT_ORDER=0xf16which sets the boot order (boot order).

[all]
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0
BOOT_ORDER=0xf16

In Pi 5, it was set to BOOT_ORDER=0xf461and so on. The order of activation is prioritized from right to left in the numerical part.

Typically, 1 is a microSD card, 4 is USB-connected storage, and 6 is an NMVe-connected SSD drive.
This time BOOT_ORDER=0xf16represents the boot order of NVMe → microSD card.

For more information, see the official documentation at
URL: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#boot_order-fields

Export of 256 hash values

Write out the hash value of pieprom.bin as a .sig file with the sha256sum command.

sha256sum pieeprom.bin | awk '{print $1}' > pieeprom.sig

To check for consistency.

Export to pieprom.bin

Finally, write it down.
Write out pieprom-2024-10-21.bin including the boot.conf you just appended as pieprom.bin.

rpi-eeprom-config -c boot.conf -o pieeprom.bin /lib/firmware/raspberrypi/bootloader-2711/default/pieeprom-2024-10-21.bin

This completes the preparation.

Execute rpiboot to write to EEPROM

Finally, run rpiboot to write the boot loader to EEPROM.
Prepare a USB cable, tweezers, display and HDMI cable.

Before executing the command, connect the PL-R4 to the display with an HDMI cable. (Do not turn on the PL-R4 itself)

cd usbboot/recovary
sudo ../rpiboot -d .

After execution, the terminal stops displaying the following Waiting message.
At this time, connect the USB cable to the PL-R4 you wish to rewrite.

The **PL-R4 does not require a power supply. Simply connect it to the Pi4 with a USB cable.

Cable connected to USB Type-C on the backside of PL-R4 main unit
sudo ../rpiboot -d .
RPIBOOT: build-date Oct 27 2024 version 20240422~085300 c944557b
Loading: ./bootcode4.bin
Waiting for BCM2835/6/7/2711/2712...

At this time, to make the EEPROM writable, switch the dip switch to the WRT side (toward the right) using tweezers or the like.

Switch DIP switch to WRT
Left: Pi 4 (USB Type-A) and Right: PL-R4 (USB-Type-C) connected by cable

When the USB cable is connected, it will be recognized as external media and writing will begin. (It will be mounted in /media/username/.)

If the following is a correct example of the result, it is OK. I was able to write.

sudo ../rpiboot -d .

RPIBOOT: build-date Oct 27 2024 version 20240422~085300 c944557b
Loading: ./bootcode4.bin
Waiting for BCM2835/6/7/2711/2712...
Loading: ./bootcode4.bin
Sending bootcode.bin
Successful read 4 bytes 
Waiting for BCM2835/6/7/2711/2712...
Loading: ./bootcode4.bin
Second stage boot server
Loading: ./config.txt
File read: config.txt
Loading: ./pieeprom.bin
Loading: ./pieeprom.bin
Loading: ./pieeprom.sig
File read: pieeprom.sig
Loading: ./pieeprom.bin
File read: pieeprom.bin
Second stage boot server done

When writing is successfully completed, the display connected to PL-R4 lights up green.
The write-in has now been successfully completed.

Items to check for failure

The two things that did not go well and got us into trouble were

One, unlike a few years ago, I git clone to build, but I get an error if I don’t include the submodule as well.
For the latest information, check the page of

usbboot: https: //github.com/raspberrypi/usbboot

The other thing is that since bookworm, the path to the .bin file in /lib/firmware/raspberrypi/bootloader~ was changed.
So now it is divided into the bootloader-2712 directory for Pi5 and the bootloader-2711 directory for the rest.

The last thing to check is the USB cable.
It may not work well if the cable is of poor quality or if you are using a converter connector to TypeC.
This is rarely the case, but if the error is unavoidable, check with another cable.

Confirmation of rewriting on PL-R4 side

The above procedure successfully rewrites the EEPROM.

Disconnect the USB cable that was connected.
Note: Remember to return the PL-R4 DIP switch to its original position before starting up the PL-R4. (Now to the left.)

To confirm that it was done correctly, turn on the PL-R4 and start it up to check.

Check the version of the boot loader.
Is the firmware “pieprom-2024-10-21.bin” used in this case?

vcgencmd bootloader_version

BOOT_ORDER was also written as specified.

sudo rpi-eeprom-config
[all]
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0
BOOT_ORDER=0xf16

NVMe connection and speed comparison

Having rewritten the boot sequence, prepare an SSD drive to connect to NVMe and write the OS image file.

PL-R4″ can be downloaded by model and version from the PiLink download page.
The OS written was downloaded from the OS image (bullseye) available for the PL-R4 and written with a custom Raspberry Pi Imager.

To write to SSD drives, it is best to have a disk station dedicated to M.2 SSDs.
As a substitute, this time I put it in a commercial SSD drive external case and wrote from another PC.

By connecting it with a USB cable, it can be recognized as external media in the same way as a microSD card. Simply write with Raspberry Pi Imager as usual.

Confirmation of SSD drive startup

Now that the boot loader has been changed to boot from the SSD drive and the OS has been written to the SSD drive, let’s check it by connecting it to the Pl-R4 main unit.

There is a connector on the bottom of the main unit to which an SSD drive can also be connected. Screw it in securely and you are ready to go.

Bottom of PL-R4 main unit (back side of carry board)
*The inside of the unit is visible when stored for demonstration purposes.

Before turning on the power, do not forget to return the eMMC write DIP switch to OFF (left side). If it is left writable (WRT side), it will not start up.

Now, let’s check it out with a few commands.

You can see that /dev/nvme0n1p1 is mounted on /boot with the df command.

df -h

Filesystem       Size  Used  Avail  Use% Mounted on
/dev/root        118G  4.2G  109G    4% /
devtmpfs         1.8G     0  1.8G    0% /dev
tmpfs            1.8G     0  1.8G    0% /dev/shm
tmpfs            724M  1.2M  723M    1% /run
tmpfs            5.0M  4.0K  5.0M    1% /run/lock
/dev/nvme0n1p1   255M   32M  224M   13% /boot
tmpfs            362M   28K  362M    1% /run/user/1000

The fdisk command gives a little more detail.

sudo fdisk -l /dev/nvme0n1

Disk /dev/nvme0n1: 119.24 GiB, 128035676160 bytes, 250069680 sectors
Disk model: TS128GMTE452T2                          
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: *******

Device         Boot  Start       End   Sectors  Size Id Type
/dev/nvme0n1p1        8192    532479    524288  256M  c W95 FAT32 (LBA)
/dev/nvme0n1p2      532480 250069679 249537200  119G 83 Linux

The lsblk command shows that there is a microSD card, eMMC, and NVMe connected SSD drive, with /boot mounted on NVMe.

lsblk

NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
mmcblk2      179:0    0  29.1G  0 disk 
└─mmcblk2p1  179:1    0  29.1G  0 part /media/pi/9016-4EF8
mmcblk0      179:32   0  29.1G  0 disk 
├─mmcblk0p1  179:33   0   512M  0 part 
└─mmcblk0p2  179:34   0  28.6G  0 part 
mmcblk0boot0 179:64   0     4M  1 disk 
mmcblk0boot1 179:96   0     4M  1 disk 
nvme0n1      259:0    0 119.2G  0 disk 
├─nvme0n1p1  259:1    0   256M  0 part /boot
└─nvme0n1p2  259:2    0   119G  0 part /

Check the speed

Then examine the speed of the SSD drive.
We measured the read speed with the hdparm command.

Measurement is made by executing commands on the PL-R4 main unit.

NVMe

NVMe-connected SSD drives are showing 375 MB/s.

sudo hdparm -t /dev/nvme0n1

/dev/nvme0n1:
 Timing buffered disk reads: 1130 MB in  3.00 seconds = 376.54 MB/sec

It’s easy to forget now that the Pi 5 is on sale, but this time it’s an SSD drive booting into the CM4, and since it doesn’t have full PCIe support like the Pi 5, it’s an order of magnitude different, if not a bit less fast.

eMMC

Next is eMMC.

sudo hdparm -t /dev/mmcblk0

/dev/mmcblk0:
 Timing buffered disk reads: 250 MB in  3.02 seconds =  82.88 MB/sec

eMMC was approximately 82 MB/s.

microSD card

Finally, there is the microSD card.

sudo hdparm -t /dev/mmcblk2

/dev/mmcblk2:
Timing buffered disk reads:  28 MB in  3.09 seconds =   9.06 MB/sec

Unlike the Pi 4 microSD card startup on the market, the PL-R4 is not as fast, approximately 10 MB/s. This is unique to the product.
For a typical Raspberry Pi, it would be approximately 20 to 30 MB/s.

Results of this speed test

Needless to compare, the speed of the SSD drive was fast enough.

ResultSSDeMMCmicroSD
Transcend32GB
Read SpeedApprox. 376 MB/sApprox. 82 MB/sApprox. 10MB/s

Values for microSD card are based on “PL-R4” specifications and adjustments.


Multiple runs
hdparm is a simple measurement. You can try running it multiple times with the following command.

for i in $(seq 5); do sudo hdparm -t /dev/mmcblk2; sleep 5; done

Compare by Score

I also took another measurement with a scripting tool that allows you to share measurements over the net.
This one does the same thing as the hdparm command. It provides a more detailed display and a unique score scoring method to compare speeds.
The result here was the same: 355 MB/s read.

The script tool has been registered so far as user name raspida, with Pi4, Pi5 and SSD drives with different conditions.
Please feel free to use it as a reference.

PiBenchmarks: https://pibenchmarks.com/user/raspida/

SSD drive startup also in CM4

Unlike the general Pi 4, changing the boot loader in CM4 may seem a bit cumbersome.
Still, the PL-R4 is easier to understand than using the CM4 with an expansion logic board because of the DIP switches for write switching.

At the time of writing, we were able to complete the process without any problems if we followed the procedure.
Besides, the startup order is written to EEPROM, so it only needs to be changed once.

An SSD drive will be essential depending on what you are using the Raspberry Pi for, as the speed benefit is greater than a microSD card or eMMC boot.
I can imagine that having the microSD card as a separate data area would make it easier to use, including in case of emergency.

Let’s make CM4 also SSD drive bootable.


Article contributed by Raspida

Raspberry Pi information site that even non-engineers can enjoy using raspida.com a Raspberry Pi information site that even non-engineers can enjoy and handle. He contributes to the PiLink site about the Raspberry Pi for industrial use.

Tags: