Manage cookies

We use cookies to improve our services. Read more about how we use cookies and how you can refuse them.

The necessary cookies help to make the website work properly.

Anonymous statistical cookies help to understand how visitors use the website.

  • Raspberry Pi
    10 min | 104600

    #Raspberry Pi 4: booting from an SSD with enabled TRIM

    Raspberry Pi | 10 min | 104600


    There are already many tutorials that explain how to boot the Raspberry Pi 4 from an SSD disk without any MicroSD card. This is just another one; but, it includes some tips that I founded on Google, because the other tutorials didn't work, nor did they explain the details.

    Let's start with some technical details. The Raspberry Pi 4 has an SPI-attached EEPROM, which contains code to boot up the system and more. The firmware of this EEPROM can be updated to include more functionalities or solves problems. The firmware release notes describe the functionalities added to each new release. As you can see, the beta release 2020-05-15 included the option to boot the Raspberry Pi from an SSD disk without any MicroSD card.

    This article is part of a series of tutorials for Raspberry Pi 4:

    Software and Hardware

    This software and hardware will be needed in this tutorial.

    DIY Instructions: Boot from SSD

    To get the Raspberry Pi 4 to boot from an SSD, you need (at the time of writing this article) to update the EEPROM. To do that, you need first to boot the Raspberry Pi using a MicroSD card. This card should have the Raspberry Pi OS installed. You need this card only once to update the EEPROM.

    Update 17.10.20:
    • New Raspberry Pis should have arrived with the newer EEPROM firmware. Thus, you don't need to update the EEPROM and you can skip these steps.
    • If you need to update because your Raspberry Pi doesn't find your SSD. You can follow the following steps, or you can create an MicroSD card using the recovery option of the Raspberry Pi Imager. The MicroSD card will then upgrade the EEPROM firmware currently to version 000138a1, which has SSD support. I recommend you to use that option described below (Recovery or easy upgrade).

    After booting the Raspberry Pi using the MicroSD card, open a Terminal and type the following:

    sudo apt-get install rpi-eeprom
    You don't need to use the beta release anymore. The 2020-07-31 release was promoted to STABLE and this release includes the SSD booting option.

    Then, you need to switch the firmware to the beta "branch", editing the rpi-eeprom-update file:

    sudo nano /etc/default/rpi-eeprom-update

    to change the file section as follows:

    FIRMWARE_RELEASE_STATUS="critical"
    # To:
    FIRMWARE_RELEASE_STATUS="beta"

    Finally, run:

    sudo rpi-eeprom-update

    I got the following:

    BCM2711 detected
    Dedicated VL805 EEPROM detected
    *** UPDATE AVAILABLE ***
    BOOTLOADER: update available
    CURRENT: Thu 16 Apr 2020 05:11:26 PM UTC (1587057086)
     LATEST: Fri 31 Jul 2020 01:43:39 PM UTC (1596203019)
     FW DIR: /lib/firmware/raspberrypi/bootloader/beta
    VL805: update available
    CURRENT: 000137ad
     LATEST: 000138a1

    To flash the EEPROM firmware, just type the following:

    sudo rpi-eeprom-update -d -f /lib/firmware/raspberrypi/bootloader/beta/pieeprom-2020-07-31.bin

    and reboot the system.

    The EEPROM firmware version changes rapidly. Therefore, if you don't find the pieeprom-2020-07-31.bin file, just ls the folder to see the last beta version available i.e. ls /lib/firmware/raspberrypi/bootloader/beta/.

    After rebooting the system, you can check if the EEPROM firmware was installed, typing:

    $ vcgencmd bootloader_version
    
    Jul 31 2020 14:43:39
    version f8d1dc69144b10b4bd78b4b6d08658ac3005726d (release)
    timestamp 1596203019

    which matches the latest firmware (LATEST: Fri 31 Jul 2020 01:43:39 PM UTC (1596203019)) described above.

    Additionally, if you type:

    $ vcgencmd bootloader_config
    
    [all]
    BOOT_UART=0
    WAKE_ON_GPIO=1
    POWER_OFF_ON_HALT=0
    DHCP_TIMEOUT=45000
    DHCP_REQ_TIMEOUT=4000
    TFTP_FILE_TIMEOUT=30000
    ENABLE_SELF_UPDATE=1
    DISABLE_HDMI=0
    BOOT_ORDER=0xf41

    The BOOT_ORDER with value 0xf41 means the following:

    0x1 - SD CARD
    0x4 - USB mass storage boot # needed for booting using an SSD
    0xf - RESTART (loop) - start again with the first boot order field

    Then, shut down the Raspberry Pi and remove the MicroSD card.

    Now, you need to flash Raspberry Pi OS on the SSD. You can use Rufus or the Raspberry Pi Imager on your PC and flash the OS image on the SSD.

    After you flash the image, replace the boot files with the newest *.dat and *.elf files from here: raspberrypi/firmware. Otherwise, you'll get the error USB-MSD boot requires newer software.

    Therefore, follow these steps:

    1. Download or clone the raspberrypi/firmware repository:
      git clone https://github.com/raspberrypi/firmware.git --depth 1
    2. Mount the SSD to the PC on which you flashed Raspberry Pi OS (just connect it to a USB port).
    3. Replace the files of the boot partition with the files inside the downloaded boot folder.

    Connect the SSD to a blue USB port of the Raspberry Pi and switch on the system. The Raspberry Pi will boot from the SSD without any MicroSD card.

    Recovery or easy upgrade

    If the Raspberry Pi 4 doesn't not boot, it is possible that the SPI EEPROM has become corrupted. To check, remove the SD card, disconnect the device from power, then reconnect it. If the green LED does not flash, this indicates that the EEPROM has become corrupted. Don't panic, you can repair it!

    Raspberry Pi Imager provides an easy way to fix this problem, by automatically preparing an SD card that will reprogram your Raspberry Pi 4’s EEPROM:

    1. Find an SD card that is empty, or does not contain any data you want to keep; it will be completely erased of all data during this process.
    2. Download Raspberry Pi Imager for your operating system on a host PC.
    3. Click CHOOSE OS and select Misc utility images then Pi 4 EEPROM boot recovery.
    4. Insert an SD card, click CHOOSE SD CARD, select the card you have inserted, then click WRITE.
    5. Once the SD card is ready, insert it into your Raspberry Pi 4 then connect the Raspberry Pi to power.
    6. Once complete, the green LED will blink rapidly in a steady pattern. Disconnect the device from power. Now you can remove the recovery SD card, insert your usual SD card, and resume using your Raspberry Pi.

    UASP vs USB-STORAGE (BOT)

    You can easily check to see whether the kernel is using uas or usb-storage for SSD device typing the lsusb -t command on a terminal:

    pi@raspberrypi:~ $ lsusb -t
    /:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
        |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
    /:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/1p, 480M
        |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=uas, 480M
    

    In my case, I attached two USB devices to my Raspberry Pi:

    • Samsung MUF-64BA 64GB USB 3.0
    • Samsung SSD 840 EVO 250GB

    The USB stick does not support the UAS protocol and the kernel uses the old BOT (usb-storage driver) mode. For the SSD, the kernel uses the UASP mode, which implements more efficient disk accesses, and therefore and you'll get the best speed.

    Most newer USB 3.0 adapters support UASP, but it's not always easy to see in the specs. Therefore, check this, otherwise you'll have an slower access to the hard drive.

    Raspberry Pi 3.0

    Raspberry Pis older than the Pi 4 have only USB 2.0 ports, and I found that they mounted the USB adapters using the usb-storage driver (BOT mode). Therefore, if you have an older Raspberry Pi, your options for fast external storage are limited.

    DIY Instructions: TRIM

    A trim command allows an OS to inform the SSD which data blocks are "no longer considered" in use and can be deleted/wiped internally or marked as free for rewriting. Thus, the solid-state drive can access only the blocks holding the data, and the next time the operating system tries to write new data in an area, it does not have to wait first to delete it (Find more about TRIM here). This ensures a faster writing speed when new data is stored in an area.

    If TRIM is NOT supported in the firmware of your drive, the following steps can damage your drive (it won't mount and can't be formatted anymore).
    Some drives (especially Flash Drives) indicate that TRIM is supported in firmware. However, it is not always the case :S.
    Supported and Enabled in firmware are two different things about TRIM. The following steps deal with cases in which TRIM is supported but it is NOT enabled out of the box. Thus, you'll able to manually enable TRIM on your SSD.

    Check: TRIM supported and enabled

    In Linux, to check if TRIM is currently supported and enabled on your device run the following on a terminal:

    pi@raspberrypi:~ $ sudo fstrim -v /
    

    It is assumed that you are booting your Raspberry Pi from an SSD, thus the device root is / - if you are not booting, change the path to e.g. media/pi/.....

    Usually, this command will report back with fstrim: /: the discard operation is not supported, then TRIM is not enabled.

    You can also run lsblk -D (more info) and check the values of DISC-GRAN (discard granularity) and DISC-MAX (discard max bytes) columns. Non-zero values indicate TRIM support.

    Check: TRIM supported by device firmware

    To check if the device firmware supports TRIM, install the following tools:

    pi@raspberrypi:~ $ sudo apt-get install -y sg3-utils lsscsi

    Then, run the following command:

    pi@raspberrypi:~ $ sudo su
    root@raspberrypi:/home/pi# sg_vpd -p bl /dev/sda
    Block limits VPD page (SBC):
      [...]
      Maximum unmap LBA count: 4194240
      Maximum unmap block descriptor count: 1
      [...]

    and check and write the Maximum unmap LBA count down, you'll need it.

    You can use sudo before most of the next steps. However, you should switch to the root user (sudo su). Otherwise, you'll have a problem with the echo commands.

    Then, run the following command to check the Unmap command supported (LBPU):

    root@raspberrypi:/home/pi# sg_vpd -p lbpv /dev/sda
    Logical block provisioning VPD page (SBC):
      Unmap command supported (LBPU): 1
      [...]

    If the Maximum unmap LBA count is greater than 0, and Unmap command supported (LBPU) is 1, then you are lucky and your device firmware is likely to support TRIM.

    Enabling TRIM

    After the checks above, you can probably now be sure if your device supports TRIM but it is not enabled. Therefore, follow these steps to manually enable it.

    Check the current provisioning_mode for all drives attached to the Pi:

    root@raspberrypi:/home/pi# find /sys/ -name provisioning_mode -exec grep -H . {} + | sort
    /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.2/1-1.2:1.0/host0/target0:0:0/0:0:0:0/scsi_disk/0:0:0:0/provisioning_mode:full
    /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.4/1-1.4:1.0/host1/target1:0:0/1:0:0:0/scsi_disk/1:0:0:0/provisioning_mode:full
    

    To make the process simple, you should only have the SSD attached to your Raspberry Pi, on which you want to enable TRIM. But if you have more than one drive attached, you need to confirm which drive you want to use, typing lsscsi:

    root@raspberrypi:/home/pi# lsscsi
    [0:0:0:0]    disk    Samsung  SSD 840 EVO 250G 0     /dev/sda 
    [1:0:0:0]    disk    Samsung  Flash Drive      1100  /dev/sdb

    In my case, I'll change the Samsung SSD 840 EVO, which is attached to the [0:0:0:0] interface.

    Once you've confirmed the drive, you'll need to change the provisioning_mode from full to unmap in the path that the find above returned:

    root@raspberrypi:/home/pi# echo unmap > /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.2/1-1.2:1.0/host0/target0:0:0/0:0:0:0/scsi_disk/0:0:0:0/provisioning_mode
    If you didn't switched to the root user, this line is not going to work using the sudo before the line.

    You can run the find line again to check if it worked.

    Now, you need to update the discard_max_bytes value for the drive as described:

    discard_max_bytes = Maximum unmap LBA count x Logical block length

    in my case (your values may be different): 2147450880 = 4194240 x 512.

    You got the Maximum unmap LBA count value from the sg_vpd -p bl /dev/sda command earlier, times the Logical block length value, you get from the sg_readcap -l /dev/sda command as:

    root@raspberrypi:/home/pi# sg_readcap -l /dev/sda
    Read Capacity results:
       [...]
       Logical block length=512 bytes
       [...]

    Then, write that value into the drive's discard_max_bytes setting as e.g. (your value may be different):

    root@raspberrypi:/home/pi# echo 2147450880 > /sys/block/sda/queue/discard_max_bytes

    Finally, you can confirm that TRIM is enabled, running:

    root@raspberrypi:/home/pi# fstrim -v /
    /: 212.1 GiB (227697213440 bytes) trimmed
    

    You should not get the error above, and depending on how many blocks it needs to clean up, it could take a few seconds longer (so be patient!).

    Persistent and management by systemctl

    The above values will be reset by the time you reboot your Raspberry Pi. But first, you'll need to check some drive information. Just run the following command:

    root@raspberrypi:/home/pi# lsusb
    [...]
    Bus 001 Device 003: ID 174c:55aa ASMedia Technology Inc. [...]
    [...]
    root@raspberrypi:/home/pi# 
    

    and write down the ID values 174c:55aa. These are the idVendor:idProduct values that you'll need in the next step. You can get more drive information typing: lsusb -vd 174c:55aa.

    To make the changes persistent, create the following rule:

    root@raspberrypi:/home/pi# nano /etc/udev/rules.d/10-trim.rules

    and add the following in that file (change the 174c and 55aa values with yours):

    ACTION=="add|change", ATTRS{idVendor}=="174c", ATTRS{idProduct}=="55aa", SUBSYSTEM=="scsi_disk", ATTR{provisioning_mode}="unmap"

    Save the 10-trim.rules file (CTRL+X), and then reboot the Pi. You can check if everything is working, running the fstrim -v / command again.

    TRIM is managed by systemctl, so to make the command runs automatically (by default, weekly)in the background, run the following line:

    sudo systemctl enable fstrim.timer

    By default, it will run weekly. But you can change that, editing the file fstrim.timer:

    pi@raspberrypi:~ $ sudo nano /lib/systemd/system/fstrim.timer
    
    [Unit]
    Description=Discard unused blocks once a week
    Documentation=man:fstrim
    
    [Timer]
    OnCalendar=weekly
    AccuracySec=1h
    Persistent=true
    
    [Install]
    WantedBy=timers.target

    Persistent of discard_max_bytes

    I didn't get any error with the configuration above, but some readers are reporting that after rebooting the system throws I/O errors and it disables TRIM.

    So, for those, I can suggest that they add the following lines to the /etc/udev/rules.d/10-trim.rules (change the values 174c, 55aa, and 2147450880 (discard_max_bytes) with your values).

    KERNEL=="sda", SUBSYSTEM=="block", ATTR{queue/discard_max_bytes}="2147450880"
    ACTION=="add|change", ATTRS{idVendor}=="174c", ATTRS{idProduct}=="55aa", SUBSYSTEM=="block", ATTR{queue/discard_max_bytes}="2147450880"

    In my case, the discard_max_bytes automatically updated after setting the "provisioning_mode" to "unmap" and the value is 2x the one that I provided... I'm checking that again.

    If you have more information or so tips about this, please leave me a comment!

    Performance

    I tested the SSD speed using agnostics (sudo apt-get install agnostics) and I got following results on an old SSD 840 EVO 250G:

    Raspberry Pi Diagnostics - version 0.4
    Sat Aug  8 17:21:22 2020
    
    Test : SD Card Speed Test
    Run 1
    prepare-file;0;0;29062;56
    seq-write;0;0;29088;56
    rand-4k-write;0;0;16574;4143
    rand-4k-read;22890;5722;0;0
    Sequential write speed 29088 KB/sec (target 10000) - PASS
    Random write speed 4143 IOPS (target 500) - PASS
    Random read speed 5722 IOPS (target 1500) - PASS
    Test PASS

    You can check your system using a MicroSD card, typing the following:

    sudo apt-get install agnostics
    
    agnostics

    Additionally, you can check the boot-up performance:

    pi@raspberrypi:~ $ systemd-analyze
    Startup finished in 2.604s (kernel) + 13.800s (userspace) = 16.405s 
    graphical.target reached after 13.720s in userspace
    

    If you got considerably worse results, you should think about changing your MicroSD card or using this tutorial to boot from an SSD :).


    Comments

    Empty