Setup: I was booted on archlinux-2022.02.01-x86_64.iso using a multiboot USB flash drive made with Ventoy 1.0.65, and had selected the “Copy to RAM” option in the ISO menu with the idea of reusing the drive as a backup destination for a system backup.

The issue: I was unable to normally eject the drive, both eject and udisksctl power-off (after installing udisks2) gave the error Device or resource busy, and lsof | grep -e ventoy -e sdb didn’t return anything. Here is what it looked like:

root@archiso ~ # lsblk
NAME          MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0           7:0    0 657.4M  1 loop /run/archiso/airootfs
sda             8:0    0 111.8G  0 disk
├─sda1          8:1    0     1G  0 part
│ └─boot-arch 254:1    0   252M  0 lvm  
└─sda2          8:2    0   100G  0 part
sdb             8:16   0  59.6G  0 disk
├─sdb1          8:17   0  59.6G  0 part
│ └─ventoy    254:0    0 812.3M  1 dm    <-- problematic entry
└─sdb2          8:18   0    32M  0 part
sr0            11:0    1  1024M  0 rom

root@archiso ~ # eject -v /dev/sdb
eject: device name is `/dev/sdb'
eject: /dev/sdb: not mounted
eject: /dev/sdb: is whole-disk device
eject: cannot open /dev/sdb: Device or resource busy

root@archiso ~ # eject -v /dev/mapper/ventoy
eject: device name is `/dev/mapper/ventoy'
eject: /dev/mapper/ventoy: not mounted
eject: /dev/mapper/ventoy: is whole-disk device
eject: /dev/mapper/ventoy: is not hot-pluggable device

root@archiso ~ # ls -l /dev/mapper/ventoy  
lrwxrwxrwx 1 root root 7 Feb 24 20:24 /dev/mapper/ventoy -> ../dm-0

root@archiso ~ # ls -l /dev/dm-0
brw-rw---- 1 root disk 254, 0 Feb 24 20:24 /dev/dm-0

root@archiso ~ # cat /sys/class/block/dm-0/dm/name
ventoy

So, I wasn’t sure what was causing the issue, and I didn’t want to simply pull the drive out as it equates to playing Russian roulette with your devices (do you like getting the rug pulled under you or being knocked inconscious by a stealthy sucker punch? Well, neither do most filesystems).

At first I had worked around the issue by using this trick :

echo offline > /sys/block/sdb/device/state
echo 1 > /sys/block/sdb/device/delete

See this post on the Arch forums for a more thorough procedure along the same vein.

Still, I wasn’t entirely convinced: it didn’t address the root cause of the issue.

Digging a little deeper led me to dmsetup:

root@archiso ~ # dmsetup table
boot-arch: 0 516096 linear 8:1 2048
ventoy: 0 1663640 linear 8:17 59447552

root@archiso ~ # dmsetup info /dev/mapper/ventoy  
Name:              ventoy
State:             ACTIVE (READ-ONLY)
Read Ahead:        256
Tables present:    LIVE
Open count:        0
Event number:      0
Major, minor:      254, 0
Number of targets: 1

Well, that READ-ONLY state was already good news, but checking dmsetup manpage I saw it offered a remove command:

root@archiso ~ # dmsetup remove ventoy
root@archiso ~ # lsblk
NAME          MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0           7:0    0 657.4M  1 loop /run/archiso/airootfs
sda             8:0    0 111.8G  0 disk
├─sda1          8:1    0     1G  0 part
│ └─boot-arch 254:1    0   252M  0 lvm  
└─sda2          8:2    0   100G  0 part
sdb             8:16   0  59.6G  0 disk
├─sdb1          8:17   0  59.6G  0 part
└─sdb2          8:18   0    32M  0 part
sr0            11:0    1  1024M  0 rom  

Victory, the mapping is gone! And now…

root@archiso ~ # eject /dev/sdb -v
eject: device name is `/dev/sdb'
eject: /dev/sdb: not mounted
eject: /dev/sdb: is whole-disk device
eject: /dev/sdb: trying to eject using CD-ROM eject command
eject: CD-ROM eject command failed
eject: /dev/sdb: trying to eject using SCSI commands
eject: SCSI eject succeeded

root@archiso ~ # lsblk  
NAME          MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0           7:0    0 657.4M  1 loop /run/archiso/airootfs
sda             8:0    0 111.8G  0 disk
├─sda1          8:1    0     1G  0 part
│ └─boot-arch 254:1    0   252M  0 lvm  
└─sda2          8:2    0   100G  0 part
sdb             8:16   0  59.6G  0 disk
sr0            11:0    1  1024M  0 rom  

Success! I saw the LED on my thumb drive flash quickly, and then it fell silent. But wait, what is all this red in my logs?

archiso kernel: sd 6:0:0:0: [sdb] Media removed, stopped polling
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel: ldm_validate_partition_table(): Disk read failed.
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel:  sdb: unable to read partition table
archiso kernel: sd 6:0:0:0: [sdb] Media removed, stopped polling
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel: ldm_validate_partition_table(): Disk read failed.
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: I/O error, dev sdb, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
archiso kernel: Buffer I/O error on dev sdb, logical block 0, async page read
archiso kernel:  sdb: unable to read partition table
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed
archiso kernel: sd 6:0:0:0: [sdb] tag#0 device offline or changed

Hum, looks like eject is doing something kind of dirty… Let’s move ahead for now, we’ll come back to it later.

Now, since this is a laptop laying on my desk, I certainly can manually unplug the drive before plugging it back again to make it available. But what if I am lazy, or the machine is faraway, or I just want to take extra care of that super frail USB plug and prevent a superfluous actuation cycle?

Reattaching an ejected USB device from the CLI

Here we are going to make use of the ability to bind and unbind drivers from devices manually from user space. First, get the drive Bus number:

root@archiso ~ # lsusb  
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 003: ID 0781:5580 SanDisk Corp. SDCZ80 Flash Drive  <-- here is my device
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Then Port for “Bus 4 Device 3”:

root@archiso ~ # lsusb -t  
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M  <-- spot it?
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/8p, 480M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/6p, 480M

Now that I know that my USB device is identified as 4-1 (bus-port) (this can be confirmed by checking dmesg | grep usb-storage), we can make it go through an unbinding/binding cycle:

root@archiso ~ # echo "4-1" > /sys/bus/usb/drivers/usb/unbind
root@archiso ~ # lsblk
NAME          MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0           7:0    0 657.4M  1 loop /run/archiso/airootfs
sda             8:0    0 111.8G  0 disk
├─sda1          8:1    0     1G  0 part
│ └─boot-arch 254:1    0   252M  0 lvm  
└─sda2          8:2    0   100G  0 part
sr0            11:0    1  1024M  0 rom  
root@archiso ~ # echo "4-1" > /sys/bus/usb/drivers/usb/bind  
root@archiso ~ # lsblk
NAME          MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0           7:0    0 657.4M  1 loop /run/archiso/airootfs
sda             8:0    0 111.8G  0 disk
├─sda1          8:1    0     1G  0 part
│ └─boot-arch 254:1    0   252M  0 lvm  
└─sda2          8:2    0   100G  0 part
sdb             8:16   0  59.6G  0 disk
├─sdb1          8:17   0  59.6G  0 part
└─sdb2          8:18   0    32M  0 part
sr0            11:0    1  1024M  0 rom

Et voilà, the flash drive is once again available for mount without having to physically remove it!

Note

Trying to bind directly after eject without unbind leads to a write error: device or resource busy . So now I am really left wondering what eject is doing to my drive…

An alternative to manually writing to /sys/bus/usb/drivers/usb/ is to use usb_modeswitch (included on the Arch installation media). In that case we only need the ID pair from lsusb:

root@archiso ~ # lsusb  
...
Bus 004 Device 003: ID 0781:5580 SanDisk Corp. SDCZ80 Flash Drive
                       ==== ==== ------ ---------
                                      |         |
root@archiso ~ # usb_modeswitch -v 0x0781 -p 0x5580 --reset-usb
Look for default devices ...
 Found devices in default mode (1)
Access device 003 on bus 004
Get the current device configuration ...
Current configuration number is 1
Use interface number 0
 with class 8
Warning: no switching method given. See documentation
Reset USB device .
 Device was reset
-> Run lsusb to note any changes. Bye!

Props to this AskUbuntu answer, confirmed and extended via this Stackoverflow one.

Wrap up

After a few trial and errors, I realized ejecting the drive was entirely unnecessary: I could get away with just dmsetup remove ventoy and an unbind/bind cycle.

Now if you ever have to cleanly power-off a device on the command line, here is the ultimate method:

% udisksctl power-off -h
Utilisation :
  udisksctl power-off [OPTION…]

Safely power off a drive.

Options :
  -p, --object-path         Object path for ATA device
  -b, --block-device        Device file for ATA device
  --no-user-interaction     Do not authenticate the user if needed

E.g. udisksctl power-off -b /dev/sdb.

For the life of me I don’t know why this utility isn’t included by default on the arch installation media, but in any case you are just one pacman -S udisks2 away from safely-removed device nibbana.

That’s all folks for today’s learning diary. Thanks for reading and take care of your devices!