"Error setting up the device" on QEMU
depau opened this issue · 40 comments
I would have tried to troubleshoot this issue on my own but I won't have a lot of time for it in the next days, I'll leave it here in case you do have some time.
The issue doesn't look like a regression as it doesn't work with any build of EtchDroid (both with my old fork of libaums and with upstream + wrapper) but also with your sample app.
As I mentioned in #206 I wasn't able to emulate a USB drive using the standard Android emulator, so I just used stock QEMU with Android-x86.
Steps to get the emulator running:
Set up using QEMU builds
- Grab a Bliss OS QEMU rpm build (I ended up using Bliss instead of Android-x86 as the guys behind it are maintaining a lot more actively than upstream, though they do mainline their contributions eventually; I've had a bunch of issues with upstream Ax86 that Bliss has fixed already)
- Extract the RPM using rpmextract
cdinto it ([rpm]/bliss-something)- Create virtual disk images for userdata and the emulated USB drive (make sure you use a filesystem that handles sparse files):
qemu-img create -f raw data.img 10G
mkfs.ext4 -L userdata data.img
qemu-img create -f raw usb.img 4G
- I also
unsquashfsd the system image in order to remove some bloat
In order to run the emulator with an emulated USB drive, I patched the script that comes with the Ax86 RPM builds: https://gist.github.com/Depau/6c0ccc7377949c68bb30608bef87b9cd
The script expects the disk images to be in the current working directory, so cd into the android image directory and run it.
Set up using ISO images
- Download an Android-x86 or Bliss-x86 ISO
- You need to run the emulator once with the iso to install it, then run it again with the USB drive attached to run tests
qemu-img create -f raw hdd.img 10G
qemu-img create -f raw usb.img 4G
# BIOS boot
qemu-system-x86_64 -enable-kvm -boot d -hda hdd.img -m 1024 -display sdl,gl=on -serial mon:stdio -cdrom build.iso
# UEFI boot in case BIOS doesn't boot
# You need to install `ovmf`; paths may vary on your distro; append:
-bios /usr/share/ovmf/x64/OVMF_CODE.fd
# USB 3.0 SCSI bulk only drive; append:
-drive if=none,id=usbstick,file=usb.img,format=raw \
-device nec-usb-xhci,id=xhci \
-device usb-storage,bus=xhci.0,drive=usbstick
# export adb tcp; append
-netdev user,id=mynet,hostfwd=tcp::5555-:5555 -device virtio-net-pci,netdev=mynet
When you're done installing, remove -cdrom
This produces an emulated device that looks very close to my real USB drive:
lsusb -v (qemu)
$ sudo lsusb -v -d 46f4:
Bus 002 Device 002: ID 46f4:0001
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.00
bDeviceClass 0
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 9
idVendor 0x46f4
idProduct 0x0001
bcdDevice 0.00
iManufacturer 1 QEMU
iProduct 2 QEMU USB HARDDRIVE
iSerial 3 1-0000:00:04.0-1
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x002c
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 6 Super speed config (usb 3.0)
bmAttributes 0xc0
Self Powered
MaxPower 0mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 0x0016
bNumDeviceCaps 2
USB 2.0 Extension Device Capability:
bLength 7
bDescriptorType 16
bDevCapabilityType 2
bmAttributes 0x00000002
HIRD Link Power Management (LPM) Supported
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x00
wSpeedsSupported 0x000e
Device can operate at Full Speed (12Mbps)
Device can operate at High Speed (480Mbps)
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 1
Lowest fully-functional device speed is Full Speed (12Mbps)
bU1DevExitLat 10 micro seconds
bU2DevExitLat 32 micro seconds
can't get debug descriptor: Resource temporarily unavailable
Device Status: 0x0001
Self Powered
lsusb -v (real, known working)
sudo lsusb -v -d 0dd8:
Bus 002 Device 003: ID 0dd8:3201 Netac Technology Co., Ltd
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.00
bDeviceClass 0
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 9
idVendor 0x0dd8 Netac Technology Co., Ltd
idProduct 0x3201
bcdDevice 11.00
iManufacturer 1 Depau
iProduct 2 EtchDrive
iSerial 3 DEPAU.00666EKMW
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x002c
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 104mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x01 EP 1 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 8
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 8
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 0x0016
bNumDeviceCaps 2
USB 2.0 Extension Device Capability:
bLength 7
bDescriptorType 16
bDevCapabilityType 2
bmAttributes 0x00000002
HIRD Link Power Management (LPM) Supported
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x00
wSpeedsSupported 0x000c
Device can operate at High Speed (480Mbps)
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 2
Lowest fully-functional device speed is High Speed (480Mbps)
bU1DevExitLat 4 micro seconds
bU2DevExitLat 4 micro seconds
can't get debug descriptor: Resource temporarily unavailable
Device Status: 0x000c
(Bus Powered)
U1 Enabled
U2 Enabled
All Android-x86 builds I tested run with network ADB already enabled on port 5555, so you should be able to adb connect localhost 5555 and install the test app to reproduce it.
Logcat with stack trace
08/23 02:05:49: Launching 'app' on QEMU Standard PC (i440FX + PIIX, 1996).
$ adb shell am start -n "com.github.mjdev.usbfileman/com.github.mjdev.libaums.usbfileman.MainActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER
Waiting for process to come online...
Connected to process 4133 on device 'qemu-standard_pc__i440fx___piix__1996_-localhost:5555'.
Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page.
W/jdev.usbfilema: Long monitor contention with owner main (4133) at boolean android.os.BinderProxy.transactNative(int, android.os.Parcel, android.os.Parcel, int)(Binder.java:-2) waiters=0 in void android.app.ActivityThread$ApplicationThread.updatePendingConfiguration(android.content.res.Configuration) for 193ms
D/UsbDocumentProvider: onCreate()
D/UsbDocumentProvider: discoverDevice() UsbDevice[mName=/dev/bus/usb/003/002,mVendorId=18164,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB HARDDRIVE,mVersion=0.00,mSerialNumber=1-0000:00:04.0-1,mConfigurations=[
UsbConfiguration[mId=1,mName=Super speed config (usb 3.0),mAttributes=192,mMaxPower=0,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]]]
W/System.err: java.io.FileNotFoundException: /jacoco.exec (Read-only file system)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:308)
at java.io.FileOutputStream.<init>(FileOutputStream.java:238)
at org.jacoco.agent.rt.internal_8ff85ea.output.FileOutput.openFile(FileOutput.java:67)
at org.jacoco.agent.rt.internal_8ff85ea.output.FileOutput.startup(FileOutput.java:49)
at org.jacoco.agent.rt.internal_8ff85ea.Agent.startup(Agent.java:122)
at org.jacoco.agent.rt.internal_8ff85ea.Agent.getInstance(Agent.java:50)
at org.jacoco.agent.rt.internal_8ff85ea.Offline.<clinit>(Offline.java:31)
W/System.err: at org.jacoco.agent.rt.internal_8ff85ea.Offline.getProbes(Offline.java:51)
at com.github.mjdev.libaums.UsbMassStorageDevice.$jacocoInit(Unknown Source:13)
at com.github.mjdev.libaums.UsbMassStorageDevice.<clinit>(Unknown Source:0)
at com.github.mjdev.libaums.UsbMassStorageDevice.getMassStorageDevices(Unknown Source:0)
at com.github.mjdev.libaums.storageprovider.UsbDocumentProvider.discoverDevice(UsbDocumentProvider.java:385)
at com.github.mjdev.libaums.storageprovider.UsbDocumentProvider.onCreate(UsbDocumentProvider.java:310)
at android.content.ContentProvider.attachInfo(ContentProvider.java:1920)
at android.content.ContentProvider.attachInfo(ContentProvider.java:1895)
at android.provider.DocumentsProvider.attachInfo(DocumentsProvider.java:177)
at android.app.ActivityThread.installProvider(ActivityThread.java:6446)
at android.app.ActivityThread.installContentProviders(ActivityThread.java:5993)
at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5908)
at android.app.ActivityThread.access$1100(ActivityThread.java:201)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1657)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:193)
at android.app.ActivityThread.main(ActivityThread.java:6724)
at java.lang.reflect.Method.invoke(Native Method)
W/System.err: at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:495)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:859)
I/UsbMassStorageDevice: found usb device: /dev/bus/usb/003/002=UsbDevice[mName=/dev/bus/usb/003/002,mVendorId=18164,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB HARDDRIVE,mVersion=0.00,mSerialNumber=1-0000:00:04.0-1,mConfigurations=[
UsbConfiguration[mId=1,mName=Super speed config (usb 3.0),mAttributes=192,mMaxPower=0,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]]]
I/UsbMassStorageDevice: Found usb interface: UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]
Found usb endpoint: UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
I/UsbMassStorageDevice: Found usb endpoint: UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
found usb device: /dev/bus/usb/001/002=UsbDevice[mName=/dev/bus/usb/001/002,mVendorId=1575,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB Tablet,mVersion=0.00,mSerialNumber=28754-0000:00:01.2-1,mConfigurations=[
UsbConfiguration[mId=1,mName=HID Tablet,mAttributes=160,mMaxPower=50,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=3,mSubclass=0,mProtocol=0,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=3,mMaxPacketSize=8,mInterval=10]]]]
D/UsbDocumentProvider: discoverDevice() UsbDevice[mName=/dev/bus/usb/001/002,mVendorId=1575,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB Tablet,mVersion=0.00,mSerialNumber=28754-0000:00:01.2-1,mConfigurations=[
UsbConfiguration[mId=1,mName=HID Tablet,mAttributes=160,mMaxPower=50,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=3,mSubclass=0,mProtocol=0,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=3,mMaxPacketSize=8,mInterval=10]]]]
I/UsbMassStorageDevice: found usb device: /dev/bus/usb/003/002=UsbDevice[mName=/dev/bus/usb/003/002,mVendorId=18164,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB HARDDRIVE,mVersion=0.00,mSerialNumber=1-0000:00:04.0-1,mConfigurations=[
UsbConfiguration[mId=1,mName=Super speed config (usb 3.0),mAttributes=192,mMaxPower=0,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]]]
Found usb interface: UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]
Found usb endpoint: UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
I/UsbMassStorageDevice: Found usb endpoint: UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
I/UsbMassStorageDevice: found usb device: /dev/bus/usb/001/002=UsbDevice[mName=/dev/bus/usb/001/002,mVendorId=1575,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB Tablet,mVersion=0.00,mSerialNumber=28754-0000:00:01.2-1,mConfigurations=[
UsbConfiguration[mId=1,mName=HID Tablet,mAttributes=160,mMaxPower=50,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=3,mSubclass=0,mProtocol=0,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=3,mMaxPacketSize=8,mInterval=10]]]]
W/jdev.usbfilema: Verification of boolean androidx.fragment.app.FragmentActivity.markState(androidx.fragment.app.FragmentManager, androidx.lifecycle.Lifecycle$State) took 101.721ms
W/jdev.usbfilema: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (light greylist, reflection)
W/jdev.usbfilema: Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (light greylist, reflection)
I/UsbMassStorageDevice: found usb device: /dev/bus/usb/003/002=UsbDevice[mName=/dev/bus/usb/003/002,mVendorId=18164,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB HARDDRIVE,mVersion=0.00,mSerialNumber=1-0000:00:04.0-1,mConfigurations=[
UsbConfiguration[mId=1,mName=Super speed config (usb 3.0),mAttributes=192,mMaxPower=0,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]]]
Found usb interface: UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]
Found usb endpoint: UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
Found usb endpoint: UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
I/UsbMassStorageDevice: found usb device: /dev/bus/usb/001/002=UsbDevice[mName=/dev/bus/usb/001/002,mVendorId=1575,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB Tablet,mVersion=0.00,mSerialNumber=28754-0000:00:01.2-1,mConfigurations=[
UsbConfiguration[mId=1,mName=HID Tablet,mAttributes=160,mMaxPower=50,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=3,mSubclass=0,mProtocol=0,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=3,mMaxPacketSize=8,mInterval=10]]]]
I/jdev.usbfilema: Background concurrent copying GC freed 14288(1391KB) AllocSpace objects, 3(72KB) LOS objects, 49% free, 2040KB/3MB, paused 30us total 391.725ms
W/View: requestLayout() improperly called by android.widget.ListView{670905e IFED.VC.. ......ID -240,0-0,888 #7f08007e app:id/left_drawer} during layout: running second layout pass
D/MainActivity: on service connected ComponentInfo{com.github.mjdev.usbfileman/com.github.mjdev.libaums.server.http.UsbFileHttpServerService}
D/UsbDocumentProvider: discoverDevice() UsbDevice[mName=/dev/bus/usb/003/002,mVendorId=18164,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB HARDDRIVE,mVersion=0.00,mSerialNumber=1-0000:00:04.0-1,mConfigurations=[
UsbConfiguration[mId=1,mName=Super speed config (usb 3.0),mAttributes=192,mMaxPower=0,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]]]
I/UsbMassStorageDevice: found usb device: /dev/bus/usb/003/002=UsbDevice[mName=/dev/bus/usb/003/002,mVendorId=18164,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB HARDDRIVE,mVersion=0.00,mSerialNumber=1-0000:00:04.0-1,mConfigurations=[
UsbConfiguration[mId=1,mName=Super speed config (usb 3.0),mAttributes=192,mMaxPower=0,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]]]
Found usb interface: UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=8,mSubclass=6,mProtocol=80,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]]
Found usb endpoint: UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
Found usb endpoint: UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=1024,mInterval=0]
found usb device: /dev/bus/usb/001/002=UsbDevice[mName=/dev/bus/usb/001/002,mVendorId=1575,mProductId=1,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=QEMU,mProductName=QEMU USB Tablet,mVersion=0.00,mSerialNumber=28754-0000:00:01.2-1,mConfigurations=[
UsbConfiguration[mId=1,mName=HID Tablet,mAttributes=160,mMaxPower=50,mInterfaces=[
UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=3,mSubclass=0,mProtocol=0,mEndpoints=[
UsbEndpoint[mAddress=129,mAttributes=3,mMaxPacketSize=8,mInterval=10]]]]
D/UsbDocumentProvider: addRoot() com.github.mjdev.libaums.UsbMassStorageDevice@4356d5b
D/UsbMassStorageDevice: setup device
I/UsbMassStorageDevice: MAX LUN 0
D/ScsiBlockDevice: inquiry response: ScsiInquiryResponse [peripheralQualifier=0, peripheralDeviceType=0, removableMedia=false, spcVersion=5, responseDataFormat=2]
E/UsbDocumentProvider: error setting up device
java.io.IOException: Could not read from device, result == -1 errno 0 null
at com.github.mjdev.libaums.usb.JellyBeanMr2Communication.bulkInTransfer(JellyBeanMr2Communication.kt:47)
at com.github.mjdev.libaums.driver.scsi.ScsiBlockDevice.transferCommand(ScsiBlockDevice.kt:159)
at com.github.mjdev.libaums.driver.scsi.ScsiBlockDevice.init(ScsiBlockDevice.kt:108)
at com.github.mjdev.libaums.UsbMassStorageDevice.setupDevice(UsbMassStorageDevice.kt:145)
at com.github.mjdev.libaums.UsbMassStorageDevice.init(UsbMassStorageDevice.kt:100)
at com.github.mjdev.libaums.storageprovider.UsbDocumentProvider.addRoot(UsbDocumentProvider.java:416)
at com.github.mjdev.libaums.storageprovider.UsbDocumentProvider.discoverDevice(UsbDocumentProvider.java:388)
at com.github.mjdev.libaums.storageprovider.UsbDocumentProvider.access$000(UsbDocumentProvider.java:53)
at com.github.mjdev.libaums.storageprovider.UsbDocumentProvider$1.onReceive(UsbDocumentProvider.java:287)
at android.app.LoadedApk$ReceiverDispatcher$Args.lambda$getRunnable$0(LoadedApk.java:1391)
at android.app.-$$Lambda$LoadedApk$ReceiverDispatcher$Args$_BumDX2UKsnxLVrE6UJsJZkotuA.run(Unknown Source:2)
at android.os.Handler.handleCallback(Handler.java:873)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:193)
at android.app.ActivityThread.main(ActivityThread.java:6724)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:495)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:859)
From what I've seen, the issue occurs when libaums tries to get the lun size; bulkInTransfer times out returning -1, no errno.
Note that I've also experienced the issue on EtchDroid with known working USB drives/Android versions, for example when re-using the same USB drive after the app was stopped for whatever reason without reattaching the USB drive and killing the app.
The LUN size is queried using a controlTransfer. Do you mean that or the very first bulkTransfer when initializing the device?
I have seen this error as well but I did not find any solution except replugging the device (#95 (comment))
I blindly tried to run it again using the debugger and this is what I get, for what it matters:
I haven't checked what exactly that command is supposed to be doing, but since this is an emulator I guess I can try stuff 🤷♂️ haha
This might be related (page 155): https://www.seagate.com/files/staticfiles/support/docs/manual/Interface%20manuals/100293068j.pdf
I'll see if I can implement and test the 16 byte command.
Not sure if that is the problem here. Every device should be able to support the READ CAPACITY(10) command. Also I would assume that it responds with an error and not time out.
What does the version field of the SCSI Inquiry Response say?
D/ScsiBlockDevice: inquiry response: ScsiInquiryResponse [peripheralQualifier=0,
peripheralDeviceType=0, removableMedia=false, spcVersion=5, responseDataFormat=2]
Also, after READ CAPACITY (10) fails, it seems that INQUIRY also subsequently fails on new executions.
Ok I see. That sounds more like the error I mentioned earlier. Some people suggested that sleeping for a couple of seconds helped in between sending commands to the device.
What might also be worth trying if you can get total commander or USB Media explorer or similar running. Then we know that it has to work somehow...
Could it be worth implementing something similar to Linux sd driver's spinup sequence?
https://github.com/torvalds/linux/blob/master/drivers/scsi/sd.c#L2066
It does seem to include some quirks for weird USB flash drives which actually do not require to be spun up.
It looks that in this particular case it simply waits one second between TEST UNIT READY and returning (and then it goes on with READ CAPACITY (16) which falls back to READ CAPACITY (10)).
Reimplementing the whole thing in Kotlin might be a big effort considering that it considers a lot of different cases.
For now I'll see what happens if I Thread.sleep(1000) after ScsiTestUnitReady.
Nope, it still fails with the 1 sec delay and READ CAPACITY 16
I attached a real USB drive to QEMU and analyzed stuff with Wireshark. The USB drive worked for a while, then the app crashed because of filesystem issues.
Which means that what happens next is IOException: Could not write to device, result == -1 errno 0 null
So, this is what I see on Wireshark when this occurs:
[app is started]
[communication times out]
[app crashes]
In this case, the app times out at transferCommand for ScsiInquiry:
D/UsbMassStorageDevice: setup device
I/UsbMassStorageDevice: MAX LUN 0
E/MainActivity: error setting up device
java.io.IOException: Could not write to device, result == -1 errno 0 null
at com.github.mjdev.libaums.usb.JellyBeanMr2Communication.bulkOutTransfer(JellyBeanMr2Communication.kt:34)
at com.github.mjdev.libaums.driver.scsi.ScsiBlockDevice.transferCommand(ScsiBlockDevice.kt:144)
at com.github.mjdev.libaums.driver.scsi.ScsiBlockDevice.init(ScsiBlockDevice.kt:75)
at com.github.mjdev.libaums.UsbMassStorageDevice.setupDevice(UsbMassStorageDevice.kt:145)
at com.github.mjdev.libaums.UsbMassStorageDevice.init(UsbMassStorageDevice.kt:100)
at com.github.mjdev.libaums.usbfileman.MainActivity.setupDevice(MainActivity.java:761)
at com.github.mjdev.libaums.usbfileman.MainActivity.selectDevice(MainActivity.java:717)
at com.github.mjdev.libaums.usbfileman.MainActivity.access$600(MainActivity.java:99)
at com.github.mjdev.libaums.usbfileman.MainActivity$4.onItemClick(MainActivity.java:681)
[... unrelated]
And this is the regular flow before the app crashes (except for the READ CAPACITY (10) command which I have replaced with READ CAPACITY (16)):
Alright, in order to capture the specific case of the emulated USB drive on Wireshark, I did a qemu-inception: I booted a GNU/Linux VM with the emulated USB drive attached, then booted the Android VM inside of it and connected the emulated drive as a host USB device.
Wireshark pcap: qemu-usb-capture.pcapng.gz
If you check the timestamps you can clearly see when the app was started.
I'll look into all this info, in the meantime I'll just drop this here in case it might be useful.
I think I found the issue. Apparently, the device is STALLing the transfer
https://www.kernel.org/doc/html/latest/driver-api/usb/URB.html#how-to-submit-an-urb
This library apparently works around this by simply clearing the fault and retrying up to 5 times.
I'm not sure how to do it on Android, but I will look into it in the next few days.
Hey there, I had a look at the file. But I am not sure what exactly is happening.
Interesting are the URB errors
- URB status: Operation now in progress (-EINPROGRESS) (-115)
- URB status: No such file or directory (-ENOENT) (-2)
For BULK transfer first and then also for control transfer. What that really means I do not know.
Did you have time to try Total Commander?
For some reason Wireshark does not parse the bulk in transfers from device to host to USBMS protocol frames.
Sorry did not see your latest comment.
Clearing the halt seems to be fairly low level and might not be supported in Android. Maybe releasing the USB interface/device altogether can help here. At least it sounds different from #95 (comment) so there is a chance that this might be possible.
Thanks for the support I really appreciate it!
Hey there, I had a look at the file. But I am not sure what exactly is happening.
Interesting are the URB errors1. URB status: Operation now in progress (-EINPROGRESS) (-115)
That doesn't seem to be an error. See this capture from a valid communication (with the kernel):
2. URB status: No such file or directory (-ENOENT) (-2)
https://www.kernel.org/doc/html/latest/driver-api/usb/error-codes.html
I also noticed that the interface is set to 0xFFFF in our requests, which also seems to be the reason WireShark is not parsing the payloads as they look just fine to me.
For BULK transfer first and then also for control transfer. What that really means I do not know.
Did you have time to try Total Commander?
No, I haven't purchased it.
I think Total Commander with the USB plugin is free (or at least some test version)
Sorry did not see your latest comment.
Clearing the halt seems to be fairly low level and might not be supported in Android. Maybe releasing the USB interface/device altogether can help here. At least it sounds different from #95 (comment) so there is a chance that this might be possible.
Thanks for the support I really appreciate it!
Reading this it seems to be just a matter of sending a controlTransfer command: https://developer.electricimp.com/resources/usberrors
However, checking libusb's source code it seems it can be done with an ioctl:
https://github.com/libusb/libusb/blob/958ec5255ba5707197486dfe4827d153f48e91ba/libusb/os/linux_usbfs.c#L1604
Which we can do, given that UsbCommunication exposes the underlying file descriptor (which can also be passed to libusb eventually)
2. URB status: No such file or directory (-ENOENT) (-2)https://www.kernel.org/doc/html/latest/driver-api/usb/error-codes.html
I see, so this actually seems to indicate that the endpoint is stalled currently and not available for communication.
Sorry did not see your latest comment.
Clearing the halt seems to be fairly low level and might not be supported in Android. Maybe releasing the USB interface/device altogether can help here. At least it sounds different from #95 (comment) so there is a chance that this might be possible.
Thanks for the support I really appreciate it!Reading this it seems to be just a matter of sending a controlTransfer command: https://developer.electricimp.com/resources/usberrors
This might be convenient because sending a controlTransfer is possible within the Android API.
However, checking libusb's source code it seems it can be done with an ioctl:
https://github.com/libusb/libusb/blob/958ec5255ba5707197486dfe4827d153f48e91ba/libusb/os/linux_usbfs.c#L1604Which we can do, given that UsbCommunication exposes the underlying file descriptor (which can also be passed to libusb eventually)
Which also means we might be able to just use libusb :D
I can confirm it works with Total Commander.
... a little reverse engineering never killed nobody 🤷♂️
Which also means we might be able to just use libusb :D
That's what I was thinking. I was also thinking that with enough infrastructure and wrappers in place we might just be able to link Linux's sd + scsi driver to libusb and use that instead, and enjoy the stability of heavily production tested code. But I haven't investigated how hard it would be, considering that the Linux USB stack is a completely separate driver and that shims for it need to be written.
Also this would mean that GPLv2 code would have to be embedded in this Apache-licensed library, which would require a relicensing to GPLv2 or GPLv3 unless I missed something.
EDIT: libusb is also GPL-licensed
EDIT2: actually it's LGPL so it should be fine
Ok awesome, then we just need to figure out what exactly they do differently.
I feel like this might a fairly big change especially designing it in a way that we can easily merge upstream back to libaums every now and then. That is why I would suggest go for the control transfer first and then maybe try ioctl
libusb even has a readme for Android https://github.com/libusb/libusb/tree/master/android
I feel like this might a fairly big change especially designing it in a way that we can easily merge upstream back to libaums every now and then. That is why I would suggest go for the control transfer first and then maybe try
ioctl
That's the hardest thing. I quickly checked how all the components talk to each other and it's quite messy, it's likely not a matter of copying files and wrapping them.
libusb even has a readme for Android https://github.com/libusb/libusb/tree/master/android
Yep, but they don't provide real JNI wrappers and we would have to write them on our own. We'd need to see how bad Android's API gets in our way, because if we need just a bunch of ioctls we might just as well duct-tape them onto the API.
I opened the USB plugin for Total Commander on Jadx and it looks like it's completely unobfuscated.
Also there are strings that refer to libusb, but I haven't seen anything JNI related so it might be just bad naming.
Anyway, that's it for me for today, I should really be doing other things right now. I should free in two weeks and I will gladly look deeper into it :)
Interesting, if that is really the case we might even be able to ask them for the source code, given they linked libusb statically (LGPL).
I did some RE on the TotalCommander plugin, I'm taking messy notes here: https://md.depau.eu/ZkErsTYJQLC7AQJhcyn7VA (editable if you wanna add comments)
It looks like the culprit is sending that reset control sequence whenever communication fails. Also, it looks like it sends a get sense command whenever anything looks weird during setup.
It seems that the libusb references assume that Android uses it underneath; the app does not include it. I haven't checked whether it actually does.
In the weekend or next week I'm gonna check a bunch of other things then see if I can patch the library ;)
Hey there, it seems that your note server is down :/
Hi, thanks for catching that, I'm taking it back up.
EDIT: docker is throwing a fit, this is gonna take a while, the whole server is borked lol
EDIT2: cleaned everything up, everything's up and running ;)
btw, I talked to the developer of Nexus Media Importer a couple of years back:
- UsbDeviceConnection.bulkTransfer is sometimes returning -1, so no bytes have been transferred. After that only re-plugging device helps. I can reproduce this pretty well when I try to communicate with multiple mass storages devices using an USB hub.
His reply
Yes, I'm painfully aware of #1. According to the Mass Storage spec, you should be able
to a Bulk-Only Mass Storage Reset and recover. Unfortunately, it doesn't work or at least I could never get it work. What I learned later on is that there is the Mass Storage spec and what Windows does. A great number of (cheap) devices only work reliably if you do what Windows does. FYI: Windows does a port reset to recover from this. I couldn't figure out how to do that without root.
Maybe this helps at some point :D
Thanks for pointing that out. I think I read about the Windows behavior somewhere. BTW libusb does have a function to perform port reset.
As I said in the other thread I won't be working on this until the next year, but I did start tinkering on it last time I got the chance, I'll see if I can find some diffs in the meantime.
EDIT: nope, can't find it :/
Here's what I use to run Android on QEMU, as discussed in #242: https://gist.github.com/Depau/77b06b18320f511db8fdae58e8bceead
Note that I'm using Android v8.1-r2, not the latest one, I'll test it 9 too.
D/ScsiBlockDevice: Reset bulk-only mass storage
W/ScsiBlockDevice: sending bulk only mass storage request
D/ScsiBlockDevice: Trying to clear halt on both endpoints
W/native_libusbcom: jint Java_me_jahnen_libaums_libusbcommunication_LibusbCommunication_nativeClearHalt(JNIEnv *, jobject, jlong, jint):98 libusb clear halt
D/LibusbCommunication: libusb clearFeatureHalt returned -5
W/native_libusbcom: jint Java_me_jahnen_libaums_libusbcommunication_LibusbCommunication_nativeClearHalt(JNIEnv *, jobject, jlong, jint):98 libusb clear halt
D/LibusbCommunication: libusb clearFeatureHalt returned -5
D/ScsiBlockDevice: inquiry response: ScsiInquiryResponse [peripheralQualifier=0, peripheralDeviceType=0, removableMedia=false, spcVersion=5, responseDataFormat=2]
E/native_libusbcom: jint Java_me_jahnen_libaums_libusbcommunication_LibusbCommunication_nativeBulkTransfer(JNIEnv *, jobject, jlong, jint, jbyteArray, jint, jint, jint):73 libusb_bulk_transfer returned -7
E/ScsiBlockDevice: Error transferring command; errno 0 null
D/ScsiBlockDevice: Reset bulk-only mass storage
W/ScsiBlockDevice: sending bulk only mass storage request
D/ScsiBlockDevice: Trying to clear halt on both endpoints
W/native_libusbcom: jint Java_me_jahnen_libaums_libusbcommunication_LibusbCommunication_nativeClearHalt(JNIEnv *, jobject, jlong, jint):98 libusb clear halt
D/LibusbCommunication: libusb clearFeatureHalt returned -5
W/native_libusbcom: jint Java_me_jahnen_libaums_libusbcommunication_LibusbCommunication_nativeClearHalt(JNIEnv *, jobject, jlong, jint):98 libusb clear halt
an inquiry response at some point is all I can get :/
When using genymotion and a real usb pen drive I get this
D/ScsiBlockDevice: inquiry response: ScsiInquiryResponse [peripheralQualifier=0, peripheralDeviceType=0, removableMedia=false, spcVersion=4, responseDataFormat=2]
Maybe there is also some protocol version libaums does not support which is the problem
Maybe there is also some protocol version libaums does not support which is the problem
There is indeed another protocol and that is UAS which should be what is used by hard drive caddies, IDK how it's different from regular mass storage devices protocol. QEMU can emulate that too.
Anyway, is that different from what you get if you connect it directly to an Android device?
I meant that the spc/sbc version (scsi block commads) is different, but I did some research and libaums is only using the bare minimum which should be supported by every device. The only thing I noticed is that with spc>=5 there is a REPORT LUNS command, maybe the control transfer to find out the luns is a problem then. Although I doubt that ..
For what it's worth, here it says:
The SCSI REPORT LUNS command is important for Logical Unit (LU) discovery. After a target device is discovered (usually via some transport specific mechanism), a REPORT LUNS command should either be sent to LUN 0 (which is Peripheral device addressing method with bus_id=0 and target/lun=0) or to the REPORT LUNS well known LUN (i.e. 0xc101000000000000). SAM-5 requires that one of these responds with an inventory of LUNS that are contained in this target device.
I haven't read the specifications not I can test it at the moment so I don't know if this may actually be useful, I'll leave it here just in case.
I've got some free time to work on libaums 🎉 (but just for a few weeks)
I'll try to work on QEMU testing again, hopefully it will also resolve the issues on real devices as we predicted.