ap--/python-seabreeze

ADC1000 error with external interrupt

drmcnelson opened this issue · 10 comments

drmcnelson commented

spectrometer and system information

  • model: (i.e. USB2000, STS, etc...) ADC1000USB
  • operating system: (i.e. Windows 7 64bit) Linux, Fedora 37, Cinnamon
  • python version: (output of python --version) Python 3.11.2
  • python-seabreeze version: (i.e. 0.5.3 or 'current master') 2.2.0
  • installed-via: (conda, wheel, or python setup.py install) pip install

After setting interrupt mode 3, the call to self.dev.intensities() sometimes produces the following error. Other times it requires two pulses on the input before it will return one spectrum.

 File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 431, in readspectrum_
    self.ydata = self.dev.intensities()/self.dev.max_intensity
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/spectrometers.py", line 212, in intensities
    out = self._dev.f.spectrometer.get_intensities()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/pyseabreeze/features/spectrometer.py", line 621, in get_intensities
    tmp = self._get_spectrum_raw()
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/pyseabreeze/features/spectrometer.py", line 171, in _get_spectrum_raw
    self.protocol.receive(
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/pyseabreeze/protocol.py", line 707, in receive
    return self.transport.read(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/nelson/.local/lib/python3.11/site-packages/seabreeze/pyseabreeze/transport.py", line 203, in read
    ret: bytes = self._device.pyusb_device.read(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/usb/core.py", line 1029, in read
    ret = fn(
          ^^^
  File "/usr/lib/python3.11/site-packages/usb/backend/libusb1.py", line 846, in bulk_read
    return self.__read(self.lib.libusb_bulk_transfer,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/usb/backend/libusb1.py", line 954, in __read
    _check(retval)
  File "/usr/lib/python3.11/site-packages/usb/backend/libusb1.py", line 604, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 75] Overflow
S

current problem

detailed description of what doesn't work

steps to reproduce

Step by step instructions to reproduce the error. The more detailed the better, but please use some common sense:

  1. run code example
  2. ...

minimal code example and error (very helpful if available)

copy of minimal code example and error messages
ap-- commented

Hi @drmcnelson

Thank you for reporting the issue. I believe this is related to and should be addressed via #190

import json

import seabreeze
seabreeze.use("pyseabreeze")

from seabreeze.spectrometers import Spectrometer


spec = Spectrometer.from_first_available()


def endpoint_sizes(s):
    sbtp = s._dev._transport
    usbdev = s._dev._raw_device.pyusb_device

    ep = {}
    for i in usbdev.get_active_configuration():
        for e in i.endpoints():
            ep[repr((i, e))] = e.wMaxPacketSize

    return {
        "model": s.model,
        "default_read_size": sbtp._default_read_size,
        "default_read_endpoint": sbtp._default_read_endpoint,
        "default_read_spectrum_endpoint": sbtp._default_read_spectrum_endpoint,
        "read_endpoints": sbtp._read_endpoints,
        "endpoint_map": {
            k: f"0x{v:02x}"
            for k, v in vars(sbtp._endpoint_map).items()
        },
        "usb_endpoints": ep
    }

print(json.dumps(endpoint_sizes(spec), indent=2))

Please run the above script with your spectrometer connected, and report the output here.
If this confirms my suspicion that the hardcoded endpoint read size doesn't match the usb endpointsize we would know how to proceed to fix this issue.

Cheers,
Andreas 😃

pbeaucage commented

Hi Andreas,

I'm not sure if it is the same issue, but I have an overflow as well when trying to use an ADC1000 connected to a dual channel SD2000 spectrometer. Output of the endpoint map:

{
  "model": "ADC1000-USB",
  "default_read_size": {
    "low_speed": 64,
    "high_speed": 512,
    "high_speed_alt": 512
  },
  "default_read_endpoint": "low_speed",
  "default_read_spectrum_endpoint": "high_speed",
  "read_endpoints": {
    "low_speed": "lowspeed_in",
    "high_speed": "highspeed_in",
    "high_speed_alt": "highspeed_in2"
  },
  "endpoint_map": {
    "primary_out": "0x02",
    "ep_out": "0x02",
    "primary_in": "0x87",
    "lowspeed_in": "0x87",
    "secondary_out": "0x02",
    "secondary_in": "0x82",
    "highspeed_in": "0x82",
    "secondary_in2": "None",
    "highspeed_in2": "None"
  },
  "usb_endpoints": {
    "(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x2: Bulk OUT>)": 64,
    "(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x82: Bulk IN>)": 64,
    "(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x7: Bulk OUT>)": 64,
    "(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x87: Bulk IN>)": 64
  }
}

Hope this helps; let me know if I should open a separate issue for this. This is on a Mac using manually-forced pyseabreeze backend.

ap-- commented

Hi @pbeaucage

Thank you so much for reporting!
The output you posted confirms my suspicion that the actual endpoint read sizes differ from the hardcoded ones.

Could you try running the script that you normally use for measuring, but with the following modification:

import seabreeze
seabreeze.use("pyseabreeze")

from seabreeze.spectrometers import Spectrometer



def _update_default_endpoint_sizes(s):
    # haven't tested this code yet...
    t = s._dev._transport
    rs = t._default_read_size
    re = t._read_endpoints
    em = vars(t._endpoint_map)

    ud = s._dev._raw_device.pyusb_device
    pm = {
        e.bEndpointAddress: e.wMaxPacketSize
        for i in ud.get_active_configuration()
        for e in i.endpoints()
    }

    for e in rs:
        rs[e] = pm[em[re[e]]]



spec = Spectrometer.from_first_available()
_update_default_endpoint_sizes(spec)

Basically for your spectrometer this should set all values in spec._dev._transport._default_read_size to 64, after the spectrometer instance is created.

let me know if this prevents the overflow errors. If yes, I'll prepare a fix.

pbeaucage commented

Hey @ap--,
Thanks for the quick reply!
Yep, that fixes it, with a couple oddities that are, I think, separate issues.

  1. On my setup, I have entries in the t._endpoint_map that map to None, specifically secondary_in2 and highspeed_in2. I had to manually patch these to a dummy value for the remapping to work, crudely:
   ...:     for k,v in em.items():
   ...:         if v is not None:
   ...:             new_em[k] = v
   ...:         else:
   ...:             new_em[k] = 130 # where 130 is a valid endpoint on my spectrometer
   ...:     em = new_em
  1. Curiously, the first call to .intensities() still produces an overflow, but subsequent calls work fine and appear to return valid data.
drmcnelson commented

Why does the first call to intensities produce an overflow?

Are we sure which spectrum is returned in the second call to "intensities">

I gave up on the device when I saw this happening. I am triggering to get detailed time dependence, and I need some certainty in what I am reading from the instrument.

The one that I have is nice for the work I am doing because it has no slit, lots of signal at low excitation. But it has to be rock solid reliable.

drmcnelson commented

P/S I am going to try the patches now. if it works great. But I am thinking about designing a board, probably using an ARM7, to interface to the SD2000 directly, even both units, and connect to the host over USB. Simultaneous triggering could be pretty useful for example. If someone wants to sponsor the effort, it can probably happen quickly.

ap-- commented

Thanks for posting the results! The first overflow likely occurs because the hot fix I posted only fixes the endpoint size mismatch after the Spectrometer.init is executed.

I'll work on a PR and then the issue should be fixed.

Cheers,
Andreas 😊

drmcnelson commented

Traceback (most recent call last):
File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 1240, in
seabreeze = SeaBreezeInstrument( graphics=args.graph, debug=args.debug)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 119, in init
print(json.dumps(endpoint_sizes(self.dev), indent=2))
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 95, in endpoint_sizes
"endpoint_map": {
^
File "/home/nelson/Projects/TeensyDataAcquistion/Python_Programs/./SeaBreezeInstrument.py", line 96, in
k: f"0x{v:02x}"
^^^^^^^^^^^^
TypeError: unsupported format string passed to NoneType.format

drmcnelson commented

Okay, got it working, patch the line
k: f"0x{v:02x}"
to
k: f"0x{v:02x}" if v is not None else None

The output is:

{
"model": "ADC1000-USB",
"default_read_size": {
"low_speed": 64,
"high_speed": 512,
"high_speed_alt": 512
},
"default_read_endpoint": "low_speed",
"default_read_spectrum_endpoint": "high_speed",
"read_endpoints": {
"low_speed": "lowspeed_in",
"high_speed": "highspeed_in",
"high_speed_alt": "highspeed_in2"
},
"endpoint_map": {
"primary_out": "0x02",
"ep_out": "0x02",
"primary_in": "0x87",
"lowspeed_in": "0x87",
"secondary_out": "0x02",
"secondary_in": "0x82",
"highspeed_in": "0x82",
"secondary_in2": null,
"highspeed_in2": null
},
"usb_endpoints": {
"(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x2: Bulk OUT>)": 64,
"(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x82: Bulk IN>)": 64,
"(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x7: Bulk OUT>)": 64,
"(<INTERFACE 0: Vendor Specific>, <ENDPOINT 0x87: Bulk IN>)": 64
}
}

drmcnelson commented

The patch also has an error, at

 rs[e] = pm[em[re[e]]]

So, we modify that to

for e in rs:
    try:
        rs[e] = pm[em[re[e]]]
    except Exception as exc:
        print( exc)
        if e is not None:
            print( 'e', e )
            if e in re:
                print( 're[e]', re[e] )
                if re[e] in em:
                    print( 'em[re[e]]', em[re[e]] )
                    if em[re[e]] in pm:
                        print( 'pm[em[re[e]]]', pm[em[re[e]]] )

And then the output before and after running the patch is
1a2,5

None
e high_speed_alt
re[e] highspeed_in2
em[re[e]] None
6c10
< "high_speed": 512,
"high_speed": 64,

In other words, the patch reduced the high_speed setting, whatever it is, from 512 to 64.

Is that what was intended?