trial_data function only returns empty lists

Question

trial_data function only returns empty lists

Closed this issue 6 years ago · 6 comments

Hi,

I am trying to use cbpy to obtain continuous and event data in real-time from a Blackrock NSP.

So far, I have been able to acquire signals in real-time using the trial_continuous function in cbpy, but I have not been able to use trial_data successfully. When calling trial_data, I never receive any samples, but instead always get empty lists. If I change to trial_continuous, I am able to get samples, but I do not get timestamps with the data points, which is something I'd like to have. I was informed that trial_data should be able to give me timestamps for neural channel samples, which is why I'd like to get that working if possible (please correct me if this is not true).

Below is the script that I am using to try and obtain samples:

# Imports
import traceback
from cerebus import cbpy

# Connection parameters
connection_params = {
    'central-addr'        : '255.255.255.255',
    'central-port'        : 51002,
    'inst-addr'           : '192.168.137.128',
    'inst-port'           : 51001,
    'receive-buffer-size' : 8388608
}

# Trial configuration parameters
trial_config_params = {
    'buffer_parameter' : {
        'double'            : True,
        'continuous_length' : 30000,
        'absolute'          : True,
    }
}

# Setup
all_data = []

# NSP interface initialization
try:
    connection_num, connection_info = cbpy.open(parameter=connection_params)
    cbpy.trial_config(**trial_config_params)
    
    # Displays incoming data while doing a quality check on the machine ID
    while True:
        machine_id, new_data = cbpy.trial_continuous(connection_num, reset=True)
        assert machine_id == 0
        all_data.append(new_data)

# Handles any errors (including KeyboardInterrupt, which is used to stop the acquisition)
except Exception as e:
    print(traceback.format_exc(e))

# Tries to ensure that the cbpy interface is closed upon termination (regardless of error)
finally:
    cbpy.close()

Here are some additional details that may be relevant:

The machine controlling the Blackrock NSP is a Windows machine.
The machine that I would like to acquire the signals on (in real-time) is an Ubuntu machine. On this machine, I am running Python 3.6. I used the following commands to install the implementation that I'm using:

wget https://github.com/dashesy/CereLink/releases/download/7.0/cerebus-0.0.3-cp36-cp36m-linux_x86_64.whl
pip install ./cerebus-0.0.3-cp36-cp36m-linux_x86_64.whl

I am currently able to run nPlayServer on the Windows machine to simulate data through the system, and I am able to acquire data from that using trial_continuous. I have also acquired data from a Blackrock Digital Neural Signal Simulator. I have confirmed that the data I acquire on the real-time machine is identical to the data stored in the .nsX files.

Answer 1 · 2019-02-26T00:28:35.000Z

Can you also add event_length in the buffer params, this is the docstring. Also, you can use Central to turn on events on a channel, or you can use get/set_channel_config.

If you decided to use set_channel_config I suggest first try Central and compare get_channel_config output before and after a change in Central, to see what fields needs to be changed.

Answer 2 · 2019-02-26T03:41:49.000Z

Wow, it really was that simple. I am also now seeing that part in the trial_data function that requires a length greater than 0. With your change, I am able to start getting samples and events!

I think that this issue can be considered resolved, but I have a followup issue. This issue might be better handled by the Blackrock support team directly (I don't want to take up too much of your time here), but I may as well ask before closing this issue. My goal is to get the timestamps for each neural data sample, not for events. I am not looking at spikes in my current project, but I was hoping to be able to acquire timestamps for each data sample primarily for quality checking (for example, making sure there were no missed packets). Is this something I can easily achieve within the trial_data function? Or is this a more involved issue that I should bring up with Blackrock?

Thanks again for the help. In case it is useful for anyone else in the future, below is the revised version of the above code block that ended up working properly for me:

# Imports
import time
import traceback

from cerebus import cbpy

# Connection parameters
connection_params = {
    'central-addr'        : '255.255.255.255',
    'central-port'        : 51002,
    'inst-addr'           : '192.168.137.128',
    'inst-port'           : 51001,
    'receive-buffer-size' : 8388608
}

# Trial configuration parameters. For best performance, it seems the
# lengths should be greater than the maximum amount of samples that is
# expected across all of the channels given the refresh rate.
trial_config_params = {
    'buffer_parameter' : {
        'double'            : True,
        'continuous_length' : 35000,
        'event_length'      : 35000,
        'absolute'          : True,
    }
}

# The rate at which the local data buffer is queried (in Hz)
refresh_rate = 1.

# Setup
all_data = []
refresh_interval = 1. / refresh_rate

# Performs the data acquisition
try:
    
    # Initializes and configures the NSP interface
    connection_num, connection_info = cbpy.open(parameter=connection_params)
    cbpy.trial_config(**trial_config_params)
    print('NSP interface initialized. Beginning data collection...')
    
    # Obtains the current time
    cur_time = time.time()
    
    # Loops continuously
    while True:
        
        # Sleeps for a certain amount of time while the NSP interface collects
        # data into its buffer. This amount of time is based on the desired
        # refresh rate.
        time.sleep(max((cur_time + refresh_interval) - time.time(), 0.))
        cur_time = time.time()
        
        # Acquires incoming data while doing a quality check on the machine ID.
        machine_id, new_events, new_data = cbpy.trial_data(connection_num, reset=True)
        assert machine_id == 0
        
        # If the acquired data sample is empty, a RuntimeError is raised
        if len(new_data) == 0:
            raise RuntimeError('An empty data sample was acquired.')
        
        # Appends the data and events to the running list of data samples
        all_data.append((new_data, new_events))

# Termines the data collection if a KeyboardInterrupt occurs
except KeyboardInterrupt:
    print('Keyboard interrupt detected. Halting data collection...')
    
# Handles any other errors
except:
    print(traceback.format_exc())

# Tries to ensure that the cbpy interface is closed upon termination
# (regardless of the occurrence of any errors)
finally:
    cbpy.close()
    print('NSP interface successfully closed.')

Answer 3 · 2019-02-26T04:21:26.000Z

trial_data() returns trial_event and trial_cont.

trial_cont is a list of [channel_number, numpy_array] doublets, one for each channel and doesn't have any timing info.
Note, however, that trial_data calls cbsdk_get_trial_data here, wherein the API fills in the cbSdkTrialCont struct defined here, and that struct includes a variable for time.

trial_event is quite different. It is a list of list-doublets, each: ([ch, {'timestamps':timestamps, 'events':dig_events}]. So trial_event will give you timestamps for each spike, if your system is configured to get spikes.

It wouldn't take too much work to add the cbSdkTrialCont.time value to the list of returned items, but to not break the API, it would have to only be returned when an extra kwarg is provided, and that kwarg defaults to False.
e.g.
def trial_data(int instance=0, bool reset=False, bool return_cont_ts=False):
then below,

if return_cont_ts:
    return <int>res, trial_event, trial_cont, trialcont.time
else:
    return <int>res, trial_event, trial_cont

You could make those changes then rebuild and install cerebus and see if it gives you what you want.

Answer 4 · 2019-02-26T22:56:17.000Z

Thanks for the tip, we can try that on our end and get back to you soon.

To make this discussion more appropriate in this issue, I've renamed the issue.

Answer 5 · 2019-02-26T23:00:55.000Z

@DavidMoses There is no reason not to create new issues :--) I suggest keeping the original issue name, close this issue and open a new one.
Also, for timing test I suggest connecting your input to a signal generator and capture few seconds, knowing the sample rate you set for the channel you can verify if the timing is correct. If you want fancier, you can strobe in a time code to a channel and then decipher the code

Answer 6 · 2019-02-26T23:05:01.000Z

Sounds good, I will close this issue here (after reverting to the original name) and open a new one.

Yes, we were considering doing some of the things you were suggesting if we could not get this done in software, but if a small modification can get us this, we may try for that solution first.

Thanks again!