neuromore/studio

EDF Support in FileWriter broken

Closed this issue · 2 comments

Issue

Setting File Writer node to EDF format creates a zero bytes file on disk.
The according node stays in a re-initialization loop, because the write fails:

if (mFileWriter.WriteSamples(mFileFormat, mWriteChannels, numNewSamples, mFile, mFileName) == false)
{
// write failed
mHasWriteError = true;
}

Problem 1

success = success && edfclose_file(handle) == -1;
edfclose_file() actually returns 0 on success.

Solution:
So the line should be: success = success && edfclose_file(handle) == 0;
This will create a ~2kb file on disk which seems to get refreshed with exactly one last sample.

Problem 2

Calling edfopen_file_writeonly() will recreate the file from scratch. It does NOT append to an existing file.

bool ChannelFileWriter::WriteSamplesEDF(const Core::Array<Channel<double>*>& inChannels, uint64 numSamples, const char* fileName)
{
const uint32 numChannels = inChannels.Size();
bool success = true;
// open file
const int handle = edfopen_file_writeonly(fileName, EDFLIB_FILETYPE_EDFPLUS, numChannels);
if (handle < 0)
success = false;
// set required recording metadata
for (uint32 i=0;i<numChannels;++i)
{
success = success && edf_set_samplefrequency(handle, i, inChannels[i]->GetSampleRate()) == 0;
success = success && edf_set_physical_minimum(handle, i, inChannels[i]->GetMinValue()) == 0;
success = success && edf_set_physical_maximum(handle, i, inChannels[i]->GetMaxValue()) == 0;
success = success && edf_set_label(handle, i, inChannels[i]->GetName()) == 0;
success = success && edf_set_digital_minimum(handle, i, -32768) == 0;
success = success && edf_set_digital_maximum(handle, i, 32767) == 0;
success = success && edf_set_physical_dimension(handle, i, "uV") == 0;
Array<double>& sampleValues = inChannels[i]->GetRawArray();
success = success && edfwrite_physical_samples(handle, sampleValues.GetPtr()) == 0;
}
// close file
success = success && edfclose_file(handle) == -1;
return success;
}

Solution:
Both, edfopen_file_writeonly() and edfclose_file() must NOT be called inside WriteSamplesEDF() which is supposed to append the last buffer to the file. But instead be called once before the writing starts (respectively once after the writing ends). For reference, see how WriteSamplesCSV() works:

bool ChannelFileWriter::WriteSamplesCSV(const Array<Channel<double>*>& inChannels, uint64 numSamples, bool useTimestamps, uint32 numDigits, FILE* outFile)

It takes an existing FILE handle and appends to it.
It also has a dedicated version that writes the header once:
bool ChannelFileWriter::WriteHeaderCSV(const Array<Channel<double>*>& inChannels, bool useTimestamps, FILE* outFile)

I'm on 1.6.0, and this is still broken for me. Neuromore just crashes, I don't get anything to the terminal, which is interesting - no mention of a segfault or unhandled exception etc. Just goes blank and exits. I may be able t break out the debugger later. All I have is a basic OpenBCI via BrainFlow to the EDF writer:

image

As of 13797ce, am getting the "Device not connected" error in lieu of a crash. Interestingly trying to stream data from my Cyton results in this; whilst using the synthetic board and it outputs a file fine (both backed by Brainflow input node). I've also tried just dividing the data through by 1000 in the classifier, given the issues with min/max in #138 (I get values in excess of 10^6 from the Cyton), but problem still persists.

Also finding it hard to debug this, as I keep getting the "realtime functionality lost" message when the debugger locks up the application. Any tips on this?