Errors while saving emitter as csv

[cleterrier]

Hi,

I have a notebook to batch filter emitters that I have updated for DECODE 0.10.0. It takes h5 as input (used to take pt with 0.9.4) and saves filtered emitters as csv. Here is the relevant code (loop on h5 files, filtering, saving):

for h5_path in filelist : 

    h5_path = h5_path.replace("\\", "/")
    print("processing file " + h5_path)

    #Load the .h5 file
    emitter = decode.EmitterSet.load(h5_path) # path to the h5 file
    
    csvout_path = h5_path.replace(".h5", ".csv") 
    print('filtered csv will be saved as: ' + csvout_path)
    
    # filter on uncertainties (absolute) for EMCCD
    sigma_x_high_threshold = 25 # be aware that these values will be *0.4 when converting into TS, like 50 = 20 nm, 22.5 = 9 nm
    sigma_y_high_threshold = 25 
    sigma_z_high_threshold = 100
    
    em_sub = emitter[(emitter.xyz_sig_nm[:, 0] <= sigma_x_high_threshold) * (emitter.xyz_sig_nm[:, 1] <= sigma_y_high_threshold) * (emitter.xyz_sig_nm[:, 2] <= sigma_z_high_threshold)]
    
    # Save filtered emitters as csv
    csvout_path = h5_path.replace(".h5", ".csv") 
    decode.utils.emitter_io.save_csv(csvout_path, em_sub.to_dict())
    #em_sub.save('csvout_path')
    print('saved filtered csv as: ' + csvout_path)

You see that the first saving line is the one from 0.9.4:
decode.utils.emitter_io.save_csv(csvout_path, em_sub.to_dict())
It was working fine on 0.9.4, now results in this error in 0.10.0:

processing file D:/210319 ENDO#10 TIF/C1b_N10_ctrl_clat-a2s-NF_clat647_MEA25.h5
filtered csv will be saved as: D:/210319 ENDO#10 TIF/C1b_N10_ctrl_clat-a2s-NF_clat647_MEA25.csv
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-18-591330129d10> in <module>
    131     # Save filtered emitters as csv
    132     csvout_path = h5_path.replace(".h5", ".csv")
--> 133     decode.utils.emitter_io.save_csv(csvout_path, em_sub.to_dict())
    134     #em_sub.save('csvout_path')
    135     print('saved filtered csv as: ' + csvout_path)

TypeError: save_csv() missing 1 required positional argument: 'metadata'

If I comment out this line and replace it by the new saving command from 0.10.0 that accepts etiher pt, h5 or csv: em_sub.save('csvout_path')
I get another error:

processing file D:/210319 ENDO#10 TIF/C1b_N10_ctrl_clat-a2s-NF_clat647_MEA25.h5
filtered csv will be saved as: D:/210319 ENDO#10 TIF/C1b_N10_ctrl_clat-a2s-NF_clat647_MEA25.csv
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-17-340fa0e8eb0c> in <module>
    132     csvout_path = h5_path.replace(".h5", ".csv")
    133     #decode.utils.emitter_io.save_csv(csvout_path, em_sub.to_dict())
--> 134     em_sub.save('csvout_path')
    135     print('saved filtered csv as: ' + csvout_path)

~\.conda\envs\decode10_env\lib\site-packages\decode\generic\emitter.py in save(self, file)
    255             emitter_io.save_csv(file, self.data, self.meta)
    256         else:
--> 257             raise ValueError
    258 
    259     @staticmethod

ValueError:

What am I doing wrong here? How should I save the emitter (or is it a bug)?

[Haydnspass]

Can you show me the error?

One thing: #decode.utils.emitter_io.save_csv(csvout_path, em_sub.to_dict()) will not work anymore because the signature has changed https://decode.readthedocs.io/en/release-0.10/decode.utils.html#decode.utils.emitter_io.save_csv
But you are calling .save directly, so that is unrelated

[cleterrier]

Sorry I was completing/editing the post. Should be complete with the two errors now

[Haydnspass]

You are parsing a string to .save() it should be em_sub.save(csvout_path) (i.e. without marks).

You are still free to use the emitter_io.save_csv function, but you would need to call it like this

decode.utils.emitter_io.save_csv(path, em_sub.data, em_sub.meta):

[cleterrier]

Ha! Sorry for that

[Haydnspass]

No worries :P