Playback error
jennieli421 opened this issue · 13 comments
Hello,
I have encountered an issue with running a simulation in MinKNOW. This is what I did:
- add the
simulation =line to “sequencing_MIN106_DNA.toml” - reload script in minknow API
- in the terminal, connect to Guppy by running guppy_basecall_server --config dna_r9.4.1_450bps_fast.cfg -p 5555 -l /tmp/guppy -x 'cuda:0' (this might not be necessary)
- connect the configuration test flow cell to the desktop.
- select flow cell type as FLO-MIN106, a random kit, and hit start.
- The flow cell temperature begin to rise, once this step is finished, the run stops and it says "run error".
Please let me know what you think might be causing this error. Thank you very much.
Hello,
If you look in your logs, it might have some more information.
Try looking somewhere like /var/log/minknow/ and look at the latest modified files/position.
Does your simulation = line have the path to your bulk fast5 file in there correctly?
Otherwise, it could be while using the configuration test cell. I've never tried to do a simulation with that before.
Thanks for replying!
I will check the logs once I get a chance.
I've checked that the path is correct, and used h5dump to inspect the file looks normal.
It could be the problem of using a test cell. But a tutorial I have been following says "Insert a configuration test flowcell into the sequencing device" (https://github.com/LooseLab/readfish). I am also hesitant to use a real flow cell before I know things can work. Can I use a used one for testing purposes?
Hi
Perhaps you gave a BLOW5 file to the simulation - I thought so because this issue is opened under slow5tools?
simulation = should point to a bulk-FAST5 file, not a BLOW5 file or a regular FAST5. Bulk-FAST5 is quite different to regular FAST5 files.
Readfish repository has a link to an example bulk-FAST5 file you can use. If you have an old used flowcell, you can use that too, instead of a new flowcell. I remember having some trouble with the config test cell with bulk-FAST5, 2 years ago, so cannot exactly remember what the issue was.
Yea, and if you don't have any old flowcells, you can also simulate positions too and don't even need to plug a device in. Handy if benchmarking 100 MinION runs at once 🤣
Thanks! Good to know the difference between bulk- and regular fast5, I am using the bulk-fast5 provided in Readfish repo. I will try with an old flowcell when I can get one. Also, @Psy-Fer do you mind sharing share a tutorial on how to "simulate positions"? Couldn't find useful info online.
Again, thanks very much for helping out and being patient with a beginner.
Hey,
yea sure.
Just don't run this while you are running other flowcells or anything. I use this when testing real-time applications on my laptop without access to a minion.
This is also assuming running on linux
# in a separate terminal that you don't close until you are done simulating run the following
# stop the minkow service
sudo systemctl stop minknow.service
# start a simulated integrated device (like on a GridION)
sudo /opt/ont/minknow/bin/mk_manager_svc -d --print-rpc-ports --simulated-integrated-devices 1Refresh your minknow and you should now see a new device called something like M00001
Have fun :)
Amazing! It is working very well!
A follow up:
For "simulate position" function, will it be simulating using the fast5 I specified? Is there anyway to make sure?
Also, How should I reset back to the normal mode? I noticed that under "connection manager" there is no "saved hosts". Don't remember editing this page when I installed minknow so not sure how to add it or find Mk1c ID.
When I try to run the simulation again, I got this issue:
$ sudo systemctl stop minknow.service
$ sudo /opt/ont/minknow/bin/mk_manager_svc -d --print-rpc-ports --simulated-integrated-devices 1
Port 9502 appears to be in use, retrying for 2 minutes as it might be in TIME_WAIT state and will become available shortly.
Port 9502 appears to be in use.
Is there another instance of MinKNOW running?
Rebooting the desktop appears to fix this issue.
In terms of simulation using an old flowcell, it also produces "run error". I checked the most recently modified log file:
2023-05-29 17:00:12.129826 INFO: starting_up (control)
hostname: LX310327
system: ubuntu 22.04
Distribution: 23.04.5 (STABLE)
MinKNOW Core: 5.5.3
Bream: 7.5.9
Protocol configuration: 5.5.13
Guppy (build): 6.5.5+3a1e121
Guppy (connected): 6.5.7+ca6d6af
2023-05-29 17:00:12.138173 WARNING: partially_raised_file_limits (analyser)
actual: 1024
target: 4096
2023-05-29 17:06:20.364178 INFO: finished (analyser)
/var/log/minknow/MN34448/analyser_log-0.txt (END)
Any new ideas on what is going on?
Also appending the head of log generated from running position simulation:
2023-05-29 16:41:32.443116 INFO: starting_up (control)
hostname: LX310327
system: ubuntu 22.04
Distribution: 23.04.5 (STABLE)
MinKNOW Core: 5.5.3
Bream: 7.5.9
Protocol configuration: 5.5.13
Guppy (build): 6.5.5+3a1e121
Guppy (connected): 6.5.7+ca6d6af
2023-05-29 16:41:32.451353 WARNING: partially_raised_file_limits (analyser)
actual: 1024
target: 4096
2023-05-29 16:42:14.143543 INFO: switching_to_guppy_config (basecalling)
new_filename: dna_r9.4.1_450bps_fast.cfg
2023-05-29 16:42:14.144469 INFO: guppy_connection_changed_state (basecalling)
client_name: MS00000_b5b77cae-b1f8-41ef-acc4-5c275a26d6ef
guppy_error_string: [no_error]
next_state: preparing_connection (1)
previous_state: disconnected (0)
2023-05-29 16:42:14.144769 INFO: guppy_connection_changed_state (basecalling)
client_name: MS00000_b5b77cae-b1f8-41ef-acc4-5c275a26d6ef
guppy_error_string: [no_error]
next_state: loading_config (2)
previous_state: preparing_connection (1)
2023-05-29 16:42:14.145785 INFO: guppy_connection_changed_state (basecalling)
client_name: MS00000_b5b77cae-b1f8-41ef-acc4-5c275a26d6ef
guppy_error_string: [no_error]
next_state: loading_alignment_index (3)
previous_state: loading_config (2)
@jennieli421 where do you see the "run error"? On Minknow, under the messages tab is there anything useful about the error?
Check the most recent of each of the all the following logs:
- /var/log/minknow/basecall_manager_log-*.txt
- /var/log/minknow/basecall_manager_log-*.txt
- /var/log/minknow/MN34448/bream-*.txt
- /var/log/minknow/MN34448/control_server_log-*.txt
One of those will have the error. Can you also double-check if you have given the path to the bulk-FAST5 correctly in the toml file - I remember due to a typo I had a similar trouble some time ago.
Also @alexomics and @mattloose
mattloose might want to comment about this, but I see they are perhaps too busy LooseLab/readfish#240
In /var/log/minknow/MN34448/control_server_log-0.txt:
023-05-29 17:06:20.125127 WARNING: hdf5_error (fast5)
cause: unable to open file: name = '/home/readuntil/readfish_files/PLSP57501_20170308_FNFAF14035_MN16458_sequencing_run_NOTT_Hum_wh1rs2_60428.fast5', errno = 13, error message = 'Permission denied', flags = 0, o_flags = 0
function: H5Fopen
message: unable to open file
stack_trace:
#000: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5F.c line 620 in H5Fopen(): unable to open file
major: File accessibility
minor: Unable to open file
2023-05-29 17:06:20.126132 ERROR: exception_during_observer_callback (state_control)
callback_mode: transitioning
cause: Dynamic exception type: boost::filesystem::filesystem_error
std::exception::what: boost::filesystem::status: Permission denied [system:13]: "/home/readuntil/readfish_files/PLSP57501_20170308_FNFAF14035_MN16458_sequencing_run_NOTT_Hum_wh1rs2_60428.fast5"
Then I changed the permission of the fast5 file by running chmod 777 PLSP57501_20170308_FNFAF14035_MN16458_sequencing_run_NOTT_Hum_wh1rs2_60428.fast5
Tried simulation using a minion device, still the same run error. the minknow message says
"MinION Mk1B · MN19362
Stopping protocol due to internal error. All data will be saved."
In the control_server_log-0.txt of MN19362, I see a different error:
2023-06-01 14:40:26.304513 WARNING: hdf5_error (fast5)
cause: object 'duty_time' doesn't exist
function: H5Gopen2
message: unable to open group
stack_trace:
#000: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5G.c line 510 in H5Gopen2(): unable to open group
major: Symbol table
minor: Can't open object
#001: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5VLcallback.c line 4187 in H5VL_group_open(): group open failed
major: Virtual Object Layer
minor: Can't open object
#002: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5VLcallback.c line 4154 in H5VL__group_open(): group open failed
major: Virtual Object Layer
minor: Can't open object
#003: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5VLnative_group.c line 123 in H5VL__native_group_open(): unable to open group
major: Symbol table
minor: Can't open object
#004: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5Gint.c line 270 in H5G__open_name(): group not found
major: Symbol table
minor: Object not found
#005: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5Gloc.c line 442 in H5G_loc_find(): can't find object
major: Symbol table
minor: Object not found
#006: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5Gtraverse.c line 837 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#007: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5Gtraverse.c line 613 in H5G__traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#008: /builds/informatics/conan-hdf5/9721449/.conan/data/hdf5/1.12.1/nanopore/stable/build/4cb52770e5d65c257e2250813532b9e31047d0eb/source_subfolder/src/H5Gloc.c line 399 in H5G__loc_find_cb(): object 'duty_time' doesn't exist
major: Symbol table
minor: Object not found
2023-06-01 14:40:26.305008 WARNING: hdf5_error (fast5)
cause: object 'read_length_histograms' doesn't exist
function: H5Gopen2
message: unable to open group
stack_trace:
Was this issue fixed @jennieli421?
Thanks for following up! Just fixed the playback error yesterday, as described in this thread.
Thank you again, and I'll close this issue.