Luciphor2 support for mixed 64-bit and 32-bit data arrays in mzML files
Closed this issue · 34 comments
Hello,
I ran into an issue using the OpenMS LuciphorAdapter.
Following error occurred several times and the LuciphorAdapter failed:
ERROR:QExactiveHF02_03926.mzML Scan: 12873
# of mz values != # intensity values: 1830 != 457
Skipping this scan...
This was related to the float precision of the m/z array and the intensity array (see below) and Luciphor2, which may only take 32-bit or 64-bit arrays [1].
We have a lot of files using a 64-bit float for the m/z array and a 32-bit float for the intensity array and this combination is commonly used in mzML-files, would it be possible to add that to luciphor? We can provide you a file if needed.
[1] 24c4b69
Hello Damian,
I send the right link to your google mail account, since I originally didn't want to post the link.
I hope I found the right address, If you can't find it there please let me know!
Cheers
Oliver
Hello Damian,
sorry about that!
I zipped the file now and send you another link!
Cheers
Oliver
Hello Damian,
have you been able to work with the file?
If you need anything else, please do not hesitate to contact me.
Oliver
Hello Damian,
no problem!
I tested it with the luciphorAdapter in OpenMs - there the issue arises (see attachment).
In the link you will find the following files:
test_luciphor_64_32bit.mzml
test_luciphor_32bit.mzml
test_luciphor.idxml
test_luciphor_64_32bit.pepxml
test_luciphor_32bit.pepxml
If the precision is formatted mz: 64bit and int: 32bit (test_luciphor_64_32bit) the luciphorAdapter fails. If the precision is formatted the same mz & int 32 bit or 64 bit the Adapter finish without an issue (test_luciphor_32bit.mzml).
That his why I suggested:
This was related to the float precision of the m/z array and the intensity array (see below) and Luciphor2, which may only take 32-bit or 64-bit arrays [1].
[1] 24c4b69
I hope this helps! If you need another file format or other files please let me know!
Hello Damian,
as far as I know the Luciphor code was not altered. The LuciphorAdapter is "just" a wrapper which writes a pepXML based on the idxml and mzml and forwards it to luciphor2 [1].
The following LuciPHOr2 version is required: luciphor2 (JAVA-based version of Luciphor) (1.2014Oct10). At the time of writing, it could be downloaded from http://luciphor2.sourceforge.net.
Input spectra for LuciPHOr2 have to be in pepXML file format. The input mzML file must be the same as the one used to create the pepXML input file.
Hello Damian,
sorry for the late response!
I get the following error message, when I try to run the LuciphorAdapter with the new version of lucXor:
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Multimap
at lucxor.LucXor.main(LucXor.java:54)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Multimap
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
Maybe a dependency is missing?
Oliver
Hello Damian,
I still get the error, even when started from the command line:
New version:
u-082-c025:luciphor_fix_1 alka$ java -jar luciphor2.jar
luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net
USAGE: java -jar luciphor2.jar <input_file>
Generate a luciphor2 input file with: java -jar luciphor2.jar -t
Modify the input file to suit your needs and submit it to the program.
Example: java -jar luciphor2.jar input_file_you_edited
u-082-c025:luciphor_fix_1 alka$ java -jar luciphor2.jar -t
luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Multimap
at lucxor.LucXor.main(LucXor.java:49)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Multimap
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
For comparison "old" version:
u-082-c025:luciphor alka$ java -jar luciphor2.jar
luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net
USAGE: java -jar luciphor2.jar <input_file>
Generate a luciphor2 input file with: java -jar luciphor2.jar -t
Modify the input file to suit your needs and submit it to the program.
Example: java -jar luciphor2.jar input_file_you_edited
u-082-c025:luciphor alka$ java -jar luciphor2.jar -t
luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net
Please edit the input file: luciphor2_input_template.txt with your favorite text editor
Hi Damian,
i previously renamed the lucXor.jar to luciphor2.jar - for the TOPPAS workflow.
Just for testing I downloaded your update again and get the same results.
Java-version:
unicorn:Downloads alka$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
Test not renamed file:
unicorn:Downloads alka$ java -jar lucXor.jar
luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net
USAGE: java -jar luciphor2.jar <input_file>
Generate a luciphor2 input file with: java -jar luciphor2.jar -t
Modify the input file to suit your needs and submit it to the program.
Example: java -jar luciphor2.jar input_file_you_edited
-> this should be java -jar lucXor.jar -t in this case or?
unicorn:Downloads alka$ java -jar lucXor.jar -t
luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Multimap
at lucxor.LucXor.main(LucXor.java:49)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Multimap
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 1 more
Not sure if I do something wrong?
I can hopefully test it tomorrow on another machine.
I am using osx 10.12.
Cheers
This is the whole output without the grep command:
unicorn:Downloads alka$ jar tf lucXor.jar
META-INF/MANIFEST.MF
lucxor/
lucxor/constants.class
lucxor/FLRClass.class
lucxor/globals.class
lucxor/LucXor.class
lucxor/ModelData_CID.class
lucxor/ModelData_HCD.class
lucxor/ModelParameterWorkerThread.class
lucxor/mzMLreader$1.class
lucxor/mzMLreader$SpectrumStruct.class
lucxor/mzMLreader.class
lucxor/NormalDensityWorkerThread.class
lucxor/PeakClass$1.class
lucxor/PeakClass$2.class
lucxor/PeakClass$3.class
lucxor/PeakClass$4.class
lucxor/PeakClass$5.class
lucxor/PeakClass.class
lucxor/Peptide.class
lucxor/PepXML.class
lucxor/PSM.class
lucxor/ScoringWorkerThread.class
lucxor/SpectrumClass.class
lucxor/statsFunctions.class
META-INF/
combinatoricslib-2.0.jar
commons-codec-1.8.jar
commons-collections-3.2.1.jar
commons-lang3-3.2.1.jar
commons-math3-3.2.jar
guava-15.0.jar
hotsax.jar
junit.jar
MSDataStructures.jar
trove-3.0.3.jar
With your grep command it seems that nothing is found:
unicorn:Downloads alka$ jar tf lucXor.jar | grep -i 'com/google/common/collect/Multimap'
unicorn:Downloads alka$
Oliver
Seems to work fine now!
unicorn:Downloads alka$ md5 luciphorUpdate.zip
MD5 (luciphorUpdate.zip) = d29bcce5c4ae92060a6b6cefb495ffea
unicorn:Downloads alka$ java -jar lucXor.jar -t
luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net
Please edit the input file: luciphor2_input_template.txt with your favorite text editor
I will test it tomorrow with the 64/32 bit files in the LuciphorAdapter.
Thank you very much for your help!
Oliver
Hello Damian,
the new version works for 32bit & 64bit float precision but unfortunately not for the mixed version 64bit_32bit.
The same error as shown above occures:
ERROR:QExactiveHF02_03926.mzML Scan: 12873
# of mz values != # intensity values: 1830 != 457
Skipping this scan...
I made another dropbox link with the input (mzml, idxml, pepxml) & output files (idxml).
Link: https://www.dropbox.com/sh/4ottvb2d2eh7daw/AAAjtpKu4fgBd98-jnMomgO9a?dl=0
May you have look on it?
Thanks
Oliver
Thank you for pointing that out - there seem to be an issue with selecting the right score. I will work on that.
To work with the data and reproduce the issue we are having using mixed 64bit/32bit precision as stated above - you could use the hack: Setting the modeling_score_threshold to 0. (I know it is not supposed to be used that way - but until I fixed the score - you could have a look on it).
Oliver
Hello Damian,
I updated the files - the pepXML should now contain the probabilities.
Please have a look!
Link: https://www.dropbox.com/sh/4ottvb2d2eh7daw/AAAjtpKu4fgBd98-jnMomgO9a?dl=0
Hello Damian,
unfortunately the link doesn't seem to work.
Cheers
Oliver
The files with mixed precision (64/32bit) work with your new version.
Thank you very much!
Cheers
Oliver
I have to find time to update the code to run with Java 13.
In the mean time use Java 8.
Regarding the second question, Luciphor2 extracts the file name from the scan ID in the pepXML file. So if your scanID in the pepXML file is somefile.001.001.03
then you need to provide it the path to somefile.mzML
.
Thanks a lot for the explanation and having a look!
Sorry for reopening this issue - if you like I can open one - specific for the Java 13 question (which may be easier to find for others.)
Let me know if you have any other questions or something ready to test down the road!
edit: In regard to file extensions, would it be possible to support different spellings of .mzML (e.g. mzml), since it seems to throw an error / does not work if ".mzml" is used.
@dfermin In theory it should be enough to update MSFTBX library to a newer version. It had Java 9+ compatibility for quite a while now.
Tested in #4 - Thanks!