dfermin/lucXor

Luciphor2 support for mixed 64-bit and 32-bit data arrays in mzML files

Closed this issue · 34 comments

Hello,

I ran into an issue using the OpenMS LuciphorAdapter.
Following error occurred several times and the LuciphorAdapter failed:

 ERROR:QExactiveHF02_03926.mzML Scan: 12873
 # of mz values != # intensity values: 1830 != 457
 Skipping this scan...

This was related to the float precision of the m/z array and the intensity array (see below) and Luciphor2, which may only take 32-bit or 64-bit arrays [1].

We have a lot of files using a 64-bit float for the m/z array and a 32-bit float for the intensity array and this combination is commonly used in mzML-files, would it be possible to add that to luciphor? We can provide you a file if needed.

[1] 24c4b69

Hello Damian,
I send the right link to your google mail account, since I originally didn't want to post the link.
I hope I found the right address, If you can't find it there please let me know!

Cheers
Oliver

Hello Damian,

sorry about that!
I zipped the file now and send you another link!

Cheers
Oliver

Hello Damian,

have you been able to work with the file?
If you need anything else, please do not hesitate to contact me.

Oliver

Hello Damian,

no problem!

I tested it with the luciphorAdapter in OpenMs - there the issue arises (see attachment).

In the link you will find the following files:
test_luciphor_64_32bit.mzml
test_luciphor_32bit.mzml
test_luciphor.idxml
test_luciphor_64_32bit.pepxml
test_luciphor_32bit.pepxml

If the precision is formatted mz: 64bit and int: 32bit (test_luciphor_64_32bit) the luciphorAdapter fails. If the precision is formatted the same mz & int 32 bit or 64 bit the Adapter finish without an issue (test_luciphor_32bit.mzml).

That his why I suggested:

This was related to the float precision of the m/z array and the intensity array (see below) and Luciphor2, which may only take 32-bit or 64-bit arrays [1].
[1] 24c4b69

I hope this helps! If you need another file format or other files please let me know!

Here a screenshot:
luciphor_error_64_32bit_mzml

Hello Damian,

as far as I know the Luciphor code was not altered. The LuciphorAdapter is "just" a wrapper which writes a pepXML based on the idxml and mzml and forwards it to luciphor2 [1].

The following LuciPHOr2 version is required: luciphor2 (JAVA-based version of Luciphor) (1.2014Oct10). At the time of writing, it could be downloaded from http://luciphor2.sourceforge.net.
Input spectra for LuciPHOr2 have to be in pepXML file format. The input mzML file must be the same as the one used to create the pepXML input file.

[1]
https://github.com/OpenMS/OpenMS/blob/7854aff8eca32d4e723553389316b18d7d7ca865/src/topp/LuciphorAdapter.cpp

Hello Damian,

sorry for the late response!

I get the following error message, when I try to run the LuciphorAdapter with the new version of lucXor:

Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Multimap
	at lucxor.LucXor.main(LucXor.java:54)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Multimap
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

Maybe a dependency is missing?

Oliver

Hello Damian,

I still get the error, even when started from the command line:
New version:

u-082-c025:luciphor_fix_1 alka$ java -jar luciphor2.jar 

luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net

USAGE: java -jar luciphor2.jar <input_file>

	Generate a luciphor2 input file with: java -jar luciphor2.jar -t
	Modify the input file to suit your needs and submit it to the program.
	Example: java -jar luciphor2.jar input_file_you_edited

u-082-c025:luciphor_fix_1 alka$ java -jar luciphor2.jar -t

luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net

Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Multimap
	at lucxor.LucXor.main(LucXor.java:49)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Multimap
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 1 more

For comparison "old" version:

u-082-c025:luciphor alka$ java -jar luciphor2.jar 

luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net

USAGE: java -jar luciphor2.jar <input_file>

	Generate a luciphor2 input file with: java -jar luciphor2.jar -t
	Modify the input file to suit your needs and submit it to the program.
	Example: java -jar luciphor2.jar input_file_you_edited

u-082-c025:luciphor alka$ java -jar luciphor2.jar -t

luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net


Please edit the input file: luciphor2_input_template.txt with your favorite text editor

Hi Damian,

i previously renamed the lucXor.jar to luciphor2.jar - for the TOPPAS workflow.
Just for testing I downloaded your update again and get the same results.

Java-version:

unicorn:Downloads alka$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Test not renamed file:

unicorn:Downloads alka$ java -jar lucXor.jar 

luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net

USAGE: java -jar luciphor2.jar <input_file>

	Generate a luciphor2 input file with: java -jar luciphor2.jar -t
	Modify the input file to suit your needs and submit it to the program.
	Example: java -jar luciphor2.jar input_file_you_edited

-> this should be java -jar lucXor.jar -t in this case or?

unicorn:Downloads alka$ java -jar lucXor.jar -t

luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net

Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/Multimap
	at lucxor.LucXor.main(LucXor.java:49)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Multimap
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 1 more

Not sure if I do something wrong?
I can hopefully test it tomorrow on another machine.
I am using osx 10.12.

Cheers

This is the whole output without the grep command:

unicorn:Downloads alka$ jar tf lucXor.jar 
META-INF/MANIFEST.MF
lucxor/
lucxor/constants.class
lucxor/FLRClass.class
lucxor/globals.class
lucxor/LucXor.class
lucxor/ModelData_CID.class
lucxor/ModelData_HCD.class
lucxor/ModelParameterWorkerThread.class
lucxor/mzMLreader$1.class
lucxor/mzMLreader$SpectrumStruct.class
lucxor/mzMLreader.class
lucxor/NormalDensityWorkerThread.class
lucxor/PeakClass$1.class
lucxor/PeakClass$2.class
lucxor/PeakClass$3.class
lucxor/PeakClass$4.class
lucxor/PeakClass$5.class
lucxor/PeakClass.class
lucxor/Peptide.class
lucxor/PepXML.class
lucxor/PSM.class
lucxor/ScoringWorkerThread.class
lucxor/SpectrumClass.class
lucxor/statsFunctions.class
META-INF/
combinatoricslib-2.0.jar
commons-codec-1.8.jar
commons-collections-3.2.1.jar
commons-lang3-3.2.1.jar
commons-math3-3.2.jar
guava-15.0.jar
hotsax.jar
junit.jar
MSDataStructures.jar
trove-3.0.3.jar

With your grep command it seems that nothing is found:

unicorn:Downloads alka$ jar tf lucXor.jar | grep -i 'com/google/common/collect/Multimap'
unicorn:Downloads alka$ 

Oliver

Seems to work fine now!

unicorn:Downloads alka$ md5 luciphorUpdate.zip 
MD5 (luciphorUpdate.zip) = d29bcce5c4ae92060a6b6cefb495ffea
unicorn:Downloads alka$ java -jar lucXor.jar -t

luciphor2 (JAVA-based version of Luciphor)
Version: 1.2014Oct10
Original C++ version available at: http://luciphor.sf.net

Please edit the input file: luciphor2_input_template.txt with your favorite text editor

I will test it tomorrow with the 64/32 bit files in the LuciphorAdapter.
Thank you very much for your help!

Oliver

Hello Damian,
the new version works for 32bit & 64bit float precision but unfortunately not for the mixed version 64bit_32bit.
The same error as shown above occures:

 ERROR:QExactiveHF02_03926.mzML Scan: 12873
 # of mz values != # intensity values: 1830 != 457
 Skipping this scan...

I made another dropbox link with the input (mzml, idxml, pepxml) & output files (idxml).
Link: https://www.dropbox.com/sh/4ottvb2d2eh7daw/AAAjtpKu4fgBd98-jnMomgO9a?dl=0

May you have look on it?

Thanks
Oliver

Thank you for pointing that out - there seem to be an issue with selecting the right score. I will work on that.

To work with the data and reproduce the issue we are having using mixed 64bit/32bit precision as stated above - you could use the hack: Setting the modeling_score_threshold to 0. (I know it is not supposed to be used that way - but until I fixed the score - you could have a look on it).

Oliver

Hello Damian,

I updated the files - the pepXML should now contain the probabilities.

Please have a look!

Link: https://www.dropbox.com/sh/4ottvb2d2eh7daw/AAAjtpKu4fgBd98-jnMomgO9a?dl=0

Hello Damian,

unfortunately the link doesn't seem to work.

Cheers
Oliver

The files with mixed precision (64/32bit) work with your new version.

Thank you very much!

Cheers
Oliver

I have to find time to update the code to run with Java 13.
In the mean time use Java 8.

Regarding the second question, Luciphor2 extracts the file name from the scan ID in the pepXML file. So if your scanID in the pepXML file is somefile.001.001.03 then you need to provide it the path to somefile.mzML.

Thanks a lot for the explanation and having a look!

Sorry for reopening this issue - if you like I can open one - specific for the Java 13 question (which may be easier to find for others.)

Let me know if you have any other questions or something ready to test down the road!

edit: In regard to file extensions, would it be possible to support different spellings of .mzML (e.g. mzml), since it seems to throw an error / does not work if ".mzml" is used.

chhh commented

@dfermin In theory it should be enough to update MSFTBX library to a newer version. It had Java 9+ compatibility for quite a while now.

@chhh has fixed the problem.
If you download the mavenize branch of luciphor and run mvn clean package in the root directory you will get a working jar at lucxor/target/lucxor-2.1-SNAPSHOT.one-jar.jar

Thanks @chhh !

Tested in #4 - Thanks!