IHTSDO/snomed-owl-toolkit

Silenced out of memory error

Closed this issue · 6 comments

fanavarro commented

Hi all, the last week I created an issue because I was not able to generate the owl ontology file for the last version of Snomed (#37). The generated owl showed an incorrect hierarchy of concepts, where most of them were outside the Snomed root concept. This was provoked because there was an out of memory error, which was silenced in the process.

I am doing experiments with the different versions of snomed along the time, and I repeated the process by assigning 4GB of memory. Then, I noticed that the owl files generated with 2GB of memory were, in general, different to the files generated with 4GB. For example, the following figure shows the snomed 2019-01-31 owl file generated with 2Gb and with 2GB:
imagen

As you can see, annotations are created incorrectly, but they could be legit so I did not detect the error.

Would it be possible to show this out of memory error, at least in the log?

kaicode commented

Hi @fanavarro,
Thanks for reporting this. This is a bug which needs to be fixed.

The workaround using the logs is to check you have a line containing the string "reference set members read from sct2_sRefset_OWLExpressionSnapshot". If this string is not present in the log output then the OWL axioms have not been completely read and the ontology file will be incomplete.

We will try to implement better exception handling for this in the next couple of weeks. Pull requests welcome.

Kind regards,
Kai

grahamponting commented

Hi all,
I am getting the "out of memory" error using using the 2.9.0 file, detaiIs included. Is there any timeline for a fix. ?

Many thanks, Graham Ponting
error track 13Aug2020.txt

kaicode commented

Hi @grahamponting,

Your log shows this command is being used: java -jar snomed-owl-toolkit-2.9.0-executable.jar -rf2-to-owl -rf2-snapshot-archives SnomedCT_InternationalRF2_PRODUCTION_20200309T120000Z.zip.
Please give 4g of memory as shown in the example in the README.

The corrected version of the command in your log would be: java -Xms4g -jar snomed-owl-toolkit-2.9.0-executable.jar -rf2-to-owl -rf2-snapshot-archives SnomedCT_InternationalRF2_PRODUCTION_20200309T120000Z.zip.

It is not possible for Java to use more memory than it's given. It is also not possible for this program to use less memory.
I'm pleased that the program reported the error effectively. I'll close this ticket now.

Kind regards,
Kai Kewley

grahamponting commented
CoderMChu commented

Hi @grahamponting,

Can you check the Java version installed on your PC please using the following command in a command line?
java -d64

If you get an error when running above command then you will not be able to use 4g due to "On most modern 32-bit Windows systems the maximum heap size will range from 1.4G to 1.6G".

See more details via the link here Why can't I get a larger heap with the 32-bit JVM?

If you have got 32 bit JRE then please install the 64 bit. Hopefully this will solve your issue.

Kind regards,

Michael

grahamponting commented