Out of Memory Error
hmqq opened this issue · 9 comments
I have a file consisting of 26618 rules and when I execute it with Alpha it gave me a memory exception:
“Exception in thread "main" java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar replaced objects”
I also try increasing the heap size and use java -Xmx16g -jar alpha.jar -i rules.lp but still I am getting the same error.
Can you provide some more information on this issue? For example, are your rules already ground? What are your rules intended to compute? How/with what command-line arguments are you calling Alpha?
With the information you provided so far, we only see that Alpha runs out of memory. Since Alpha allows nonground rules with function symbols, it is clear that one can write programs that run out of memory. In order to help you, we need some more details about the problem you encountered.
I am basically passing three input arguments like this:
"java -Xmx16g -jar alpha.jar -i MEF0.lp -i rules.lp -i query0.lp"
Where MEF0.lp is the instance file, rules.lp contains the rules and query0.lp contains the query that I wanted to execute on the former arguments. (I have changed the extensions of the file because apparently, I am unable to upload with .lp extension)
MEF0.txt
rules.txt
query0.txt
I am basically querying the rules and instance file. I have encountered the heap problem so far. Maybe the files that i have attached could provide you more insight.
From what I have seen of your inputs, I think the "problem" is really that with the amount of facts and rules you have, you just simply run out of memory. This is due to the fact that Alpha (and every other ASP solver that I know of), by it's very nature, has to keep every atom derived by your rules in memory to be able to display a full answer set. (at least until we get around to implementing specialized facilities for query-answering)
In order to solve your problem, I'd suggest the following workaround:
Because none of your rules contains negation, your program can be split into sub-programs along rule dependencies, i.e. instead of your single, massive, program you'd have a few partial programs where each takes the answer set of the previous one as its input.
To easily find out how your rules depend on each other, you could use Alpha's -wdg ("i.e. " write dependendy graph") command-line switch which will write a .dot (graphviz) file visualizing dependencies between predicates in your program. For those rules that have single-literal bodies (e.g. isacCC(C1,"owl:Thing"):- isacCI(C1,R2,C2).) you can even split the evaluation across multiple solver runs, i.e. have a program consisting of just this rule and feed it 5000 facts per run instead of all at once. (This might also work for other rules, but is potentially more invloved yince you have to take care that you don't lose joining pairs when splitting your facts).
More detailed (if unfortunately tedious to read) information on splitting of ASP programs can be found in these papers:
Towards a Theory of Declarative Knowledge by Apt, Blair, Walker and
Splitting a Logic Program by Lifschitz and Turner
Thankyou for the detailed answer. As per suggestion, I am trying to use the "-wdg" command but Alpha throwing "unrecognized option". How can I use this command?
Second The idea of splitting seems interesting. I have to look more into it. Thankyou.
Thankyou for the detailed answer. As per suggestion, I am trying to use the "-wdg" command but Alpha throwing "unrecognized option". How can I use this command?
Second The idea of splitting seems interesting. I have to look more into it. Thankyou.
Have you specified a target file for the dependency graph?
I tried it out now, following command works for me:
java -jar Alpha-bundled.jar -i rules.txt -wdg rules.txt.depgraph.dot
This is the generated file (txt extension at the end to satisfy github):
rules.txt.depgraph.dot.txt
It looks like you have quite a lot of recursive rules, so splitting should still work (since you don't have negation), but might be a bit more involved unfortunately.
yes, i have specified the dependency graph but did not work java -jar alpha.jar -i rules.lp -wdg rules.lp.dot the command you share is also not working for me. I am still getting the same error.
Thankyou for the file.
Yes you are right.
oh, I see the problem now.. I'm assuming you were using the jar file from the latest (0.5.0) release.
The wdg command (along with a more efficient evaluation strategy for programs like yours) were only added in October 2020 and are not yet part of any official release.
It might be worth a try to build the jar from the latest version of the master branch (everything in there is tested and works to the best of our knowledge). Though I can't guarantee that, even your original heap space problem might be solved with the latest version.
Sorry for the confusion!
I build the jar from the latest version of the "master" and wdg start working now. Unfortunately the original problem hasn't been resolved. Thankyou.
Closing this issue since there's no technical solution with the current system architecture due to the sheer amount of data involved which simply requires a lot of heap space.