Timeout error on dataset

[navdeepkjohal]

Hello I ran Popper on a Knowledge Graph dataset which is a pretty large dataset. I turned recursion on. I got the following error. Could you please suggestion some changes so that Popper can be run on large datasets

[andrewcropper]

You need to provide more details than a screenshot. How large in the dataset? Is it large in the number of examples or the BK? What bias settings did you use? Also, what have you tried so far to improve the efficiency?

…

On 20 Sep 2022, at 11:10, navdeepkjohal ***@***.***> wrote: Hello I ran Popper on a Knowledge Graph dataset which is a pretty large dataset. I turned recursion on. I got the following error. Could you please suggestion some changes so that Popper can be run on large datasets — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[navdeepkjohal]

Hello,

Thank you for replying back to me.

I have tried the following setting in bias.pl

max_body(3).
enable_recursion.

Also, the bk.pl has 2.1M facts. There are 1900 pos examples and 3800 neg examples.

[andrewcropper]

Can you please paste the full bias file here so we can look at it?

…

On 20 Sep 2022, at 11:24, navdeepkjohal ***@***.***> wrote: Hello, Thank you for replying back to me. I have tried the following setting in bias.pl max_body(3). enable_recursion. Also, the bk.pl has 2.1M facts. There are 1900 pos examples and 3800 neg examples. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[navdeepkjohal]

ismarriedto.zip

Hello Andrew,

Sorry about the delay in the response. We wanted to take some more time to set different flags to see if we could run the model on our own. The timeout error is not showing anymore and the Popper model runs on the data without error.

The entire dataset including the bias file is attached with this message. This is data extracted from YaGo11k dataset which is temporal dataset. As you know the KGs are huge and, at times, noisy, that's why we have not been able to learn rules from the model. We ran the model for 12 hours by setting the --timeout flag to 43200 but the model did not learn any rules. With lesser timeout value it returns 'no rule' and completes. We ran the code on Ubuntu installed on a VMWare. Please suggest any settings in the bias that could help us in obtaining rules on this dataset.

Many Thanks
Navdeep

[andrewcropper]

Hi, Popper is not well-suited to this problem. There is too much noise, i.e. the signal is too weak for Popper to learn meaningful rules. For instance, I looked at learning rules for each of the first few examples but there are no suitable rules in the search space. For this problem, it might be worth trying Aleph (https://www.cs.ox.ac.uk/activities/programinduction/Aleph/aleph.html). However, I doubt it will learn anything meaningful from this sample of data. I am curious, have you tried other learning systems? If so, has any learned a set of rules that generalises well on unseen data? Thanks, Andrew

…

On 21 Sep 2022, at 05:51, navdeepkjohal ***@***.***> wrote: ismarriedto.zip Hello Andrew, Sorry about the delay in the response. We wanted to take some more time to set different flags to see if we could run the model on our own. The timeout error is not showing anymore and the Popper model runs on the data without error. The entire dataset including the bias file is attached with this message. This is data extracted from YaGo11k dataset which is temporal dataset. As you know the KGs are huge and, at times, noisy, that's why we have not been able to learn rules from the model. We ran the model for 12 hours by setting the --timeout flag to 43200 but the model did not learn any rules. With lesser timeout value it returns 'no rule' and completes. We ran the code on Ubuntu installed on a VMWare. Please suggest any settings in the bias that could help us in obtaining rules on this dataset. Many Thanks Navdeep — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[navdeepkjohal]

Hello Andrew,

Thank you for replying back to me. Your makes sense.

Before using Popper, I used the ILP model available at https://github.com/starling-lab/BoostSRL/tree/development/src/edu/wisc/cs/will/ILP to learn the rules. It learnt the following rule as one of the rule for the ismarriedto predicate:

ismarriedto(A, B, C) :- wasbornin(A, D, E), before(E, C), diedin(B, F, G), diedin(A, F, H), finishes(H, C)

But I had hard time extracting the recursive rules from the above software, that's why I decided to move to Popper. I will try Aleph next.

Thank you for your help.

Best
Navdeep

[andrewcropper]

Is that rule not nonsense? I do not see how it generalises. Anyway, if you want recursive rules, then Popper is probably the most well-suited approach. However, there needs to be a strong signal / sufficient information to learn from.

…

On 21 Sep 2022, at 08:32, navdeepkjohal ***@***.***> wrote: Hello Andrew, Thank you for replying back to me. Your makes sense. Before using Popper, I used the ILP model available at https://github.com/starling-lab/BoostSRL/tree/development/src/edu/wisc/cs/will/ILP to learn the rules. It learnt the following rule as one of the rule for the ismarriedto predicate: ismarriedto(A, B, C) :- wasbornin(A, D, E), before(E, C), diedin(B, F, G), diedin(A, F, H), finishes(H, C) But I had hard time extracting the recursive rules from the above software, that's why I decided to move to Popper. I will try Aleph next. Thank you for your help. Best Navdeep — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[navdeepkjohal]

I agree with you that it is not a very strong rule. According to that software, the rule does not have to satisfy all the positive examples. If the coverage of the rule is above a certain threshold, then the model generates the rule as output.