Study 784 Experiments not showing but exist

Question

Study 784 Experiments not showing but exist

Closed this issue 2 months ago · 43 comments

https://bugsigdb.org/Study_784 shows no experiments on the page but they do exist: https://bugsigdb.org/Study_784/Experiment_1

I tried refreshing the page under ... > Refresh but that did not resolve the issue.

tosfos commented 7 months ago

Checking

Answer 1 · 2024-03-12T17:10:25.000Z

This probably has the same cause as #196. We'll check this Study in particular, but a permanent solution has been elusive.

Answer 2 · 2024-03-12T23:05:08.000Z

We re-saved the experiment page and it's showing up now. I'm not sure if you tried to refresh the Experiment page or only the Study page. In the future, that would be more likely to succeed.

We're going to keep researching how this happened.

Answer 3 · 2024-05-30T13:15:42.000Z

@tosfos I'm not sure if this is the same bug but we're seeing something similar for https://bugsigdb.org/Study_1069 where https://bugsigdb.org/Study_1069/Experiment_3 and https://bugsigdb.org/Study_1069/Experiment_4 are not displaying. I tried refreshing both the study page and the experiment page and it's still not showing.

Answer 4 · 2024-06-01T21:12:56.000Z

@cmirzayi @tosfos
Experiments 7 and 8 also have this issue
https://bugsigdb.org/Study_1079

Answer 5 · 2024-06-03T12:06:09.000Z

Hello @cmirzayi
I am currently having the same issue. Experiments 4,5 and 6 are not showing but they exist.
Study link: https://bugsigdb.org/Study_940

Answer 6 · 2024-06-05T17:24:31.000Z

Hello,

In this study, https://bugsigdb.org/Study_1013, experiments and signatures intermittently failed to display. Reloading the page sometimes resolves this issue, but sometimes they don't display after reloading.

Answer 7 · 2024-06-06T18:10:20.000Z

Signatures for experiments 5, 6 and 7 are not displaying but exist; https://bugsigdb.org/Study_950

Answer 8 · 2024-06-10T12:02:18.000Z

Signature 2 of experiment 1 in this study, https://bugsigdb.org/Study_901, is not also displaying but exist.

Answer 9 · 2024-06-10T16:41:31.000Z

This study is also having the same issue;
https://bugsigdb.org/Study_915

Experiments 3 and 4 and their signatures are not showing but exist.

Answer 10 · 2024-06-11T17:47:44.000Z

Signature 2 of experiment 1 here is not displaying;
https://bugsigdb.org/Study_890

Answer 11 · 2024-06-18T12:54:55.000Z

Signature 1 of experiment 1, 2 and 3 are not displaying for this study;
https://bugsigdb.org/Study_1054

Answer 12 · 2024-06-20T11:17:41.000Z

Signature 2 of experiment 4 is not displaying for this study;
https://bugsigdb.org/Study_1026

Answer 13 · 2024-06-20T17:02:58.000Z

Experiments 2, 3 and 4 with their signatures are not displaying in this curation;
https://bugsigdb.org/Study_916

Answer 14 · 2024-06-20T19:28:50.000Z

We'll check on it and see how we can resolve all of these issues permanently.

Answer 15 · 2024-06-25T14:25:20.000Z

Signature 2 of experiments 1 and 3 are not displaying in this study;
https://bugsigdb.org/Study_1009

Answer 16 · 2024-06-30T15:54:30.000Z

Signature 1 of experiment 6 is not displaying in this study;
https://bugsigdb.org/Study_997

Answer 17 · 2024-07-05T09:21:32.000Z

Experiment 2 and its signature 1 is not displaying for this study;
https://bugsigdb.org/Study_1062

Answer 18 · 2024-07-18T13:16:07.000Z

I checked all of these and they seem to be appearing now so closing this issue.

Answer 19 · 2024-08-02T12:39:32.000Z

@Scholarpat is facing the same issue again. Scholastica, please report it here.

Answer 20 · 2024-08-02T12:45:36.000Z

Some signatures are not displaying in this study;
https://bugsigdb.org/Study_1018

Signature 1 and 2 of experiment 2
Signature 1 of experiments 3, 4 and 5

Answer 21 · 2024-08-05T11:50:51.000Z

Signature 1 of experiments 11, 12 and 13 are not displaying in this study;
https://bugsigdb.org/Study_943

Answer 22 · 2024-08-05T16:44:01.000Z

We have a fix in progress. We're hoping it will be resolved by Thursday.

Answer 23 · 2024-08-13T10:46:12.000Z

Signatures 1 and 2 for Experiment 13 and Signature 2 for Experiment 14 in this study is not visible
https://bugsigdb.org/Study_1086

Answer 24 · 2024-08-13T14:35:46.000Z

I don’t know if this is a temporal fix or if it will work for other studies with same issues, but I noticed, when I edited an experiment with invisible signatures and saved the page the signatures appeared.

Currently all the signatures are visible for this study https://bugsigdb.org/Study_1086

Answer 25 · 2024-08-16T09:49:55.000Z

I added a signature for experiment 7 but it's not showing
https://bugsigdb.org/Study_1079

Update *
So, I tried out what Anashie said she did and yes the signature displays. I'm concerned this study would have more issues like this because it is still under the category "Pages using DynamicPageList3 parser function"
Not sure if this should also display in the study

Answer 26 · 2024-08-28T22:22:56.000Z

We're in a catch-22 here. None of the versions of the extension we use for dependency updates will cover all situations. But the present situation is clearly not workable. We're going to update to the latest version of the extension which will work better overall, and then we'll improve it.

Answer 27 · 2024-08-29T14:59:11.000Z

The server is being restarted as part of this work. It will be down for a couple of minutes.

Answer 28 · 2024-08-30T18:33:22.000Z

This issue should be basically resolved. I'll follow up with more details.

Answer 29 · 2024-09-09T17:02:05.000Z

This issue is continuing. Today's export is missing nearly 1000 records. This bug has become a critical issue that is impacting our ability to work with the BugSigDB data.

Answer 30 · 2024-09-09T17:47:47.000Z

Please provide more details. Is it the Signature CSV?

Answer 31 · 2024-09-09T19:49:00.000Z

Signatures.csv
We are seeing that the CSV download link is returning a different number of rows each time it is run. I attached an updated CSV that has all the Signatures. We'll research a better solution.

Answer 32 · 2024-09-10T13:48:07.000Z

It looks like we were running out of resources on the server. We raised the resources and it looks like that alleviates the issue. I just generated a CSV for the Signatures that downloaded the full set.

Answer 33 · 2024-09-10T19:34:18.000Z

@tosfos Thank you so much. It does seem to be resolved. I was wondering if/when (hopefully) BugSigDB doubles in size over the next few years, are we going to run into this issue again?

Just wondering if there's any groundwork or processes we can put into play in the near to mid-future to prevent this from becoming a problem a few years down the line?

Answer 34 · 2024-09-10T20:54:52.000Z

I think that @jwokaty's idea of tracking export sizes with an alert if thing break is a good one. See waldronlab/BugSigDBExports#34 for discussion.

Answer 35 · 2024-09-10T21:30:10.000Z

Thanks Sean. I agree. Jen and I met earlier today to discuss this issue and how we can detect it faster going forward. They and I are working on implementing some detection solutions for the exports and bugsigdbr. However, since this seems to have stemmed from a resource issue upstream of the exports, I wanted to check to see if there was anything else we can do to ameliorate this once and future issue upstream as well.

Answer 36 · 2024-09-25T19:41:05.000Z

@tosfos Unfortunately the issue is not resolved. I just noticed the signatures.csv is once again returning differing rows each time it's run. I ran it in the past 10 minutes and got 2081 rows, 4106 rows, and 5352 rows.

Answer 37 · 2024-09-25T23:31:35.000Z

We adjusted the settings again and did a restart. It looks like the problem is resolved for now, but we need to do more research because it seems our solution is not permanent.

Answer 38 · 2024-09-29T14:55:50.000Z

We believe a permanent solution is in place as of yesterday afternoon. We're still monitoring.

Answer 39 · 2024-10-02T20:40:08.000Z

Have any issues been seen since 9/28?

Answer 40 · 2024-10-02T20:49:38.000Z

Not that I have observed. @jwokaty you are more closely monitoring this. Anything you've noticed?

Answer 41 · 2024-10-02T20:55:24.000Z

No issues yet. Let's watch it over the next two weeks.

Answer 42 · 2024-11-14T19:10:25.000Z

Let's close this and create a new issue if anything happens.