Data missing and provenance problems
Opened this issue · 0 comments
Hey Adrian, I found some problems with the webservice and I have some suggestions:
-
Some projects are missing in the public projects tab (https://orthology.benchmarkservice.org/proxy/projects/2020/)
that are present in the public results tab (https://orthology.benchmarkservice.org/proxy/results/2020/)
For example SonicParanoid2-sens, SonicParanoid2-... (only SonicParanoid is present) -
The names of the tools do not perfectly match (e.g. SonicParanoid_sensitive = sonicparanoid-sens) between the projects and results tab. Maybe crosslink them, so there is no confusion or use short IDs.
-
Version numbers are missing for almost all projects (e.g. Domainoid+ has no tool version assigned).
- idea: you could add this as a requirement field when submitting to the database
- Parameters are inconsistent, some projects are well described and others not.
- idea: make the full command that generates the output a requirement to submit data, so that others can reproduce the results
- I tried to reproduce the SonicParanoid-sens results with the description but got very different results, probably different versioning or some other post-processing is missing...
- RBH/BBH: what is the program that generates these results? My RBH graph with 99% similarity and the specified e-value differs from this result in the benchmarks.
-
It is clear to me that *-g represents a grouped upload, but maybe separate the results or make it clearer what is a paired upload and what is a grouped one.
-
The 'EnsemblCompara-e56' has multiple entries and some of the data links go to your paper instead of the data uploaded for this result
Maybe this helps for the next qfo release!