refactor: organize repo as standard-GEM
Closed this issue · 13 comments
Description of the issue:
Efforts are made to standardize the format of [Git(Hub)] repositories containing genome-scale models: standard-GEM. This is being discussed and formalized at https://github.com/MetabolicAtlas/standard-GEM. yeast-GEM largely already complies with standard-GEM, so applying the latest guidelines will not result in drastic changes.
Any comments on the standard-GEM format should be discussed on that repo: https://github.com/MetabolicAtlas/standard-GEM/issues.
Main changes:
Note that this is a non-exhaustive list, and standard-GEM is still under development.
- Rename folders (
ComplementaryScripts-->code;ComplementaryData-->data;modelFiles-->model). - Replace folder names in all scripts (should be no problem due to uniqueness of the old folder names).
- Refactor minor changes
README.md. - Additional changes specified in
.standard-GEM.mdtemplate.
@edkerk @mihai-sysbio I realized a problem (in this repo) with changing /modelFiles to /model: A condition for the memote report to work properly when showing the project's history is that the model's .xml path remains unaltered: So if we would change the model path, we would only be able to see improvements in memote from that moment onwards, losing our previous history. Do you think this is worth it? Personally I don't have strong feelings towards either direction, but I'm wondering which would be the advantage of having the path be /model instead of /modelFiles (other than it looks much better)
/model looks better indeed :), but it would also be good to have yeast-GEM follow the standard-GEM standard. Would it work if we include a symbolic link of modelFiles pointing to model for the purpose of memote?
but it would also be good to have yeast-GEM follow the standard-GEM standard
I guess that was my question: what would be the advantage of following said standard? It will probably take some extra work on the yeast-GEM side (solving this memote problem + some conflicts with master), so would be good to know the benefits of doing it :)
Would it work if we include a symbolic link of modelFiles pointing to model for the purpose of memote?
No idea, but I'm not a fan of having both folders model and modelFiles, IMO would look confusing.
@Midnighter would it be possible to adapt the travis deploy script to catch this file migration when computing the history report? e.g. try to run memote on model/yeastGEM.xml and if not possible try with modelFiles/yeastGEM.xml? If not, is it a feature that you have considered adding?
Let me see if I understood the issue: the path to the .xml file is in the memote.ini. If the /modelFiles gets renamed to /model, that change would have to be reflected into an update in that file.
When running
memote report history
is the behaviour that the latest version of the model gets added to the database, or that all previous versions are run sequentially?
Also, from what I'm seeing in the gh-pages branch, neither history_report.html nor results.db seem to have been updated. Maybe it wouldn't be such a bad time to migrate to GitHub Actions (not that it would be required in any way, but it would be part of the same platform).
@BenjaSanchez following a standard that is adopted across open-sourced GEMs facilitates automation, be it integration into different websites or workflows. If we were to imagine that such a standard is already in place, the standard could eg be taking care of running Memote on all releases in that repo that follow said standard, without expecting users to understand the intricacies of having automation and gh-pages branches and a Travis account. So by joining a standard, everybody who sees such a GEM repository can expect the same thing, while benefiting from a simple setup. In essence, it's not too far from having an SBML format, but this is one level of abstraction higher.
@mihai-sysbio sounds indeed very nice. As the shift from modelFiles to model is rather quick to do on the yeast-GEM side, perhaps an easy solution would be to keep modelFiles as it is right now, and to make the transition to model as soon as the plans you mention become implemented? That will give us more time to figure out the proper memote implementation :)
is the behaviour that the latest version of the model gets added to the database, or that all previous versions are run sequentially?
not really sure how it works: maybe @Midnighter can shed some light here. Hopefully it is the former, so that we can run the history for everything so far in /modelFiles, and it stays there once we switch to /model
from what I'm seeing in the gh-pages branch, neither history_report.html nor results.db seem to have been updated
yes, as the branch of memote has still not been merged (#162 was in standby). I will be working on re-implementing it in the coming weeks now that BiGG ids are implemented and the model can be properly loaded with cobrapy.
I am great fan of how yeast-GEM is on the forefront of open-sourcing, and how it has inspired other GEMs to follow suit. And yeast-GEM already follows a setup that standard-GEM is based on, so the adoption should be pretty straightforward (short to-do list in OP). But, to be direct, the current setup is closed-source, which is painful for others to copy-paste, and prone to errors (forgot to copy file, or misnamed file - I've heard this directly from other GEM authors). With the formalization of the adoption of a standard, it would be a lot easier yeast-GEM to continue to inspire other GEMs, like it has already done for Human-GEM.
@BenjaSanchez the automated validation mentioned above in the thread needs some repositories to start adopting the standard (chicken and egg). It looks like Human-GEM might take the lead on standardization - it has already enabled proof of concept standard-GEM automated validation with Actions. These is also an issue for automated validation to contribute to - any feature there (eg inspired from #162) will benefit all GEMs following the standard.
Another core feature of the yeast-GEM setup I value is the git flow, with develop and feat branches allowing work to be integrated gradually into master. This discussion of folder renaming makes a lot of sense once we understand the implications of merging into master, and potentially breaking feat/memote, but it shouldn't make any difference for the current branch.
@mihai-sysbio nice to see there is some automated validation starting in Human-GEM, would be good to have that here as well. As soon as it is safe to switch from modelFiles to model we can proceed :)
After a call with @Midnighter I confirmed that indeed we can switch the model file path, but only after memote is running in the repo (as otherwise the history report command won't work for all of the old commits). Currently there's issue opencobra/memote#689 that needs to be handled but as soon as that is done we can move forward.
@mihai-sysbio I like your idea of moving from Travis CI to GH actions (I read online that it performs faster, which is crucial for our setup), however I have no experience with the latter. Would you be interested in helping me to set it up? We can have a call to discuss the nuts and bolts ;)
With opencobra/memote#689 closed, yeast-GEM can be transferred to standard-GEM?
Will close when merged with master