sys-bio/roadrunner

Roadrunner allows L2 events to fire at t0.

Closed this issue · 6 comments

eltix commented

Issue description

Simulating several times the same SBML model with simulate() yields different results.

To reproduce

Download this SBML document

In a Python script or interpreter:

from roadrunner import RoadRunner
r = RoadRunner("BIOMD0000000087_url.xml")
r.simulate(selections=["recovery"], times=[0,1,2,3,4,5,6,7,8,9,10])

Run the above 3-4 times and you should see recovery arbitrarily take values 0 or 1.

My two-cents

To be confirmed but I suspect roadrunner's event scheduler. In this SBML model there are 9 events updating each other's trigger variables leading a mind-boggling event cascade. If several events are triggered at the same time and the order in which the events are resolved (i.e. the eventAssignments are applied) is not reproducible this could explain what we observe.
Again, this is just an intuition and it could very well be untrue.

Versions

  • Python 3.8.10
  • roadrunner 2.2.0
luciansmith commented

Thanks for the report!

So, the cause of this turns out to be a combination of things:

  • There are two events in particular that are both true at the beginning of the simulation
  • They both assign a different value to 'recovery'.
  • The model is encoded in SBML Level 2

If I convert the model to Level 3 (either L3v1 or L3v2), the problem goes away: roadrunner realizes that no event is supposed to fire at t0, because nothing changes from false to true then (without the L3 flag that says it's false until calculated otherwise).

It is also simply the case that some models are going to be not reproducible when they depend on events firing in a particular order, without 'priority' values (which can only be set in L3). I'm not sure if that problem normally arises in this model or not; it might be OK, if the only conflict is at t0 when nothing (in L2) is supposed to fire.

So, an immediate fix to this particular issue is to up-convert the model to level 3. But I also need to fix the bug in roadrunner that's letting L2 models fire events at t0, which should be impossible.

eltix commented

Thank you so much @luciansmith for taking a look at this.
I will ask the Biomodels team whether it is possible to replace the model with its L3V2 counterpart.

luciansmith commented

I've actually found and fixed the bug in roadrunner that makes this happen, I just haven't released the new version of roadrunner yet. If you want to grab the wheel file for your platform with the fix, you can find it at

https://dev.azure.com/TheRoadrunnerProject/roadrunner/_build/results?buildId=1933&view=artifacts&pathAsName=false&type=publishedArtifacts

or you can wait for a new release of roadrunner. Thanks again for the report!

eltix commented

@luciansmith We use nix to manage our dependencies, can we use commit f6b4684 instead?

luciansmith commented

Yup! It's failing for a completely different reason (the VM is running out of space; still trying to figure out how to fix that.)

luciansmith commented

OK, finally got the build system back up and running, and folded the fix into develop. The next release will have the fix incorporated. Thanks again!

  • 👍1