missing 0th frame in .xml after "Export mamut"

[xulman]

Hi,
this dataset consists of frames t1.tif till t9.tif with timepoints 1 till 9 (according to its DBV's dataset.xml. However, when opened as "New project" in Mastodon, the timepoints are treated 0 till 8 (in BDV as well as in TS). Now, when exporting into Mamut's .xml, the spot definitions are with timepoints 1 till 9, e.g.
<SpotsInFrame frame="9" />,
and definitions of spots from the first frame are missing (but their ID's are referenced from definitions of tracks). Also the
<AllSpots nspots="154">
counts all spots and is larger than number of lines with individual spot definitions in the created Mamut's .xml.

[xulman]

It gets even more funky when input dataset.xml consists of timepoints e.g. 5,6,7 (then 0,1,2) is (attempted to get) exported...

[xulman]

I've patched that in a very nasty way with these 3 commits, check out their commit messages

[tinevez]

Hi @xulman
Tobias and I rewrote a very large chunks of mastodon core and merged it to master recently.
Since this work focused on numerical features, there are little chance that it fixed this bug, but just to be sure, would you mind trying again?
Thanks!

[tpietzsch]

@tinevez The correct solution would be to use timepoint indices when querying the Mastodon model (and using timepoint ids (==names) when referencing them in the XML).

So this

final SpatioTemporalIndex< Spot > spots = model.getSpatioTemporalIndex();
for ( final TimePoint tp : tps )
{
	final Element frameSpotsElement = new Element( SPOT_FRAME_COLLECTION_TAG );
	frameSpotsElement.setAttribute( FRAME_ATTRIBUTE, tp.getName() );

	for ( final Spot spot : spots.getSpatialIndex( tp.getId() ) )
	{
		final Element spotElement = spotToXml( spot );
		frameSpotsElement.addContent( spotElement );
	}
	spotCollectionElement.addContent( frameSpotsElement );
}

should become

final SpatioTemporalIndex< Spot > spots = model.getSpatioTemporalIndex();
for ( int i = 0; i < tps.size(); i++ )
{
	final TimePoint tp = tps.get( i );

	final Element frameSpotsElement = new Element( SPOT_FRAME_COLLECTION_TAG );
	frameSpotsElement.setAttribute( FRAME_ATTRIBUTE, tp.getName() );

	for ( final Spot spot : spots.getSpatialIndex( i ) )
	{
		final Element spotElement = spotToXml( spot );
		frameSpotsElement.addContent( spotElement );
	}
	spotCollectionElement.addContent( frameSpotsElement );
}

and maybe other places too. Could you take care of it?

[tinevez]

Yes.

[tinevez]

@xulman I am lazy, can you re-upload the dataset that allows to test the solution? Thanks.

[xulman]

Hi @tinevez,
thanks for working on it... here's the new testing folder:

Dataset is defined on timepoints 5 till 8, and the Mamut export seems to keep trying (polling) some other timepoints as the output .xml contains this:

    <AllSpots nspots="8">
      <SpotsInFrame frame="5" />
      <SpotsInFrame frame="6" />
      <SpotsInFrame frame="7" />
      <SpotsInFrame frame="8" />
    </AllSpots>

[xulman]

BTW: have you tried in Mastodon to add linked spot beyond the last frame?
(just in the test dataset, move to the last timepoint, move mouse over some spot in BDV, press A (add linked spot) and it gets a bit crazy... would expect nothing to happen actually)

[tinevez]

And this, Vlado, is a new issue :)
Thanks for reporting. Can you submit it since you have found it?

[tinevez]

Right,
There are still some issues even with @tpietzsch fix. I try to list the situation here.

Whatever you do, in Mastodon, we always display the timepoint indices, not their names. This is the case for image and track data.

In your BDV data set, the first time-point is 5, so

• its name is "5"
• its index is 0.

In Mastodon (& MaMuT actually), the BDV opens it as the first frame and displays "t = 0" in the status. If you create a spot in this frame, it will receive timepoint = 0. The same goes for TrackScheme.

So all of the timepoints names are 'forgotten'. The 't' simply lists timepoint indices in order.

First question: do we want to change that?

[xulman]

I (as a user, if you will) do not insist on preserving the original time point names. Since both TS and BDV is showing indices, I hasn't even occurred to me that I could have seen names and so I was not expecting time point names in any output...

[tpietzsch]

First question: do we want to change that?

No. I would keep it like it is. My feeling is that it will only get more complicated if we switch to timepoint names. Let's solve it in the export.

[xulman]

@tinevez, I had a look on it again and came to the same conclusion as @tpietzsch above, do you recall what was not working on it for you?
I've now tried on the fake dataset above and also on some production data (42k spots), and it works well...
I'll create a PR :)