Issues loading some OME-TIFF files
xinaesthete opened this issue · 13 comments
I'm trying to load this ome-tiff with AVIVATOR but it seems perhapsgeotiff is not interpreting the metadata appropriately. I'm not entirely clear on how I'd determine whether or not the file is a form that would be expected to be supported.
Actually, I'm now seeing that entering that URL directly in AVIVATOR is having problems because of CORS, but downloading the file and loading it locally has Error: Failed to parse OME-XML metadata.. It seems like it's internally trying to parse
ImageJ=
hyperstack=true
images=609
channels=3
slices=203
frames=1
as XML.
I've tried re-exporting with various different options with Fiji Bio-Formats plugin but haven't fully followed your documentation. It seems like it has Creator="OME Bio-Formats 6.11.0", which I would hope should conform.
Ok, so the issue here is that this does not appear to be an OME-TIFF, but an ImageJ tiff. I've had people have trouble with Fiji before for exporting. My best advice would be to run bioformats2raw and then raw2ometiff pipeline and then move from there, which it sounds like you may not have done (that's what's in our docs). That should automatically create an OME-TIFF since it runs a converter internally. You'll also get an image pyramid for free which is always nice. I don't use ImageJ that much, sorry. If you want to pursue this route though, someone on image.sc can certainly help you!
OK, many thank @ilan-gold and appologies for that - I did see the documentation suggesting bioformats2raw and raw2ometiff and was somewhat hesitant to post an issue without properly following through. I'll try that soon. Perhaps it might be worth noting explicitly that ImageJ exports are known to be problematic.
No worries! This stuff is super hairy and overwhelming! Just lmk if things work!
Irritatingly, it seems like there are no conda osx-arm64 binaries for bioformats2raw and last time I tried to sort this stuff out on this laptop I ended up wasting a bit of time and still not getting a useful result. I'll have a go on linux desktop tomorrow.
Hmmm... so I've been trying to convert starting with a nii file of that data, and I do get something that loads, but not in an entirely useful way. In Avivator, the non-volumetric rendering isn't showing anything at all, and the volumetric rendering shows a 3x3 grid. It's treating the data as signed for the contrastLimits etc. There are warnings in the console about tex(Sub)Image[23]D: ArrayBufferView type Int8 not compatible with type UNSIGNED_BYTE. when rendering 2d.
I was a little bit confused by what arguments are and aren't needed by bioformats2raw & raw2ometiff. The LuCa data you mention in the documentation now seems to work with no options apart from input and output file-names. I think the names of options (and ability to infer sane defaults) might have changed since that was written. For instance, bioformats2raw says it'll take -h, --tile_height=<tileHeight>, I don't see any mention of -tiley.
I've added --rgb to raw2ometiff, which doesn't seem to help. Also tried various --pixel-types for bioformats2raw, all with seemingly similar results.
I can put files somewhere accessible if that helps.
Interesting, we can handle signed data fine from what I remember although I haven't tried it in 3D. Yeah, if you want to put the files up somewhere I can curl them (or something), that'd be great (I won't have enough space locally to just hit download in a browser). Google drive works.
Could you check the OMEXML? It should be serialized to JSON in the browser console. Try to see if everything looks how you'd expect it to.
To be clear, I don't think the data should be signed, and I'm not sure why it's treating it as such - for instance, I tried --pixel-type=uint32 (as well as uint8).
An image generated something like
bioformats2raw brain.nii brain_raw --pixel-type=uint32
raw2ometiff brain_raw brain.ome.tiff --rgb --compression=LZW
is getting OME with BigEndian: true, Channels: Array(3), Type: "int8"... I don't know what raw2ometiff does with its --rgb option, but that seems to be the only relevant option it has and I think I get pretty similar result with or without it.
I'm getting warnings about An illegal reflective access operation occurred in bioformats2raw output, which I wasn't at first, so I don't know what's going on there or if it's confounding things somehow.
The files are relatively small (20-60mb depending on compression etc), I'll try to put them somewhere sensible... this is the nii looks like that's not there now.
Ok, let me know when it's posted and I'll look. Do you know what the data type on the nii file is? Is it 8 bit or is that an error? I have the ImageJ tiff from before but it also interprets your data as RGB, i.e I see Photometric Interpretation: RGB color. Do you know what assay produced your data? I'd be kind of surprised if it were natively 8 bit.
`
Thanks for helping with this. Managed to lose previous draft reply, and am still a little confused about some of what I’m seeing (or at what point things changed).
At the moment, I have at least got a couple of ome.tiff files rendering in Viv deployed on vercel, where they load ok. When running from a local server on my laptop and viewing in Firefox or Safari, I get 416 ‘range not satisfiable’ errors at consistent points in each of my two current test files (/t1-head-imj.ome.tiff fails early, and brain_uint32rgb.ome.tiff at 83.82%). I’m using Vite, and doing most of my development with the Vite dev server, although I also seem to see similar results with the preview server on my (M1 pro) laptop. That is, Chrome always loads the data ok, the other browsers always fail with the local server but work ok with the Vercel version.
I’m a bit confused as to how this relates to my previous experience from last week, or what causes that inconsistency in behaviour. This current problem is at least not a blocker for me, and far from clear that Viv is implicated.
I’m still waiting on real data to work with from collaborators on this project, and trying to establish that I’m able to process at least some form of data… sorry that I’m not a bit clearer on the form of the data; what I have is very much a placeholder, it seems like it is 8-bit-per-channel RGB, at least in the form that I got it in that nii.
The imj one is made with ImageJ, by loading the sample ‘T1 head’ and ‘Save As…/OME-TIFF’ (not using the BioFormats exporter). brain_uint32rgb was made from the nii file (which I hadn’t looked at properly earlier - I realise now that the 3x3 grid is not an artefact - that did seem a particularly odd glitch), with raw2ometiff → bioformats2raw.
Amazing! That looks so good, the UI is beautiful! Great stuff! I'll have a look at the two files, but I know for a fact (unless something has changed....which would be unfortunate) that the blocks that geotiff.js fetches of the tiff file should not overrun the size of the file if the size of the file is attached as part of the returned request. So it's possible that your server is neither returning the size of the file nor handling overruns, which is kind of terrible! Really not much we can do about that unfortunately, I think.
For example when you look at the content-range returned from wherever you are hosting the data remotely, you'll see the range and then a slash after which is the total size of the file:
Is this present locally?
I would put my money on this being the problem since I have no issues with the headers using http-server --cors="*": https://www.npmjs.com/package/http-server
As for your data, I would try to ascertain (1) where the data is coming (seems like fmri) and (2) why they are giving it to you in this format i.e determine any pre-processing they might be doing and why. I would be somewhat suspicious that there are actually 3 channels of fmri data of type uint8 - it sounds like they are processing the data into this format which you probably don't want. That's one of the selling points of viv - we can view your "raw" data. It's possible I'm wrong here, but this is where I'd start.
For example I went here, downloaded a random dataset, and ran bfconvert. It took about 2 minutes. There are not 3 channels and they are not uint8 (notice how the max value is >255 in the hovered slider):
Closing for inactivity.