natverse/nat.flybrains

Add JRC2018U temblate brain

tomka opened this issue · 11 comments

tomka commented

It would be nice if the JRC2018U (unisex) template brain could be included. I added a bridging registration to the BridingRegistration package, but need a templatebrain object to use it with some nat functionality (e.g. writing NRRD files) in order to have meta-information like the voxdims available.

I tried to add this templatebrain object following the existing JRC2018F example, but couldn't get R to make this object available and use it when the bridging registration is used. Therefore I add this as an issue here including the information I collected so far.

This is the information I get when loading the reference brain NRRD file with as.tenplatebrain():

=== Template Brain ===                
Name: JRC2018_UNISEX_20x_HR           
Short Name: JRC2018_UNISEX_20x_HR     
Type:                                 
Sex:                                  
Dimensions:1210 x 566 x 174 voxels    
Voxel size:                           
  x = 0.518916129032258 microns       
  y = 0.51891610619469 microns        
  z = 1 microns                       
Bounding box (microns):               
  x = 0, y = 0, z = 0,                
  x = 627.3696, y = 293.1876, z = 173.
Description:

I also extraced the NRRD header of the original JRC2018U reference NRRD file (data-raw/RC2018_UNISEX_20x_HR.nrrd):

NRRD0004
# Created by Nrrd_Writer at Thu Nov 29 15:08:01 EST 2018
type: uint16
encoding: gzip
endian: big
dimension: 3
sizes: 1210 566 174
space dimension: 3
space directions: (0.5189161,0,0) (0,0.5189161,0) (0,0,1.0)
space units: "microns" "microns" "microns"

This is the templatebrain object I added to data-raw/jrc2018.R, based on JRC2018F:

JRC2018U <-
  as.templatebrain('data-raw/JRC2018_UNISEX_20x_HR.nrrd',
    regName = 'JRC2018U',
    type = 'Average template brain',
    sex = 'U',
    doi = 'https://doi.org/10.1101/376384',
    description="Average template brain constructed from JRC2018F and JRC2018M."
  )
use_data(JRC2018U, overwrite = T

This didn't work though, the JRC2018U templatebrain wasn't available after installing a new version of this package. I likely forgot to add something, but figured this might be easier solvable here. Also, I am not sure the meta information above is correct, I merely guessed what it could be.

jefferis commented
jefferis commented
tomka commented

Thanks Greg, I will find out if there are other variants of the image data and make them available. Also, I found the problem with my first attempts above: I didn't realize I had to add the data/JRC2018U.rda file. After fighting save/load compatibility problems between different R versions, this seems to work now: I can export NRRD files of data in this space without errors (mirroring fails for FAFB14 -> JRC2018U though, but it seems this might not be needed here anyway. Improving the documentation is still on the to-do list, but I got roxygen2 to work as well as pkgdown in case that would be needed. Given that my background knowledge on how this template space came about is rather limited, the description in the docs will be rather short though. I suppose I can file a PR once all this is done.

jefferis commented

Hi, Tom. That all sounds good. Sorry I didn't guess the problem. Normally if you do use_data then the object is added to the data folder. I would be happy to add this template (it was on my todo), so you could do a PR if you would like.

tomka commented

Hi Greg, I was talking to @bogovicj and he kindly provided me with the NRRD headers of both the original files and their downsampled versions. It looks like there are the following variants for the female, male and unisex versions: 20x_gen1_iso, 20x_gen1, 20xHR_iso, 20xHR, 38um_iso, 40x, HRaniso, HRiso and p88umiso. According to John, the HRiso files are the non-donsampled ones. They look loke this for all three types:

JRC2018_FEMALE_HRiso.nhdr:

NRRD0004
# Created by Nrrd_Writer at Mon Apr 09 13:52:02 EDT 2018
type: float
encoding: raw
endian: big
dimension: 3
sizes: 3333 1550 965
space dimension: 3
space directions: (0.188268,0,0) (0,0.188268,0) (0,0,0.188268)
space units: "microns" "microns" "microns

JRC2018_MALE_HRiso.nhdr:

NRRD0004
# Created by Nrrd_Writer at Mon Oct 29 11:55:37 EDT 2018
type: float
encoding: raw
endian: big
dimension: 3
sizes: 3150 1500 960
space dimension: 3
space directions: (0.188268,0,0) (0,0.188268,0) (0,0,0.188268)
space units: "microns" "microns" "microns"

JRC2018_UNISEX_HRiso.nhdr:

NRRD0004
# Created by Nrrd_Writer at Mon Apr 09 13:08:09 EDT 2018
type: float
encoding: raw
endian: big
dimension: 3
sizes: 3333 1560 919
space dimension: 3
space directions: (0.188268,0,0) (0,0.188268,0) (0,0,0.188268)
space units: "microns" "microns" "microns

I checked how the existing JRC2018F template brain is defined and it seems it is based on the JRC2018_FEMALE_38um_iso_16bit.nrrd file in the repo. It's content matches the file I got from John (JRC2018_FEMALE_38um_iso.nhdr), except for the datatype (here float, in the repo uint16):

NRRD0004
# Created by Nrrd_Writer at Tue Apr 10 13:36:13 EDT 2018
type: float
encoding: raw
endian: big
dimension: 3
sizes: 1652 768 479
space dimension: 3
space directions: (0.38,0,0) (0,0.38,0) (0,0,0.38)
space units: "microns" "microns" "microns"

The same variant (38um_iso) for both the male and the unisex version follow below.

JRC2018_MALE_38um_iso.nhdr:

NRRD0004
# Created by Nrrd_Writer at Mon Oct 29 13:54:46 EDT 2018
type: float
encoding: raw
endian: big
dimension: 3
sizes: 1561 744 476
space dimension: 3
space directions: (0.38,0,0) (0,0.38,0) (0,0,0.38)
space units: "microns" "microns" "microns"

JRC2018_UNISEX_38um_iso.nhdr:

NRRD0004
# Created by Nrrd_Writer at Tue Apr 10 12:08:16 EDT 2018
type: float
encoding: raw
endian: big
dimension: 3
sizes: 1652 773 456
space dimension: 3
space directions: (0.38,0,0) (0,0.38,0) (0,0,0.38)
space units: "microns" "microns" "microns"

If you are happy with one these headers, I could go ahead and create the necessary definitions for this R package (and replace the existing JRC2018U I came up with and create a PR. I could also create the JRC2018M variant while I am at it (if this is useful). Let me know if you need more information

Could I just use the bridging registration in jefferislab/BridgingRegistrations#7 with the new definition of JRC210U as well?

  • 👍1
jefferis commented

Dear @tomka thanks a lot for digging into this (and @bogovicj for providing the details). It would be great if you could make a template definition for the JRC2018U/JRC2018M based on the 0.38 µm images. It would be perfect if you could include the nhdr files in data-raw.

jefferis commented

However this does not completely solve the issue of using the bridging registration! The thing is that we do not currently distinguish between what you might call an abstract space (to coin a term) and multiple template images (with different dimensions) that might all be consistent with that space. We probably should, but we would need to think about it. Furthermore amongst the different registrations running around at Janelia in particular, there are cases where the same abstract space also has templates with incompatible dimensions, typically ones where the Z dimensions have squashed or stretched.

jefferis commented

What this means in practice is that the registration that you have already added should work for point data but not image data. It is true that is less likely that people will use image data, but it is still sometimes the case. I think for now, I would therefore prefer to go with a separate template brain definition for each case where we are likely to encounter image data. This means that you would

  1. need to define separate template brain objects for the the 20x templates. I would suggest calling it JRC2018UDS by analogy with some others.
  2. you would need to update jefferislab/BridgingRegistrations/pull/7 to use JRC2018UDS not JRC2018U as the name.
  3. If you were up for it you could add JRC2018FDS and JRC2018MDS for good measure.

This would be a very helpful addition if it's good with you.

jefferis commented

Dear @tomka, I realise that we have some unfinished business here and I'm not sure how you ended up handling this for CATMAID. In any case I have now merged feature #41 which adds the JRC2018U template in what I consider canonical form. This can be used with the high quality registrations from @bogovicj, which are now available via the nat.jrcbrains package. This would be my recommendation and I will therefore close this issue. You can see some more discussion here:
https://groups.google.com/forum/#!topic/nat-user/zNaCyQnZeVg

tomka commented
jefferis commented

OK! Note that the new approach is

  1. more accurate
  2. more robust
    but is also
  3. slower
  4. with additional dependencies.

I hope that 3/4 will not prevent you from using this approach. The dependencies now include a JDK and the hdf5 library. Let me know if you have any trouble.