ccpem/mrcfile

Unable to fix mrc file header

GenevieveBuckley opened this issue · 3 comments

GenevieveBuckley commented

Hi! I've been trying to fix the header of an mrc file, but the example in the docs doesn't seem to be working for me.

I know you can just ignore this type of error using the permissive=True keyword argument, but I wanted to fix the file properly, since it was causing a bit of confusion.

Details

Operating system: M1 Mac Monteray 12.3.1 and also tried on Linux Ubuntu 20.04 LTS
python version: 3.9
mrcfile version: 1.4.2

Data

I've been working with this example data, untarred with tar -xzvf examples.tar.gz
This data was a segmentation exported from Amira, which apparently doesn't produce a very nice file header in the mrc files.

To reproduce

I used this example from the mrcfile documentaion

  1. First, open the mrcfile and observe the error
import mrcfile
with mrcfile.open(filename) as mrc:
    print(mrc.data)
# ValueError: Map ID string not found - not an MRC file, or file is corrupt
  1. Then, attempt the header fix suggested in the mrcfile docs
with mrcfile.open(filename, mode='r+', permissive=True) as mrc:
    mrc.header.map = mrcfile.constants.MAP_ID
  1. Try opening the (hopefully fixed) file again
with mrcfile.open(filename) as mrc:
    print(mrc.data)
#  ValueError: Unrecognised machine stamp: 0x90 0x42 0x4d 0xc3

I also tried this suggestion in the comment

import mrcfile
with mrcfile.open('<name>', mode='r+', permissive=True) as mrc:
    mrc.update_header_from_data()

This second option did not seem to update the header at all, because when I tried to read the file back in again, I got the same VAlueError about the Map ID string as I did originally.

Related discussions

GenevieveBuckley commented

Additional information: here is what the file headers look like immediately after untarring the example data.

In [1]: import mrcfile

In [2]: mrcfile.__version__
Out[2]: '1.4.2'

In [3]: with mrcfile.open("TE1.mrc", permissive=True) as mrc:
   ...:     print(mrc.header)
   ...:
/Users/genevieb/mambaforge/envs/morphometrics/lib/python3.9/site-packages/mrcfile/mrcinterpreter.py:209: RuntimeWarning: Map ID string not found - not an MRC file, or file is corrupt
  warnings.warn(msg, RuntimeWarning)
/Users/genevieb/mambaforge/envs/morphometrics/lib/python3.9/site-packages/mrcfile/mrcinterpreter.py:219: RuntimeWarning: Unrecognised machine stamp: 0x90 0x42 0x4d 0xc3
  warnings.warn(str(err), RuntimeWarning)
(960, 928, 312, 0, 0, 0, 0, 960, 928, 312, (11942.4, 11544.32, 3881.28), (0., 0., 0.), 1, 2, 3, 0., 3., 0.00433542, 0, 0, b'\x00\x00\x00\x00\x00\x00\x00\x00', b'', 0, b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x20\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', (0., 0., 0.), b'|\x94\xf2\xc4', [144,  66,  77, 195], 195.93001, 0, [b'', b'', b'', b'', b'', b'', b'', b'', b'', b''])

In [4]: with mrcfile.open("TF1.mrc", permissive=True) as mrc:
   ...:     print(mrc.header)
   ...:
/Users/genevieb/mambaforge/envs/morphometrics/lib/python3.9/site-packages/mrcfile/mrcinterpreter.py:209: RuntimeWarning: Map ID string not found - not an MRC file, or file is corrupt
  warnings.warn(msg, RuntimeWarning)
/Users/genevieb/mambaforge/envs/morphometrics/lib/python3.9/site-packages/mrcfile/mrcinterpreter.py:219: RuntimeWarning: Unrecognised machine stamp: 0x00 0x00 0x00 0x00
  warnings.warn(str(err), RuntimeWarning)
(928, 960, 350, 0, 0, 0, 0, 928, 960, 350, (12435.199, 12864., 4690.), (0., 0., 0.), 1, 2, 3, 0., 3., 0.00568473, 0, 0, b'\x00\x00\x00\x00\x00\x00\x00\x00', b'', 0, b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x20\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', (0., 0., 0.), b'', [0, 0, 0, 0], 0., 0, [b'', b'', b'', b'', b'', b'', b'', b'', b'', b''])
colinpalmer commented

Hello, and thanks for the thorough and well explained report of the issue. I wonder if the problem is simply that you need to apply both of the fixes you tried at the same time? update_header_from_data should fix the machine stamp but it doesn't change the map ID field. If you call update_header_from_data and also reset the map ID, it could be enough to give you a valid file. Give that a try, and if it still doesn't work let me know and when I have a chance I'll take a look and see if I can see what's going on.

GenevieveBuckley commented

Ah, you're right! This works:

with mrcfile.open(filename, mode='r+', permissive=True) as mrc:
    mrc.header.map = mrcfile.constants.MAP_ID
    mrc.update_header_from_data()

Thank you, I'm not sure why I didn't think to try them both together earlier.