Reading large matrix from op4 fails when in binary format

[drcavaliere]

Hello,

First of all, thank you for your support in advance.

I am trying to read the stiffness and mass matrix from a large binary file (the model has 6035743 DOFs) and I get this error:

_File "pyYeti_test.py", line 25, in get_pyyeti
dct = o4.load(op4_filename, namelist=['kaa', 'maa'], into='dct', justmatrix=False, sparse=sparse)
File "/pyyeti/nastran/op4.py", line 1877, in load
return self.dctload(filename, namelist, justmatrix, sparse)
File "pyyeti/nastran/op4.py", line 1703, in dctload
self._op4open_read(filename)
File "pyyeti/nastran/op4.py", line 271, in _op4open_read
line = self.fileh.readline().decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb2 in position 1: invalid start byte

How could I fix it?

Best regards,
Fabiola

[twmacro]

Hi Fabiola,

Would you please post the first say 100 bytes or so of this file? That might help me see what's going on.

Thank you!
Best regards,
Tim

[drcavaliere]

Hi Tim,

Thank you so much for your fast reply. I attach a file with the first 100 bytes: first_100_bytes.txt
To extract them I used the following command:

hexdump -n100 filename

If it's not what you need, I'll be happy to follow your suggestion!

Best regards,
Fabiola

[twmacro]

Hi Fabiola,

What a puzzle! I'm unable to decode that data (so far). It doesn't match any op4 format that I'm familiar with. A few questions:

1. Do you know the name of the first matrix in the file and its dimensions?
2. Have you tried pyNastran? Does that work?
3. Can Nastran itself read it? Which version?
4. How big is the file?
5. How was it created? I'd like to see the ASSIGN statement.

I'm positive I can think of more questions ...

Thank you!
Best regards,
Tim

[drcavaliere]

Hi Tim!

I have tried to replicate the same task with a smaller model.
I have generated two different files with the same matrices but one is in ascii and the other in binary format. Pyyeti is able to read the ascii file, but I get the error with the binary file.
Here you have the files: ISSUE_PyYeti.zip

One note: In the original issue, I was trying to extract the Stiffness and Mass matrices on the ASET, which gave me the error.
In the small model that I am sharing now:

• I CAN extract K and M from the binary file
• I get the failing decoding error if I try to extract other matrices (mm: mass, ll: lower diagonal matrix from decomposed stiffness, dd: diagonal matrix from decomposed stiffness)

I answer now your questions for both examples:

1- Do you know the name of the first matrix in the file and its dimensions?
BIG MODEL: Name of first matrix: KAA, Dimension: (6035743 x 6035743) - sparse
SMALL MODEL: Name of first matrix: MAA, Dimension: (240 x 240) - sparse

2 - Have you tried pyNastran? Does that work?
I have tried it and it does not work (for both models).

3 - Can Nastran itself read it? Which version?
It can read it. I am using version 2019.1

4 - How big is the file?
BIG MODEL: 3.6GB
SMALL MODEL: 114KB

5 - How was it created? I'd like to see the ASSIGN statement.
ASSIGN OUTPUT4='filename.op4',unit=30,form=unformatted

I hope my answer is clear! Happy to answer any extra question :)

Thank you a lot!
Fabiola

[twmacro]

Thank you so much Fabiola! I will try to look at this today or tomorrow! :-)

[twmacro]

Hi Fabiola,

I've updated pyYeti to version 1.2.2 and it can now read the small file and I hope it can read the larger one too. It turns out there was a bug in the reader for binary, 64-bit integer files. The files I had in the test suite didn't trigger the bug by just bad luck. For future protection, I added the two files you sent to the test suite. I hope you don't mind! :-)

Also, as an aside, I either don't understand "hexdump" or it worked improperly. In any case, it gave us garbled bytes: for example, the name of the matrix from the hexdump was "AK A" ... which makes no sense. xxd seemed to work more reliably than hexdump on my machine.

Please let me know if this new version works for you!

Many thanks for reporting this issue.

Best Regards,
Tim

[drcavaliere]

Hi Tim,

Sorry for the delay in giving you a feedback! I've just tested the new version and it seems to work fine! No problem about using my files for test.

Thank you so much for the fast response and the amazing job you have done with pyyeti :)

Best regards,
Fabiola

[twmacro]

Thank you so much for your help, Fabiola! :)