Problem running GenPK
Closed this issue · 32 comments
Hey,
Could you help me with this issue: When I give the command : "./gen-pk -i run1_005 -o /GenPk " it gives me an error saying "Failed to create block at Header:Failed to open physical file run1_005/Header/header' with mode r' (Not a directory) @(bigfile.c:1628) @(bigfile.c:291)
Could not load header"
What mistake am I making? Thanks for your help.
Hi,
Are you actually trying to open a BigFile formatted snapshot? Or some other format?
Its a snapshot generated at the end of a Gadget simulation. I normally use GadgetViewer for opening snapshots. When I changed the extention to .0 and ran the same command "./gen-pk -i run1_005.0 /GenPK" it gave me this error:
[GadgetReader]: WARNING: Reading Gadget-I file using pre-computed
block order, which may not correspond to the actual order of your file!
Boxsize=150, NPart=(0,128,0,0,0,0)**3
Masses=[0 13.4 0 ]
redshift=0, Ω_M=0.3
FFT grid dimension: 256
total_mass in type 1 = 1
Error opening file: /GenPK/PK-DM-run1_005
Interesting - does the header match what you think the snapshot should be, and is it indeed Format 1?
If it is possible, can you paste the output of:
head -c 400
to me?
It has been running for an hour now and still no output.
Your filename is a little unusual for a gadget snapshot. Are you sure it exists?
You need to pass GenPK the actual snapshot name, not the snapshot directory. So there should be a file called: run1_005.0 , not a directory called run1_005 containing run1_005.0
It looks like the problem with your second command is that the subdirectory GenPK does not exist. Try creating it, or try:
./gen-pk -i run1_005.0 -o .
I have two gadget snapshots: run1_005 and run1_005.0. When I run
-
./gen-pk -i run1_005.0 -o I got:
./gen-pk: option requires an argument -- 'o'
Usage: ./gen-pk -i filenames -j other_filenames -o outdir -c (optional) cross-corr type
Outputs one file per particle type, with the name PK-$TYPE-$INPUT
Each output file has three columns, for each bin, k_eff, P(k) and N_modes
if -j is specified it cross-correlates files specified under -i with files specifed under
-j (one output per particle type).
If -c is specified the code computes the cross-correlation of that particle type with
the CDM (type 1) (within the file specified by -i) -
./gen-pk -i run1_005 -o /home/user/GenPK
[GadgetReader]: WARNING: Reading Gadget-I file using pre-computed
block order, which may not correspond to the actual order of your file!
Boxsize=150, NPart=(0,128,0,0,0,0)**3
Masses=[0 13.4 0 ]
redshift=0, Ω_M=0.3
FFT grid dimension: 256
total_mass in type 1 = 1
Error opening file: /home/user/GenPk/PK-DM-run1_005
3../gen-pk -i run1_005.0 -o /home/user/GenPk
Failed to create block at Header:Failed to open physical file run1_005.0/Header/header' with mode r' (Not a directory) @(bigfile.c:1628) @(bigfile.c:291)
Could not load header
I renamed the snapshots in that specific run with decreasing redshift hence the _005.
The '-o' option specifies where the file it output to. It must be a directory which exists.
So you should try: "./gen-pk -i run1_005.0 -o ." <- NOTE THE PERIOD HERE!
Yes I tried that. Gives the same errors:
Failed to create block at Header:Failed to open physical file run1_005.0/Header/header' with mode r' (Not a directory) @(bigfile.c:1628) @(bigfile.c:291)
Could not load header
AND
[GadgetReader]: WARNING: Reading Gadget-I file using pre-computed
block order, which may not correspond to the actual order of your file!
Boxsize=150, NPart=(0,128,0,0,0,0)**3
Masses=[0 13.4 0 ]
redshift=0, Ω_M=0.3
FFT grid dimension: 256
total_mass in type 1 = 1
Error opening file: ./PK-DM-run1_005
The error means that you could not open the output file for writing. It's loading the snapshot and generating the power spectrum just fine, it just isn't writing the output. So I would check that you have write permissions to the directory, that the filesystem isn't full, and so on. You could also check that you can create a file with that name. If you can do all those things, something a little weird is going on.
Exactly! Well I checked the write permissions they are create and delete files, the system isnt full. I created a file with that name. Everything seems fine. I have no clue why there is an error opening the file.
Do you still get the same error if there is an empty file in the right place?
Also, what operating system are you running on?
Yup same error even if there is an empty folder with that name. I'm using Ubuntu 16.04
That's a little strange. Is there a way you can send the snapshot to me?
Wait - when you ran "./gen-pk -i run1_005 -o /home/user/GenPK" why did it print:
"Error opening file: /home/user/GenPk/PK-DM-run1_005" ?
The case in GenPK has changed and by default ubuntu is case-sensitive, so it should not have done that. Are you doing anything unusual with ubuntu? (eg. running it on windows or on a live cd)?
Its a dual boot with windows 10.
Ok, I see it was just a paste error.
Can you show me the output of 'df -h' ?
Should I try removing everything and see if there were installation problems?
No no, that looks fine to me.
Can I see:
ls -al ~/GenPK
?
That looks fine too.
Can you try:
rm -R ~/GenPK/PK-DM-run1_005
touch ~/GenPK/PK-DM-run1_005
and then try running GenPK again (just in case)?
Okay, Worked for ./gen-pk -i run1_005.0 -o . It gave me a file with 3 columns but
Did not work for
./gen-pk -i run1_005.0 -o /home/user/GenPk
Ok. If you try it with consistent capitalisation, eg, ./gen-pk -i run1_005.0 -o /home/user/GenPK, it should work too.
That's good. It seems that the directory you want to write to must exist (so ~/GenPK must be there), and you must be able to create a file in that directory, which means that there must not be a subdirectory of the same name (so ~/GenPK/PK-DM-run1_005/ must not exist).
Linux filesystems are case-sensitive, so you need to be careful to use the same cases as the filesystem
Everything looks ok now, so closing.
Thank you so much for your help. :) I'm new to cosmological simulations and analysis. Its taking some time to get familiar.
No problem! And good luck.
If you are new and don't have a legacy of old simulations running around, one thing I would recommend is using format 2 or HDF5 (format 3) snapshot outputs - it makes a lot of things easier.
You can change that by altering the SnapFormat option in Gadget's parameter file to 3.