Segmentation fault when convert ERA5 data into ARL

Question

Segmentation fault when convert ERA5 data into ARL

haochiche opened this issue 5 years ago · 11 comments

Hi

I downloaded the data with the example script successfully, but in the converting, I got the error messages as below:

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x7F137AB5F697
#1  0x7F137AB5FCDE
#2  0x7F137A4AD27F
#3  0x7F137B300F77
#4  0x7F137B304244
#5  0x405DD5 in MAIN__ at era52arl.f:648
Segmentation fault

I'm running Hysplit on centOS Linux, and the example data for reproducing this error is in here.. I tested era52arl by just execute it, and it showed me the usage information, so I guess the era52arl works fine?

Many thanks
HC

Answer 1 · 2020-02-10T20:48:09.000Z

Hey,

are you running the script with superuser command?
Is the era52arl program correctly linked to the eccodes libraries? (see https://ready.arl.noaa.gov/data/web/models/hysplit4/linux/README_external_libraries.txt)
It should look like this:

~/Trajektorien_git/Hysplit_data_conversion/era52arl $ ldd era52arl
	linux-vdso.so.1 =>  (0x00007fffa39ca000)
	libeccodes_f90.so => /lib/eccodes/lib/libeccodes_f90.so (0x00007fd3d5927000)
	libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007fd3d55d1000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd3d52c8000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd3d50b1000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd3d4ce7000)
	libeccodes.so => /lib/eccodes/lib/../lib/libeccodes.so (0x00007fd3d485a000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd3d463c000)
	libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fd3d43fd000)
	/lib64/ld-linux-x86-64.so.2 (0x0000555d978d5000)

Are you calling the era52arl program in the right /dir ? When you try to start the program and it can't find the .grib data it also ends in a Segmentation fault error call.

Hope this will help you find your error,
Thomas

Answer 2 · 2020-02-16T07:34:39.000Z

Hey,

are you running the script with superuser command?
Is the era52arl program correctly linked to the eccodes libraries? (see https://ready.arl.noaa.gov/data/web/models/hysplit4/linux/README_external_libraries.txt)
It should look like this:
~/Trajektorien_git/Hysplit_data_conversion/era52arl $ ldd era52arl
	linux-vdso.so.1 =>  (0x00007fffa39ca000)
	libeccodes_f90.so => /lib/eccodes/lib/libeccodes_f90.so (0x00007fd3d5927000)
	libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007fd3d55d1000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd3d52c8000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd3d50b1000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd3d4ce7000)
	libeccodes.so => /lib/eccodes/lib/../lib/libeccodes.so (0x00007fd3d485a000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd3d463c000)
	libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fd3d43fd000)
	/lib64/ld-linux-x86-64.so.2 (0x0000555d978d5000)
Are you calling the era52arl program in the right /dir ? When you try to start the program and it can't find the .grib data it also ends in a Segmentation fault error call.

Hope this will help you find your error,
Thomas

Thanks for the reply! I'm sure I give it the correct location and also all libraries are linked. But it still keeps crashing. Maybe just because I'm converting only a small domain data instead of global?

Answer 3 · 2020-02-17T09:36:16.000Z

Hi,
do you fix your probelms? I used the new 'era52arl.f' file and met the same segmentation fault, I got probelms on the 'CALL PAKSET(lunit,ARLCFG_NAME,1,NXP,NYP,NZP)' part and I don't know why.

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0 0x7f73e600b2da in ???
#1 0x7f73e600a503 in ???
#2 0x7f73e566ef1f in ???
#3 0x7f73e57be73d in ???
#4 0x7f73e56cd9fd in ???
#5 0x7f73e600aa42 in ???
#6 0x7f73e618938d in ???
#7 0x7f73e6176eeb in ???
#8 0x7f73e6a0a7c7 in ???
#9 0x7f73e6a06404 in era52arl
at /mnt/d/ckwork/hysplit_metdata/hysplit/data2arl/era52arl/era52arl.f:827
#10 0x7f73e6a01b32 in main
at /mnt/d/ckwork/hysplit_metdata/hysplit/data2arl/era52arl/era52arl.f:102

Answer 4 · 2020-02-17T09:47:04.000Z

Hi

Unfortunately no, the era52arl always throw the segment fault error.
However, I managed to convert the data with another Jpython based software, using the following script: http://meteothink.org/examples/meteoinfolab/trajectory/grib2arl.html
This would be a good option for converting the era5 data as we can custom the script.

Answer 5 · 2020-02-17T14:09:57.000Z

Thank you very much, but ERA5 has 3d and 2d files, but the grib2arl script has only one input. I combined these two data into one grib file, but I still got error messages.Would you mind tell me how to solve this problem?

Time index: 0
2018-12-31 19:00
PRSS Pressure_surface
Pressure_surface is not a variable name
Traceback (most recent call last):
File "", line 55, in

Answer 6 · 2020-02-17T14:22:01.000Z

Thank you very much, but ERA5 has 3d and 2d files, but the grib2arl script has only one input. I combined these two data into one grib file, but I still got error messages.Would you mind tell me how to solve this problem?

Time index: 0
2018-12-31 19:00
PRSS Pressure_surface
Pressure_surface is not a variable name
Traceback (most recent call last):
File "", line 55, in

Hi

There is no need to combine the data into one file, you can read two files instead. Also, note some variables need to convert the unit to fit the ARL format. I have attached my modified script, hope this would help!

To use the script, rename the format to .py

Cheers
Haochi
era52arl.py.txt

Answer 7 · 2020-02-17T14:29:23.000Z

It works! I'd really appreciate your help!!

Answer 8 · 2020-02-20T13:02:16.000Z

Hi

The author of MeteoInfo has updated the conversion program, which supports to use DIFF as the difference between the original data and the packed data.

The new version will provide more accurate precision of the data and is highly recommended.

The example script, as well as the comparison between converted ARL and ERA5 data, can be found here: http://meteothink.org/examples/meteoinfolab/trajectory/grib2arl.html

To use the script, the latest version of MeteoInfo 2.1.1 is required.

Cheers
Haochi

Answer 9 · 2021-07-30T00:13:57.000Z

Hi, I came across the same issue above and now have resolved it: please find the solution in my forked repo.

BTW, I have also tried the package suggested @haochiche, which can do the conversion; however, the covered files are not recognised by HYSPLIT.

Answer 10 · 2021-11-28T14:18:57.000Z

Hi, I came across the same issue above and now have resolved it: please find the solution in my forked repo.

BTW, I have also tried the package suggested @haochiche, which can do the conversion; however, the covered files are not recognised by HYSPLIT.

Thanks for your solution. Please carefully check details if you want to compare the MeteoInfo package, because you are the only user who reported the converted files are not recognised by HSPLIT.

Many thanks

Answer 11 · 2022-01-21T22:38:55.000Z

Hi all

I have some questions regarding ERA52ARL.
First I want to were you able to run HYSPLIT using the converted ERA5 data? @haochiche
My second question is that when I try to run the script for python, the function "addfile" is not defined in my python. What should I do? which package should I install before? @haochiche
@Alliekaca please let me know if you have a most update script in python for converting ERA5 to ARL for run in HYSPLIT.

Thank you all
Nasim