fix one cause for GammaUnknownError: calloc_2d: memory allocation error for data values
Closed this issue · 2 comments
The following is caused by only no-data values in an S1 scene over the DEM/Area of interest to process the data.
Using the geometry of the scene's metadata, as in #185, instead of the footprint, would not consider those scenes in a select, reducing unnecessary data processing up to the shell commands below.
- which operating system are you using?
Centos - which environment is pyroSAR running in?
Micromamba env - which version of pyroSAR are you using?
0.22.1 - which function of pyroSAR did you call with which parameters?
geocode(scene = f,
dem = dem,
outdir = outdir,
spacing = 10,
scaling = 'db',
export_extra = ['ls_map_geo', 'pix_area_gamma0_geo'],
tmpdir = '/work/tempdir' + site_name,
osvdir = '/work/osvs')
- GAMMA 20221129
- the full error message
File "/work/dems/process_data.py", line 52, in <module>
geocode(scene = f,
^^^^^^^^^^^^^^^^^^
File "/home/micromamba/envs/s1proc/lib/python3.12/site-packages/pyroSAR/gamma/util.py", line 835, in geocode
gc_map_wrap(image=reference, namespace=n, dem=dem, spacing=spacing, exist_ok=exist_ok,
File "/home/micromamba/envs/s1proc/lib/python3.12/site-packages/pyroSAR/gamma/util.py", line 1477, in gc_map_wrap
diff.gc_map2(**gc_map_args)
File "/home/.pyrosar/gammaparse/diff.py", line 1421, in gc_map2
process(cmd, logpath=logpath, outdir=outdir, shellscript=shellscript)
File "/home/micromamba/envs/s1proc/lib/python3.12/site-packages/pyroSAR/gamma/auxil.py", line 360, in process
gammaErrorHandler(out, err)
File "/home/micromamba/envs/s1proc/lib/python3.12/site-packages/pyroSAR/gamma/error.py", line 74, in gammaErrorHandler
raise GammaUnknownError(err_out)
pyroSAR.gamma.error.GammaUnknownError: calloc_2d: memory allocation error for data values
shell commands:
# this script was created automatically by pyroSAR on Wed Nov 29 13:00:45 2023
export base=/work/tempdirAG/S1A__IW___A_20171119T171511
export GAMMA_HOME=/cluster/apps/GAMMA_SOFTWARE/20221129
$GAMMA_HOME/ISP/bin/par_S1_GRD $base/S1A_IW_GRDH_1SDV_20171119T171511_20171119T171536_019337_020C59_1951.SAFE/measurement/s1a-iw-grd-vh-20171119t171511-20171119t171536-019337-020c59-002.tiff $base/S1A_IW_GRDH_1SDV_20171119T171511_20171119T171536_019337_020C59_1951.SAFE/annotation/s1a-iw-grd-vh-20171119t171511-20171119t171536-019337-020c59-002.xml $base/S1A_IW_GRDH_1SDV_20171119T171511_20171119T171536_019337_020C59_1951.SAFE/annotation/calibration/calibration-s1a-iw-grd-vh-20171119t171511-20171119t171536-019337-020c59-002.xml - $base/S1A__IW___A_20171119T171511_VH_grd.par $base/S1A__IW___A_20171119T171511_VH_grd - - - - - 2 - -
$GAMMA_HOME/ISP/bin/par_S1_GRD $base/S1A_IW_GRDH_1SDV_20171119T171511_20171119T171536_019337_020C59_1951.SAFE/measurement/s1a-iw-grd-vv-20171119t171511-20171119t171536-019337-020c59-001.tiff $base/S1A_IW_GRDH_1SDV_20171119T171511_20171119T171536_019337_020C59_1951.SAFE/annotation/s1a-iw-grd-vv-20171119t171511-20171119t171536-019337-020c59-001.xml $base/S1A_IW_GRDH_1SDV_20171119T171511_20171119T171536_019337_020C59_1951.SAFE/annotation/calibration/calibration-s1a-iw-grd-vv-20171119t171511-20171119t171536-019337-020c59-001.xml - $base/S1A__IW___A_20171119T171511_VV_grd.par $base/S1A__IW___A_20171119T171511_VV_grd - - - - - 2 - -
$GAMMA_HOME/ISP/bin/S1_OPOD_vec $base/S1A__IW___A_20171119T171511_VH_grd.par $base/osv/S1A_OPER_AUX_POEORB_OPOD_20210305T045727_V20171118T225942_20171120T005942.EOF -
$GAMMA_HOME/ISP/bin/S1_OPOD_vec $base/S1A__IW___A_20171119T171511_VV_grd.par $base/osv/S1A_OPER_AUX_POEORB_OPOD_20210305T045727_V20171118T225942_20171120T005942.EOF -
$GAMMA_HOME/ISP/bin/multi_look_MLI $base/S1A__IW___A_20171119T171511_VH_grd $base/S1A__IW___A_20171119T171511_VH_grd.par $base/S1A__IW___A_20171119T171511_VH_grd_mli $base/S1A__IW___A_20171119T171511_VH_grd_mli.par 1 1 - - - -
$GAMMA_HOME/ISP/bin/multi_look_MLI $base/S1A__IW___A_20171119T171511_VV_grd $base/S1A__IW___A_20171119T171511_VV_grd.par $base/S1A__IW___A_20171119T171511_VV_grd_mli $base/S1A__IW___A_20171119T171511_VV_grd_mli.par 1 1 - - - -
$GAMMA_HOME/DIFF/bin/gc_map2 $base/S1A__IW___A_20171119T171511_VH_grd_mli.par /work/dems/AG_2019_p50_3x3_corr_swap.par /work/dems/AG_2019_p50_3x3_corr_swap $base/S1A__IW___A_20171119T171511_dem_seg_geo.par $base/S1A__IW___A_20171119T171511_dem_seg_geo $base/S1A__IW___A_20171119T171511_lut_init 0.99959192 0.99959192 $base/S1A__IW___A_20171119T171511_ls_map_geo - $base/S1A__IW___A_20171119T171511_inc_geo - - - - - - - - - - 8 - - - -
I see a problem with the solution in #185 as there is a backwards incompatability if the geometry column is simply added. I tried to work around this via checks and the special flag to use the geometry. but this is already required at creation of the archive.
Another idea is to just add a refinement to the select method:
- find scenes by the footprint and
- get only those geometry on the fly from stored metadata and refine search.
The second option would leave the database as-is and reduce the needed ajustments to a read method for the geometry and a call within the select funtion, See #282.
What do you think?
I guess the issue is not really closed yet but is being addressed in #296.