new icebrgs io produces a restart file even for io tiles that have no icebergs

[nikizadehgfdl]

We get a icebergs.res.nc* restart file for every io tile including the ones that have no icebergs.
Hence some restart files have
i = UNLIMITED ; // (0 currently)
Hence the general method of ncrcating them together at the end of the run (in the postprocess section of the xml) crashes:

gaea6: /lustre/f1/Niki.Zadeh/work/ulm_accumGTfluxes_mom6_2015.05.27/OM4_SIS2_baseline_1x0m10d_2997pe.o7085905/RESTART % ncrcat icebergs.res.nc.0000 icebergs.res.nc.0001 icebergs.res.nc
ncrcat: ERROR nco_lmt_sct_mk() reports record variable exists and is size zero, i.e., has no records yet.
ncrcat: HINT: Perform record-oriented operations only after file has valid records.
ncrcat: cnt < 0 in nco_lmt_sct_mk()
/bin/sh: line 1: 17130 Segmentation fault      $b_op $b_argv 2> $b_logfile.out

We need to either:

1. Avoid writing a restart file on an empty tile
2. Write a combine tool for iceberg restarts to handle empty file.

We have to fix this otherwise we cannot run models with the new icebergs:

/lustre/f1/Niki.Zadeh/ulm_accumGTfluxes_mom6_2015.05.27_0/OM4_SIS2_baseline/ncrc2.intel-prod/stdout/run/OM4_SIS2_baseline_1x0m10d_2997pe.o7085905

[jwdGFDL]

Sorry. While at this point I forget exactly why, hHaving a restart file
for every tile is a requirement of the restart infrastructure. The
combine needs to ignore files with no icebergs.

On 06/10/2015 05:27 PM, Niki Zadeh wrote:

We get a icebergs.res.nc* restart file for every io tile including the
ones that have no icebergs.
Hence some restart files have
i = UNLIMITED ; // (0 currently)
Hence the general method of ncrcating them together at the end of the
run (in the postprocess section of the xml) crashes:

|gaea6: /lustre/f1/Niki.Zadeh/work/ulm_accumGTfluxes_mom6_2015.05.27/OM4_SIS2_baseline_1x0m10d_2997pe.o7085905/RESTART % ncrcat icebergs.res.nc.0000 icebergs.res.nc.0001 icebergs.res.nc
ncrcat: ERROR nco_lmt_sct_mk() reports record variable exists and is size zero, i.e., has no records yet.
ncrcat: HINT: Perform record-oriented operations only after file has valid records.
ncrcat: cnt < 0 in nco_lmt_sct_mk()
/bin/sh: line 1: 17130 Segmentation fault $b_op $b_argv 2> $b_logfile.out
|

We need to either:

1. Avoid writing a restart file on an empty tile
2. Write a combine tool for iceberg restarts to handle empty file.

We have to fix this otherwise we cannot run models with the new icebergs:

/lustre/f1/Niki.Zadeh/ulm_accumGTfluxes_mom6_2015.05.27_0/OM4_SIS2_baseline/ncrc2.intel-prod/stdout/run/OM4_SIS2_baseline_1x0m10d_2997pe.o7085905

—
Reply to this email directly or view it on GitHub
#2.

Jeff Durachta
Engineering Lead for Modeling Services
NOAA Geophysical Fluid Dynamics Lab
Forrestal Campus, Princeton University
201 Forrestal Road
Princeton, NJ 08540
Office: +1-609-987-5054

[nikizadehgfdl]

Added the following logic in the xml to avoid ncrcat of empty files:

         set non_empty_restarts = ""
         foreach file ( `ls icebergs.res.nc.????` )
            if ( `ncdump -h $file | grep UNLIMITED | awk '{gsub(/\(/," ");print $6}'` ) set non_empty_restarts = "$non_empty_restarts $file"
         end
         ncrcat $non_empty_restarts icebergs.res.nc