build quadrature + validation for 3 storms

Question

build quadrature + validation for 3 storms

Opened this issue 3 years ago · 23 comments

Answer 1 · 2021-12-02T16:32:33.000Z

list of COASTAL Act storms:

Michael 2018
Florence 2018
Irma 2017
Maria 2017
Harvey 2017
Sandy 2012
Irene 2011
Ike 2008
Isabel 2003

Answer 2 · 2021-12-02T18:04:17.000Z

@zacharyburnettNOAA Thanks, I have made the subsetted meshes and got the storm data, but seems like cannot access Orion at the moment. Will let you know once I can login to Orion and transfer them over.

Answer 3 · 2021-12-03T02:35:47.000Z

@zacharyburnettNOAA Please find the subsetted meshes for each storm on Orion at: /work/noaa/nosofs/share/wpringle/HSOFS_Subset_Storms

This directory is actually something I made for the paper, where you can select any storm you want and run the besttrack_subset_merge.py python script to subset and merge the mesh in a few minutes. But need MATLAB and OceanMesh2D.

Answer 4 · 2021-12-03T15:13:28.000Z

excellent, thanks!

Answer 5 · 2021-12-13T23:03:49.000Z

@zacharyburnettNOAA Just wondering if have any update on Sandy and Irma simulations?

Answer 6 · 2021-12-14T15:38:05.000Z

@zacharyburnettNOAA Just wondering if have any update on Sandy and Irma simulations?

I had a few issues with running aswip but I have now figured out the problems, I should have the model results soon

Answer 7 · 2021-12-14T16:40:58.000Z

@zacharyburnettNOAA Just wondering if have any update on Sandy and Irma simulations?

I had a few issues with running aswip but I have now figured out the problems, I should have the model results soon

Great, thanks.
But just a note if we do use nws=8 standard holland model then aswip is not needed.

Answer 8 · 2021-12-22T16:26:32.000Z

I've tracked down the issue; when I was reading in the Sandy and Irma ATCF, the parser was parsing the storm name as a single newline \n for some reason, which was messing up the output. I'm fixing it now so we should have Sandy and Irma results running soon

Answer 9 · 2021-12-23T15:14:08.000Z

Spinup for Sandy 2012 fails at 0.11% on the subsetted mesh with the following error:

(ensembleperturbation)  Zachary.Burnett@hfe12 � /scratch2/COASTAL/coastal/scrub/Zachhary.Burnett/working/zach/nems_adcirc/run_20211202_sandy_besttrack_250m_quadrature�                                                                                
 > check_completion spinup --verbose

{
    "spinup": {
        "status": "error",
        "progress": "0.11%",
        "error": {
            "ADCIRC_COLDSTART_SPINUP.err.log": [
                " TIME STEP =     639     ITERATIONS =   16     TIME =  0.12780000E+04\n",
                "  ELMAX =  2.0421E+001 AT NODE   557516  SPEEDMAX =  1.5602E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     640     ITERATIONS =   16     TIME =  0.12800000E+04\n",
                "  ELMAX =  2.2397E+001 AT NODE   557516  SPEEDMAX =  1.6384E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     641     ITERATIONS =   16     TIME =  0.12820000E+04\n",
                "  ELMAX =  2.4613E+001 AT NODE   557516  SPEEDMAX =  1.5678E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     642     ITERATIONS =   17     TIME =  0.12840000E+04\n",
                "  ELMAX =  2.7103E+001 AT NODE   557516  SPEEDMAX =  1.5741E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     643     ITERATIONS =   16     TIME =  0.12860000E+04\n",
                "  ELMAX =  2.9921E+001 AT NODE   557516  SPEEDMAX =  1.5810E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     644     ITERATIONS =   16     TIME =  0.12880000E+04\n",
                "  ELMAX =  3.3123E+001 AT NODE   557516  SPEEDMAX =  1.5891E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     645     ITERATIONS =   16     TIME =  0.12900000E+04\n",
                "  ELMAX =  3.6771E+001 AT NODE   557516  SPEEDMAX =  1.5983E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     646     ITERATIONS =   17     TIME =  0.12920000E+04\n",
                "  ELMAX =  4.1013E+001 AT NODE   557516  SPEEDMAX =  1.6086E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     647     ITERATIONS =   17     TIME =  0.12940000E+04\n",
                "  ELMAX =  4.6434E+001 AT NODE   557516  SPEEDMAX =  1.6199E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     648     ITERATIONS =   17     TIME =  0.12960000E+04\n",
                "  ELMAX =  5.3241E+001 AT NODE   557516  SPEEDMAX =  1.6321E+000 AT NODE   944594  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     649     ITERATIONS =   18     TIME =  0.12980000E+04\n",
                "  ELMAX =  6.4476E+001 AT NODE   557516  SPEEDMAX =  1.8708E+000 AT NODE   557516  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     650     ITERATIONS =   23     TIME =  0.13000000E+04\n",
                "  ELMAX =  8.2971E+001 AT NODE   557516  SPEEDMAX =  2.7947E+000 AT NODE   557518  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     651     ITERATIONS =   22     TIME =  0.13020000E+04\n",
                "  ELMAX =  1.1504E+002 AT NODE   557516  SPEEDMAX =  4.5430E+000 AT NODE   557518  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     652     ITERATIONS =   21     TIME =  0.13040000E+04\n",
                "  ELMAX =  1.7060E+002 AT NODE   557516  SPEEDMAX =  7.0660E+000 AT NODE   557518  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     653     ITERATIONS =   19     TIME =  0.13060000E+04\n",
                "  ELMAX =  2.6385E+002 AT NODE   557516  SPEEDMAX =  1.0913E+001 ATNODE   557518  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     654     ITERATIONS =   18     TIME =  0.13080000E+04\n",
                "  ELMAX =  4.1041E+002 AT NODE   557516  SPEEDMAX =  1.6903E+001 AT NODE   557518  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     655     ITERATIONS =   18     TIME =  0.13100000E+04\n",
                "  ELMAX =  6.1703E+002 AT NODE   557518  SPEEDMAX =  2.2961E+001 AT NODE   557518  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     656     ITERATIONS =   16     TIME =  0.13120000E+04\n",
                "  ELMAX =  8.5691E+002 AT NODE   557518  SPEEDMAX =  3.4116E+001 AT NODE   557516  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     657     ITERATIONS =   19     TIME =  0.13140000E+04\n",
                "  ELMAX =  1.0244E+003 AT NODE   557518  SPEEDMAX =  3.8499E+001 AT NODE   557516  ON MYPROC =    9   ** WARNING: Elevation.gt.WarnElev **\n",
                " TIME STEP =     657     ITERATIONS =   19     TIME =  0.13140000E+04\n",
                "  ELMAX =  1.0244E+003 AT NODE   557518  SPEEDMAX =  3.8499E+001 AT NODE   557516  ON MYPROC =    9\n",
                "  ** ERROR: Elevation.gt.ErrorElev, ADCIRC stopping. **\n",
                "srun: error: h15c16: tasks 0-10: Exited with exit code 1\n",
                "srun: launch/slurm: _step_signal: Terminating StepId=26799133.0\n"
            ]
        }
    }
}

I will attempt to use the full 250m HSOFS mesh to rule out problems with the mesh subsetting.

Answer 10 · 2021-12-27T21:49:50.000Z

@zacharyburnettNOAA Thanks that's a good idea to try the non-subsetted version. Also you may try to reduce time step (e.g., if using 6 sec reduce by half to 3 sec).

Answer 11 · 2021-12-27T23:05:14.000Z

Also you may try to reduce time step (e.g., if using 6 sec reduce by half to 3 sec).

good idea, I will try that as well.

I also found out that the parsed best track data for Sandy 2012 had incorrect Rmax (it was set to 0 across the entire track), which made ADCIRC break upon hotstart. I manually updated the file and perturbations and am running the new ensemble for Sandy 2012 now.

Answer 12 · 2021-12-29T17:14:58.000Z

quadrature + validation runs for Sandy 2012 and Irma 2017 have completed at a 3 second time interval and proper best track (manually fixed the Rmax) over the full HSOFS 250m mesh (i will retry the subsets later).

I am now running combine_results to build the necessary NetCDF files from which we can fit a surrogate

Answer 13 · 2022-01-05T16:53:47.000Z

@zacharyburnettNOAA Can you please share with me (wpringle@anl.gov) the google slides we were making? My NOAA account was temporally disabled. Thanks.

Answer 14 · 2022-01-05T21:13:46.000Z

wpringle@anl.gov

sure thing, sent

Answer 15 · 2022-01-05T21:15:40.000Z

@zacharyburnettNOAA Thanks I got it. I'll try out your edits to the code. The joint KLPC seems to be working anyway. Actually gives very similar result to without using KL. I guess that's expected, just less computationally expensive.

Answer 16 · 2022-01-07T17:52:07.000Z

@zacharyburnettNOAA Hi Zach, can you do the Florence best track again for another 50 random samples? I want to try build training without the quadrature (maybe also try UQTk again and then convert to the chaospy ndpoly type). So then I can use 50 of samples for training and 50 for validation. Or even do another 100 to make sure.

Answer 17 · 2022-01-07T17:53:20.000Z

@zacharyburnettNOAA Are the Sandy combined results available on Orion?

Answer 18 · 2022-01-10T22:40:04.000Z

@zacharyburnettNOAA I noticed that the along_track perturbation doesn't seem to be doing anything in the Florence best track example (the model results and the fort.22 for location is same as other perturbations for different along_track and same cross_track). Can please make sure that along_track is being called correctly.

Answer 19 · 2022-01-11T14:49:11.000Z

@zacharyburnettNOAA Are the Sandy combined results available on Orion?

@WPringle sorry I forgot to reply to this. I put the Sandy results on Orion at /work/noaa/nosofs/share/working/hera/nems_adcirc/run_20211227_sandy_besttrack_250m_quadrature/. The Irma results still have some issues that I have been working through.

I will build another 100-member validation run for Florence once Hera comes out of its current maintenance period, which should be tomorrow

Answer 20 · 2022-01-11T16:16:29.000Z

@zacharyburnettNOAA Zach I had some other thoughts now.
Let's try to simplify things and start with a 2 variable ensemble. That way quadrature is just 16 members. We can also try other sampling methods like latin_hypercube.

It seems like cross_track and max_wind_speed are the most impactful variables, so can we work on producing an ensemble of just perturbing those, with various sampling methods, and then can do a thorough investigation of best methodology.

Answer 21 · 2022-01-11T18:06:40.000Z

that sounds like a simpler way to start, good idea

I will focus on compiling an ensemble for those two variables

Answer 22 · 2022-01-11T19:15:14.000Z

@zacharyburnettNOAA I implemented sample rule options in 4aa8ad3. So now we can try the different options to try and determine the best one.

Answer 23 · 2022-01-12T03:59:08.000Z

@zacharyburnettNOAA FYI,I'm trying out the different sampling methods and running the two variable ensembles on Argonne machine right now

Storm	mesh subsetted	ensemble built	ensemble completed	results combined	surrogate fit
Florence 2018
Sandy 2012
Irma 2017