add spack test to verify functionality of E4S 23.05

[shahzebsiddiqui]

Let's add a simple test to verify E4S 23.05 stack a few things we can run

1. List spack environment spack env list
2. Activate each spack environment spack env activate gcc and then run spack find

To access the e4s stack for time being we can do the following

(buildtest) siddiq90@login25> module use /global/common/software/nersc/pe/modulefiles/latest
(buildtest) siddiq90@login25> ml e4s/23.05
    _______________________________________________________________________________________________________
     The Extreme-Scale Scientific Software Stack (E4S) is accessible via the Spack package manager.

     In order to access the production stack, you will need to load a spack environment. Here are some tips to get started:


     'spack env list' - List all Spack environments
     'spack env activate gcc' - Activate the "gcc" Spack environment
     'spack env status' - Display the active Spack environment
     'spack load amrex' - Load the "amrex" Spack package into your user environment

     For additional support, please refer to the following references:

       NERSC E4S Documentation: https://docs.nersc.gov/applications/e4s/
       E4S Documentation: https://e4s.readthedocs.io
       Spack Documentation: https://spack.readthedocs.io/en/latest/
       Spack Slack: https://spackpm.slack.com

    ______________________________________________________________________________________________________
(buildtest) siddiq90@login25> spack env list
==> 7 environments
    cce  cuda  data  gcc  math-libs  nvhpc  tools

Let's add the buildspec in https://github.com/buildtesters/buildtest-nersc/blob/devel/buildspecs/apps/spack/perlmutter_spack_check.yml and have it run on both perlmutter and muller login nodes via bash executor.

[shahzebsiddiqui]

The e4s stack is now on Perlmutter so there is no need to set MODULEPATH

Previously the example test https://github.com/buildtesters/buildtest-nersc/blob/devel/buildspecs/apps/spack/perlmutter_spack_check.yml was running a version check and doing a regex which i dont think is very useful it would be more useful running some command spack commands to ensure the module is working correctly and one can switch between spack environments and query spack packages.

@etpalmer63 @jscook2345 what do you think about this approach in order to test 23.05 stack and any subsequent stacks in future

[jscook2345]

Go ahead and put together an MR showing what you mean and we'll take a look. Thanks!

[shahzebsiddiqui]

closing this issue now, the test is already merged