USEPA/CMAQ

undefined references, missing libraries for pnetcdf and openmp

lizadams opened this issue · 2 comments

lizadams commented

Obtained the following type of message when trying to compile CMAQ
/home/lizadams/CMAQ_REPO/linuxbrew_mpif90_gcc/lib/x86_64/gcc/ioapi/lib/libioapi.a(wrtflag.o): In function wrtflag_': wrtflag.f:(.text+0x75): undefined reference to GOMP_critical_name_start'
wrtflag.f:(.text+0xab): undefined reference to GOMP_critical_name_end' wrtflag.f:(.text+0x104): undefined reference to GOMP_critical_name_start'
wrtflag.f:(.text+0x125): undefined reference to GOMP_critical_name_end' wrtflag.f:(.text+0x396): undefined reference to GOMP_critical_name_start'
wrtflag.f:(.text+0x870): undefined reference to GOMP_critical_name_end' /home/lizadams/CMAQ_REPO/linuxbrew_mpif90_gcc/lib/x86_64/gcc/ioapi/lib/libioapi.a(modatts3.o): In function modatts3_MOD_pn_getcmaq':
modatts3.F90:(.text+0x139): undefined reference to nfmpi_get_att_int_' modatts3.F90:(.text+0x171): undefined reference to nfmpi_get_att_text'
modatts3.F90:(.text+0x19f): undefined reference to nfmpi_get_att_text_' modatts3.F90:(.text+0x1cd): undefined reference to nfmpi_get_att_text'
modatts3.F90:(.text+0x1fb): undefined reference to nfmpi_get_att_text_' modatts3.F90:(.text+0x22c): undefined reference to nfmpi_get_att_text_'

When I added the following to the Makefile: -pnetcdf -fopenmp, then the undefined references were satisfied.
NETCDF = -L$(LIB)/netcdf/lib -lnetcdff -lnetcdf -lpnetcdf -fopenmp

Note, to obtain netcdf and pnetcdf, I used the following method:
Install Linuxbrew on a linux machine following instructions on this link: http://linuxbrew.sh/
Install netcdf using linuxbrew command:
brew install netcdf
Install pnetcdf using linuxbrew command from this link: MPAS-Dev/MPAS-Model#34
brew install pwolfram/mpas/parallel-netcdf
All of the libraries, executables, and include files are available under:
/home/linuxbrew/.linuxbrew
Download ioapi, using the github clone command
git clone https://github.com/cjcoats/ioapi-3.2
Install csh on ubuntu
sudo apt-get install csh
Edit the Makefile to specify the Makeinclude that you will be using
BIN = Linux4_x86_64gfortmpi

cd ioapi
cp Makeinclude.Linux2_x86_64gfortmpi Makeinclude.Linux4_x86_64gfortmpi
I made two modifications to the ioapi-3.2/ioapi/Makeinclude.Linux4_x86_64gfortmpi
Changed:
CXX = mpicxx
to remove the path to the default system libraries-L/usr/lib64
ARCHLIB = -dynamic -lm -lpthread –lc

Edit the ioapi-3.2/Makefile to add the pnetcdf library
NCFLIBS = -lnetcdf -lnetcdff -lpnetcdf
Run make to build the ioapi library and m3tools
make
Next step was to download the 5.3beta2 version of CMAQ

git clone -b 5.3.b2 https://github.com/USEPA/CMAQ.git CMAQ_REPO

cd CMAQ_REPO
edit bldit_project.csh to set CMAQ_HOME
set CMAQ_HOME = /home/lizadams/linuxbrew_mpi_gcc
./bldit_project.csh
cd /home/lizadams/linuxbrew_mpi_gcc
I made a modification to the config_cmaq.csh under the section:
case gcc:

    #> I/O API, netCDF, and MPI library locations
    setenv IOAPI_MOD_DIR   /home/lizadams/ioapi-3.2/Linux4_x86_64gfortmpi #> I/O API precompiled modules
    setenv IOAPI_INCL_DIR  /home/lizadams/ioapi-3.2/ioapi/fixed_src  #> I/O API include header files
    setenv IOAPI_LIB_DIR   /home/lizadams/ioapi-3.2/Linux4_x86_64gfortmpi  #> I/O API libraries
    setenv NETCDF_LIB_DIR  /home/linuxbrew/.linuxbrew/lib #> netCDF directory path
    setenv NETCDF_INCL_DIR /home/linuxbrew/.linuxbrew/include  #> netCDF directory path
    setenv MPI_LIB_DIR     /home/linuxbrew/.linuxbrew/lib    #> MPI directory path

    setenv netcdf_lib "-lnetcdff -lnetcdf -lpnetcdf"
    setenv myLINK_FLAG "-fopenmp"

Built CMAQ using the following:

cd CCTM/scripts
./bldit_cctm.csh gcc |& tee ./bldit_cctm.log

Successful build with no undefined references.

mdzz2268 commented

aaaaa......
there is too many troubles builting CMAQ5.2 ,such as undefined references you said above,and cant find open mpif.h when i built cctm.csh,,

lizadams commented
  1. build netcdf fortran, netcdf C and ioapi using the ./configure command with the options --disable-netcdf-4 --disable-dap using a consistent set of compilers (it helps to use the module load command if your system supports modules, and then set the compiler using the environment variables "setenv CC and setenv FC to specifically set the path to the compiler for C and Fotran that you are using.
  2. build ioapi using the Makeinclude file that does not use mpiCC or mpif90 (uses the same CC and FC environment variable used to build netcdf fortran and netcdf C libraries to compile the library.
    Don't use any Makeinclude file that contains the following option -DIOAPI_PNCF=1 in the FPPFlags to avoid needing pnetcdf.

See the following instructions for additional help.
https://github.com/lizadams/CMAQ/blob/v53_UG/DOCS/Users_Guide/Tutorials/CMAQ_Build_Using_Module_Load_GNU.md