compilation error of vasp-6.4.3 with aocc 5.0.0

[rodrigo_mejia]

Dear vasp support team,

I am trying to compile vasp 6.4.3 in a new cluster with AMD EPYC 9454 processors. I am using the aocc 5.0.0 compiler with the aocl 5.0.0 libraries (liblis, libflame, libscalapack, lilbfftw), which were provided by the cluster admin team.

Unfortunately the compilation fails with the following error message:

Code: Select all

Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)
./incar_reader.inc:142:44:

  142 |             call xml_incar_v(key, TYPE_CHECK, val, rdum, cdum, ldum, sdum, size(val))
      |                                            1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (rank-1 and scalar)
make[2]: *** [makefile:171: incar_reader.o] Error 1
make[2]: Leaving directory '/pfs/data6/home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/vasp/vasp-6.4.3/vasp.6.4.3/build/std'
cp: cannot stat 'vasp': No such file or directory
make[1]: *** [makefile:131: all] Error 1
make[1]: Leaving directory '/pfs/data6/home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/vasp/vasp-6.4.3/vasp.6.4.3/build/std'
make: *** [makefile:17: std] Error 2

You can find the complete compilation log, and corresponding makefile.include, within the attached zip file.

There is a somewhat related issue in the forum (see https://w.vasp.at/forum/viewtopic.php?p=23217#p23229), where the accepted answer suggests the following:

- switch to a different preprocessor e.g. cpp.
- Add -ffree-form to the preprocessor (CPP in the makefile.include).
- Rename the files that fail to .F90 (note that would require changes to the makefile as well).

Thus I would like ask what does the phrase "note that would require changes to the makefile as well" explicitly refers to?
Maybe these changes could help with my problem. Or, if not the case, what else could I try to complete my installation?

One more thing I want to comment on is the makefile.include for aocc.
I am using a makefile.include which was adapted from https://www.vasp.at/wiki/index.php/Make ... _ompi_aocl .
In my file I needed to adapt the FFLAGS in line 25 from:

Code: Select all

-w -fno-fortran-main -Mbackslash

to:

Code: Select all

-w  -fbackslash

I did this because I was getting this other error message:

Code: Select all

gfortran: error: unrecognized command-line option ‘-fno-fortran-main’
gfortran: error: unrecognized command-line option ‘-Mbackslash’; did you mean ‘-fbackslash’?

Since this is my first time compiling for an AMD architecture I am not sure if something changed in version 5.0.0 of aocl with respect to the previous version which causes these option conlficts.
I just thought you might want to know this happened.

Best regards,
Rodrigo

[michael_wolloch]

Dear Rodrigo Mejia,

I am very sorry for the delay!

I think a hint to the solution to your problem can be found here:

I did this because I was getting this other error message:

Code: Select all

gfortran: error: unrecognized command-line option ‘-fno-fortran-main’
gfortran: error: unrecognized command-line option ‘-Mbackslash’; did you mean ‘-fbackslash’?

This indicates that you are not using AMD flang/clang, but GNU gfortran.

Please check if the mpif90 wrapper is using the right compiler executable by running

Code: Select all

mpif90 --version

on the command line.
E.g. for me, when I load our aocc 5.0 toolchain:

Code: Select all

$ mpif90 --version
AMD clang version 17.0.6 (CLANG: AOCC_5.0.0-Build#1377 2024_09_24)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/share/spack-0.23/opt/spack/linux-rocky8-x86_64/gcc-8.5.0/aocc-5.0.0-weyxlvlhkb4btty2nomuhgfilcackxr5/bin

You should see something similar. Especially no mentioning of "gfortran"

Is it possible that you have loaded the wrong openMPI module?

If the correct aocc compiler is used, the original FFLAGS line

Code: Select all

FFLAGS      = -w -fno-fortran-main -Mbackslash

will work and your code should compile.

Let me know if this helps, and again sorry for the delayed answer.

Cheers, Michael

[rodrigo_mejia]

Dear Michael,

Many thanks for your answer. I have contacted the technical team of our HPC cluster to request the right mpi module.
I will post an update as soon as I am able to test the proposed solution.

Best regards,
Rodrigo

[rodrigo_mejia]

Dear Michael,

The module has been fixed and the reported error no longer occurs.

Now there I have encountered another problem concerning compilation with HDF5 1.14.6 support.
My first attempt was done using a serial version of HDF5 1.14.6.
I built this serial HdF5 with the modules and commands below:

Code: Select all

$> module load compiler/aocc/5.0.0
$> module load  lib/aocl/5.0.0
$> ./configure CC=clang CXX=clang++ CFLAGS="-O2 -march=znver4" --prefix=/home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/hdf5/1.14.6_aocc-5.0.0_serial --enable-fortran --enable-cxx --enable-shared --with-zlib=/usr/include,/usr/lib64
$> make
$> cd test #To test the libraries
$> make check 2>&1 | tee make_check.log
$> make install

Then I set up my makefile.include accordingly:

Code: Select all

# HDF5-support (optional but strongly recommended)
CPP_OPTIONS+= -DVASP_HDF5
HDF5_ROOT  ?= /home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/hdf5/1.14.6_aocc-5.0.0_serial
LLIBS      += -L$(HDF5_ROOT)/lib -lhdf5_fortran
INCS       += -I$(HDF5_ROOT)/include

And attempted to compile vasp-6.4.3_std using the following modules:
1) compiler/aocc/5.0.0
2) lib/aocl/5.0.0
3) mpi/openmpi/5.0.7-aocc-5.0.0

For which I got the following error message:

Code: Select all

F90-F-0004-Corrupt or Old Module file /home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/hdf5/1.14.6_aocc-5.0.0_serial/include/hdf5.mod (vhdf5_base.F: 9)
F90/x86-64 Linux Flang - 1.5 2017-05-01: compilation aborted
make[2]: *** [makefile:171: vhdf5_base.o] Error 1
make[2]: Leaving directory '/pfs/data6/home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/vasp/vasp-6.4.3/vasp.6.4.3/build/std'
cp: cannot stat 'vasp': No such file or directory
make[1]: *** [makefile:131: all] Error 1
make[1]: Leaving directory '/pfs/data6/home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/vasp/vasp-6.4.3/vasp.6.4.3/build/std'
make: *** [makefile:17: std] Error 2

I then tested another version of HDF5 which was compiled using OpenMPI using the following modules and commands:

Code: Select all

$> module load compiler/aocc/5.0.0
$> module load lib/aocl/5.0.0
$> module load mpi/openmpi/5.0-aocc-5.0.0
$> ./configure CC=mpicc --prefix=/home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/hdf5/1.14.6_aocc-5.0.0_openmpi --enable-parallel --enable-fortran --enable-shared --with-pic --with-zlib=/usr/include,/usr/lib64 CPPFLAGS=-I/opt/bwhpc/common/mpi/openmpi/5.0.7-aocc-5.0.0/include LDFLAGS=-L/opt/bwhpc/common/mpi/openmpi/5.0.7-aocc-5.0.0/lib64
$> make
$> make check
$> make install

And also adjusted my makefile.include accordingly:

Code: Select all

# HDF5-support (optional but strongly recommended)
CPP_OPTIONS+= -DVASP_HDF5
HDF5_ROOT  ?= /home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/hdf5/1.14.6_aocc-5.0.0_openmpi
LLIBS      += -L$(HDF5_ROOT)/lib -lhdf5_fortran
INCS       += -I$(HDF5_ROOT)/include

This HDF5 version with OpenMPI support was then used to attempt a new compilation of VASP 6.4.3.
This time things seem to have worked just fine, as there are no error messages in the build log.

So I am guessing that, even though the manual wiki/index.php/Category:HDF5_support states "As of VASP 6.2.0 we only support reading/writing of hdf5 files in serial mode."
this only refers to the capabilities of VASP, but then HDF5 should be compiled with MPI/OpenMPI support? Or have there been any changes on this regard? The available CHANGELOG only allows to go back intil VASP 6.4.3 wiki/index.php/Changelog.

Do you have any advice on this regard? If HDF5 needs to be compiled with MPI/OpenMPI support then I will need to request the corresponding modules from our technical team. Or, is there a way to include HDF5 statically in the VASP binary? Some input on this regard would be appreciated.

Best,
Rodrigo

[michael_wolloch]

Dear Rodrigo,

So I am guessing that, even though the manual wiki/index.php/Category:HDF5_support states "As of VASP 6.2.0 we only support reading/writing of hdf5 files in serial mode."
this only refers to the capabilities of VASP, but then HDF5 should be compiled with MPI/OpenMPI support? Or have there been any changes on this regard? The available CHANGELOG only allows to go back intil VASP 6.4.3 wiki/index.php/Changelog.

No, you do not need to compile with MPI support to use HDF5 with VASP.

My guess is that you had some problems with your Fortran compiler in the first attempt. Can you check your HDF5 config.log file for FC=flang? If it might be FC=gfortran instead, which would then not work with using flang for VASP compilation.
If you want to try again, please specify the Fortran compiler as well like:

Code: Select all

./configure CC=clang CXX=clang++ FC=flang CFLAGS="-O2 -march=znver4" --prefix=/home/ka/ka_scc/ka_an9294/Programs/uc3-software_admin/hdf5/1.14.6_aocc-5.0.0_serial --enable-fortran --enable-cxx --enable-shared --with-zlib=/usr/include,/usr/lib64

My guess is that for the parallel build, the default FC might be mpif90, which was probably configured to wrap around flang for the modules that you loaded.

Anyhow, as it is working now, no real need to change anything, right?

If all is solved now, please let me know, so I can lock the topic.