Intel MPI startup freeze issue
If you are observing issues where Intel MPI applications freeze during startup on large core counts, please try to set I_MPI_HYDRA_BOOTSTRAP=ssh in order to use the SSH bootstrap mechanism instead of blaunch.
Intel MPI is currently having issues with blaunch on larger jobs, the issue will be fixed in upcoming releases.
Common error for segmentation violation
It mainly happens to the fortran programs as static arrays are allocated on stack by default. When the job runs on the cluster it might exceed the stack size and therefore run into a segmentation violation (SIGSEV - 11). You can address this problem by setting "ulimit -s unlimited" in your ~/.bashrc .
Please keep in mind to specify the target architecture when compiling in order to allow the compiler to automatically vectorize your code with the best suitable vector instruction set. This can be done by using the "-x" compiler flag followed by the desired architecture. For the Aaditya cluster with the SandyBridge architecture, you may use "-xAVX". Alternatively you can always use "-xhost" in order to compile for the current architecture.
In case you observe MPI crashes where DAPL library error messages occour, please try to use the most recent DAPL library instead. This can be accomplished by setting
LD_LIBRARY_PATH=/gpfs1/home/Libs/GNU/DAPL/lib:$LD_LIBRARY_PATH in your ~/.bashrc