Zenn
😸

OpenCFDのコンパイル

2022/03/15に公開
WSL
#
OpenFOAM
#
freecad
tech

FreeCADのアドオンCfdOFで並列計算するとエラーになったので、試しにOpenCFDをコンパイルして、確認してみます。(コンパイルに必要なものはインストール済みのつもり。)

手元の環境はWindows 11 WSL Ubutun 20.04になります。

ESI-OpenCFD版のv2112を使います。ソースをダウンロードして解凍します。

$ wget https://dl.openfoam.com/source/v2112/OpenFOAM-v2112.tgz
$ wget https://dl.openfoam.com/source/v2112/ThirdParty-v2112.tgz
$ mkdir ~/src
$ cd ~/src
$ tar zxvf ~/OpenFOAM-v2112.tgz
$ tar zxvf ~/ThirdParty-v2112.tgz

標準の設定でコンパイル

とりあえず、デフォルトのままやってみます。

大きな設定はOpenFOAM-v2112/etc/bashrcに記載してあります。一部抜粋すると、デフォルトの設定はこんなです。

OpenFOAM-v2112/etc/bashrc
# [WM_COMPILER_TYPE] - Compiler location:
# = system | ThirdParty
export WM_COMPILER_TYPE=system

# [WM_COMPILER] - Compiler:
# = Gcc | Clang | Icc | Icx | Amd | Arm | Cray | Fujitsu |
#   Gcc<digits> | Clang<digits>
#   [Not well tested: Pgi | Nvidia]
export WM_COMPILER=Gcc

# [WM_MPLIB] - MPI implementation:
# = SYSTEMOPENMPI | OPENMPI | SYSTEMMPI | MPI | MPICH | MPICH-GM |
#   HPMPI | CRAY-MPICH | FJMPI | QSMPI | SGIMPI | INTELMPI | USERMPI
# Specify SYSTEMOPENMPI1, SYSTEMOPENMPI2 for internal tracking (if desired)
# Can also use INTELMPI-xyz etc and define your own wmake rule
export WM_MPLIB=SYSTEMOPENMPI

ThirdPartyのコンパイル

初めにThirdPartyをコンパイルします。

$ source ~/src/OpenFOAM-v2112/etc/bashrc
$ cd ~/src/ThirdParty-v2112
$ ./Allwmake -j

METISとADIOS2でエラーが出ました。

METISのエラー
========================================
METIS decomposition
Did not find ... metis-5.1.0
Missing sources: 'metis-5.1.0'
Possible download locations for metis :
    http://glaros.dtc.umn.edu/gkhome/metis/metis/overview
    http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/metis-5.1.0.tar.gz

    ---------------------------------------------------
    Optional component (METIS) had build issues
    OpenFOAM will nonetheless remain largely functional
    ---------------------------------------------------
ADIOS2のエラー
========================================
ADIOS2
Did not find ... ADIOS2-2.6.0

========================================
Build adios library ADIOS2-2.6.0 for sys-openmpi

Starting build: ADIOS2-2.6.0 (using cmake)

./makeAdios2: 191: cd: can't cd to $HOME/src/ThirdParty-v2112/ADIOS2-2.6.0
Error building: ADIOS2-2.6.0

    ---------------------------------------------------
    Optional component (ADIOS2) had build issues
    OpenFOAM will nonetheless remain largely functional
    ---------------------------------------------------

どちらもソースがない模様。

METISのソースをダウンロードして、ThirdParty-v2112/sourcesに展開。

$ wget http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/metis-5.1.0.tar.gz
$ tar zxvf metis-5.1.0.tar.gz -C sources

ADIOS2は2.6.0ではなく2.7.1があったので、そちらを使うことにします。解凍するとADIOS2-2.7.1というディレクトリができます。

$ wget https://github.com/ornladios/ADIOS2/archive/refs/tags/v2.7.1.tar.gz
$ tar zxvf v2.7.1.tar.gz

使用するADIOS2のバージョンをデフォルトから変更するので、OpenFOAM-v2112/etc/config.sh/adios2を編集します。

OpenFOAM-v2112/etc/config.sh/adios2
--- OpenFOAM-v2112/etc/config.sh/adios2,orig    2021-12-21 03:57:43.000000000 +0900
+++ OpenFOAM-v2112/etc/config.sh/adios2 2022-03-13 21:49:25.779751600 +0900
@@ -21,7 +21,7 @@
 #------------------------------------------------------------------------------
 # USER EDITABLE PART: Changes made here may be lost with the next upgrade

-adios2_version=ADIOS2-2.6.0
+adios2_version=ADIOS2-2.7.1
 export ADIOS2_ARCH_PATH="$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER/$adios2_version"

 # END OF (NORMAL) USER EDITABLE PART

OpenFOAM-v2112/etc/bashrcの再読み込みは必要ないかもしれないけど、いちおう。

$ source ~/src/OpenFOAM-v2112/etc/bashrc
$ ./Allwmake -j

で、コンパイルできました。

OpenFOAMのコンパイル

ThirdPartyのコンパイルが終わったので、次にOpenFOAMをコンパイルします。

$ cd ~/src/OpenFOAM-v2112
$ ./Allwmake -j -s -l

CfdOFで動作確認

OpenFOAMのコンパイルも終わったら、CfdOFで参照するOpenFOAMのインストールディレクトリの設定を修正して、動作確認をしてみます。

む、ダメです。並列計算の実行でエラーとなりました。

CfdOFから並列計算の実行でのエラー
--> FOAM FATAL ERROR: (openfoam-2112)
attempt to run parallel on 1 processor

    From static bool Foam::UPstream::init(int&, char**&, bool)
    in file UPstream.C at line 297.

FOAM aborting

#0  Foam::error::printStack(Foam::Ostream&)

--> FOAM FATAL ERROR: (openfoam-2112)
attempt to run parallel on 1 processor

    From static bool Foam::UPstream::init(int&, char**&, bool)
    in file UPstream.C at line 297.

FOAM aborting

 at ??:?
#1  Foam::error::simpleExit(int, bool)#0  Foam::error::printStack(Foam::Ostream&)

--> FOAM FATAL ERROR: (openfoam-2112)
attempt to run parallel on 1 processor

    From static bool Foam::UPstream::init(int&, char**&, bool)
    in file UPstream.C at line 297.

FOAM aborting

#0  Foam::error::printStack(Foam::Ostream&) at ??:?
#2  Foam::error::exiting(int, bool) at ??:?
#1  Foam::error::simpleExit(int, bool)

--> FOAM FATAL ERROR: (openfoam-2112)
attempt to run parallel on 1 processor

    From static bool Foam::UPstream::init(int&, char**&, bool)
    in file UPstream.C at line 297.

FOAM aborting

#0  Foam::error::printStack(Foam::Ostream&) at ??:?
 at ??:?
#1  Foam::error::simpleExit(int, bool)#2  Foam::error::exiting(int, bool) at ??:?
#3  Foam::UPstream::init(int&, char**&, bool) at ??:?
#4  Foam::argList::argList(int&, char**&, bool, bool, bool) at ??:?
#2  Foam::error::exiting(int, bool) at ??:?
#3  Foam::UPstream::init(int&, char**&, bool) at ??:?
#5   at ??:?
#4  Foam::argList::argList(int&, char**&, bool, bool, bool) at ??:?
#3  Foam::UPstream::init(int&, char**&, bool) at ??:?
#4  Foam::argList::argList(int&, char**&, bool, bool, bool) at ??:?
#1  Foam::error::simpleExit(int, bool)? at ??:?
#5   in ~/src/OpenFOAM-v2112/platforms/linux64GccDPInt32Opt/bin/potentialFoam
#6  __libc_start_main at ??:?
#5   in /lib/x86_64-linux-gnu/libc.so.6
#7  ?? at ??:?
#2  Foam::error::exiting(int, bool)? in ~/src/OpenFOAM-v2112/platforms/linux64GccDPInt32Opt/bin/potentialFoam
#6  __libc_start_main in ~/src/OpenFOAM-v2112/platforms/linux64GccDPInt32Opt/bin/potentialFoam
[OPTIPLEX:00691] *** Process received signal ***
[OPTIPLEX:00691] Signal: Aborted (6)
[OPTIPLEX:00691] Signal code:  (-6)
[OPTIPLEX:00691] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x143c0)[0x7f82150b83c0]
[OPTIPLEX:00691] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f8214ef503b]
[OPTIPLEX:00691] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f8214ed4859]
[OPTIPLEX:00691] [ 3] /home/zzzzz/src/OpenFOAM-v2112/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam5error10simpleExitEib+0x17a)[0x7f8215823d2a]
[OPTIPLEX:00691] [ 4] /home/zzzzz/src/OpenFOAM-v2112/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam5error7exitingEib+0x4a)[0x7f821582732a]
[OPTIPLEX:00691] [ 5] /home/zzzzz/src/OpenFOAM-v2112/platforms/linux64GccDPInt32Opt/lib/sys-openmpi/libPstream.so(_ZN4Foam8UPstream4initERiRPPcb+0x1ad9)[0x7f8214ea2dc9]
[OPTIPLEX:00691] [ 6] /home/zzzzz/src/OpenFOAM-v2112/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam7argListC1ERiRPPcbbb+0x7e2)[0x7f8215853f42]
[OPTIPLEX:00691] [ 7] potentialFoam(+0x31ed3)[0x56249e35aed3]
[OPTIPLEX:00691] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f8214ed60b3]
[OPTIPLEX:00691] [ 9] potentialFoam(+0x347ee)[0x56249e35d7ee]
[OPTIPLEX:00691] *** End of error message ***
 in ~/src/OpenFOAM-v2112/platforms/linux64GccDPInt32Opt/bin/potentialFoam
#6  __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 690 RUNNING AT OPTIPLEX
=   EXIT CODE: 9
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Terminated (signal 15)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestion

使用するOpenMPIを変更

OpenMPIをコンパイルして使うようにしてみます。

OpenFOAM-v2112/etc/bashrc
--- OpenFOAM-v2112/etc/bashrc,orig      2021-12-21 03:57:43.000000000 +0900
+++ OpenFOAM-v2112/etc/bashrc   2022-03-14 21:55:31.189751600 +0900
@@ -95,7 +95,7 @@
 #   HPMPI | CRAY-MPICH | FJMPI | QSMPI | SGIMPI | INTELMPI | USERMPI
 # Specify SYSTEMOPENMPI1, SYSTEMOPENMPI2 for internal tracking (if desired)
 # Can also use INTELMPI-xyz etc and define your own wmake rule
-export WM_MPLIB=SYSTEMOPENMPI
+export WM_MPLIB=OPENMPI


 #------------------------------------------------------------------------------

再度、ThirdPartyのコンパイル

$ source ~/src/OpenFOAM-v2112/etc/bashrc
$ cd ~/src/ThirdParty-v2112
$ ./Allwmake -j

なんか、SCOTCHでエラーがあったみたいです。

SCOTCHのエラー
make                                    \
                                CC="/home/zzzzz/src/ThirdParty-v2112/platforms/linux64Gcc/openmpi-4.1.2/bin/mpicc"  \
                                CFLAGS="-m64 -DOPENFOAM=2112 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -O3 -fPIC -fPIC -O3 -UCOMMON_FILE_COMPRESS -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_DETERMINISTIC -DSCOTCH_RENAME -DIDXSIZE64 -Drestrict=__restrict -DSCOTCH_PTSCOTCH"           \
                                CCDFLAGS="-m64 -DOPENFOAM=2112 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -O3 -fPIC -fPIC -O3 -UCOMMON_FILE_COMPRESS -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_DETERMINISTIC -DSCOTCH_RENAME -DIDXSIZE64 -Drestrict=__restrict -DSCOTCH_PTSCOTCH" \
                                ptscotch.h                                      \
                                ptscotchf.h                                     \
                                libptscotch.so                          \
                                libptscotcherr.so                               \
                                libptscotcherrexit.so
make[2]: ディレクトリ '/home/zzzzz/src/ThirdParty-v2112/sources/scotch/scotch_6.1.0/src/libscotch' に入ります
/home/zzzzz/src/ThirdParty-v2112/platforms/linux64Gcc/openmpi-4.1.2/bin/mpicc -m64 -DOPENFOAM=2112 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -O3 -fPIC -fPIC -O3 -UCOMMON_FILE_COMPRESS -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_DETERMINISTIC -DSCOTCH_RENAME -DIDXSIZE64 -Drestrict=__restrict -DSCOTCH_PTSCOTCH -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=1 -DSCOTCH_PATCHLEVEL_NUM=0 dummysizes.c -o ptdummysizes -Xlinker --no-as-needed -m64 -lm -lrt
In file included from dummysizes.c:92:
common.h:123:21: fatal error: mpi.h: そのようなファイルやディレクトリはありません
  123 | #include            <mpi.h>
      |                     ^~~~~~~
compilation terminated.

ThirdParty-v2112に入っているSCOTCHはバージョン6.1.0ですが、バージョン7.0.1と6.1.3が出ているようなので、6.1.3を使ってみます。
https://gitlab.inria.fr/scotch/scotch/-/releases

$ wget https://gitlab.inria.fr/scotch/scotch/-/archive/v6.1.3/scotch-v6.1.3.tar.gz
$ tar zxvf scotch-v6.1.3.tar.gz -C sources/scotch
$ mv sources/scotch/scotch-v6.1.3 sources/scotch/scotch_6.1.3

使用するSCOTCHのバージョンを変えるので、OpenFOAM-v2112/etc/config.sh/scothを修正します。

OpenFOAM-v2112/etc/config.sh/scotch
--- OpenFOAM-v2112/etc/config.sh/scotch,orig    2021-12-21 03:57:43.000000000 +0900
+++ OpenFOAM-v2112/etc/config.sh/scotch 2022-03-14 23:09:30.169751600 +0900
@@ -33,7 +33,7 @@
 #------------------------------------------------------------------------------
 # USER EDITABLE PART: Changes made here may be lost with the next upgrade

-SCOTCH_VERSION=scotch_6.1.0
+SCOTCH_VERSION=scotch_6.1.3
 export SCOTCH_ARCH_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER$WM_PRECISION_OPTION$WM_LABEL_OPTION/$SCOTCH_VERSION

 # END OF (NORMAL) USER EDITABLE PART

で、再度、コンパイル。

$ ./Allwmake -j

再度、OpenFOAMのコンパイル

ThirdPartyのコンパイルが終わったら、OpenFOAMを再度コンパイルします。

$ cd ~/src/OpenFOAM-v2112
$ ./Allwmake -j -s -l

CfdOFで動作確認

今度はうまく並列計算を計算実行できました！

Discussion