Open6

Hark 3.5.0 のソースコードからのビルドとインストールとDocker Imageビルド

PINTOPINTO

HARK is open-sourced robot audition software consisting of sound source localization modules, sound source separation modules and automatic speech recognition modules of separated speech signals that work on any robot with any microphone configuration.

Since a robot with ears may be deployed to various auditory environments, the robot audition system should provide an easy way to adapt to them. HARK provides a set of modules to cope with various auditory environments by using an open-sourced middleware, FlowDesigner and reduces the overheads of data transfer between modules.

HARK has been open-sourced since April 2008. The resulting implementation of HARK with MUSIC-based sound source localization, GHDSS-based sound source separation and Missing-Feature-Theory-based automatic speech recognition (ASR) on several robots like HRP-2, SIG, SIG2, and Robovie R2 attains recognizing three simultaneous utterances in real time.

https://hark.jp/install/linux/

https://hark.jp/kaldidecoder-source-compilation/

https://qiita.com/keinko/items/7710d4099d4782b79b0d

https://qiita.com/daisukehark/questions/115ff13fe3a44de903f6

https://github.com/alphacep/vosk-api/issues/1134

PINTOPINTO

Hark 3.5.0 CPUバージョンをソースコードからすべてビルドして Docker 環境として構築する。
下記のページの
https://hark.jp/install/linux/
Installation from Source Compilation の箇所から順番にトレースしていく。

mkdir -p work/audio/hark && cd work/audio/hark
PINTOPINTO

手順の最後の Kaldi のビルドで下記のエラーが発生する。(エラーメッセージを省略表記)
解決策はこれ。かなり面倒。
https://github.com/alphacep/vosk-api/issues/1134

In file included from tp-matrix.cc:24:
../matrix/cblas-wrappers.h: In function 'void kaldi::clapack_Xtptri(KaldiBlasInt*, float*, KaldiBlasInt*)':
../matrix/cblas-wrappers.h:386:10: error: too few arguments to function 'void stptri_(const char*, const char*, const int*, float*, int*, size_t, size_t)'
  386 |   stptri_(const_cast<char *>("U"), const_cast<char *>("N"), num_rows, Mdata, result);
      |   ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/lapack.h:11,
                 from /usr/include/lapacke.h:36,
                 from ../matrix/kaldi-blas.h:100,
                 from ../matrix/cblas-wrappers.h:29,
                 from tp-matrix.cc:24:
/usr/include/lapack.h:21096:6: note: declared here
21096 | void LAPACK_stptri_base(
      |      ^~~~~~~~~~~~~~~~~~
In file included from tp-matrix.cc:24:
../matrix/cblas-wrappers.h: In function 'void kaldi::clapack_Xtptri(KaldiBlasInt*, double*, KaldiBlasInt*)':
../matrix/cblas-wrappers.h:389:10: error: too few arguments to function 'void dtptri_(const char*, const char*, const int*, double*, int*, size_t, size_t)'
  389 |   dtptri_(const_cast<char *>("U"), const_cast<char *>("N"), num_rows, Mdata, result);
      |   ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  :
  :

/usr/include/lapack.h:15908:6: note: declared here
15908 | void LAPACK_ssptrf_base(
      |      ^~~~~~~~~~~~~~~~~~
In file included from ../matrix/jama-svd.h:34,
                 from kaldi-matrix.cc:27:
../matrix/cblas-wrappers.h: In function 'void kaldi::clapack_Xsptrf(KaldiBlasInt*, double*, KaldiBlasInt*, KaldiBlasInt*)':
../matrix/cblas-wrappers.h:451:10: error: too few arguments to function 'void dsptrf_(const char*, const int*, double*, int*, int*, size_t)'
  451 |   dsptrf_(const_cast<char *>("U"), num_rows, Mdata, ipiv, result);
      |   ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/lapack.h:11,
                 from /usr/include/lapacke.h:36,
                 from ../matrix/kaldi-blas.h:100,
                 from ../matrix/cblas-wrappers.h:29,
                 from ../matrix/jama-svd.h:34,
                 from kaldi-matrix.cc:27:
/usr/include/lapack.h:15892:6: note: declared here
15892 | void LAPACK_dsptrf_base(
      |      ^~~~~~~~~~~~~~~~~~
make[1]: *** [<builtin>: tp-matrix.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [<builtin>: qr.o] Error 1
make[1]: *** [<builtin>: packed-matrix.o] Error 1
make[1]: *** [<builtin>: sp-matrix.o] Error 1
make[1]: *** [<builtin>: kaldi-vector.o] Error 1
make[1]: *** [<builtin>: kaldi-matrix.o] Error 1
make[1]: Leaving directory '/workdir/kaldi/src/matrix'
make: *** [Makefile:164: matrix] Error 2
PINTOPINTO

Kaldi を Fork して問題点のソースコードをすべて修正コミット。

https://github.com/PINTO0309/kaldi

修正点は kaldi フォルダ直下で下記のコマンドを実行することで確認可能。

# fix_openblas というブランチをデフォルトブランチに切り替え済み
git clone https://github.com/PINTO0309/kaldi.git
cd kaldi
git diff HEAD~1

diff ファイルに出力する場合は下記。

git diff HEAD~1 > diff_output.diff
PINTOPINTO
Dockerfile.cpu
# Hark 3.5.0 - CPU ver

FROM ubuntu:22.04

ENV DEBIAN_FRONTEND=noninteractive
ENV USERNAME=user
ARG WKDIR=/workdir
ARG HARKVER=3.5.0
ARG DISTRO=jammy

# https://hark.jp/install/linux/
RUN apt-get update \
    && apt-get install -y \
        build-essential \
        gfortran \
        libtool \
        cmake \
        cmake-extras \
        curl \
        git \
        pkgconf \
        nano \
        libxml2-dev \
        libzip-dev \
        libasound2-dev \
        libsndfile1-dev \
        libsdl2-dev \
        libopenblas-dev \
        liblapack-dev \
        liblapacke-dev \
        libeigen3-dev \
        libfmt-dev \
        libgtk2.0-dev \
        libpopt-dev \
        python-is-python3 \
        mosquitto \
        libmosquittopp-dev \
        lsb-release \
        pybind11-dev \
        python2.7 \
        python3-dev \
        python3-daemon \
        python3-paho-mqtt \
        python3-pybind11 \
        python3-pkgconfig \
        python3-pip \
        python3-setuptools \
        python3-scipy \
        python3-matplotlib \
        python3-kivy \
        sudo \
        automake \
        autoconf \
        zlib1g-dev \
        libopenblas-base \
        libopenblas-dev \
        sox \
        subversion \
        unzip \
        wget \
    && sed -i 's/# set linenumbers/set linenumbers/g' /etc/nanorc

RUN python -m pip install kivy_garden.graph \
        --extra-index-url https://kivy-garden.github.io/simple/ \
    && pip install numpy==1.24.3

RUN echo "root:root" | chpasswd \
    && adduser --disabled-password --gecos "" "${USERNAME}" \
    && echo "${USERNAME}:${USERNAME}" | chpasswd \
    && echo "%${USERNAME}    ALL=(ALL)   NOPASSWD:    ALL" >> /etc/sudoers.d/${USERNAME} \
    && chmod 0440 /etc/sudoers.d/${USERNAME} \
    && mkdir -p ${WKDIR} \
    && chown ${USERNAME}:${USERNAME} ${WKDIR}

USER ${USERNAME}
WORKDIR ${WKDIR}

RUN sudo curl -sSL http://archive.hark.jp/harkrepos/public.gpg -o /usr/share/keyrings/hark-archive-keyring.asc \
    && sudo bash -c 'echo -e "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hark-archive-keyring.asc] \
    http://archive.hark.jp/harkrepos $(lsb_release -cs) non-free\ndeb-src [arch=$(dpkg --print-architecture) \
    signed-by=/usr/share/keyrings/hark-archive-keyring.asc] http://archive.hark.jp/harkrepos \
    $(lsb_release -cs) non-free" > /etc/apt/sources.list.d/hark.list'

RUN sudo apt-get update

# build & install libhark-netapi3
RUN sudo apt-get source libhark-netapi3 \
    && cd libhark-netapi3-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir -p build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=/usr .. \
    && make \
    && sudo make install

# build & install libharkio3
RUN sudo apt-get source libharkio3 \
    && cd libharkio3-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=/usr .. \
    && make \
    && sudo make install

# build & install hark-base3
RUN sudo apt-get source hark-base3 \
    && cd hark-base3-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=/usr .. \
    && make \
    && sudo make install

# build & install harkmw
RUN sudo apt-get source harkmw \
    && cd harkmw-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && python setup.py build \
    && sudo python setup.py install

# build & install hark-core
RUN sudo apt-get source hark-core \
    && cd hark-core-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=/usr .. \
    && make \
    && sudo make install

# build & install hark-linux
RUN sudo apt-get source hark-linux \
    && cd hark-linux-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=/usr -DENABLE_RASP=OFF .. \
    && make \
    && sudo make install

# build & install hark-gtkplot
RUN sudo apt-get source hark-gtkplot \
    && cd hark-gtkplot-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=/usr .. \
    && make \
    && sudo make install

# build & install python3-harkpython, hark-python3
RUN sudo apt-get source python3-harkpython \
    && sudo apt-get source hark-python3 \
    && cd python3-harkpython-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && sudo python setup.py install \
    && cd .. \
    && cd hark-python3-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=/usr .. \
    && make \
    && sudo make install

# build & install harktool5
RUN sudo apt-get source harktool5 \
    && cd harktool5-*.*.* \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=/usr .. \
    && make \
    && sudo make install

RUN \
    ## Kaldi’s external tool
    git clone https://github.com/PINTO0309/kaldi.git \
    && cd kaldi/tools \
    && extras/check_dependencies.sh \
    && make \
    ## Kaldi with the set up to use openblas library
    && cd ../src \
    && mkdir -p fakeroot \
    && ln -s /usr/include/x86_64-linux-gnu/ fakeroot/include \
    && ln -s /usr/lib/x86_64-linux-gnu/ fakeroot/lib \
    && ./configure --mathlib=OPENBLAS --openblas-root=fakeroot \
    && make clean -j $(nproc) \
    && make depend -j $(nproc) \
    && make -j $(nproc) \
    # build & install KaldiDecoder
    && cd .. \
    && wget http://archive.hark.jp/harkrepos/dists/${DISTRO}/non-free/source/kaldidecoder-hark_${HARKVER}.tar.xz \
    && tar -Jxvf kaldidecoder-hark_${HARKVER}.tar.xz \
    && cd kaldidecoder3 \
    && sudo chown -R ${USERNAME}:${USERNAME} . \
    && mkdir -p build \
    && cd build \
    && cmake .. -DOPENBLAS_ROOT_DIR:STRING=/usr -DCMAKE_VERBOSE_MAKEFILE:BOOL=TRUE \
    && make \
    && sudo make install

# GUI install
RUN sudo apt-get install -y \
    hark-designer \
    harktool5-gui
PINTOPINTO

以下、完成したシーケンスで docker イメージを生成するコマンド。kaldi のビルドに20スレッドで10分以上掛かる。

docker build -t pinto0309/hark_cpu:3.5.0 -f Dockerfile.cpu .

万人のため、ビルドしたイメージを docker hub へ push しておく。僕以外の人たちはビルドする必要なし。

docker push pinto0309/hark_cpu:3.5.0