OpenGrok→Herokuにデプロイ

Yusuke Iwaki 2021/04/17

FROM adoptopenjdk/openjdk11:alpine

RUN mkdir -p /opengrok/dist
RUN wget https://github.com/oracle/opengrok/releases/download/1.6.9/opengrok-1.6.9.tar.gz \
    && tar -C /opengrok/dist --strip-components=1 -xzf opengrok-1.6.9.tar.gz \
    && rm opengrok-1.6.9.tar.gz

RUN mkdir /usr/local/tomcat
RUN wget https://www-eu.apache.org/dist/tomcat/tomcat-10/v10.0.5/bin/apache-tomcat-10.0.5.tar.gz \
    && tar -C /usr/local/tomcat --strip-components=1 -xzf apache-tomcat-10.0.5.tar.gz \
    && rm apache-tomcat-10.0.5.tar.gz

RUN apk add --no-cache git ctags python3 py3-pip
RUN python3 -m pip install /opengrok/dist/tools/opengrok-tools.tar.gz

# compile and install universal-ctags
RUN git clone https://github.com/universal-ctags/ctags /tmp/ctags \
    && cd /tmp/ctags \
    && apk add --no-cache --virtual=build-dependencies autoconf automake build-base pkgconfig \
    && ./autogen.sh && ./configure && make && make install \
    && apk del build-dependencies && rm -rf /tmp/ctags

ENV ENTRYKIT_VERSION 0.4.0
RUN wget https://github.com/progrium/entrykit/releases/download/v${ENTRYKIT_VERSION}/entrykit_${ENTRYKIT_VERSION}_Linux_x86_64.tgz \
    && tar -xvzf entrykit_${ENTRYKIT_VERSION}_Linux_x86_64.tgz \
    && rm entrykit_${ENTRYKIT_VERSION}_Linux_x86_64.tgz \
    && mv entrykit /bin/entrykit \
    && chmod +x /bin/entrykit \
    && entrykit --symlink

# environment variables
ENV SRC_ROOT /opengrok/src
ENV DATA_ROOT /opengrok/data
ENV OPENGROK_WEBAPP_CONTEXT /
ENV OPENGROK_TOMCAT_BASE /usr/local/tomcat
ENV OPENGROK_INDEX_OPTIONS="-H -P -S -G"
ENV CATALINA_HOME /usr/local/tomcat
ENV CATALINA_BASE /usr/local/tomcat
ENV CATALINA_TMPDIR /usr/local/tomcat/temp
ENV PATH $CATALINA_HOME/bin:$PATH
ENV JRE_HOME /opt/java/openjdk
ENV CLASSPATH /usr/local/tomcat/bin/bootstrap.jar:/usr/local/tomcat/bin/tomcat-juli.jar

# disable all file logging
RUN wget https://raw.githubusercontent.com/oracle/opengrok/476443edddbb606ccf738095d424034494b8dd71/docker/logging.properties -O /usr/local/tomcat/conf/logging.properties
RUN sed -i -e 's/Valve/Disabled/' /usr/local/tomcat/conf/server.xml

# put source codes. （ここが検索対象のソースコードをDockerイメージにぶちこむところ！）
RUN mkdir -p /opengrok/src /opengrok/data
COPY lib /opengrok/src/lib
COPY spec /opengrok/src/spec

# custom deployment to / with redirect from /source
RUN rm -rf /usr/local/tomcat/webapps/* \
    && mkdir -p /opengrok/etc \
    && opengrok-deploy -c /opengrok/etc/configuration.xml /opengrok/dist/lib/source.war /usr/local/tomcat/webapps/ROOT.war \
    && mkdir "/usr/local/tomcat/webapps/source" \
    && echo '<% response.sendRedirect("/"); %>' > "/usr/local/tomcat/webapps/source/index.jsp"

# indexing
RUN opengrok-indexer \
    -a /opengrok/dist/lib/opengrok.jar -- \
    -c /usr/local/bin/ctags \
    -s /opengrok/src \
    -d /opengrok/data \
    $OPENGROK_INDEX_OPTIONS \
    -W /opengrok/etc/configuration.xml

RUN echo "#!/bin/sh" > /start.sh \
    && echo "cd /usr/local/tomcat" >> /start.sh \
    && echo "sed -i -e 's/Connector port=\"8080\"/Connector port=\"\${httpPort}\"/' conf/server.xml" >> /start.sh \
    && echo "JAVA_OPTS=\"-DhttpPort=\$PORT\" catalina.sh run" >> /start.sh

WORKDIR $CATALINA_HOME
CMD ["sh", "/start.sh"]

ローカルだとこうなるのに

herokuだとこうなる

Yusuke Iwaki 2021/04/17

Heroku側のファイル構成を覗いてみる

    && echo "ls -l /usr/local/tomcat" >> /start.sh \
    && echo "ls -l /usr/local/tomcat/webapps" >> /start.sh \
    && echo "ls -l /usr/local/tomcat/webapps/source" >> /start.sh \

2021-04-16T15:16:21.537491+00:00 heroku[web.1]: Starting process with command `/bin/sh -c sh\ /start.sh`
2021-04-16T15:16:24.149335+00:00 app[web.1]: total 152
2021-04-16T15:16:24.149509+00:00 app[web.1]: -rw-------    1 u29849   dyno         19010 Mar 30 08:19 BUILDING.txt
2021-04-16T15:16:24.149509+00:00 app[web.1]: -rw-------    1 u29849   dyno          5644 Mar 30 08:19 CONTRIBUTING.md
2021-04-16T15:16:24.149509+00:00 app[web.1]: -rw-------    1 u29849   dyno         60269 Mar 30 08:19 LICENSE
2021-04-16T15:16:24.149510+00:00 app[web.1]: -rw-------    1 u29849   dyno          2333 Mar 30 08:19 NOTICE
2021-04-16T15:16:24.149510+00:00 app[web.1]: -rw-------    1 u29849   dyno          3372 Mar 30 08:19 README.md
2021-04-16T15:16:24.149510+00:00 app[web.1]: -rw-------    1 u29849   dyno          7008 Mar 30 08:19 RELEASE-NOTES
2021-04-16T15:16:24.149510+00:00 app[web.1]: -rw-------    1 u29849   dyno         16517 Mar 30 08:19 RUNNING.txt
2021-04-16T15:16:24.149511+00:00 app[web.1]: drwx------    2 u29849   dyno          4096 Apr 16 15:14 bin
2021-04-16T15:16:24.149511+00:00 app[web.1]: drwx------    1 u29849   dyno          4096 Apr 16 15:15 conf
2021-04-16T15:16:24.149511+00:00 app[web.1]: drwx------    2 u29849   dyno          4096 Apr 16 15:14 lib
2021-04-16T15:16:24.149512+00:00 app[web.1]: drwx------    2 u29849   dyno          4096 Mar 30 08:19 logs
2021-04-16T15:16:24.149512+00:00 app[web.1]: drwx------    2 u29849   dyno          4096 Apr 16 15:14 temp
2021-04-16T15:16:24.149512+00:00 app[web.1]: drwx------    1 u29849   dyno          4096 Apr 16 15:15 webapps
2021-04-16T15:16:24.149512+00:00 app[web.1]: drwx------    2 u29849   dyno          4096 Mar 30 08:19 work
2021-04-16T15:16:24.150762+00:00 app[web.1]: total 46808
2021-04-16T15:16:24.151034+00:00 app[web.1]: drwx------    3 u29849   dyno          4096 Apr 16 15:14 ROOT
2021-04-16T15:16:24.151035+00:00 app[web.1]: -rw-------    1 u29849   dyno      47904969 Apr 16 15:15 ROOT.war
2021-04-16T15:16:24.151035+00:00 app[web.1]: drwx------   15 u29849   dyno          4096 Apr 16 15:14 docs
2021-04-16T15:16:24.151036+00:00 app[web.1]: drwx------    7 u29849   dyno          4096 Apr 16 15:14 examples
2021-04-16T15:16:24.151036+00:00 app[web.1]: drwx------    6 u29849   dyno          4096 Apr 16 15:14 host-manager
2021-04-16T15:16:24.151036+00:00 app[web.1]: drwx------    6 u29849   dyno          4096 Apr 16 15:14 manager
2021-04-16T15:16:24.151037+00:00 app[web.1]: drwx------    2 u29849   dyno          4096 Apr 16 15:15 source
2021-04-16T15:16:24.151829+00:00 app[web.1]: total 4
2021-04-16T15:16:24.151915+00:00 app[web.1]: -rw-------    1 u29849   dyno            34 Apr 16 15:15 index.jsp
2021-04-16T15:16:24.161129+00:00 app[web.1]: NOTE: Picked up JDK_JAVA_OPTIONS:  --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED

ローカル環境だと↓

$ docker run --rm -it -p 8081:8081 -e PORT=8081 opengrok sh
/usr/local/tomcat # ls -l
total 152
-rw-r-----    1 root     root         19010 Mar 30 08:19 BUILDING.txt
-rw-r-----    1 root     root          5644 Mar 30 08:19 CONTRIBUTING.md
-rw-r-----    1 root     root         60269 Mar 30 08:19 LICENSE
-rw-r-----    1 root     root          2333 Mar 30 08:19 NOTICE
-rw-r-----    1 root     root          3372 Mar 30 08:19 README.md
-rw-r-----    1 root     root          7008 Mar 30 08:19 RELEASE-NOTES
-rw-r-----    1 root     root         16517 Mar 30 08:19 RUNNING.txt
drwxr-x---    2 root     root          4096 Apr 16 13:26 bin
drwx------    1 root     root          4096 Apr 16 13:32 conf
drwxr-x---    2 root     root          4096 Apr 16 13:26 lib
drwxr-x---    2 root     root          4096 Mar 30 08:19 logs
drwxr-x---    2 root     root          4096 Apr 16 13:26 temp
drwxr-x---    1 root     root          4096 Apr 16 14:44 webapps
drwxr-x---    2 root     root          4096 Mar 30 08:19 work
/usr/local/tomcat # ls -l webapps
total 46788
-rw-r--r--    1 root     root      47904969 Apr 16 14:44 ROOT.war
drwxr-xr-x    2 root     root          4096 Apr 16 14:44 source
/usr/local/tomcat # ls -l webapps/source/
total 4
-rw-r--r--    1 root     root            34 Apr 16 14:44 index.jsp
/usr/local/tomcat # exit

Heroku側には ROOT/ っていう見に覚えのないディレクトリがある？？！！

Yusuke Iwaki 2021/04/17

ちなみに、HerokuにDockerizedなTomcatデプロイするとなんかおかしいって書いてる人は、居るにはいる。

Yusuke Iwaki 2021/04/17に更新

Heroku側の ROOT/ の中身

2021-04-16T15:22:48.645580+00:00 app[web.1]: -rw-------    1 u23714   dyno          7008 Mar 30 08:19 RELEASE-NOTES.txt
2021-04-16T15:22:48.645581+00:00 app[web.1]: drwx------    2 u23714   dyno          4096 Apr 16 15:20 WEB-INF
2021-04-16T15:22:48.645581+00:00 app[web.1]: -rw-------    1 u23714   dyno         27235 Mar 30 08:19 asf-logo-wide.svg
2021-04-16T15:22:48.645582+00:00 app[web.1]: -rw-------    1 u23714   dyno           713 Mar 30 08:19 bg-button.png
2021-04-16T15:22:48.645582+00:00 app[web.1]: -rw-------    1 u23714   dyno          1918 Mar 30 08:19 bg-middle.png
2021-04-16T15:22:48.645583+00:00 app[web.1]: -rw-------    1 u23714   dyno          1401 Mar 30 08:19 bg-nav.png
2021-04-16T15:22:48.645583+00:00 app[web.1]: -rw-------    1 u23714   dyno          3103 Mar 30 08:19 bg-upper.png
2021-04-16T15:22:48.645583+00:00 app[web.1]: -rw-------    1 u23714   dyno         21630 Mar 30 08:19 favicon.ico
2021-04-16T15:22:48.645584+00:00 app[web.1]: -rw-------    1 u23714   dyno         12167 Mar 30 08:19 index.jsp
2021-04-16T15:22:48.645584+00:00 app[web.1]: -rw-------    1 u23714   dyno          5542 Mar 30 08:19 tomcat.css
2021-04-16T15:22:48.645585+00:00 app[web.1]: -rw-------    1 u23714   dyno         67795 Mar 30 08:19 tomcat.svg
2021-04-16T15:22:48.651466+00:00 app[web.1]: total 4

デフォルトコンテンツっぽい。いらん。

Yusuke Iwaki 2021/04/17

2021-04-16T15:34:39.079039+00:00 app[web.1]: 16-Apr-2021 15:34:39.074 INFO [main] org.hibernate.validator.internal.util.Version.<clinit> HV000001: Hibernate Validator 7.0.0.Final
2021-04-16T15:34:39.583973+00:00 heroku[web.1]: Process running mem=859M(167.9%)
2021-04-16T15:34:39.586330+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2021-04-16T15:34:39.615842+00:00 heroku[router]: at=error code=H12 desc="Request timeout" method=GET path="/" host=opengrok-heroku-play.herokuapp.com request_id=40824d3a-6537-492c-b66e-0a2e6af779c9 fwd="92.203.52.226" dyno=web.1 connect=3ms service=30000ms status=503 bytes=0 protocol=https
2021-04-16T15:34:46.528441+00:00 app[web.1]: 16-Apr-2021 15:34:46.527 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/usr/local/tomcat/webapps/ROOT.war] has finished in [37,946] ms
2021-04-16T15:34:46.530595+00:00 app[web.1]: 16-Apr-2021 15:34:46.530 INFO [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/docs]
2021-04-16T15:34:47.373671+00:00 app[web.1]: 16-Apr-2021 15:34:47.369 WARNING [main] org.apache.tomcat.util.scan.StandardJarScanner.processURLs Failed to scan [file:/usr/local/tomcat/webapps/ROOT/WEB-INF/lib/annotations-19.0.0.jar] from classloader hierarchy

起動したと思いきや、メモリ不足で死亡...

Yusuke Iwaki 2021/04/17

たとえば、最小ヒープ・サイズを128MB、最大ヒープ・サイズを1024MBに設定するには、次のように記述してください。

JAVA_OPTS="-Xms128m -Xmx1024m"

ということなので、Xmx512mを指定してみる。

Yusuke Iwaki 2021/04/17

ｷﾀ━━━━(ﾟ∀ﾟ)━━━━!!

ｷﾀ━━━━(ﾟ∀ﾟ)━━━━!!ｷﾀ━━━━(ﾟ∀ﾟ)━━━━!!検索できる！！

Yusuke Iwaki 2021/04/17

Review appsも。

にあるように、app.jsonをかきかきしないと、heroku-20スタックでビルドしようとして失敗してしまう

Yusuke Iwaki 2021/04/17に更新

2021-04-16T23:39:33.540129+00:00 app[web.1]: 16-Apr-2021 23:39:33.540 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [69027] milliseconds
2021-04-16T23:39:40.673193+00:00 heroku[router]: at=error code=H12 desc="Request timeout" method=GET path="/" host=opengrok-her-replace-co-sieg64.herokuapp.com request_id=ac4d1bfd-a03e-4f31-99ec-a3d34b664d14 fwd="92.203.52.226" dyno=web.1 connect=1ms service=30004ms status=503 bytes=0 protocol=https
2021-04-16T23:39:58.173031+00:00 heroku[web.1]: Process running mem=517M(100.9%)
2021-04-16T23:39:58.193552+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)

初回アクセス時にリクエストタイムアウトしちゃう...けどとりあえず、レビュー対象のソースコードを検索するということはできそう。

このスクラップは2021/04/17にクローズされました