Zenn
Open1

ISUCON過去問メモ

こばこば

環境構築（ISUCON7）

感想戦用に1VMで建てられる。
https://github.com/isucon/isucon7-qualify

まずGoogle CloudにVMを用意。

  • とりあえず1コア？
  • ubuntu 16.04
  • isuconユーザのgroupは？とりあえずsudoも付けとく。

ubuntuわからん

isuconユーザはadduserで作った方が良さそう。
https://qiita.com/white_aspara25/items/c1b9d02310b4731bfbaa

mysqlとnginxをインストール

MySQLは5.7、nginxは何のverだこれ。

$ sudo systemctl status mysql.service
● mysql.service - MySQL Community Server
   Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-02-11 09:49:26 UTC; 4min 39s ago
 Main PID: 13471 (mysqld)
    Tasks: 29
   Memory: 130.3M
      CPU: 329ms
   CGroup: /system.slice/mysql.service
           └─13471 /usr/sbin/mysqld

Feb 11 09:49:25 isucon7-1 systemd[1]: Starting MySQL Community Server...
Feb 11 09:49:26 isucon7-1 systemd[1]: Started MySQL Community Server.
$ sudo systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-02-11 09:51:50 UTC; 2min 33s ago
 Main PID: 14028 (nginx)
    Tasks: 2
   Memory: 3.8M
      CPU: 17ms
   CGroup: /system.slice/nginx.service
           ├─14028 nginx: master process /usr/sbin/nginx -g daemon on; master_process on
           └─14029 nginx: worker process                           

Feb 11 09:51:50 isucon7-1 systemd[1]: Starting A high performance web server and a reverse proxy server...
Feb 11 09:51:50 isucon7-1 systemd[1]: nginx.service: Failed to parse PID from file /run/nginx.pid: Invalid argume
Feb 11 09:51:50 isucon7-1 systemd[1]: Started A high performance web server and a reverse proxy server.

パッケージが一部入らない

pythonは使わないとしても、libmcrypt-devは大丈夫か。phpで使う奴かな、一旦skip。

$ sudo apt install -y git curl libreadline-dev pkg-config autoconf automake build-essential libmysqlclient-dev \
> libssl-dev python3 python3-dev python3-venv openjdk-8-jdk-headless libxml2-dev libcurl4-openssl-dev \
>         libxslt1-dev re2c bison libbz2-dev libreadline-dev libssl-dev gettext libgettextpo-dev libicu-dev \
> libmhash-dev libmcrypt-dev libgd-dev libtidy-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package python3-venv is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'python3-venv' has no installation candidate
E: Unable to locate package libmcrypt-dev

xbuildでgoをインストール。このツールなんだろう。

$ cd
$ ls
isucon7
$ git clone https://github.com/tagomoris/xbuild.git
Cloning into 'xbuild'...
remote: Enumerating objects: 205, done.
remote: Total 205 (delta 0), reused 0 (delta 0), pack-reused 205
Receiving objects: 100% (205/205), 43.15 KiB | 0 bytes/s, done.
Resolving deltas: 100% (109/109), done.
Checking connectivity... done.

$ mkdir local

$ ls -al
total 24
drwxr-xr-x 5 isucon isucon 4096 Feb 11 10:10 .
drwxr-xr-x 5 root   root   4096 Feb 11 09:30 ..
-rw------- 1 isucon isucon   47 Feb 11 09:33 .bash_history
drwxrwxr-x 3 isucon isucon 4096 Feb 11 09:47 isucon7
drwxrwxr-x 2 isucon isucon 4096 Feb 11 10:10 local
-rw-r--r-- 1 isucon isucon    0 Feb 11 09:44 .sudo_as_admin_successful
drwxrwxr-x 3 isucon isucon 4096 Feb 11 10:10 xbuild

$ xbuild/go-install     -f 1.9     /home/isucon/local/go
Start to install go 1.9 ...
go 1.9 successfully installed on /home/isucon/local/go
To use this go, do 'export PATH=/home/isucon/local/go/bin:$PATH' and 'export GOROOT=/home/isucon/local/go'.

$ export PATH=$HOME/local/go/bin:$HOME/go/bin:$PATH

ベンチマークを準備する。

$ go get github.com/constabulary/gb/...
$ cd ~/isucon7/isubata/bench/
$ gb vendor restore
Getting github.com/PuerkitoBio/goquery
Getting golang.org/x/net/html
Getting github.com/andybalholm/cascadia
Getting github.com/marcw/cachecontrol
$ make
-su: make: command not found
$ sudo apt install -y make
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following package was automatically installed and is no longer required:
  grub-pc-bin
Use 'sudo apt autoremove' to remove it.
Suggested packages:
  make-doc
The following NEW packages will be installed:
  make
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/151 kB of archives.
After this operation, 365 kB of additional disk space will be used.
Selecting previously unselected package make.
(Reading database ... 82973 files and directories currently installed.)
Preparing to unpack .../archives/make_4.1-6_amd64.deb ...
Unpacking make (4.1-6) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up make (4.1-6) ...
$ make
GOPATH=`pwd`:`pwd`/vendor go install ./src/cmd/...

$ cd ~/isucon7/isubata/bench/
$ ./bin/gen-initial-dataset 
2021/02/11 10:19:36 datapath ./data

とりあえずデータ準備まで完了。

mysql> status
--------------
mysql  Ver 14.14 Distrib 5.7.33, for Linux (x86_64) using  EditLine wrapper

Connection id:          22
Current database:       isubata
Current user:           root@localhost
SSL:                    Not in use
Current pager:          stdout
Using outfile:          ''
Using delimiter:        ;
Server version:         5.7.33-0ubuntu0.16.04.1 (Ubuntu)
Protocol version:       10
Connection:             Localhost via UNIX socket
Server characterset:    latin1
Db     characterset:    latin1
Client characterset:    utf8
Conn.  characterset:    utf8
UNIX socket:            /var/run/mysqld/mysqld.sock
Uptime:                 3 hours 10 min 13 sec

mysql> SELECT
    ->   TABLE_NAME
    -> , ENGINE
    -> , TABLE_ROWS                                                      -- レコード数
    -> , AVG_ROW_LENGTH                                                  -- 平均レコード容量
    -> , FLOOR((DATA_LENGTH + INDEX_LENGTH) / 1024 / 1024) AS SIZE       -- 合計容量
    -> , FLOOR((DATA_LENGTH) / 1024 / 1024)                AS DATA_SIZE  -- データ容量
    -> , FLOOR((INDEX_LENGTH) / 1024 / 1024)               AS INDEX_SIZE -- インデックス容量
    -> FROM
    ->   INFORMATION_SCHEMA.TABLES
    -> WHERE
    ->   TABLE_SCHEMA = DATABASE()
    -> ORDER BY
    ->   (DATA_LENGTH + INDEX_LENGTH) DESC
    -> ;
+------------+--------+------------+----------------+------+-----------+------------+
| TABLE_NAME | ENGINE | TABLE_ROWS | AVG_ROW_LENGTH | SIZE | DATA_SIZE | INDEX_SIZE |
+------------+--------+------------+----------------+------+-----------+------------+
| image      | InnoDB |        994 |         339168 |  321 |       321 |          0 |
| message    | InnoDB |       9772 |            484 |    4 |         4 |          0 |
| user       | InnoDB |       1000 |            212 |    0 |         0 |          0 |
| haveread   | InnoDB |          0 |              0 |    0 |         0 |          0 |
| channel    | InnoDB |         10 |           1638 |    0 |         0 |          0 |
+------------+--------+------------+----------------+------+-----------+------------+
5 rows in set (0.00 sec)

go実装を起動する。
https://github.com/isucon/isucon7-qualify/tree/master/files/app のenv.shとisubata.golang.serviceを編集。

$ sudo cp isubata.golang.service /etc/systemd/system/
$ sudo systemctl start isubata.golang.service

$ sudo systemctl status isubata.golang.service
● isubata.golang.service - isucon7 qualifier main application in golang
   Loaded: loaded (/etc/systemd/system/isubata.golang.service; disabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-02-11 13:50:26 UTC; 7s ago
 Main PID: 18836 (isubata)
    Tasks: 3
   Memory: 1.3M
      CPU: 5ms
   CGroup: /system.slice/isubata.golang.service
           └─18836 /home/isucon/isucon7/isubata/webapp/go/isubata

Feb 11 13:50:26 isucon7-1 isubata[18836]: 2021/02/11 13:50:26 Succeeded to connect db.
Feb 11 13:50:26 isucon7-1 isubata[18836]:    ____    __
Feb 11 13:50:26 isucon7-1 isubata[18836]:   / __/___/ /  ___
Feb 11 13:50:26 isucon7-1 isubata[18836]:  / _// __/ _ \/ _ \
Feb 11 13:50:26 isucon7-1 isubata[18836]: /___/\__/_//_/\___/ v3.2.1
Feb 11 13:50:26 isucon7-1 isubata[18836]: High performance, minimalist Go web framework
Feb 11 13:50:26 isucon7-1 isubata[18836]: https://echo.labstack.com
Feb 11 13:50:26 isucon7-1 isubata[18836]: ____________________________________O/_______
Feb 11 13:50:26 isucon7-1 isubata[18836]:                                     O\
Feb 11 13:50:26 isucon7-1 isubata[18836]: ⇨ http server started on [::]:5000

benchがエラーになる。

$ ./bin/bench -remotes=127.0.0.1 -output result.json
[isu7q-bench] 2021/02/11 14:19:40.557775 dataset.go:37: datapath ./data
[isu7q-bench] 2021/02/11 14:19:40.817950 bench.go:472: Remotes [127.0.0.1]
[isu7q-bench] 2021/02/11 14:19:40.818103 bench.go:346: State.Init()
[isu7q-bench] 2021/02/11 14:19:40.826649 bench.go:348: State.Init() Done
[isu7q-bench] 2021/02/11 14:19:40.826786 bench.go:350: requestInitialize()
[isu7q-bench] 2021/02/11 14:19:40.835175 bench.go:358: requestInitialize() Done
[isu7q-bench] 2021/02/11 14:19:40.835279 bench.go:363: preTest()
[isu7q-bench] 2021/02/11 14:19:46.185583 bench.go:491: {"job_id":"","ip_addrs":"127.0.0.1","pass":false,"score":0,"message":"負荷走行前のバリデーションに失敗しました。2021-02-11 14:19:46.18544448 +0000 UTC m=+5.629833717 リク Post http://isubata.example.com/add_channel: EOF (POST /add_channel )","error":["2021-02-11 14:19:46.18544448 +0000 UTC m=+5.629833717 リクエストに失敗しました Post http://isubata.example.com/add_channel: EOF (POST /add_channel )"],"log":null,"load_level":0,"start_time":"2021-02-11T14:19:40.818096428Z","end_time":"2021-02-11T14:19:46.185467884Z"}
[isu7q-bench] 2021/02/11 14:19:46.185732 bench.go:498: result json saved to  result.json
$ sudo view /var/log/nginx/access.log