Open1
docker jupyerhub
karamawanu特定少数に演習等で使ってもらうことを意図して、jupyterhubを試すことにした。
jupyter notebookそのままでも運用は可能だが、他人のファイルが丸見えなので、カンニングを助長することになる。後半はともかく序盤は独習してほしいものだ。
dockerで起動するらしいので試してみる。
$ sudo docker run -it -p 8000:8000 jupyterhub/jupyterhub
[I 2023-03-02 03:43:10.184 JupyterHub app:2810] Running JupyterHub version 3.1.1
[I 2023-03-02 03:43:10.184 JupyterHub app:2840] Using Authenticator: jupyterhub.auth.PAMAuthenticator-3.1.1
[I 2023-03-02 03:43:10.184 JupyterHub app:2840] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-3.1.1
[I 2023-03-02 03:43:10.184 JupyterHub app:2840] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-3.1.1
[I 2023-03-02 03:43:10.194 JupyterHub app:1694] Writing cookie_secret to /srv/jupyterhub/jupyterhub_cookie_secret
[I 2023-03-02 03:43:10.209 alembic.runtime.migration migration:205] Context impl SQLiteImpl.
[I 2023-03-02 03:43:10.209 alembic.runtime.migration migration:208] Will assume non-transactional DDL.
[I 2023-03-02 03:43:10.224 alembic.runtime.migration migration:619] Running stamp_revision -> 651f5419b74d
[I 2023-03-02 03:43:10.377 JupyterHub proxy:556] Generating new CONFIGPROXY_AUTH_TOKEN
[I 2023-03-02 03:43:10.406 JupyterHub app:1969] Not using allowed_users. Any authenticated user will be allowed.
[I 2023-03-02 03:43:10.427 JupyterHub app:2879] Initialized 0 spawners in 0.002 seconds
[I 2023-03-02 03:43:10.432 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.twenty_four_hours
[I 2023-03-02 03:43:10.433 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.seven_days
[I 2023-03-02 03:43:10.433 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.thirty_days
[W 2023-03-02 03:43:10.434 JupyterHub proxy:746] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else...
[I 2023-03-02 03:43:10.434 JupyterHub proxy:750] Starting proxy @ http://:8000
03:43:10.620 [ConfigProxy] info: Proxying http://*:8000 to (no default)
03:43:10.622 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
[I 2023-03-02 03:43:10.817 JupyterHub app:3130] Hub API listening on http://127.0.0.1:8081/hub/
03:43:10.817 [ConfigProxy] info: 200 GET /api/routes
03:43:10.818 [ConfigProxy] info: 200 GET /api/routes
[I 2023-03-02 03:43:10.819 JupyterHub proxy:477] Adding route for Hub: / => http://127.0.0.1:8081
03:43:10.820 [ConfigProxy] info: Adding route / -> http://127.0.0.1:8081
03:43:10.820 [ConfigProxy] info: Route added / -> http://127.0.0.1:8081
03:43:10.821 [ConfigProxy] info: 201 POST /api/routes/
[I 2023-03-02 03:43:10.821 JupyterHub app:3197] JupyterHub is now running at http://:8000
[I 2023-03-02 03:43:19.735 JupyterHub log:186] 302 GET / -> /hub/ (@::ffff:192.168.0.18) 0.77ms
[I 2023-03-02 03:43:19.746 JupyterHub log:186] 302 GET /hub/ -> /hub/login?next=%2Fhub%2F (@::ffff:192.168.0.18) 0.86ms
[I 2023-03-02 03:43:19.797 JupyterHub log:186] 200 GET /hub/login?next=%2Fhub%2F (@::ffff:192.168.0.18) 36.26ms
[W 2023-03-02 03:45:35.134 JupyterHub auth:1111] PAM Authentication failed (honda@::ffff:192.168.0.18): [PAM Error 7] Authentication failure
[W 2023-03-02 03:45:35.135 JupyterHub base:825] Failed login for honda
[I 2023-03-02 03:45:35.141 JupyterHub log:186] 200 POST /hub/login?next=%2Fhub%2F (@::ffff:192.168.0.18) 2693.85ms
[W 2023-03-02 03:45:48.224 JupyterHub auth:1111] PAM Authentication failed (jupyter@::ffff:192.168.0.18): [PAM Error 7] Authentication failure
[W 2023-03-02 03:45:48.225 JupyterHub base:825] Failed login for jupyter
[I 2023-03-02 03:45:48.230 JupyterHub log:186] 200 POST /hub/login?next=%2Fhub%2F (@::ffff:192.168.0.18) 2374.18ms
[I 2023-03-02 03:47:29.954 JupyterHub roles:238] Adding role user for User: jupy
[I 2023-03-02 03:47:29.971 JupyterHub base:819] User logged in: jupy
[I 2023-03-02 03:47:29.972 JupyterHub log:186] 302 POST /hub/login?next=%2Fhub%2F -> /hub/ (jupy@::ffff:192.168.0.18) 68.83ms
[I 2023-03-02 03:47:29.997 JupyterHub log:186] 302 GET /hub/ -> /hub/spawn (jupy@::ffff:192.168.0.18) 13.23ms
/usr/local/lib/python3.10/dist-packages/jupyterhub/oauth/provider.py:652: SAWarning: The argument signature for the "ConnectionEvents.engine_connect" event listener has changed as of version 2.0, and conversion for the old argument signature will be removed in a future release. The new signature is "def engine_connect(conn)" (This warning originated from the Session 'autoflush' process, which was invoked automatically in response to a user-initiated operation.)
self.db.query(orm.OAuthClient).filter_by(identifier=client_id).one_or_none()
dockerプロセスでjupyterhubが動いてる。^cとか押すとjupyterhubが終了するので注意。それだと困るので、実際の運用では常駐起動する必要があるが、とりあえずこのまま進めることにする。
ログの序盤から察するに、PAM認証が有効になってるらしい。しかし今回はdocker起動してるので、PAM参照先はdockerの中.....
[I 2023-03-02 03:43:10.184 JupyterHub app:2840] Using Authenticator: jupyterhub.auth.PAMAuthenticator-3.1.1
仕方ないのでdockerの中を探検する。
$ sudo docker exec -it 30e6fb52aaa6 bash
root@30e6fb52aaa6:/srv/jupyterhub# ps axf
PID TTY STAT TIME COMMAND
17 pts/1 Ss 0:00 bash
26 pts/1 R+ 0:00 \_ ps axf
1 pts/0 Ssl+ 0:00 /usr/bin/python3 /usr/local/bin/jupyterhub
9 ? Ssl 0:00 node /usr/local/bin/configurable-http-proxy --ip --port 8000 --api-ip 127.0.0.1 --api-port 8001 --error-
どうやらjupyterhubの正体は、nodejsで書いたフロントエンドのようだ。それが、notebook本体を多重起動してくれるらしい。
ユーザを追加しよう。名前考えるのが面倒だったんで"jupy"さんで。
root@30e6fb52aaa6:/srv/jupyterhub# adduser jupy
Adding user `jupy' ...
Adding new group `jupy' (1000) ...
Adding new user `jupy' (1000) with group `jupy' ...
Creating home directory `/home/jupy' ...
Copying files from `/etc/skel' ...
New password:
Retype new password:
passwd: password updated successfully
Changing the user information for jupy
Enter the new value, or press ENTER for the default
Full Name []:
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] Y
root@30e6fb52aaa6:/srv/jupyterhub#
このshellはまた使うかもしれないので閉じないこと。
これで host:8000 からログインできるようになったはずである。
なずなのだが....
jupyterhubのコンソールで次のようなエラーで落ちるので起動しない。
[W 2023-03-02 03:47:59.854 JupyterHub user:882] jupy's server never showed up at http://127.0.0.1:52777/user/jupy/ after 30 seconds. Giving up.
Common causes of this timeout, and debugging tips:
1. The server didn't finish starting,
or it crashed due to a configuration issue.
Check the single-user server's logs for hints at what needs fixing.
2. The server started, but is not accessible at the specified URL.
This may be a configuration issue specific to your chosen Spawner.
Check the single-user server logs and resource to make sure the URL
is correct and accessible from the Hub.
3. (unlikely) Everything is working, but the server took too long to respond.
To fix: increase `Spawner.http_timeout` configuration
to a number of seconds that is enough for servers to become responsive.
[E 2023-03-02 03:47:59.894 JupyterHub gen:630] Exception in Future <Task finished name='Task-352' coro=<BaseHandler.spawn_single_user.<locals>.finish_user_spawn() done, defined at /usr/local/lib/python3.10/dist-packages/jupyterhub/handlers/base.py:963> exception=TimeoutError("Server at http://127.0.0.1:52777/user/jupy/ didn't respond in 30 seconds")> after timeout
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/tornado/gen.py", line 625, in error_callback
future.result()
File "/usr/local/lib/python3.10/dist-packages/jupyterhub/handlers/base.py", line 970, in finish_user_spawn
await spawn_future
File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 863, in spawn
await self._wait_up(spawner)
File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 907, in _wait_up
raise e
File "/usr/local/lib/python3.10/dist-packages/jupyterhub/user.py", line 877, in _wait_up
resp = await server.wait_up(
File "/usr/local/lib/python3.10/dist-packages/jupyterhub/utils.py", line 288, in wait_for_http_server
re = await exponential_backoff(
File "/usr/local/lib/python3.10/dist-packages/jupyterhub/utils.py", line 236, in exponential_backoff
raise asyncio.TimeoutError(fail_message)
asyncio.exceptions.TimeoutError: Server at http://127.0.0.1:52777/user/jupy/ didn't respond in 30 seconds
[I 2023-03-02 03:47:59.896 JupyterHub log:186] 200 GET /hub/api/users/jupy/server/progress (jupy@::ffff:192.168.0.18) 28746.41ms
単純にjupyternotebook本体が入ってないせいらしい。
ええ....
さきほどユーザ追加したshellから操作する。
jupyter notebookはpip3で入る。condaまでは必要ない。
root@30e6fb52aaa6:/srv/jupyterhub# pip3 install jupyter
これで起動できた。
jupyterhub-singleuser というやつが増えている。
どうやらこいつがマルチユーザ用の小細工をしてくれるようだ。
root@30e6fb52aaa6:/srv/jupyterhub# ps axf
PID TTY STAT TIME COMMAND
17 pts/1 Ss 0:00 bash
110 pts/1 R+ 0:00 \_ ps axf
1 pts/0 Ssl+ 0:01 /usr/bin/python3 /usr/local/bin/jupyterhub
9 ? Ssl 0:00 node /usr/local/bin/configurable-http-proxy --ip --port 8000 --api-ip 127.0.0.1 --api-port 8001 --error-
103 ? Ssl 0:01 /usr/bin/python3 /usr/local/bin/jupyterhub-singleuser
PAM認証では、ipynbファイルは当人のユーザディレクトリに作られるようだ。
root@30e6fb52aaa6:/home/jupy# ls -la
total 36
drwxr-x--- 5 jupy jupy 4096 Mar 2 04:10 .
drwxr-xr-x 1 root root 4096 Mar 2 03:47 ..
-rw-r--r-- 1 jupy jupy 220 Mar 2 03:47 .bash_logout
-rw-r--r-- 1 jupy jupy 3771 Mar 2 03:47 .bashrc
drwxr-xr-x 2 jupy jupy 4096 Mar 2 04:09 .ipynb_checkpoints
drwxr-xr-x 3 jupy jupy 4096 Mar 2 04:09 .ipython
drwxr-xr-x 3 jupy jupy 4096 Mar 2 03:59 .local
-rw-r--r-- 1 jupy jupy 807 Mar 2 03:47 .profile
-rw-r--r-- 1 jupy jupy 978 Mar 2 04:10 Untitled.ipynb
これだったすでにjupyter notebookが入ってる環境に、jupyterhubを後付けしたほうが手間は少なさそうだ。