pysen + Python 拡張の連携ができないか試してみる

作業環境を残すのを忘れていた。

$ echo $PYENV_ROOT
/home/xeres/.local/share/pyenv
$ pyenv versions
  system
* 3.8.8 (set by /home/xeres/.local/share/pyenv/version)
$ echo $PIPX_HOME
/home/xeres/.local/share/pipx
$ echo $PIPX_DEFAULT_PYTHON
/home/xeres/.local/share/pyenv/versions/3.8.8/bin/python
$ pipx list
venvs are in /home/xeres/.local/share/pipx/venvs
apps are exposed on your $PATH at /home/xeres/.local/bin
   package aws-sam-cli 1.23.0, Python 3.8.8
    - sam
   package cfn-lint 0.49.0, Python 3.8.8
    - cfn-lint
   package git-remote-codecommit 1.15.1, Python 3.8.8
    - git-remote-codecommit
   package poetry 1.1.6, Python 3.8.8
    - poetry

Xeres

とりあえず、サンプルとして sam init したフォルダを対象にしてみる。

$ sam init
Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1
What package type would you like to use?
        1 - Zip (artifact is a zip uploaded to S3)
        2 - Image (artifact is an image uploaded to an ECR image repository)
Package type: 1

Which runtime would you like to use?
        1 - nodejs14.x
        2 - python3.8
        3 - ruby2.7
        4 - go1.x
        5 - java11
        6 - dotnetcore3.1
        7 - nodejs12.x
        8 - nodejs10.x
        9 - python3.7
        10 - python3.6
        11 - python2.7
        12 - ruby2.5
        13 - java8.al2
        14 - java8
        15 - dotnetcore2.1
Runtime: 2

Project name [sam-app]: eventbridge-hello-world

Cloning app templates from https://github.com/aws/aws-sam-cli-app-templates

AWS quick start application templates:
        1 - Hello World Example
        2 - EventBridge Hello World
        3 - EventBridge App from scratch (100+ Event Schemas)
        4 - Step Functions Sample App (Stock Trader)
        5 - Elastic File System Sample App
Template selection: 2

    -----------------------
    Generating application:
    -----------------------
    Name: eventbridge-hello-world
    Runtime: python3.8
    Dependency Manager: pip
    Application Template: eventBridge-hello-world
    Output Directory: .

    Next steps can be found in the README file at ./eventbridge-hello-world/README.md

$ cd eventbridge-hello-world/

Xeres

poetry のセットアップ。
venv 環境はローカルにプロジェクト内に作って .gitignore する派。

$ poetry config virtualenvs.in-project true --local

poetry.toml

[virtualenvs]
in-project = true

$ poetry init

This command will guide you through creating your pyproject.toml config.

Package name [eventbridge-hello-world]:
Version [0.1.0]:
Description []:
Author [Xeres <294146+xeres@users.noreply.github.com>, n to skip]:
License []:
Compatible Python versions [^3.8]:

Would you like to define your main dependencies interactively? (yes/no) [yes]
You can specify a package in the following forms:
  - A single name (requests)
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Search for package to add (or leave blank to continue):

Would you like to define your development dependencies interactively? (yes/no) [yes]
Search for package to add (or leave blank to continue):

Generated file

[tool.poetry]
name = "eventbridge-hello-world"
version = "0.1.0"
description = ""
authors = ["Xeres <294146+xeres@users.noreply.github.com>"]

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"


Do you confirm generation? (yes/no) [yes]
$ poetry install
$ source .venv/bin/activate

Xeres

pysen のセットアップ。

$ poetry add -D pysen -E lint
Using version ^0.9.1 for pysen

Updating dependencies
Resolving dependencies... (1.8s)

Writing lock file

Package operations: 25 installs, 0 updates, 0 removals

  • Installing mccabe (0.6.1)
  • Installing pycodestyle (2.7.0)
  • Installing pyflakes (2.3.1)
  • Installing smmap (4.0.0)
  • Installing appdirs (1.4.4)
  • Installing attrs (21.2.0)
  • Installing click (7.1.2)
  • Installing flake8 (3.9.1)
  • Installing gitdb (4.0.7)
  • Installing mypy-extensions (0.4.3)
  • Installing pathspec (0.8.1)
  • Installing regex (2021.4.4)
  • Installing toml (0.10.2)
  • Installing typed-ast (1.4.3)
  • Installing typing-extensions (3.10.0.0)
  • Installing black (20.8b1)
  • Installing colorlog (4.8.0)
  • Installing dacite (1.6.0)
  • Installing flake8-bugbear (21.4.3)
  • Installing gitpython (3.1.15)
  • Installing isort (5.1.4)
  • Installing mypy (0.790)
  • Installing tomlkit (0.7.0)
  • Installing unidiff (0.6.0)
  • Installing pysen (0.9.1)
$ poetry run pysen
Could not find a pyproject.toml file containing a [tool.pysen] section in this or any of its parent directories.
The `--loglevel debug option` may help.

というわけで、pyproject.toml に pysen の設定を追加。

pyproject.toml (追記)

[tool.pysen]
version = "0.9"

[tool.pysen.lint]
enable_black = true
enable_flake8 = true
enable_isort = true
enable_mypy = true
mypy_preset = "strict"
line_length = 88
py_version = "py38"
[tool.pysen.lint.source]
  excludes = [".venv/"]
[[tool.pysen.lint.mypy_targets]]
  paths = ["."]

$ poetry run pysen run lint
Running commands concurrently...
... concurrent execution done
Running: black
Checking 16 files
(以下中略)
Running: mypy
[1/1] Checking 1 entries
eventbridge-hello-world is not a valid Python package name

 ** execution summary **
isort .......... Failed (0.66 sec)
black .......... Failed (0.77 sec)
flake8 .......... Failed (0.74 sec)
mypy .......... Failed (0.15 sec)

lint finished with error(s)
Errored:
 - isort
 - black
 - flake8
 - mypy

めっちゃ怒られたｗ

Xeres

pysen-vscode のために pysen_ls を入れる。

$ poetry add -D pysen_ls
Using version ^0.1.1 for pysen-ls

Updating dependencies
Resolving dependencies... (0.4s)

Writing lock file

Package operations: 4 installs, 0 updates, 0 removals

  • Installing pydantic (1.8.1)
  • Installing typeguard (2.12.0)
  • Installing pygls (0.10.3)
  • Installing pysen-ls (0.1.1)
$ python3 -m pysen_ls
usage: pysen-ls [-h] (--tcp | --io) [--host HOST] [--port PORT] [--log-file LOG_FILE]
pysen-ls: error: one of the arguments --tcp --io is required

とりあえず動きそうな感じ。

Xeres

VS Code 側でも pysen-vscode 拡張を入れる。
先に Microsoft 製の Python を削除しておかないとコンフリクトするらしいので注意。

Python ファイルを開くと、下記のようなメッセージが出て怒られる。

The pysen language server server crashed 5 times in the last 3 minutes. The server will not be restarted.

出力

/home/xeres/.local/share/pyenv/versions/3.8.8/bin/python3: No module named pysen_ls
[Info  - 1:50:09 AM] Connection to server got closed. Server will restart.
/home/xeres/.local/share/pyenv/versions/3.8.8/bin/python3: No module named pysen_ls
[Info  - 1:50:09 AM] Connection to server got closed. Server will restart.
/home/xeres/.local/share/pyenv/versions/3.8.8/bin/python3: No module named pysen_ls
[Info  - 1:50:09 AM] Connection to server got closed. Server will restart.
/home/xeres/.local/share/pyenv/versions/3.8.8/bin/python3: No module named pysen_ls
[Info  - 1:50:09 AM] Connection to server got closed. Server will restart.
/home/xeres/.local/share/pyenv/versions/3.8.8/bin/python3: No module named pysen_ls
[Error - 1:50:10 AM] Connection to server got closed. Server will not be restarted.

venv ではなく、pyenv global で指定した python3 を見に行っているっぽい。
うーん、ちょっとこれは実プロジェクトでは使えないなあ…。

Xeres

ワークスペースの pysen.client.pythonPath で .venv/bin/python3 をフルパスで指定したところ、エラーは出なくなったが、拡張はちゃんと動かない模様。うーん…。

出力

[Info  - 2:05:30 AM] starting runtime for '/home/xeres/repos/eventbridge-hello-world/hello_world_function/hello_world/app.py'
[Info  - 2:05:30 AM] pysen runtime activated: /home/xeres/repos/eventbridge-hello-world/hello_world_function/hello_world/app.py
[Info  - 2:06:19 AM] starting runtime for '/home/xeres/repos/eventbridge-hello-world/hello_world_function/__init__.py'
[Info  - 2:06:19 AM] pysen runtime activated: /home/xeres/repos/eventbridge-hello-world/hello_world_function/__init__.py
[Info  - 2:06:31 AM] starting runtime for '/home/xeres/repos/eventbridge-hello-world/conftest.py'
[Info  - 2:06:31 AM] pysen runtime activated: /home/xeres/repos/eventbridge-hello-world/conftest.py
[Info  - 2:06:35 AM] starting runtime for '/home/xeres/repos/eventbridge-hello-world/__init__.py'
[Info  - 2:06:35 AM] pysen runtime activated: /home/xeres/repos/eventbridge-hello-world/__init__.py
[Info  - 2:06:40 AM] starting runtime for '/home/xeres/repos/eventbridge-hello-world/hello_world_function/hello_world/__init__.py'
[Info  - 2:06:40 AM] pysen runtime activated: /home/xeres/repos/eventbridge-hello-world/hello_world_function/hello_world/__init__.py

Xeres

source .venv/bin/activate した状態で立ち上げた VS Code 環境だと動作する模様。

動作画像

ソースを見ると workspace.getConfiguration('python').get<string>('pythonPath'); した値を使っているので、実装の問題じゃなさそうなんだけどなあ…。

Xeres

1～2時間コードを書いてみて、Quick Fix で pysen-vscode が自動で書きっぷりを修正してくれるのは書き手としてはめっちゃ気持ちがよいんだけど、Microsoft の Python 拡張で関数名やプロパティの補完が効く方が結局のところありがたいという結論になった。

Language Server を pysen-ls にするより、Python 拡張を使いつつ設定で pysen を使ってセットアップした linter や formatter と共存させる方が良さそう？

検証をその方向に進めるため、タイトルを変えた。

Before: poetry + venv + pysen + pysen-vscode を試してみる
After: pysen + Python 拡張の連携ができないか試してみる

Xeres

venv ではなく、pyenv global で指定した python3 を見に行っているっぽい。
うーん、ちょっとこれは実プロジェクトでは使えないなあ…。

これ、Workspace ディレクトリに .venv がないと機能しないのに、間違って違う Workspace ファイルを開いていて、そこからプロジェクトのフォルダを追加していたのが原因だった。

Where the extension looks for environments | Using Python Environments in Visual Studio Code

いずれにしても Language Server は Python 拡張のままにしておきたいので、そのまま作業を続ける。