TensorFlow 学習メモ
poetry とやらが良いらしい?
公式サイトの方法に従ってインストール
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
プロジェクトの初期化。
poetry new tutorial
% tree tutorial
.
├── README.rst
├── poetry.lock
├── pyproject.toml
├── tutorial
│ └── __init__.py
└── tests
├── __init__.py
└── test_tutorial.py
どうやら プロジェクト (tutorial) > モジュール (tutorial) という構成になるっぽい。
Python はプロジェクト毎に仮想環境を作るのがお作法っぽい。
poetry new の時点で仮想環境作ってくれるっぽい?が、python3 -m venv .venv
すると良いよって記事を見掛けたので従ってみる。
poetry config virtualenvs.in-project true
python3 -m venv .venv
poetry install
確認。
poetry env list
# .venv (Activated) ← こうなってれば OK っぽい
in-project の設定をする前に install しちゃった場合は venv 設定後に env を消すと良さそう?
(消さなくても良いのかな?)
TensorFlow 追加。
poetry add tensorflow
Using version ^2.7.0 for tensorflow
Updating dependencies
Resolving dependencies... (10.6s)
Resolving dependencies... (10.7s)
SolverProblemError
The current project's Python requirement (>=3.9,<4.0) is not compatible with some of the required packages Python requirement:
- tensorflow-io-gcs-filesystem requires Python >=3.7, <3.11, so it will not be satisfied for Python >=3.11,<4.0
- tensorflow-io-gcs-filesystem requires Python >=3.6, <3.10, so it will not be satisfied for Python >=3.10,<4.0
- tensorflow-io-gcs-filesystem requires Python >=3.6, <3.10, so it will not be satisfied for Python >=3.10,<4.0
- tensorflow-io-gcs-filesystem requires Python >=3.6, <3.10, so it will not be satisfied for Python >=3.10,<4.0
Because no versions of tensorflow-io-gcs-filesystem match >0.21.0,<0.22.0 || >0.22.0,<0.23.0 || >0.23.0,<0.23.1 || >0.23.1
and tensorflow-io-gcs-filesystem (0.21.0) requires Python >=3.6, <3.10, tensorflow-io-gcs-filesystem is forbidden.
And because tensorflow-io-gcs-filesystem (0.22.0) requires Python >=3.6, <3.10, tensorflow-io-gcs-filesystem is forbidden.
And because tensorflow-io-gcs-filesystem (0.23.0) requires Python >=3.6, <3.10
and tensorflow-io-gcs-filesystem (0.23.1) requires Python >=3.7, <3.11, tensorflow-io-gcs-filesystem is forbidden.
Because no versions of tensorflow match >2.7.0,<3.0.0
and tensorflow (2.7.0) depends on tensorflow-io-gcs-filesystem (>=0.21.0), tensorflow (>=2.7.0,<3.0.0) requires tensorflow-io-gcs-filesystem (>=0.21.0).
Thus, tensorflow is forbidden.
So, because tutorial depends on tensorflow (^2.7.0), version solving failed.
at ~/.poetry/lib/poetry/puzzle/solver.py:241 in _solve
237│ packages = result.packages
238│ except OverrideNeeded as e:
239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
240│ except SolveFailure as e:
→ 241│ raise SolverProblemError(e)
242│
243│ results = dict(
244│ depth_first_search(
245│ PackageNode(self._package, packages), aggregate_package_nodes
• Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
For tensorflow-io-gcs-filesystem, a possible solution would be to set the `python` property to ">=3.9,<3.11"
For tensorflow-io-gcs-filesystem, a possible solution would be to set the `python` property to ">=3.9,<3.10"
For tensorflow-io-gcs-filesystem, a possible solution would be to set the `python` property to ">=3.9,<3.10"
For tensorflow-io-gcs-filesystem, a possible solution would be to set the `python` property to ">=3.9,<3.10"
https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
https://python-poetry.org/docs/dependency-specification/#using-environment-markers
ガッツリ怒られるw
どうやら Python のバージョン指定方法がマズいっぽい。
pyproject.toml
の Python バージョンの指定を ^3.9
から >=3.9,<3.10
に変更してみる。
[tool.poetry.dependencies]
-python = "^3.9"
+python = ">=3.9,<3.10"
再実行。
poetry add tensorflow
Using version ^2.7.0 for tensorflow
Updating dependencies
Resolving dependencies... (0.9s)
Writing lock file
Package operations: 40 installs, 0 updates, 0 removals
• Installing certifi (2021.10.8)
• Installing charset-normalizer (2.0.9)
• Installing idna (3.3)
• Installing pyasn1 (0.4.8)
• Installing urllib3 (1.26.7)
• Installing cachetools (4.2.4)
• Installing oauthlib (3.1.1)
• Installing pyasn1-modules (0.2.8)
• Installing requests (2.26.0)
• Installing rsa (4.8)
• Installing six (1.16.0)
• Installing zipp (3.6.0)
• Installing google-auth (2.3.3)
• Installing importlib-metadata (4.10.0)
• Installing requests-oauthlib (1.3.0)
• Installing absl-py (1.0.0)
• Installing google-auth-oauthlib (0.4.6)
• Installing grpcio (1.43.0)
• Installing markdown (3.3.6)
• Installing numpy (1.21.5)
• Installing protobuf (3.19.1)
• Installing tensorboard-data-server (0.6.1)
• Installing tensorboard-plugin-wit (1.8.0)
• Installing werkzeug (2.0.2)
• Installing astunparse (1.6.3)
• Installing flatbuffers (2.0)
• Installing gast (0.4.0)
• Installing google-pasta (0.2.0)
• Installing h5py (3.6.0)
• Installing keras (2.7.0)
• Installing keras-preprocessing (1.1.2)
• Installing libclang (12.0.0)
• Installing opt-einsum (3.3.0)
• Installing tensorboard (2.7.0)
• Installing tensorflow-estimator (2.7.0)
• Installing tensorflow-io-gcs-filesystem (0.23.1)
• Installing termcolor (1.1.0)
• Installing typing-extensions (4.0.1)
• Installing wrapt (1.13.3)
• Installing tensorflow (2.7.0)
成功 🎉
PyCharm で開いてみる。
ちゃんと認識してる
とりあえず読み込めているようす。
No Interpreter
だけど、インタプリタが設定できていないっぽい。
Add Interpreter から Poetry のインタプリタを設定
Poetry のを指定するべきかどうかは分からんけど、とりあえず設定した。
続いて、Jupyter Notebook によるコーディングを試したいので jupyter をインストールする。
poetry add jupyter
Using version ^1.0.0 for jupyter
Updating dependencies
Resolving dependencies... (3.2s)
Writing lock file
Package operations: 52 installs, 0 updates, 0 removals
• Installing pyrsistent (0.18.0)
• Installing traitlets (5.1.1)
• Installing entrypoints (0.3)
• Installing ipython-genutils (0.2.0)
• Installing jsonschema (4.3.2)
• Installing jupyter-core (4.9.1)
• Installing nest-asyncio (1.5.4)
• Installing parso (0.8.3)
• Installing ptyprocess (0.7.0)
• Installing pycparser (2.21)
• Installing python-dateutil (2.8.2)
• Installing pyzmq (22.3.0)
• Installing tornado (6.1)
• Installing appnope (0.1.2)
• Installing backcall (0.2.0)
• Installing cffi (1.15.0)
• Installing decorator (5.1.0)
• Installing jedi (0.18.1)
• Installing jupyter-client (7.1.0)
• Installing markupsafe (2.0.1)
• Installing matplotlib-inline (0.1.3)
• Installing nbformat (5.1.3)
• Installing pexpect (4.8.0)
• Installing pickleshare (0.7.5)
• Installing prompt-toolkit (3.0.24)
• Installing pygments (2.10.0)
• Installing webencodings (0.5.1)
• Installing argon2-cffi-bindings (21.2.0)
• Installing bleach (4.1.0)
• Installing debugpy (1.5.1)
• Installing defusedxml (0.7.1)
• Installing ipython (7.30.1)
• Installing jinja2 (3.0.3)
• Installing jupyterlab-pygments (0.1.2)
• Installing mistune (0.8.4)
• Installing nbclient (0.5.9)
• Installing pandocfilters (1.5.0)
• Installing testpath (0.5.0)
• Installing argon2-cffi (21.3.0)
• Installing ipykernel (6.6.0)
• Installing nbconvert (6.3.0)
• Installing prometheus-client (0.12.0)
• Installing send2trash (1.8.0)
• Installing terminado (0.12.1)
• Installing notebook (6.4.6)
• Installing jupyterlab-widgets (1.0.2)
• Installing qtpy (2.0.0)
• Installing widgetsnbextension (3.5.2)
• Installing ipywidgets (7.6.5)
• Installing jupyter-console (6.4.0)
• Installing qtconsole (5.2.2)
• Installing jupyter (1.0.0)
TensorFlow Tutorials の Beginner のヤツを写経してみる。
補完候補に tf.keras.datasets が出てこない…。
型ヒントのファイル?らしき .pyi
が無いから補完されないのかな…?
find .venv -name "*tensorflow*"
.venv/lib/python3.9/site-packages/opt_einsum/backends/__pycache__/tensorflow.cpython-39.pyc
.venv/lib/python3.9/site-packages/opt_einsum/backends/tensorflow.py
.venv/lib/python3.9/site-packages/tensorflow_estimator
.venv/lib/python3.9/site-packages/tensorflow_io_gcs_filesystem
.venv/lib/python3.9/site-packages/tensorflow_io_gcs_filesystem/core/python/ops/libtensorflow_io_gcs_filesystem.so
.venv/lib/python3.9/site-packages/tensorflow_estimator-2.7.0.dist-info
.venv/lib/python3.9/site-packages/tensorflow
.venv/lib/python3.9/site-packages/tensorflow/core/protobuf/tensorflow_server_pb2.py
.venv/lib/python3.9/site-packages/tensorflow/core/protobuf/__pycache__/tensorflow_server_pb2.cpython-39.pyc
.venv/lib/python3.9/site-packages/tensorflow/python/__pycache__/pywrap_tensorflow.cpython-39.pyc
.venv/lib/python3.9/site-packages/tensorflow/python/__pycache__/pywrap_tensorflow_internal.cpython-39.pyc
.venv/lib/python3.9/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
.venv/lib/python3.9/site-packages/tensorflow/python/pywrap_tensorflow_internal.py
.venv/lib/python3.9/site-packages/tensorflow/python/pywrap_tensorflow.py
.venv/lib/python3.9/site-packages/tensorflow/python/debug/lib/__pycache__/grpc_tensorflow_server.cpython-39.pyc
.venv/lib/python3.9/site-packages/tensorflow/python/debug/lib/grpc_tensorflow_server.py
.venv/lib/python3.9/site-packages/tensorflow/lite/python/optimize/_pywrap_tensorflow_lite_calibration_wrapper.so
.venv/lib/python3.9/site-packages/tensorflow/lite/python/metrics_wrapper/_pywrap_tensorflow_lite_metrics_wrapper.so
.venv/lib/python3.9/site-packages/tensorflow/lite/python/interpreter_wrapper/_pywrap_tensorflow_interpreter_wrapper.so
.venv/lib/python3.9/site-packages/tensorflow/include/tensorflow
.venv/lib/python3.9/site-packages/tensorflow/include/tensorflow/core/protobuf/tensorflow_server.proto
.venv/lib/python3.9/site-packages/tensorflow/include/tensorflow/core/protobuf/tensorflow_server.pb.h
.venv/lib/python3.9/site-packages/tensorflow/include/tensorflow/compiler/mlir/tensorflow
.venv/lib/python3.9/site-packages/tensorflow/xla_aot_runtime_src/tensorflow
.venv/lib/python3.9/site-packages/tensorflow/libtensorflow_framework.2.dylib
.venv/lib/python3.9/site-packages/tensorflow/libtensorflow_framework.2.7.0.dylib
.venv/lib/python3.9/site-packages/tensorflow/compiler/mlir/tensorflow
.venv/lib/python3.9/site-packages/tensorflow/libtensorflow_framework.dylib
.venv/lib/python3.9/site-packages/tensorboard/compat/tensorflow_stub
.venv/lib/python3.9/site-packages/tensorboard/compat/tensorflow_stub/__pycache__/pywrap_tensorflow.cpython-39.pyc
.venv/lib/python3.9/site-packages/tensorboard/compat/tensorflow_stub/pywrap_tensorflow.py
.venv/lib/python3.9/site-packages/tensorflow_io_gcs_filesystem-0.23.1.dist-info
.venv/lib/python3.9/site-packages/tensorflow-2.7.0.dist-info
.venv/lib/python3.9/site-packages/tensorboard_plugin_wit/_vendor/tensorflow_serving
Google Colaboratory だと補完されてたのは何だったんだろ…?
sklearn と pandas も追加
poetry add sklearn
poetry add pandas
# 出力は省略
sklearn は train_test_split
を使うために入れる。
from sklearn.model_selection import train_test_split
などとしてインポートするのが常套句?
(TensorFlow 側とかにあっても良さそうなもんだけど、そういう世界の話でもないのかな?)
pandas は Python で構造化されたファイルを読み込むときの便利ライブラリらしい。
import pandas as pd
とかでインポートするのが一般的だそうな。
pd.read_csv()
の戻り値は DataFrame 型なので df
という変数で受けるコトが多いっぽい。