🚀

【Microsoft Fabric】Microsoft Fabricでcondaを使い、ライブラリーのバージョンを設定する

2024/07/08に公開

参考資料

https://learn.microsoft.com/en-us/fabric/data-engineering/library-management

前提

  • Microsoft FabricのNotebookが使える状態であること

https://zenn.dev/headwaters/articles/6c08f8ba1f8e47

手順

  1. Notebook上で以下のコードを実行する
pip list
出力結果

Package Version


absl-py 2.0.0
adal 1.2.7
adlfs 2023.4.0
aiohttp 3.8.6
aiosignal 1.3.1
alembic 1.12.0
ansi2html 0.0.0
anyio 3.7.1
appdirs 1.4.4
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
astor 0.8.1
asttokens 2.4.0
astunparse 1.6.3
async-timeout 4.0.3
attrs 23.1.0
autopage 0.5.2
azure-core 1.29.4
azure-datalake-store 0.0.51
azure-identity 1.14.1
azure-storage-blob 12.18.3
azure-storage-file-datalake 12.12.0
azure-synapse-ml-predict 1.0.0
azureml-synapse 0.0.1
backcall 0.2.0
backoff 1.11.1
backports.functools-lru-cache 1.6.5
bcrypt 4.0.1
beautifulsoup4 4.12.2
bleach 6.1.0
blinker 1.6.3
Brotli 1.0.9
cached-property 1.5.2
cachetools 5.3.2
catboost 1.1.1
certifi 2023.7.22
cffi 1.16.0
charset-normalizer 3.3.1
chroma-hnswlib 0.7.3
chromadb 0.4.13
click 8.1.7
cliff 4.2.0
cloudpickle 2.2.1
clr-loader 0.2.6
cmaes 0.10.0
cmd2 2.4.3
colorama 0.4.6
coloredlogs 15.0.1
colorlog 6.7.0
comm 0.1.4
conda-package-handling 2.2.0
conda_package_streaming 0.9.0
configparser 5.3.0
contextlib2 21.6.0
contourpy 1.1.1
cryptography 41.0.5
cycler 0.12.1
Cython 3.0.4
dash 2.14.0
dash-core-components 2.0.0
dash-cytoscape 0.2.0
dash-html-components 2.0.0
dash-table 5.0.0
databricks-cli 0.18.0
dataclasses 0.8
datasets 2.14.6
debugpy 1.8.0
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.7
diskcache 5.6.3
distlib 0.3.7
docker 6.1.3
docker-pycreds 0.4.0
entrypoints 0.4
et-xmlfile 1.1.0
exceptiongroup 1.1.3
executing 1.2.0
fastapi 0.103.2
fastjsonschema 2.18.1
filelock 3.11.0
FLAML 2.1.2+20240517
Flask 3.0.0
flatbuffers 23.5.26
fluent-logger 0.10.0
fonttools 4.43.1
fqdn 1.5.1
frozenlist 1.4.0
fsspec 2023.10.0
fsspec-wrapper 0.1.12
gast 0.4.0
geographiclib 1.52
geopy 2.3.0
gevent 23.9.0.post1
gitdb 4.0.11
GitPython 3.1.40
gmpy2 2.1.2
google-auth 2.23.3
google-auth-oauthlib 1.0.0
google-pasta 0.2.0
graphviz 0.20.1
greenlet 3.0.1
grpcio 1.54.3
gson 0.0.3
h11 0.14.0
h5py 3.10.0
holidays 0.35
html5lib 1.1
huggingface-hub 0.18.0
humanfriendly 10.0
idna 3.4
imageio 2.31.1
importlib-metadata 6.8.0
importlib-resources 6.1.0
impulse-python-handler 1.0.20.1.0.0
interpret 0.4.3
interpret-core 0.4.3
ipykernel 6.26.0
ipython 8.14.0
ipywidgets 8.0.7
isodate 0.6.1
isoduration 20.11.0
itsdangerous 2.1.2
jaraco.context 4.3.0
jax 0.4.17
jaxlib 0.4.14
jedi 0.19.1
Jinja2 3.1.2
joblib 1.3.2
joblibspark 0.5.2
json-tricks 3.17.3
jsonpointer 2.4
jsonschema 4.19.1
jsonschema-specifications 2023.7.1
jupyter_client 8.5.0
jupyter_core 5.4.0
jupyter-events 0.8.0
jupyter_server 2.7.3
jupyter_server_terminals 0.4.4
jupyter-ui-poll 0.2.2
jupyterlab-pygments 0.2.2
jupyterlab-widgets 3.0.9
keras 2.12.0
Keras-Preprocessing 1.1.2
kiwisolver 1.4.5
liac-arff 2.5.0
library-metadata-cooker 0.0.7
lightgbm 4.0.0
lime 0.2.0.1
llvmlite 0.40.1
lxml 4.9.3
Mako 1.2.4
Markdown 3.4.4
markdown-it-py 3.0.0
MarkupSafe 2.1.3
matplotlib 3.7.2
matplotlib-inline 0.1.6
mdurl 0.1.0
mistune 3.0.1
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
ml-dtypes 0.3.1
mlflow-skinny 2.6.0
monotonic 1.5
more-itertools 10.1.0
mpmath 1.3.0
msal 1.24.1
msal-extensions 1.0.0
msgpack 1.0.7
multidict 6.0.4
multiprocess 0.70.15
munkres 1.1.4
mypy 1.4.1
mypy-extensions 1.0.0
nbclient 0.8.0
nbconvert 7.9.2
nbformat 5.9.2
nest-asyncio 1.5.8
networkx 3.2
nltk 3.8.1
nni 2.10.1
notebookutils 1.1.1.34.20240605.8
numba 0.57.1
numpy 1.24.3
oauthlib 3.2.2
onnxruntime 1.16.1
openai 0.27.8
openpyxl 3.1.2
opt-einsum 3.3.0
optuna 2.8.0
overrides 7.4.0
packaging 23.2
pandas 2.0.3
pandas-stubs 2.1.1.230928
pandasql 0.7.3
pandocfilters 1.5.0
paramiko 3.3.1
parso 0.8.3
pathos 0.3.1
pathtools 0.1.2
patsy 0.5.3
pbr 5.11.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 10.0.1
pip 23.1.2
pkgutil_resolve_name 1.3.10
platformdirs 3.5.1
plotly 5.16.1
ply 3.11
pooch 1.8.0
portalocker 2.8.2
posthog 3.0.2
powerbiclient 3.1.1
pox 0.3.3
ppft 1.7.6.7
prettytable 3.8.0
prometheus-client 0.17.1
prompt-toolkit 3.0.39
protobuf 4.21.12
psutil 5.9.5
ptyprocess 0.7.0
pulsar-client 3.3.0
pure-eval 0.2.2
py4j 0.10.9.7
pyarrow 12.0.1
pyasn1 0.5.0
pyasn1-modules 0.3.0
pycosat 0.6.6
pycparser 2.21
pydantic 1.10.9
Pygments 2.16.1
PyJWT 2.8.0
PyNaCl 1.5.0
pyodbc 4.0.39
pyOpenSSL 23.2.0
pyparsing 3.0.9
pyperclip 1.8.2
PyPika 0.48.9
PyQt5 5.15.9
PyQt5-sip 12.12.2
PySocks 1.7.1
pyspark 3.4.1.5.3.20230713
python-dateutil 2.8.2
python-json-logger 2.0.7
pythonnet 3.0.1
PythonWebHDFS 0.2.3
pytz 2023.3.post1
pyu2f 0.1.5
PyYAML 6.0.1
pyzmq 25.1.1
referencing 0.30.2
regex 2023.8.8
requests 2.31.0
requests-oauthlib 1.3.1
responses 0.23.3
retrying 1.3.3
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.6.0
rouge-score 0.1.2
rpds-py 0.10.6
rsa 4.9
ruamel.yaml 0.17.32
ruamel.yaml.clib 0.2.7
ruamel-yaml-conda 0.15.80
sacremoses 0.0.53
SALib 1.4.7
schema 0.7.5
scikit-learn 1.3.0
scipy 1.10.1
seaborn 0.12.2
semantic-link-sempy 0.7.2
Send2Trash 1.8.2
sentence-transformers 2.0.0
sentry-sdk 1.32.0
seqeval 1.2.2
setproctitle 1.3.3
setuptools 68.2.2
shap 0.42.1
shellingham 1.5.4
simplejson 3.19.2
sip 6.7.12
six 1.16.0
slicer 0.0.7
smmap 5.0.0
sniffio 1.3.0
soupsieve 2.5
SQLAlchemy 2.0.22
sqlanalyticsconnectorpy 1.0.1
sqlparse 0.4.4
stack-data 0.6.2
starlette 0.27.0
statsmodels 0.14.0
stevedore 5.1.0
sympy 1.12
synapseml-cognitive 1.0.4
synapseml-core 1.0.4
synapseml-deep-learning 1.0.4
synapseml-internal 1.0.4.0.dev1
synapseml-lightgbm 1.0.4
synapseml-mlflow 1.0.23
synapseml-opencv 1.0.4
synapseml-utils 1.0.19.post1
synapseml-vw 1.0.4
tabulate 0.9.0
tenacity 8.2.3
tensorboard 2.12.3
tensorboard-data-server 0.7.0
tensorflow 2.12.1
tensorflow-estimator 2.12.0
termcolor 2.3.0
terminado 0.17.1
threadpoolctl 3.2.0
tiktoken 0.5.1
tinycss2 1.2.1
tokenizers 0.13.3
toml 0.10.2
tomli 2.0.1
toolz 0.12.0
torch 2.0.1
tornado 6.3.3
tqdm 4.66.1
traitlets 5.12.0
transformers 4.26.0
treeinterpreter 0.2.2
typed-ast 1.5.5
typeguard 2.13.3
typer 0.9.0
types-python-dateutil 2.8.19.14
types-pytz 2023.3.1.1
types-PyYAML 6.0.12.12
typing_extensions 4.5.0
typing-utils 0.1.0
tzdata 2023.3
unicodedata2 15.1.0
uri-template 1.3.0
urllib3 1.26.17
uvicorn 0.23.2
virtualenv 20.23.1
wandb 0.15.12
wcwidth 0.2.8
webcolors 1.13
webencodings 0.5.1
websocket-client 1.6.4
websockets 12.0
Werkzeug 3.0.1
wheel 0.41.2
widgetsnbextension 4.0.9
wolframalpha 5.0.0
wrapt 1.15.0
xgboost 1.7.6
xmltodict 0.13.0
xxhash 3.4.1
yarl 1.9.2
zipp 3.17.0
zope.event 5.0
zope.interface 6.1
zstandard 0.21.0

  1. openai 0.27.8を1.27.0に変更を行うために、以下のコマンドを実行する
%conda install openai==1.27.0
  1. 以下のコマンドを実行して、バージョンアップしたかを確認する
pip show openai
出力結果

Name: openai
Version: 1.27.0
Summary: The official Python library for the openai API
Home-page:
Author:
Author-email: OpenAI support@openai.com
License:
Location: /nfs4/pyenv-a0010c15-dde7-483d-8012-83e07505ad29/lib/python3.10/site-packages
Requires: anyio, distro, httpx, pydantic, sniffio, tqdm, typing-extensions
Required-by:
Note: you may need to restart the kernel to use updated packages.

  1. コードを実行しバージョンアップしたかを試す

https://zenn.dev/headwaters/articles/9d4410846fd6b9

from openai import AzureOpenAI

client = AzureOpenAI(
    api_version="2023-05-15", # 固定
    api_key= "<key>",
    azure_endpoint="<endpoint>"
)

response = client.chat.completions.create(
    model="gpt-4o",  
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me a joke."},
    ]
)
print(response.choices[0].message.content)
出力結果

Sure, here's one for you:

Why don't skeletons fight each other?

They don't have the guts!

まとめ

Condaを使い、Fabricのライブラリーのバージョンを管理してみました。
他にも環境を作ってライブラリーのバージョンを管理する方法もあるとのこと。
こっちの方が良さそう。
https://learn.microsoft.com/en-us/fabric/data-engineering/environment-manage-library

ヘッドウォータース

Discussion