✨

MotionProで画像からアニメーション作った: 2.付属デモ動かす

2025/09/05に公開

前回は環境構築まで行きましたので

今回は付属のデモをGRADIOで動かすところを開設します。といってもほぼそのまま動かし、でも自分で用意した画像だと全く動かないという問題にちょくめんしましたが、いったん付属のデモの動かし方を解説します。
COLABでは2_Gradioデモ.ipynb で保存してます。

(motionpro)$ python demo_sparse_flex_wh.py

COLABでの出力、LOCALでもGRADIOのアクセスが出力されると思います

/usr/local/lib/python3.12/dist-packages/pytorch_lightning/plugins/training_type/ddp.py:68: DeprecationWarning: `TorchScript` support for functional optimizers is deprecated and will be removed in a future PyTorch release. Consider using the `torch.compile` optimizer instead.
  from torch.distributed.optim import DistributedOptimizer
[2025-09-05 01:00:27,433] [INFO] [real_accelerator.py:219:get_accelerator] Setting ds_accelerator to cuda (auto detect)
2025-09-05 01:00:29.075220: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-09-05 01:00:29.092170: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1757034029.113547    6327 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1757034029.120042    6327 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
W0000 00:00:1757034029.136580    6327 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1757034029.136606    6327 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1757034029.136609    6327 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1757034029.136611    6327 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
2025-09-05 01:00:29.141662: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
VideoTransformerBlock is using checkpointing
open_clip_model.safetensors: 100% 3.94G/3.94G [00:21<00:00, 181MB/s]
Initialized embedder #0: FrozenOpenCLIPImagePredictionEmbedder with 683800065 params. Trainable: False
Initialized embedder #1: ConcatTimestepEmbedderND with 0 params. Trainable: False
Initialized embedder #2: ConcatTimestepEmbedderND with 0 params. Trainable: False
/usr/local/lib/python3.12/dist-packages/pytorch_lightning/core/lightning.py:2058: DeprecationWarning: `torch.distributed._sharded_tensor` will be deprecated, use `torch.distributed._shard.sharded_tensor` instead
  from torch.distributed._sharded_tensor import pre_load_state_dict_hook, state_dict_hook
Initialized embedder #3: VideoPredictionEmbedderWithEncoder with 83653863 params. Trainable: False
Initialized embedder #4: ConcatTimestepEmbedderND with 0 params. Trainable: False
Lora successfully injected into Net.
Restored from models/motionpro_ckpt/MotionPro-gs_16k.pt with 0 missing and 0 unexpected keys
Running on local URL:  http://127.0.0.1:7865
Running on public URL: https://16d1342c307b50787f.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)

手順は、画像をアップして、動かしたいところをブラシして、"Init Mask and Crop Image"ボタンを押す.そしてAdd Dragで、軌跡をマウスで書いて（右の赤い線）、RUNするだけです。
（が今回はエラーになりました。画像や、マスク、軌跡など入力などによってよくというかほとんどエラーになります）。一応このエラーはGRADIOの入力データの調整に問題があり、モデル自体の問題ではないことはわかってます。

ですので私のほうで入力（元画像、マスク画像、軌跡）をあたえればアニメーションをつくるコマンドを作成しています。

mp_client.py
また、入力を作成するHTML＋JAVASCRIPTも作成しています。
generate_motionpro_input.html
(FLASKもしくはFASTAPIなどその他のAPIサーバーへの送信はAPIサーバー＋WEBサーバーが立ち上がってないと機能しませんがリサイズした画像、マスク画像、軌跡のダウンロードはローカルで動かしても機能します。)　次回はこれらをつかって、実際に自由にアニメを作るところを解説します。

Discussion