Open1
stackflowのVLMのログ
nnn112358stackflowのVLM(arduino)
https://github.com/Abandon-ht/ModuleLLM_Development_Guide/blob/dev/ESP32/cpp/src/yolo11n.cpp
root@m5stack-LLM:~# /opt/m5stack/bin/llm_vlm
[I][operator()][ 197]: serial_zmq_url:ipc:///tmp/llm/8000.sock
[I][setup][ 247]: StackFlow::setup raw zmq_url:ipc:///tmp/llm/5556.sock raw:{"request_id":"vlm_setup","work_id":"vlm","action":"setup","object":"vlm.setup","data":{"model":"internvl2.5-1B-ax630c","response_format":"vlm.utf-8.stream","enoutput":true,"enkws":true,"max_token_len":255,"prompt":"","input":["vlm.utf-8.stream"]}}
[I][sys_register_unit][ 472]: work_id_number:1000, out_port:ipc:///tmp/llm/5010.sock.vlm.output_url, inference_port:ipc:///tmp/llm/5012.sock.vlm.inference_url
[W][load_model][ 98]: config file :./internvl2.5-1B-ax630c.json miss
[W][load_model][ 98]: config file :./mode_internvl2.5-1B-ax630c.json miss
[W][load_model][ 98]: config file :/opt/m5stack/data/internvl2.5-1B-ax630c/internvl2.5-1B-ax630c.json miss
[W][load_model][ 98]: config file :/opt/m5stack/data/internvl2.5-1B-ax630c/./mode_internvl2.5-1B-ax630c.json miss
[W][load_model][ 98]: config file :/opt/m5stack/data/models/internvl2.5-1B-ax630c.json miss
[I][load_model][ 110]: base_model /opt/m5stack/data/internvl2.5-1B-ax630c/
[I][load_model][ 141]: port_=8080 model_id=/opt/m5stack/data/internvl2.5-1B-ax630c/tokenizer content=''
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
http://localhost:8080
[I][load_model][ 148]: filename_tokenizer_model: http://localhost:8080
[I][ Init][ 127]: LLM init start
/bos_id
127.0.0.1 - - [29/Jan/2025 10:58:21] "GET /bos_id HTTP/1.1" 200 -
{"bos_id": -1}
/eos_id
127.0.0.1 - - [29/Jan/2025 10:58:21] "GET /eos_id HTTP/1.1" 200 -
{"eos_id": 151645}
bos_id: -1, eos_id: 151645
3% | ██ | 1 / 28 [0.01s<0.36s, 76.92 count/s] tokenizer init ok[I][ Init][ 26]: LLaMaEmbedSelector use mmap
64% | █████████████████████ | 18 / 28 [1.53s<2.38s, 11.74 count/s] init 15 axmodel ok,remain_cmm(1952 MB 67% | ██████████████████████ | 19 / 28 [1.64s<2.41s, 11.60 count/s] init 16 axmodel ok,remain_cmm(1936 M 71% | ███████████████████████ | 20 / 28 [1.73s<2.43s, 11.54 count/s] init 17 axmodel ok,remain_cmm(1920 75% | ████████████████████████ | 21 / 28 [1.84s<2.45s, 11.42 count/s] init 18 axmodel ok,remain_cmm(1904 78% | ██████████████████████████ | 22 / 28 [1.93s<2.46s, 11.38 count/s] init 19 axmodel ok,remain_cmm(18 82% | ███████████████████████████ | 23 / 28 [2.03s<2.47s, 11.32 count/s] init 20 axmodel ok,remain_cmm(1 85% | ████████████████████████████ | 24 / 28 [2.13s<2.48s, 11.28 count/s] init 21 axmodel ok,remain_cmm( 89% | █████████████████████████████ | 25 / 28 [2.23s<2.50s, 11.22 count/s] init 22 axmodel ok,remain_cmm 92% | ██████████████████████████████ | 26 / 28 [2.33s<2.50s, 11.18 count/s] init 23 axmodel ok,remain_cm 96% | ███████████████████████████████ | 27 / 28 [3.00s<3.11s, 9.00 count/s] init post axmodel ok,remain_100% | ████████████████████████████████ | 28 / 28 [4.67s<4.67s, 6.00 count/s] init vpm axmodel ok,remain_cmm(1353 MB)[I][ Init][ 275]: max_token_len : 255
[I][ Init][ 280]: kv_cache_size : 128, kv_cache_num: 1023
[I][ Init][ 288]: prefill_token_num : 128
[I][ Init][ 290]: vpm_height : 224,vpm_width : 224
[I][ Init][ 299]: LLM init ok
[I][setup][ 414]: load_mode success
[I][ Encode][ 351]: image encode time : 1043.864014 ms, size : 57344
[I][prompt_complete][ 192]: prompt_complete:ユーモアを交えて60文字以内で説明してください
127.0.0.1 - - [29/Jan/2025 10:58:45] "POST /encode HTTP/1.1" 200 -
{'img_prompt': True, 'text': 'ユーモアを交えて60文字以内で説明してください'}
{"token_ids": [151644, 8948, 198, 5501, 7512, 279, 2168, 19620, 13, 151645, 151644, 872, 198, 151665, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151667, 151666, 198, 125407, 24041, 95, 38780, 29412, 38109, 126600, 21, 15, 87335, 108341, 16161, 44818, 105, 30858, 134093, 151645, 151644, 77091, 198]}
[I][ Run][ 561]: ttft: 359.45 ms
127.0.0.1 - - [29/Jan/2025 10:58:45] "POST /decode HTTP/1.1" 200 -
{"text": "\u3053\u306e\u5199\u771f"}
[I][task_output][ 263]: send:この写真
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:45] "POST /decode HTTP/1.1" 200 -
{"text": "\u306f\u3001\u65e5"}
[I][task_output][ 263]: send:は、日
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:46] "POST /decode HTTP/1.1" 200 -
{"text": "\u5149\u306e\u5149"}
[I][task_output][ 263]: send:光の光
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:46] "POST /decode HTTP/1.1" 200 -
{"text": "\u304c\u6620\u308b"}
[I][task_output][ 263]: send:が映る
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:47] "POST /decode HTTP/1.1" 200 -
{"text": "\u58c1\u306b\u5199"}
[I][task_output][ 263]: send:壁に写
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:47] "POST /decode HTTP/1.1" 200 -
{"text": "\u3063\u3066\u3044\u307e\u3059\u3002\u5149"}
[I][task_output][ 263]: send:っています。光
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:47] "POST /decode HTTP/1.1" 200 -
{"text": "\u304c\u58c1\u306e"}
[I][task_output][ 263]: send:が壁の
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:48] "POST /decode HTTP/1.1" 200 -
{"text": "\u8272\u3092\u5909"}
[I][task_output][ 263]: send:色を変
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:48] "POST /decode HTTP/1.1" 200 -
{"text": "\u3048\u3066\u8f1d\u3044\u3066"}
[I][task_output][ 263]: send:えて輝いて
[I][task_output][ 275]: send stream
127.0.0.1 - - [29/Jan/2025 10:58:48] "POST /decode HTTP/1.1" 200 -
{"text": "\u3044\u307e\u3059\u3002"}
[I][task_output][ 263]: send:います。
[I][task_output][ 275]: send stream
[N][ Run][ 700]: hit eos,avg 8.25 token/s
127.0.0.1 - - [29/Jan/2025 10:58:48] "POST /decode HTTP/1.1" 200 -
{"text": "\u3053\u306e\u5199\u771f\u306f\u3001\u65e5\u5149\u306e\u5149\u304c\u6620\u308b\u58c1\u306b\u5199\u3063\u3066\u3044\u307e\u3059\u3002\u5149\u304c\u58c1\u306e\u8272\u3092\u5909\u3048\u3066\u8f1d\u3044\u3066\u3044\u307e\u3059\u3002"}
[I][task_output][ 263]: send:この写真は、日光の光が映る壁に写っています。光が壁の色を変えて輝いています。
[I][task_output][ 275]: send stream