MMDetection エラーとjsonファイル適正化

COCO Format(Jsonファイル)でエラーが出る

• 本稿は画像単体の検出ではなく、ビデオ検出の為にトレーニングデータ pthファイル作成前提です。
• This article is written for user who will use webcam detection.

COCO Annotator Exportファイルユーティリティ更新

• 色々文句言われるので、文句に応じた変換スクリプトを用意。（Faster‗RCNN）
• ファイルチェッカーで問題なかったです。

https://github.com/mi-kaneyon/coco-data/tree/main/coco-corrector

このあたりにまとめました(coco annotator関連)

coco annotatorで作っても読めない？

• coco形式といえばcoco annotatorと思いますが、エラーでTrainingできません。
• 掲示板でも聞いても文法違いとあしらわれ・・・

https://qiita.com/Manyan3/items/df894499d30da9220fc7

学習時間がかかるので、専用機を立てました

• Anaconda 環境です
• 私は母艦と同じ名前で作りました。

conda env export > ファイル名.yml

conda env create -n 環境名 -f ファイル名.yml

的はずれなエラー

python tools/train.py ./configs/yolox/cup_yolo4_modified.py --work-dir work_dirs
Traceback (most recent call last):
  File "/home/kanengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 245, in _validate_py_syntax
    ast.parse(content)
  File "/home/kanengi/anaconda3/envs/mmlab/lib/python3.8/ast.py", line 47, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 60
    model = dict(
    ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tools/train.py", line 133, in <module>
    main()
  File "tools/train.py", line 70, in main
    cfg = Config.fromfile(args.config)
  File "/home/kanengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 178, in fromfile
    cfg_dict, cfg_text, env_variables = Config._file2dict(
  File "/home/kanengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 487, in _file2dict
    for base_cfg_path in Config._get_base_files(temp_config_file.name):
  File "/home/kanengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 645, in _get_base_files
    Config._validate_py_syntax(filename)
  File "/home/kanengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 247, in _validate_py_syntax
    raise SyntaxError('There are syntax errors in config '
SyntaxError: There are syntax errors in config file /tmp/tmp0spns6hx/tmpb9bvi6q8.py: invalid syntax (<unknown>, line 60)
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f4d3cf2f160>
Traceback (most recent call last):
  File "/home/kanengi/anaconda3/envs/mmlab/lib/python3.8/tempfile.py", line 440, in __del__
    self.close()
  File "/home/kanengi/anaconda3/envs/mmlab/lib/python3.8/tempfile.py", line 436, in close
    unlink(self.name)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp0spns6hx/tmpb9bvi6q8.py'

なんか惜しいエラー

box_head.multi_level_conv_obj.2.bias:weight_decay=0.0
loading annotations into memory...
Done (t=0.03s)
creating index...
index created!
Traceback (most recent call last):
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 122, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "/home/askengi/mmdetection/mmdet/datasets/base_det_dataset.py", line 40, in __init__
    super().__init__(*args, **kwargs)
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 245, in __init__
    self.full_init()
  File "/home/askengi/mmdetection/mmdet/datasets/base_det_dataset.py", line 78, in full_init
    self.data_bytes, self.data_address = self._serialize_data()
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/dataset/base_dataset.py", line 765, in _serialize_data
    data_bytes = np.concatenate(data_list)
  File "<__array_function__ internals>", line 200, in concatenate
ValueError: need at least one array to concatenate

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 122, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/runner/loops.py", line 335, in __init__
    super().__init__(runner, dataloader)
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/runner/base_loop.py", line 26, in __init__
    self.dataloader = runner.build_dataloader(
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1346, in build_dataloader
    dataset = DATASETS.build(dataset_cfg)
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/registry/registry.py", line 548, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 144, in build_from_cfg
    raise type(e)(
ValueError: class `CocoDataset` in mmdet/datasets/coco.py: need at least one array to concatenate

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tools/train.py", line 133, in <module>
    main()
  File "tools/train.py", line 129, in main
    runner.train()
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1701, in train
    self._val_loop = self.build_val_loop(
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1521, in build_val_loop
    loop = LOOPS.build(
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/registry/registry.py", line 548, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/home/askengi/anaconda3/envs/mmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 144, in build_from_cfg
    raise type(e)(
ValueError: class `ValLoop` in mmengine/runner/loops.py: class `CocoDataset` in mmdet/datasets/coco.py: need at least one array to concatenate

微妙に足りなかったり、名前の違うパラメータがあり！

• どうもID名がMMDetectionが要求しているものと異なっている模様。
• １０００枚超えのアノテーションデータは流石に開くのが厳しい
• ツールで修正することにしました。

English summary of this Article

If you use coco annotator for annotation, you may face issue of training failure

• Here is introduce failure sample and solution to use converter.
• below Github link is converter tools introduction
• older files for label studio, latest one is for coco annotator

Label studioの方はそのままの構造を活用するのも一つの手

まずエラーが出ないのが大きいです

• ただし、ファイル名が変えられてしまうので、管理が大変かも・・・

https://github.com/mi-kaneyon/coco-data/tree/main

アノテーション結果ビューア(更新）

• 結果がファイルに正しく反映されているのか表示
  cocoanote_checker.py
• 同じリポのcoco-correctorに置きました。
• 簡易ビューアとして使っています。