Open2024/02/04にコメント追加1

YOLOX-Body-Head-Face-HandLR-Dist

epoch数。COCO + Distored のとき
nano=220, tiny=220, s=190, m=150, l=120, x=165

Flip 時に左手・右手のラベルを交換するように改造する

yolox/data/data_augment.py

def _mirror(image, boxes, labels, labels_o, prob=0.5):
    _, width, _ = image.shape
    if random.random() < prob:
        image = image[:, ::-1]
        boxes[:, 0::2] = width - boxes[:, 2::-2]
        labels[labels == 4] = -1
        labels[labels == 5] = 4
        labels[labels == -1] = 5
        labels_o[labels_o == 4] = -1
        labels_o[labels_o == 5] = 4
        labels_o[labels_o == -1] = 5
    return image, boxes, labels, labels_o

        # image_t, boxes = _mirror(image, boxes, self.flip_prob)
        image_t, boxes, labels, labels_o = _mirror(image, boxes, labels, labels_o, self.flip_prob)

YOLOX-Body-Head-Face-HandLR-Dist - Nano - Flip時のラベル交換有り

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.347
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.649
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.327
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.230
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.514
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.627
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.161
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.385
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.447
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.623
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.700
per class AP:
| class   | AP     | class     | AP     | class      | AP     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 40.216 | head      | 47.832 | face       | 38.029 |
| hand    | 31.280 | hand_left | 25.175 | hand_right | 25.566 |
per class AR:
| class   | AR     | class     | AR     | class      | AR     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 49.026 | head      | 53.489 | face       | 44.146 |
| hand    | 41.797 | hand_left | 39.967 | hand_right | 39.787 |

YOLOX-Body-Head-Face-HandLR-Dist - Tiny

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.416
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.735
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.415
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.289
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.601
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.692
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.184
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.433
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.492
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.382
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.672
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.738
per class AP:
| class   | AP     | class     | AP     | class      | AP     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 47.267 | head      | 51.593 | face       | 42.546 |
| hand    | 38.834 | hand_left | 34.936 | hand_right | 34.202 |
per class AR:
| class   | AR     | class     | AR     | class      | AR     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 53.979 | head      | 56.300 | face       | 47.929 |
| hand    | 46.630 | hand_left | 45.914 | hand_right | 44.710 |

YOLOX-Body-Head-Face-HandLR-Dist - S

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.450
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.750
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.463
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.312
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.649
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.773
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.194
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.469
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.532
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.411
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.727
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.818
per class AP:
| class   | AP     | class     | AP     | class      | AP     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 53.694 | head      | 54.517 | face       | 46.527 |
| hand    | 44.184 | hand_left | 35.947 | hand_right | 34.897 |
per class AR:
| class   | AR     | class     | AR     | class      | AR     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 59.493 | head      | 59.105 | face       | 51.442 |
| hand    | 50.578 | hand_left | 49.599 | hand_right | 48.779 |

YOLOX-Body-Head-Face-HandLR-Dist - M

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.499
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.796
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.523
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.353
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.706
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.829
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.209
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.505
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.567
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.444
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.763
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.866
per class AP:
| class   | AP     | class     | AP     | class      | AP     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 59.372 | head      | 57.107 | face       | 49.782 |
| hand    | 49.081 | hand_left | 42.296 | hand_right | 41.915 |
per class AR:
| class   | AR     | class     | AR     | class      | AR     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 63.833 | head      | 61.064 | face       | 54.233 |
| hand    | 54.206 | hand_left | 53.359 | hand_right | 53.326 |

YOLOX-Body-Head-Face-HandLR-Dist - L

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.517
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.810
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.545
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.371
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.724
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.844
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.214
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.517
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.579
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.456
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.776
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.876
per class AP:
| class   | AP     | class     | AP     | class      | AP     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 61.569 | head      | 57.917 | face       | 51.026 |
| hand    | 51.069 | hand_left | 44.588 | hand_right | 43.741 |
per class AR:
| class   | AR     | class     | AR     | class      | AR     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 65.520 | head      | 61.566 | face       | 55.260 |
| hand    | 55.558 | hand_left | 54.852 | hand_right | 54.406 |

YOLOX-Body-Head-Face-HandLR-Dist - X

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.549
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.836
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.589
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.406
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.753
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.855
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.225
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.603
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.480
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.799
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.888
per class AP:
| class   | AP     | class     | AP     | class      | AP     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 63.684 | head      | 59.497 | face       | 53.517 |
| hand    | 54.638 | hand_left | 49.313 | hand_right | 48.647 |
per class AR:
| class   | AR     | class     | AR     | class      | AR     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 66.834 | head      | 62.888 | face       | 57.599 |
| hand    | 58.656 | hand_left | 58.211 | hand_right | 57.577 |

YOLOX のラスト 15 epoch の mAP 上昇幅。8ポイントぐらい一気に上昇してトレーニングが終わった。そして、前回コミットした Distorted YOLOX のmAPから 1.9 ポイントだけ低下しているけど、左右の手の難易度を加味するとかなり健闘している気がする。

YOLOX-Body-Head-Face-HandLR-Dist - Nano - Flip時のラベル交換無し

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.315
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.590
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.298
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.206
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.453
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.586
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.140
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.382
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.446
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.337
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.622
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.682
per class AP:
| class   | AP     | class     | AP     | class      | AP     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 40.152 | head      | 47.845 | face       | 37.948 |
| hand    | 31.451 | hand_left | 15.161 | hand_right | 16.482 |
per class AR:
| class   | AR     | class     | AR     | class      | AR     |
|:--------|:-------|:----------|:-------|:-----------|:-------|
| body    | 48.957 | head      | 53.372 | face       | 43.786 |
| hand    | 41.706 | hand_left | 39.536 | hand_right | 39.953 |