【ML】Benefit of label smoothing

1. What is label smoothing?

label smoothing is a method to prevent Machine Learing model get overfitting.
It makes label a little softly.

・example

# correct label is '2'
label = [0, 0, 1] # 2(one-hot)

smoothed_label = [0.05, 0.05, 0.9]

Like this, label smoothing changes label a little softly.

2. Benefit

We understood about label smoothing, but what does it bring for us?
Let's see benefit of it.

2.1 prevents overfitting

label smoothing prevents overfitting. It is undestandable.
It makes label a little softly, and restrict so overfitting to train data.

As a result, model get more Generalization.

2.2 Handles Label Noise

If train data has incorrect label, label smoothing reduces the inpact of that.

But, I think It is secondary efficacy. The main effect is prevents overfitting.

Summary

label smoothing is a method to prevent overfitting.