bilzard

<p>reduction == "mean"の場合だと警告が出る。</p>
<blockquote>
<p>/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py:2943: UserWarning: reduction: 'mean' divides the total loss by both the batch size and the support size.'batchmean' divides only by the batch size, and aligns with the KL div math definition.'mean' will be changed to behave the same as 'batchmean' in the next major release.</p>
</blockquote>
<p>ドキュメント[1]によると、<code>reduction="batchmean"</code>の場合は集計結果をBatch数で割る。この指定だとKL divergenceの数学的な定義と一致するらしい。将来的に<code>reduction="mean"</code>も<code>reduction="batchmean"</code>と同じ振る舞いに変更されるとのこと。</p>
<h2 id="reference">
<a class="header-anchor-link" href="#reference" aria-hidden="true"></a> Reference</h2>
<ul>
<li>[1] <a href="https://pytorch.org/docs/stable/generated/torch.nn.KLDivLoss.html" target="_blank" rel="nofollow noopener noreferrer">https://pytorch.org/docs/stable/generated/torch.nn.KLDivLoss.html</a>
</li>
</ul>
