<h1 id="int4-vs-fp4" data-line="0" class="code-line">
<a class="header-anchor-link" href="#int4-vs-fp4" aria-hidden="true"></a> Int4 vs fp4</h1>
<h2 id="comparison" data-line="1" class="code-line">
<a class="header-anchor-link" href="#comparison" aria-hidden="true"></a> Comparison</h2>
<table data-line="3" class="code-line">
<thead data-line="3" class="code-line">
<tr data-line="3" class="code-line">
<th style="text-align:left">Feature</th>
<th style="text-align:left">INT4</th>
<th style="text-align:left">FP4</th>
</tr>
</thead>
<tbody data-line="5" class="code-line">
<tr data-line="5" class="code-line">
<td style="text-align:left"><strong>Distribution</strong></td>
<td style="text-align:left">Uniform</td>
<td style="text-align:left">Non-Uniform</td>
</tr>
<tr data-line="6" class="code-line">
<td style="text-align:left"><strong>Dynamic Range</strong></td>
<td style="text-align:left">Limited, linear range.</td>
<td style="text-align:left">Wide (due to exponent).</td>
</tr>
<tr data-line="7" class="code-line">
<td style="text-align:left"><strong>Calibration</strong></td>
<td style="text-align:left">Sensitive to outliers</td>
<td style="text-align:left">Robust to outlier.</td>
</tr>
<tr data-line="8" class="code-line">
<td style="text-align:left"><strong>Dequant. Cost to FP16</strong></td>
<td style="text-align:left">multiply-add operation (<code>FP16_value = (INT4_value - ZeroPoint) * ScaleFactor</code>).</td>
<td style="text-align:left">bit-shift conversion(efficient in hardware).</td>
</tr>
<tr data-line="9" class="code-line">
<td style="text-align:left"><strong>Power Consumption</strong></td>
<td style="text-align:left">lower</td>
<td style="text-align:left">higher</td>
</tr>
</tbody>
</table>
<h2 id="which-to-choose%3F" data-line="11" class="code-line">
<a class="header-anchor-link" href="#which-to-choose%3F" aria-hidden="true"></a> which to choose?</h2>
<p data-line="12" class="code-line">INT4: if the hardware doesn't support FP4 well. low power edge device<br>
FP4: basically, any model</p>


[Quantization] int4 vs fp4 which to choose?

Int4 vs fp4

Comparison

which to choose?

Discussion

Feature	INT4	FP4
Distribution	Uniform	Non-Uniform
Dynamic Range	Limited, linear range.	Wide (due to exponent).
Calibration	Sensitive to outliers	Robust to outlier.
Dequant. Cost to FP16	multiply-add operation (`FP16_value = (INT4_value - ZeroPoint) * ScaleFactor`).	bit-shift conversion(efficient in hardware).
Power Consumption	lower	higher