悪虫

雑に<code>pip install auto-gptq</code>したら<code>cuda extention not installed</code>とか言われた。 
これを回避したいならwhlからinstallしよう。
<div class="code-block-container"><pre><code> $ pip install auto_gptq-0.2.2+cu118-cp310-cp310-win_amd64.whl
</code></pre></div>ダウンロードリンク 
<iframe id="zenn-embedded__5286b69971a1e" src="https://embed.zenn.studio/card#zenn-embedded__5286b69971a1e" data-content="https%3A%2F%2Fgithub.com%2FPanQiWei%2FAutoGPTQ%2Freleases" frameborder="0" scrolling="no" loading="lazy"></iframe><a href="https://github.com/PanQiWei/AutoGPTQ/releases" style="display:none" target="_blank" rel="nofollow noopener noreferrer">https://github.com/PanQiWei/AutoGPTQ/releases</a>
poetry環境なら以下
<div class="code-block-container">
<div class="code-block-filename-container"> pyproject.toml</div>
<pre class="language-toml"><code class="language-toml">[tool.poetry.dependencies]
python = "^3.10"
auto-gptq = {url = "https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.2.2/auto_gptq-0.2.2+cu118-cp310-cp310-win_amd64.whl"}
torch = {url = "https://download.pytorch.org/whl/cu118/torch-2.0.0%2Bcu118-cp310-cp310-win_amd64.whl"}
transformers = "^4.30.2"
</code></pre>
</div>