<p>When using the GPU with various codes, it is possible that VRAM may still be in use somewhere even after you think you have released it.<br>
Then, the program outputs the error like this:<br>
・Error</p>
<div class="code-block-container"><pre><code>RESOURCE_EXHAUSTED: Out of memory while trying to allocate XXXXXXXXX bytes.
</code></pre></div><p>This time, I will introduce a method to release all VRAM(for Nvidia GPU).</p>
<h1 id="check-the-memory-usage">
<a class="header-anchor-link" href="#check-the-memory-usage" aria-hidden="true"></a> Check the memory usage</h1>
<p>・Check</p>
<div class="code-block-container"><pre class="language-bash"><code class="language-bash">nvidia-smi
</code></pre></div><p>・Memory Usage(Output)</p>
<div class="code-block-container"><pre><code>...
1591MiB /  16376MiB
...
</code></pre></div><h1 id="release-the-vram">
<a class="header-anchor-link" href="#release-the-vram" aria-hidden="true"></a> Release the VRAM</h1>
<aside class="msg message"><span class="msg-symbol">!</span><div class="msg-content">
<p>note: This command kills all instances that using VRAM, so run it after finishing all programs.</p>
</div></aside>
<p>・Install lsof</p>
<div class="code-block-container"><pre class="language-bash"><code class="language-bash"><span class="token function">sudo</span> <span class="token function">apt-get</span> <span class="token function">install</span> <span class="token function">lsof</span>
</code></pre></div><p>・Release</p>
<div class="code-block-container"><pre class="language-bash"><code class="language-bash"><span class="token function">kill</span> <span class="token parameter variable">-9</span> <span class="token variable"><span class="token variable">$(</span><span class="token function">lsof</span> <span class="token parameter variable">-t</span> /dev/nvidia*<span class="token variable">)</span></span>
</code></pre></div><p>After this, maybe you can use the GPU comfortably.</p>


【Nvidia】How to release VRAM of GPU

Check the memory usage

Release the VRAM

Discussion