まさぴょん🐱

<h2 id="dataflow%E3%81%AB%E3%81%A4%E3%81%84%E3%81%A6%F0%9F%93%9D" data-line="0" class="code-line">
<a class="header-anchor-link" href="#dataflow%E3%81%AB%E3%81%A4%E3%81%84%E3%81%A6%F0%9F%93%9D" aria-hidden="true"></a> Dataflowについて📝</h2>
<p data-line="2" class="code-line"><span class="embed-block zenn-embedded zenn-embedded-card"><iframe id="zenn-embedded__2c05ca07ba0ec" src="https://embed.zenn.studio/card#zenn-embedded__2c05ca07ba0ec" data-content="https%3A%2F%2Fcloud.google.com%2Fdataflow%2Fdocs%2Foverview%3Fhl%3Dja" frameborder="0" scrolling="no" loading="lazy"></iframe></span><a href="https://cloud.google.com/dataflow/docs/overview?hl=ja" style="display:none" target="_blank" rel="nofollow noopener noreferrer">https://cloud.google.com/dataflow/docs/overview?hl=ja</a></p>
<p data-line="4" class="code-line"><span class="embed-block zenn-embedded zenn-embedded-card"><iframe id="zenn-embedded__d26f13579baa5" src="https://embed.zenn.studio/card#zenn-embedded__d26f13579baa5" data-content="https%3A%2F%2Fblog.g-gen.co.jp%2Fentry%2Fdataflow-explained" frameborder="0" scrolling="no" loading="lazy"></iframe></span><a href="https://blog.g-gen.co.jp/entry/dataflow-explained" style="display:none" target="_blank" rel="nofollow noopener noreferrer">https://blog.g-gen.co.jp/entry/dataflow-explained</a></p>
<p data-line="6" class="code-line"><span class="embed-block zenn-embedded zenn-embedded-card"><iframe id="zenn-embedded__fabf8209aa48b" src="https://embed.zenn.studio/card#zenn-embedded__fabf8209aa48b" data-content="https%3A%2F%2Fwww.case-k.jp%2Fentry%2F2019%2F11%2F08%2F173450" frameborder="0" scrolling="no" loading="lazy"></iframe></span><a href="https://www.case-k.jp/entry/2019/11/08/173450" style="display:none" target="_blank" rel="nofollow noopener noreferrer">https://www.case-k.jp/entry/2019/11/08/173450</a></p>


<h2 id="dataflow%E6%B4%BB%E7%94%A8%E4%BA%8B%E4%BE%8B%F0%9F%93%9D" data-line="0" class="code-line">
<a class="header-anchor-link" href="#dataflow%E6%B4%BB%E7%94%A8%E4%BA%8B%E4%BE%8B%F0%9F%93%9D" aria-hidden="true"></a> Dataflow活用事例📝</h2>
<h3 id="dataflow%E3%82%92%E4%BD%BF%E3%81%A3%E3%81%9F%E5%A4%A7%E8%A6%8F%E6%A8%A1%E8%A8%80%E8%AA%9E%E3%83%A2%E3%83%87%E3%83%AB%EF%BC%88llm%EF%BC%89%E3%81%AE%E3%82%B9%E3%83%88%E3%83%AA%E3%83%BC%E3%83%9F%E3%83%B3%E3%82%B0%E5%87%A6%E7%90%86" data-line="2" class="code-line">
<a class="header-anchor-link" href="#dataflow%E3%82%92%E4%BD%BF%E3%81%A3%E3%81%9F%E5%A4%A7%E8%A6%8F%E6%A8%A1%E8%A8%80%E8%AA%9E%E3%83%A2%E3%83%87%E3%83%AB%EF%BC%88llm%EF%BC%89%E3%81%AE%E3%82%B9%E3%83%88%E3%83%AA%E3%83%BC%E3%83%9F%E3%83%B3%E3%82%B0%E5%87%A6%E7%90%86" aria-hidden="true"></a> Dataflowを使った大規模言語モデル（LLM）のストリーミング処理</h3>
<p data-line="3" class="code-line"><span class="embed-block zenn-embedded zenn-embedded-card"><iframe id="zenn-embedded__22070f233c5e3" src="https://embed.zenn.studio/card#zenn-embedded__22070f233c5e3" data-content="https%3A%2F%2Fzenn.dev%2Fcloud_ace%2Farticles%2Fbf805ea9d9450d" frameborder="0" scrolling="no" loading="lazy"></iframe></span><a href="https://zenn.dev/cloud_ace/articles/bf805ea9d9450d" style="display:none" target="_blank">https://zenn.dev/cloud_ace/articles/bf805ea9d9450d</a></p>
<h3 id="%E5%88%86%E6%9E%90%E7%94%A8%E3%83%AD%E3%82%B0%E3%82%92cloud-storage%E3%81%AB%E3%82%A2%E3%83%83%E3%83%97%E3%83%AD%E3%83%BC%E3%83%89%E3%81%97%E3%81%A6%E3%80%81dataflow%E7%B5%8C%E7%94%B1%E3%81%A7bigquery%E3%81%B8%E6%A0%BC%E7%B4%8D%E3%81%99%E3%82%8B" data-line="5" class="code-line">
<a class="header-anchor-link" href="#%E5%88%86%E6%9E%90%E7%94%A8%E3%83%AD%E3%82%B0%E3%82%92cloud-storage%E3%81%AB%E3%82%A2%E3%83%83%E3%83%97%E3%83%AD%E3%83%BC%E3%83%89%E3%81%97%E3%81%A6%E3%80%81dataflow%E7%B5%8C%E7%94%B1%E3%81%A7bigquery%E3%81%B8%E6%A0%BC%E7%B4%8D%E3%81%99%E3%82%8B" aria-hidden="true"></a> 分析用ログをCloud Storageにアップロードして、Dataflow経由でBigQueryへ格納する</h3>
<p data-line="6" class="code-line"><span class="embed-block zenn-embedded zenn-embedded-card"><iframe id="zenn-embedded__a92f33c94d982" src="https://embed.zenn.studio/card#zenn-embedded__a92f33c94d982" data-content="https%3A%2F%2Fzenn.dev%2Fpharmax%2Farticles%2F932d7095313387" frameborder="0" scrolling="no" loading="lazy"></iframe></span><a href="https://zenn.dev/pharmax/articles/932d7095313387" style="display:none" target="_blank">https://zenn.dev/pharmax/articles/932d7095313387</a></p>


【GoogleCloud/GCP】Dataflow を使ったストリーミングデータ処理について📝