GenAI Inference

a

GPU Kernel

https://developer.nvidia.com/blog/automating-gpu-kernel-generation-with-deepseek-r1-and-inference-time-scaling/

https://qiita.com/teppei_nakano/items/62e93ccceb7066fff4ce

https://arxiv.org/html/2502.11089v1

https://nvidianews.nvidia.com/news/nvidia-dynamo-open-source-library-accelerates-and-scales-ai-reasoning-models?linkId=100000349576608

https://youtu.be/1bRmskFCnqY?si=XlNB2PU9HP9lPlP6

https://youtu.be/xoBl4PYFEHU?si=svCAdyvQUSoSJlbS

https://youtu.be/SeImiPDVMCw?si=-ztp8WQdTjRXgOTG

Inf2

• ViT: https://towardsdatascience.com/ai-model-optimization-on-aws-inferentia-and-trainium-cfd48e85d5ac/

Inf1

• YOLO: https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/pytorch/yolo_v4.html
• EfficientNet: https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/models/inference-inf1-samples.html

https://chatgpt.com/share/6804e4e2-e490-8006-ada0-f8d9d60baf02

https://www.ogis-ri.co.jp/otc/hiroba/technical/detr/part1.html

https://pytorch.org/blog/flexattention-for-inference/