Half Precision PyTorch Developer podcast

Artwork

Tech PyTorch Edward Yang Team PyTorch Deep Learning Machine Learning

内容由PyTorch, Edward Yang, and Team PyTorch提供。所有播客内容（包括剧集、图形和播客描述）均由 PyTorch, Edward Yang, and Team PyTorch 或其播客平台合作伙伴直接上传和提供。如果您认为有人在未经您许可的情况下使用您的受版权保护的作品，您可以按照此处概述的流程进行操作https://zh.player.fm/legal。

PyTorch Developer Podcast « »
Half precision

4+ y ago 18:00

分享

MP3•单集首页

内容由PyTorch, Edward Yang, and Team PyTorch提供。所有播客内容（包括剧集、图形和播客描述）均由 PyTorch, Edward Yang, and Team PyTorch 或其播客平台合作伙伴直接上传和提供。如果您认为有人在未经您许可的情况下使用您的受版权保护的作品，您可以按照此处概述的流程进行操作https://zh.player.fm/legal。

In this episode I talk about reduced precision floating point formats float16 (aka half precision) and bfloat16. I'll discuss what floating point numbers are, how these two formats vary, and some of the practical considerations that arise when you are working with numeric code in PyTorch that also needs to work in reduced precision. Did you know that we do all CUDA computations in float32, even if the source tensors are stored as float16? Now you know!

Further reading.

The Wikipedia article on IEEE floating point is pretty great https://en.wikipedia.org/wiki/IEEE_754
How bfloat16 works out when doing training https://arxiv.org/abs/1905.12322
Definition of acc_type in PyTorch https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/AccumulateType.h

… continue reading

83集单集

#Tech #PyTorch #Edward Yang #Team PyTorch #Deep Learning #Machine Learning

Artwork

Half precision

PyTorch Developer Podcast

26 subscribers

published 4+ y ago

分享

MP3•单集首页

内容由PyTorch, Edward Yang, and Team PyTorch提供。所有播客内容（包括剧集、图形和播客描述）均由 PyTorch, Edward Yang, and Team PyTorch 或其播客平台合作伙伴直接上传和提供。如果您认为有人在未经您许可的情况下使用您的受版权保护的作品，您可以按照此处概述的流程进行操作https://zh.player.fm/legal。

In this episode I talk about reduced precision floating point formats float16 (aka half precision) and bfloat16. I'll discuss what floating point numbers are, how these two formats vary, and some of the practical considerations that arise when you are working with numeric code in PyTorch that also needs to work in reduced precision. Did you know that we do all CUDA computations in float32, even if the source tensors are stored as float16? Now you know!

Further reading.

The Wikipedia article on IEEE floating point is pretty great https://en.wikipedia.org/wiki/IEEE_754
How bfloat16 works out when doing training https://arxiv.org/abs/1905.12322
Definition of acc_type in PyTorch https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/AccumulateType.h

… continue reading

83集单集

#Tech #PyTorch #Edward Yang #Team PyTorch #Deep Learning #Machine Learning

所有剧集

×

欢迎使用Player FM

Player FM正在网上搜索高质量的播客，以便您现在享受。它是最好的播客应用程序，适用于安卓、iPhone和网络。注册以跨设备同步订阅。

收听超过500个主题

边探索边听这个节目