内容由Hajime Morrita , Jun Mukai提供。所有播客内容(包括剧集、图形和播客描述)均由 Hajime Morrita , Jun Mukai 或其播客平台合作伙伴直接上传和提供。如果您认为有人在未经您许可的情况下使用您的受版权保护的作品,您可以按照此处概述的流程进行操作https://zh.player.fm/legal。
Player FM -播客应用
使用Player FM应用程序离线!
使用Player FM应用程序离线!
值得一听的播客
赞助
<
<div class="span index">1</div> <span><a class="" data-remote="true" data-type="html" href="/series/advances-in-care">Advances in Care</a></span>
![<div class="span index">1</div> <span><a class="" data-remote="true" data-type="html" href="/series/advances-in-care">Advances in Care</a></span> podcast artwork](https://cdn.player.fm/images/55928701/series/ElP1AC184YuHqdZr/32.jpg 32w, https://cdn.player.fm/images/55928701/series/ElP1AC184YuHqdZr/64.jpg 64w, https://cdn.player.fm/images/55928701/series/ElP1AC184YuHqdZr/128.jpg 128w, https://cdn.player.fm/images/55928701/series/ElP1AC184YuHqdZr/256.jpg 256w, https://cdn.player.fm/images/55928701/series/ElP1AC184YuHqdZr/512.jpg 512w)
![<div class="span index">1</div> <span><a class="" data-remote="true" data-type="html" href="/series/advances-in-care">Advances in Care</a></span> podcast artwork](/static/images/64pixel.png)
On Advances in Care, epidemiologist and science communicator Erin Welsh sits down with physicians from NewYork-Presbyterian hospital to discuss the details behind cutting-edge research and innovative treatments that are changing the course of medicine. From breakthroughs in genome sequencing to the backstories on life-saving cardiac procedures, the work of these doctors from Columbia & Weill Cornell Medicine is united by a collective mission to shape the future of health care and transform the lives of their patients. Erin Welsh, who also hosts This Podcast Will Kill You, gets to the heart of her guests’ most challenging and inventive medical discoveries. Advances in Care is a show for health careprofessionals and listeners who want to stay at the forefront of the latest medical innovations and research. Tune in to learn more about some of medicine’s greatest leaps forward. For more information visit nyp.org/Advances
#111: Formal Algorithms for Transformers
Manage episode 359821334 series 2151064
内容由Hajime Morrita , Jun Mukai提供。所有播客内容(包括剧集、图形和播客描述)均由 Hajime Morrita , Jun Mukai 或其播客平台合作伙伴直接上传和提供。如果您认为有人在未经您许可的情况下使用您的受版权保护的作品,您可以按照此处概述的流程进行操作https://zh.player.fm/legal。
勤務先への脅威に怯える森田が Transformer を復習しました。ご意見ご感想などはおたより投書箱や Reddit にお寄せください。iTunes のレビューや星も歓迎です。
今回は録音に際し Adobe Podcast (beta) のバグを引き当ててしまい、向井と森田の音声トラックがずれてしまいました。ごめんなさい。次回からは non-beta の手堅いツールで録音しようと思います・・・。
- [2207.09238] Formal Algorithms for Transformers
- #15 – Neural Machine Translation by Jointly Learning to Align and Translate
- #38 – Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates
- #51 – Attention Is All You Need
- #53 – BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Jay Alammar – YouTube
- GitHub – openai/tiktoken: tiktoken is a fast BPE tokeniser for use with OpenAI’s models.
- GitHub – karpathy/nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs.
- Let’s build GPT: from scratch, in code, spelled out. – YouTube
147集单集
Manage episode 359821334 series 2151064
内容由Hajime Morrita , Jun Mukai提供。所有播客内容(包括剧集、图形和播客描述)均由 Hajime Morrita , Jun Mukai 或其播客平台合作伙伴直接上传和提供。如果您认为有人在未经您许可的情况下使用您的受版权保护的作品,您可以按照此处概述的流程进行操作https://zh.player.fm/legal。
勤務先への脅威に怯える森田が Transformer を復習しました。ご意見ご感想などはおたより投書箱や Reddit にお寄せください。iTunes のレビューや星も歓迎です。
今回は録音に際し Adobe Podcast (beta) のバグを引き当ててしまい、向井と森田の音声トラックがずれてしまいました。ごめんなさい。次回からは non-beta の手堅いツールで録音しようと思います・・・。
- [2207.09238] Formal Algorithms for Transformers
- #15 – Neural Machine Translation by Jointly Learning to Align and Translate
- #38 – Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates
- #51 – Attention Is All You Need
- #53 – BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Jay Alammar – YouTube
- GitHub – openai/tiktoken: tiktoken is a fast BPE tokeniser for use with OpenAI’s models.
- GitHub – karpathy/nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs.
- Let’s build GPT: from scratch, in code, spelled out. – YouTube
147集单集
所有剧集
×GitHub の Issue を読んでバグを直すエーアイについて森田が読みました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 [2310.06770] SWE-bench: Can Language Models Resolve Real-World GitHub Issues? [2405.15793] SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering SWE-bench Introducing SWE-bench Verified | OpenAI The new Claude 3.5 Sonnet, Computer Use, and Building SOTA Agents — with Erik Schluntz, Anthropic…
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
Rust を Linux カーネルへで使う取り組みの進捗を 向井 がウォッチしました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 An Empirical Study of Rust-for-Linux: The Success, Dissatisfaction, and Compromise | USENIX Rust for Linux
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
Google SQL の新しい文法を森田が紹介しました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 SQL Has Problems. We Can Fix Them: Pipe Syntax In SQL PRQL Pipe syntax | BigQuery | Google Cloud SQLite Forum: Interesting paper from Google on pipe syntax on SQL
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
1 #140: GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models 39:54
LLM にひっかけ算数問題を出してみる話を 向井 が読みました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 [2410.05229] GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models GitHub – openai/grade-school-math
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
写真を集めてシーンをレンダリングするニューラルネットを森田が読みました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis NeRF Tutorial ECCV 2022 illuminate.google.com
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
大きなモデルから小さなモデルを作るテクニックを 向井 が回願しました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 [1503.02531] Distilling the Knowledge in a Neural Network
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
ストリームにパーセンタイルを計算したい森田が教科書を読みました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 [1603.05346v2] Optimal Quantile Approximation in Streams Small Summaries for Big Data Data Types – Presto 0.288 Documentation Estimating Percentile Values | Snowflake Documentation KLL sketch vs t-digest – DataSketches…
学部生にも実装できるストリームの要素カウントアルゴリズムを 向井 が試しました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 [2301.10191] Distinct Elements in Streams: An Algorithm for the (Text) Book The CVM Algorithm for Estimating Distinct Elements in Streams Computer scientists invent an efficient new way to count | Hacker News…
行列の掛け算が得意なハードウェアについて森田が読みました。ご意見感想などは Reddit やおたより投書箱にお寄せください。iTunes のレビューや星もよろしくね。 [1704.04760] In-Datacenter Performance Analysis of a Tensor Processing Unit Systolic Arrays – an overview | ScienceDirect Topics Pallas: a JAX kernel language — JAX documentation About Groq – Fast AI Inference The Design Process for Google’s Training Chips: TPUv2 and TPUv3 | IEEE Journals & Magazine | IEEE Xplore…
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
巨大 ML モデルの軽量 fine-tuning 手法を 向井 が読みました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 [2106.09685] LoRA: Low-Rank Adaptation of Large Language Models
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
CUDA を書かずに済む GPU カーネルの DSL について森田が読みました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations Introducing Triton: Open-source GPU programming for neural networks | OpenAI Welcome to Triton’s documentation! — Triton documentation Hello Triton.ipynb – Colab #01: Tensor Comprehensions, Rust Belt – Misreading Chat #23 – Halide: Decoupling Algorithms from Schedules for High-Performance Image Processing – Misreading Chat #27 – Julia: A Fresh Approach to Numerical Computing – Misreading Chat…
Stable Diffusion の元論文を 向井 が読みました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 [2112.10752] High-Resolution Image Synthesis with Latent Diffusion Models [2105.05233] Diffusion Models Beat GANs on Image Synthesis Classifier-Free Diffusion Guidance | OpenReview What are Diffusion Models? | Lil’Log…
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
CUDA で書かれた PyTorch 用カーネルに森田が玉砕しました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 [2205.14135] FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness GitHub – Dao-AILab/flash-attention: Fast and memory-efficient exact attention GitHub – NVIDIA/apex: A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch [2307.08691] FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning [2112.05682] Self-attention Does Not Need $O(n^2)$ Memory GitHub – tspeterkim/flash-attention-minimal: Flash Attention in ~100 lines of CUDA (forward pass only)…
向井 が画像生成の拡散モデルに入門しました。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 Diffusion models from scratch [1503.03585] Deep Unsupervised Learning using Nonequilibrium Thermodynamics [2006.11239] Denoising Diffusion Probabilistic Models
M
Misreading Chat
![Misreading Chat podcast artwork](https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/32.jpg 32w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/64.jpg 64w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/128.jpg 128w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/256.jpg 256w, https://cdn.player.fm/images/20753125/series/K0KlVlJ2j1UfaNnu/512.jpg 512w)
![Misreading Chat podcast artwork](/static/images/64pixel.png)
森田が飽きずに CUDA の教科書を読んでます。ご意見感想などは Reddit や おたより投書箱 にお寄せください。 iTunes のレビューや星 もよろしくね。 Programming Massively Parallel Processors: A Hands-on Approach ( Amazon.co.jp , Elsevier ) NVIDIA H100 Tensor Core GPU Architecture Overview
欢迎使用Player FM
Player FM正在网上搜索高质量的播客,以便您现在享受。它是最好的播客应用程序,适用于安卓、iPhone和网络。注册以跨设备同步订阅。