))
Reinventing Stream Processing: From LinkedIn to Responsive with Apurva Mehta
Manage episode 469992700 series 3594857
内容由Kostas Pardalis, Nitay Joffe提供。所有播客内容（包括剧集、图形和播客描述）均由 Kostas Pardalis, Nitay Joffe 或其播客平台合作伙伴直接上传和提供。如果您认为有人在未经您许可的情况下使用您的受版权保护的作品，您可以按照此处概述的流程进行操作https://zh.player.fm/legal。
Summary
In this episode, Apurva Mehta, co-founder and CEO of Responsive, recounts his extensive journey in stream processing—from his early work at LinkedIn and Confluent to his current venture at Responsive.
He explains how stream processing evolved from simple event ingestion and graph indexing to powering complex, stateful applications such as search indexing, inventory management, and trade settlement.
Apurva clarifies the often-misunderstood concept of “real time,” arguing that low latency (often in the one- to two-second range) is more accurate for many applications than the instantaneous response many assume. He delves into the challenges of state management, discussing the limitations of embedded state stores like RocksDB and traditional databases (e.g., Postgres) when faced with high update rates and complex transactional requirements.
The conversation also covers the trade-offs between SQL-based streaming interfaces and more flexible APIs, and how Responsive is innovating by decoupling state from compute—leveraging remote state solutions built on object stores (like S3) with specialized systems such as SlateDB—to improve elasticity, cost efficiency, and operational simplicity in mission-critical applications.
Chapters
00:00 Introduction to Apurva Mehta and Streaming Background
08:50 Defining Real-Time in Streaming Contexts
14:18 Challenges of Stateful Stream Processing
19:50 Comparing Streaming Processing with Traditional Databases
26:38 Product Perspectives on Streaming vs Analytical Systems
31:10 Operational Rigor and Business Opportunities
38:31 Developers' Needs: Beyond SQL
45:53 Simplifying Infrastructure: The Cost of Complexity
51:03 The Future of Streaming Applications
23集单集
分享
