Why focus on schemers in particular? (Sections 1.3-1.4 of "Scheming AIs")

Joe Carlsmith Audio

#Society #Philosophy #Joe

46:39

“If we march into that village and we start trying to persecute people for using poison, something that's very illegal, nobody's going to talk to us. We're not going to find out where the poison came from. We're not going to be able to shut anything down. We should take the approach that people are using poison because they're desperate, because they see no other alternative.” – Andrew Stein Andrew Stein is a wildlife ecologist who spent the past 25 years studying human carnivore conflict from African wild dogs and lions in Kenya and Botswana to leopards and hyenas in Namibia. His work has long focused on finding ways for people and predators to coexist. He is the founder of CLAWS , an organization based in Botswana that's working at the intersection of cutting-edge wildlife research and community driven conservation. Since its start in 2014 and official launch as an NGO in 2020, CLAWS has been pioneering science-based, tech-forward strategies to reduce conflict between people and carnivores. By collaborating closely with local communities, especially traditional cattle herders, CLAWS supports both species conservation and rural livelihoods—making coexistence not just possible, but sustainable.…

大约1年之前 31:17

MP3•单集首页

This is sections 1.3-1.4 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?”

Text of the report here: https://arxiv.org/abs/2311.08379
Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power
Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power

章节

1. Why focus on schemers in particular? (Sections 1.3-1.4 of "Scheming AIs") (00:00:00)

2. 1.3 Why focus on schemers in particular? (00:00:36)

3. 1.3.1 The type of misalignment I’m most worried about (00:01:14)

4. 1.3.2 Contrast with reward-on-the-episode seekers (00:04:27)

5. 1.3.2.1 Responsiveness to honest tests (00:04:46)

6. 1.3.2.2 Temporal scope and general “ambition” (00:07:54)

7. 1.3.2.3 Sandbagging and “early undermining” (00:11:17)

8. 1.3.3 Contrast with models that aren’t playing the training game (00:17:13)

9. 1.3.4 Non-schemers with schemer-like traits (00:23:13)

10. 1.3.5 Mixed models (00:25:20)

11. 1.4 Are theoretical arguments about this topic even useful? (00:28:35)

63集单集

Why focus on schemers in particular? (Sections 1.3-1.4 of "Scheming AIs")

Joe Carlsmith Audio

published 大约1年之前

MP3•单集首页

This is sections 1.3-1.4 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?”

章节

1. Why focus on schemers in particular? (Sections 1.3-1.4 of "Scheming AIs") (00:00:00)

2. 1.3 Why focus on schemers in particular? (00:00:36)

3. 1.3.1 The type of misalignment I’m most worried about (00:01:14)

4. 1.3.2 Contrast with reward-on-the-episode seekers (00:04:27)

5. 1.3.2.1 Responsiveness to honest tests (00:04:46)

6. 1.3.2.2 Temporal scope and general “ambition” (00:07:54)

7. 1.3.2.3 Sandbagging and “early undermining” (00:11:17)

8. 1.3.3 Contrast with models that aren’t playing the training game (00:17:13)

9. 1.3.4 Non-schemers with schemer-like traits (00:23:13)

10. 1.3.5 Mixed models (00:25:20)

11. 1.4 Are theoretical arguments about this topic even useful? (00:28:35)

63集单集

#Society #Philosophy #Joe

所有剧集

Joe Carlsmith Audio

1
AI for AI safety 27:51

6 weeks之前27:51

27:51

We should try extremely hard to use AI labor to help address the alignment problem. Text version here: https://joecarlsmith.com/2025/03/14/ai-for-ai-safety

Joe Carlsmith Audio

1
Paths and waystations in AI safety 18:07

6 weeks之前18:07

18:07

On the structure of the path to safe superintelligence, and some possible milestones along the way. Text version here: https://joecarlsmith.substack.com/p/paths-and-waystations-in-ai-safety

Joe Carlsmith Audio

1
When should we worry about AI power-seeking? 46:54

9 weeks之前46:54

46:54

Examining the conditions required for rogue AI behavior. Text version here: https://joecarlsmith.substack.com/p/when-should-we-worry-about-ai-power

Joe Carlsmith Audio

1
What is it to solve the alignment problem? 40:13

10 weeks之前40:13

40:13

Also: to avoid it? Handle it? Solve it forever? Solve it completely? Text version here: https://joecarlsmith.substack.com/p/what-is-it-to-solve-the-alignment

Joe Carlsmith Audio

1
How do we solve the alignment problem? 8:43

10 weeks之前8:43

8:43

Introduction to a series of essays about paths to safe and useful superintelligence. Text version here: https://joecarlsmith.substack.com/p/how-do-we-solve-the-alignment-problem

Joe Carlsmith Audio

1
Fake thinking and real thinking 1:18:47

12 weeks之前1:18:47

1:18:47

When the line pulls at your hand. Text version here: https://joecarlsmith.com/2025/01/28/fake-thinking-and-real-thinking/.

Joe Carlsmith Audio

1
Takes on "Alignment Faking in Large Language Models" 1:27:54

18 weeks之前1:27:54

1:27:54

What can we learn from recent empirical demonstrations of scheming in frontier models? Text version here: https://joecarlsmith.com/2024/12/18/takes-on-alignment-faking-in-large-language-models/

Joe Carlsmith Audio

1
(Part 2, AI takeover) Extended audio from my conversation with Dwarkesh Patel 2:07:33

29 weeks之前2:07:33

2:07:33

Extended audio from my conversation with Dwarkesh Patel. This part focuses on the basic story about AI takeover. Transcript available on my website here: https://joecarlsmith.com/2024/09/30/part-2-ai-takeover-extended-audio-transcript-from-my-conversation-with-dwarkesh-patel

Joe Carlsmith Audio

1
(Part 1, Otherness) Extended audio from my conversation with Dwarkesh Patel 3:58:38

29 weeks之前3:58:38

3:58:38

Extended audio from my conversation with Dwarkesh Patel. This part focuses on my series "Otherness and control in the age of AGI." Transcript available on my website here: https://joecarlsmith.com/2024/09/30/part-1-otherness-extended-audio-transcript-from-my-conversation-with-dwarkesh-patel/

Joe Carlsmith Audio

1
Introduction and summary for "Otherness and control in the age of AGI" 12:23

44 weeks之前12:23

12:23

This is the introduction and summary for my series "Otherness and control in the age of AGI." Text version here: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi

Joe Carlsmith Audio

1
Second half of full audio for "Otherness and control in the age of AGI" 4:11:02

44 weeks之前4:11:02

4:11:02

Second half of the full audio for my series on how agents with different values should relate to one another, and on the ethics of seeking and sharing power. First half here: https://joecarlsmithaudio.buzzsprout.com/2034731/15266490-first-half-of-full-audio-for-otherness-and-control-in-the-age-of-agi PDF of the full series here: https://jc.gatspress.com/pdf/otherness_full.pdf Summary of the series here: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi…

Joe Carlsmith Audio

1
First half of full audio for "Otherness and control in the age of AGI" 3:07:29

44 weeks之前3:07:29

3:07:29

First half of the full audio for my series on how agents with different values should relate to one another, and on the ethics of seeking and sharing power. Second half here: https://joecarlsmithaudio.buzzsprout.com/2034731/15272132-second-half-of-full-audio-for-otherness-and-control-in-the-age-of-agi PDF of the full series here: https://jc.gatspress.com/pdf/otherness_full.pdf Summary of the series here: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi…

Joe Carlsmith Audio

1
Loving a world you don't trust 1:03:54

44 weeks之前1:03:54

1:03:54

Garden, campfire, healing water. Text version here: https://joecarlsmith.com/2024/06/18/loving-a-world-you-dont-trust This essay is part of a series I'm calling "Otherness and control in the age of AGI." I'm hoping that individual essays can be read fairly well on their own, but see here for brief text summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi…

Joe Carlsmith Audio

1
On attunement 44:14

1 year之前44:14

44:14

Examining a certain kind of meaning-laden receptivity to the world. Text version here: https://joecarlsmith.com/2024/03/25/on-attunement This essay is part of a series I'm calling "Otherness and control in the age of AGI." I'm hoping that individual essays can be read fairly well on their own, but see here for brief text summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi (Though: note that I haven't put the summary post on the podcast yet.)…

Joe Carlsmith Audio

1
On green 1:15:13

1 year之前1:15:13

1:15:13

Examining a philosophical vibe that I think contrasts in interesting ways with "deep atheism." Text version here: https://joecarlsmith.com/2024/03/21/on-green This essay is part of a series I'm calling "Otherness and control in the age of AGI." I'm hoping that individual essays can be read fairly well on their own, but see here for brief text summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi (Though: note that I haven't put the summary post on the podcast yet.)…

Joe Carlsmith Audio

1
On the abolition of man 1:09:22

1 year之前1:09:22

1:09:22

What does it take to avoid tyranny towards to the future? Text version here: https://joecarlsmith.com/2024/01/18/on-the-abolition-of-man This essay is part of a series I'm calling "Otherness and control in the age of AGI." I'm hoping that individual essays can be read fairly well on their own, but see here for brief text summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi (Though: note that I haven't put the summary post on the podcast yet.)…

Joe Carlsmith Audio

1
Being nicer than Clippy 47:30

1 year之前47:30

47:30

Let's be the sort of species that aliens wouldn't fear the way we fear paperclippers. Text version here: https://joecarlsmith.com/2024/01/16/being-nicer-than-clippy/ This essay is part of a series I'm calling "Otherness and control in the age of AGI." I'm hoping that individual essays can be read fairly well on their own, but see here for brief text summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi (Though: note that I haven't put the summary post on the podcast yet.)…

Joe Carlsmith Audio

1
An even deeper atheism 25:12

1 year之前25:12

25:12

Who isn't a paperclipper? Text version here: https://joecarlsmith.com/2024/01/11/an-even-deeper-atheism This essay is part of a series I'm calling "Otherness and control in the age of AGI." I'm hoping that individual essays can be read fairly well on their own, but see here for brief summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi…

Joe Carlsmith Audio

1
Does AI risk "other" the AIs? 13:15

1 year之前13:15

13:15

Examining Robin Hanson's critique of the AI risk discourse. Text version here: https://joecarlsmith.com/2024/01/09/does-ai-risk-other-the-ais This essay is part of a series of essays called "Otherness and control in the age of AGI." I'm hoping the individual essays can be read fairly well on their own, but see here for brief summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi…

Joe Carlsmith Audio

1
When "yang" goes wrong 21:32

1 year之前21:32

21:32

On the connection between deep atheism and seeking control. Text version here: https://joecarlsmith.com/2024/01/08/when-yang-goes-wrong This essay is part of a series of essays called "Otherness and control in the age of AGI." I'm hoping the individual essays can be read fairly well on their own, but see here for brief summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi…

Joe Carlsmith Audio

1
Deep atheism and AI risk 46:59

1 year之前46:59

46:59

On a certain kind of fundamental mistrust towards Nature. Text version here: https://joecarlsmith.com/2024/01/04/deep-atheism-and-ai-risk This is the second essay in my series “Otherness and control in the age of AGI. I’m hoping that the individual essays can be read fairly well on their own, but see here for brief summaries of the essays released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi…

Joe Carlsmith Audio

1
Gentleness and the artificial Other 22:39

1 year之前22:39

22:39

AIs as fellow creatures. And on getting eaten. Link: https://joecarlsmith.com/2024/01/02/gentleness-and-the-artificial-other This is the first essay in a series of essays that I’m calling “Otherness and control in the age of AGI.” See here for more about the series as a whole: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi.…

Joe Carlsmith Audio

1
In search of benevolence (or: what should you get Clippy for Christmas?) 52:52

1 year之前52:52

52:52

What is altruism towards a paperclipper? Can you paint with all the colors of the wind at once? (This is a recording of an essay originally published in 2021. Text here: https://joecarlsmith.com/2021/07/19/in-search-of-benevolence-or-what-should-you-get-clippy-for-christmas)

Joe Carlsmith Audio

1
Empirical work that might shed light on scheming (Section 6 of "Scheming AIs") 28:00

1 year之前28:00

28:00

This is section 6 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?” Text of the report here: https://arxiv.org/abs/2311.08379 Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power…

Joe Carlsmith Audio

1
Summing up "Scheming AIs" (Section 5) 15:46

1 year之前15:46

15:46

This is section 5 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?” Text of the report here: https://arxiv.org/abs/2311.08379 Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power…

Joe Carlsmith Audio

1
Speed arguments against scheming (Section 4.4-4.7 of "Scheming AIs") 15:19

1 year之前15:19

15:19

This is section 4.4 through 4.7 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?” Text of the report here: https://arxiv.org/abs/2311.08379 Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power…

Joe Carlsmith Audio

1
Simplicity arguments for scheming (Section 4.3 of "Scheming AIs") 19:37

1 year之前19:37

19:37

This is section 4.3 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?” Text of the report here: https://arxiv.org/abs/2311.08379 Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power…

Joe Carlsmith Audio

1
The counting argument for scheming (Sections 4.1 and 4.2 of "Scheming AIs") 10:40

1 year之前10:40

10:40

This is sections 4.1 and 4.2 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?” Text of the report here: https://arxiv.org/abs/2311.08379 Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power…

Joe Carlsmith Audio

1
Arguments for/against scheming that focus on the path SGD takes (Section 3 of "Scheming AIs") 29:03

1 year之前29:03

29:03

This is section 3 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?” Text of the report here: https://arxiv.org/abs/2311.08379 Summary of the report here: https://joecarlsmith.com/2023/11/15/new-report-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power Audio summary here: https://joecarlsmithaudio.buzzsprout.com/2034731/13969977-introduction-and-summary-of-scheming-ais-will-ais-fake-alignment-during-training-in-order-to-get-power…

Joe Carlsmith Audio

1
Non-classic stories about scheming (Section 2.3.2 of "Scheming AIs") 24:34

1 year之前24:34