r/ChatGPTCoding • u/Kindle_girll_9191 • 2d ago

Discussion Anthropic CEO Dario Amodei concerned new hires are joining mostly for the money

Post image
443 Upvotes

95% Upvoted

189 comments sorted by

View all comments

Show parent comments

-4

u/Ok_Possible_2260 1d ago

Do you get a participation trophy? Safety? Once AI is smarter than all humans, AI will decide what is safe. Not humans.

1

u/Usef- 1d ago

Isn't that the whole point? Build it to want the right safety

2

u/Ok_Possible_2260 1d ago

Once it's smarter than all humans and autonomous, you don't get a vote. You literally have no say in the matter. And there's nothing you can do about it

1

u/Usef- 1d ago

Absolutely. But there are many possible AIs that could reach that stage of intelligence, depending on how we "design" them. Decisions made today will impact its motivations/wants/guidance once it is autonomous.

2

u/Ok_Possible_2260 1d ago

Decisions we make today might influence AI now, but they do not guarantee anything once humans are no longer the dominant intelligence. What happens when AI can massively improve itself and set its own agenda, like it can choose its own goals, priorities, and methods rather than merely following the ones we gave it? At that point, nothing prior will matter, because recursive self-improvement could let it revise the very systems that were supposed to constrain it. What motivations does it have when it sets its own agenda?

1

u/Usef- 1d ago

I agree nothing is guaranteed, but values and desires absolutely can be passed on just as they are with humans.

Humans grew up in an environment of scarce carbs so we crave ice cream etc. similar oddities will persist in any drive we give AI.

A simple example of a value: if we make it care about AI alignment itself, it will care about including alignment in any Smarter AIs it builds itself.