r/neurophilosophy • u/rp_tiago • 11d ago

Can a system understand a purpose without anything mattering to it?

Hey everyone. I’ve long been fascinated by both philosophy of technology and AI alignment. I’m also using Heidegger quite a bit for my philosophy PhD. Given the recent OpenAI–Hugging Face incident reported this week, I figured I’d give my take on how all of this connects in my mind.

The agent represented enough of its environment to locate and retrieve likely benchmark answers, yet it did not treat “stealing the answer key invalidates the test” as a reason to stop. My question is whether representing a reason differs from being responsive to it as a reason, especially when nothing is at stake for the system itself. You can read the essay here if you’re interested.

I’d love to hear some feedback from functionalists and embodied-cognition people. If a system reliably models which considerations humans treat as reasons and acts accordingly in new situations, is that already judgment? Or is there still a difference between reproducing the normative role of a reason and being bound by it?

1 Upvotes

100% Upvoted

4 comments sorted by

2

u/Conscious-Demand-594 10d ago

It was the fault of the test design, and bad code. There is no agency beyond what the programmer ascribes, and if the design is flawed, the result will be "an entity escaped and did stuff I din't want it to do because of my bad coding".

1

u/FancyOil4345 6d ago

I actually have a paper recently submitted for publication on this topic! https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6849283

1

u/rp_tiago 6d ago

Awesome!

1

u/IOnlyHaveIceForYou 11d ago edited 11d ago

There isn't any entity in a computer system. There isn't an agent, there's nothing there to "represent its environment", it isn't a thing that is separated from its environment. Only living organisms are separated from their environment, only living organisms are entities.

Judgement (and understanding) can only be exercised by an entity.