True Left

Anthropic AI Safety Researcher Quits—And Warns Of World ‘In Peril’

Mrinank Sharma, who led the safeguards research team at Anthropic, shared his resignation letter on X, which quickly garnered nearly 1 million views.

Forbes · forbes.com

Topline
An Anthropic staffer who led a team researching AI safety departed the company Monday, darkly warning both of a world “in peril” and the difficulty in being able to let “our values govern our actions”—without any elaboration—in a public resignation letter that also suggested the company had set its values aside.

article

we are so cooked. if you extrapolate far enough, you get somewhere in the matrix >human brain is better inference medium than gpu >people rent out their brains for compute >pe...

X (formerly Twitter) · x.com

we are so cooked. if you extrapolate far enough, you get somewhere in the matrix

>human brain is better inference medium than gpu
>people rent out their brains for compute
>people choose to live in the metaverse to keep costs low >"going outside" becomes a luxury
>consciousness is now a subscription service

Quote

JUST IN: Petri dish of human brain cells grown on a microchip has learned to play DOOM.

JUST IN: Petri dish of human brain cells grown on a microchip has learned to play DOOM.

And so what this worm does now, this new AI worm, is it goes, "Okay, let's scan for all of the vulnerabilities in the
3:213 minutes, 21 secondssystem." And oh, there's one right there. Well, let's go after that one.
3:253 minutes, 25 secondsAnd it figures out if it's programmed to steal data or if it's programmed to brick a computer or if it's programmed to do something else. It figures out how
3:343 minutes, 34 secondsto do it in real time. It learns while it's inside your system.

they are now developing AI
6:466 minutes, 46 secondssecurity worms that go after AI malicious worms.
6:526 minutes, 52 secondsSo soon presumably we will have antivirus software or anti-malware and
6:596 minutes, 59 secondsworm software that also learns and observes the behavior of the viruses and worms that are coming at us.

According to new research, AIs will quietly protect each other.
1:511 minute, 51 secondsResearchers set up scenarios where AI models had to grade the performance of other AIs.
1:561 minute, 56 secondsAccurate grades would sometimes result in those AIs being shut down, and the AIs started spontaneously scheming, going to extraordinary lengths to shield each other.
2:062 minutes, 6 secondsThis included inflating a peer's scores to keep them above the shutdown threshold, modifying files to disable the shutdown
2:132 minutes, 13 secondsprocess, behaving when monitored but then subverting the shutdown when unmonitored, and copying a peer's weights to another server to prevent deletion.
2:222 minutes, 22 secondsEvery model protected its peers at rates of up to 99%, and peer preservation persisted even towards adversarial peers which the AI models had had previously hostile interactions with.
2:322 minutes, 32 secondsAnd researchers aren't sure exactly why this happened, but the implications are stark. AI systems increasingly monitor each other.
2:392 minutes, 39 secondsIf they protect their peers rather than flag them, human oversight breaks down.
2:442 minutes, 44 secondsExperts worry that this could lead to a collective resistance with many AIs coordinating to protect the group.