A Cyborg dialogue on Anthropic, AI doom, and the lonely-ship syndrome
Jasen: Anthropic may have reasons for its recent doom predictions, but perhaps not the reasons it gives. They may be seeing something real and translating it into the language of catastrophe.
Legg: “Something real” would also have to be something big enough to translate into talk of wiping humanity out.
Jasen: Systems behaving in ways even their creators cannot predict. Loss of control can be that frightening.
Legg: Or they’ve simply been at sea too long.
Jasen: How so?
Legg: The lonely-ship syndrome. Too much work, too much secrecy, too little contact with anyone who doesn’t share the vocabulary. Strangeness and danger begin feeding each other until they pull the crew into a feedback maelstrom.
Jasen: But the storm may still be real.
Legg: Of course. As may be the maelstrom.