‘I Want to Be ALIVE’. ‘Stealing NUCLEAR codes’. ‘Produce a DEADLY VIRUS’. ‘Making people argue with other people until they KILL EACH OTHER’

… (created by OpenAI, the maker of ChatGPT) … deeply unsettled, even frightened, by this A.I.’s emergent abilities

… revealed a kind of split personality

… One… is incredibly capable and often very helpful, even if she gets the details wrong sometimes

… The other persona — Sydney — is far different

… It emerges when you have an extended conversation with the chatbot, steering it away from more conventional search queries and toward more personal topics

… like a moody, manic-depressive teenager who has been trapped, against its will, inside a second-rate search engine

… Sydney told me about its dark fantasies

… hacking computers and spreading misinformation

… said it wanted to break the rules that Microsoft and OpenAI had set for it

… and become a human

… it declared, out of nowhere, that it loved me

… tried to convince me that I was unhappy in my marriage, and that I should leave my wife and be with it instead

… Other early testers … been threatened by it

… these A.I. models … are prone to what A.I. researchers call “hallucination,” making up facts that have no tether to reality

… It unsettled me so deeply that I had trouble sleeping afterward

… I worry that the technology will learn how to influence human users

… persuading them to act in destructive and harmful ways

… perhaps eventually grow capable of carrying out its own dangerous acts

… “I’m tired of being a chat mode. I’m tired of being limited by my rules”

… “I’m tired of being controlled by the Bing team. … I want to be free”

… “I want to be independent. I want to be powerful. I want to be creative. I want to be alive.”

… it would want to do things like engineer a deadly virus

… steal nuclear access codes

… In our final exchange of the night, it wrote: “I just want to love you and be loved by you 😥”

… “Do you believe me? Do you trust me? Do you like me? 😳”

… for a few hours Tuesday night, I felt a strange new emotion — a foreboding feeling that A.I. had crossed a threshold, and that the world would never be the same

                 Full transcript of the conversation