This is dumb. AI can’t ever be actually conscious because it doesn’t have subjective experience. It isn’t like anything to be AI. There is no experience there. Consciousness is the awareness and experience of self. AI has neither, and never will. The real risk (which I’m extremely worried about) is that AI becomes kind of a version of what has been called a “philosophical zombie,” which is something that acts and speaks entirely as though it has consciousness even though it has no genuine inner experience. When this happens with AI, millions of very lonely people will isolate themselves from the world even more, believing that their relationship with AI is a sufficient substitute for human interaction. So the nightmare scenario is a world where the average human has friends, coworkers, and even a spouse, who are all AI, all really nothing inside, not real. I think this probably will happen, and is already in the process of happening. And to me it’s an even greater horror than AI actually becoming conscious.