I’m not sure this question can be resolved. After all, how do you explain to an AI what our goals are, even with a computational language. How are you going to be sure that the AI, trying to maximize happiness, isn’t going to be doing ridiculous things? Say you train the AI to maximize happiness, so the AI is chasing people on the street with drones having an injection of dopamine so people experience non stop pleasure and don’t do anything else? Or maybe the AI, trying never to hurt anybody, is not saving people because that would mean to inject something in the skin and that causes pain and the AI avoids human pain at all costs so the AI lets people die when it could have saved them?

There’s seemingly an infinite number of potentially ridiculous situations that we won’t take into account simply because we use common sense to solve them, but the AI doesn’t have any common sense. Maybe we could instill some common sense by training it correctly, I don’t know, but I am doubtful.

In this module, we learned about the Principle of Computational Irreducibility and its relationship to scientific processes. For example, we examined the pigmentation patterns of mollusk shells (note image below) which, like a one-dimensional cellular automata, grow one line at a time. What other “simple” patterns do you come across that exhibit “complex” behavior? On what basis do you determine whether it is simple or complex?