The holy grail, I think, for artificial intelligence is true creativity. The only things that we know, for a fact, are truly creative on this planet are humans. And so it's worth asking where human creativity comes from. Broadly speaking, there are two views. One is, there's an oft-repeated saying, "Good artists borrow, but great artists steal." That expresses an idea that there really isn't any creativity, that people are just reinterpreting what other people have done ad-infinitum. I mean, it makes for a glib saying, but I don't, intuitively, think that it's true.
Could a computer ever write and compose Hamilton? You know, a Broadway show which, in a dozen ways, was really unprecedented? Would it really come up with the concept and the implementation and all of the idiosyncrasies or something like that? Is Hamilton just some kind of a synthesis of all the other Broadway shows that came before it? And could a machine do it? It doesn't feel satisfying to say that that's all it is. But it's equally unsatisfying to say that human creativity comes about in a way that we don't understand. We can't even really boil it down to a specific system that it comes from. It doesn't actually feel like a scientific process. It doesn't feel mechanistic, and yet if our brains are mechanisms then creativity must be mechanistic as well. It's a great mystery in the end, and one that I think we'll need to understand better before we can hope to make a creative machine.