Surely the problem with re-enforcement learning, is that at some point, instead of learning the problem being taught, the AI will go meta and start learning the re-enforcement learning, because that's the most obvious re-occurring pattern. Then it's just two entities teaching each other to teach while pretending to be doing something else. And presumably the AI will eventually win that war since it can do much cleverer, and devious re-enforcements than the human. Then civilisational collapse ensues. #