The limitation in doing anything interesting with this type of AI is always going to be that the simulation has limits on accuracy and fidelity, and you reach the limits of the simulation before you get anything interesting, and the AI ends up exploiting your simulation in a non-generalizable way. See also, much earlier in the current AI hype cycle, when OpenAI beat human pros in DotA, but if you actually look into it, they were mostly winning on the basis of exploiting APM more than anything else.
The limitation in doing anything interesting with this type of AI is always going to be that the simulation has limits on accuracy and fidelity, and you reach the limits of the simulation before you get anything interesting, and the AI ends up exploiting your simulation in a non-generalizable way. See also, much earlier in the current AI hype cycle, when OpenAI beat human pros in DotA, but if you actually look into it, they were mostly winning on the basis of exploiting APM more than anything else.