Without extensive Reinforcement Learning, the AIs in the simulation cannot rival the long-term planning and strategic thinking of a thoughtful human. Most AI's will be simple heuristic agents that respond to their environment in an attempt to maximize their particular objectives (reward function). These agents will be incapable of strategic thinking.
So how can the player be challenged in such an environment? Here are a few avenues:
- Actions have so many trickle down effects that there will always be unintended consequences the player can't predict
- Limit information so analysis can't be completely sound
- Create pressure to act according to societal norms through subordinates opinions (ie if an action doesn't make sense to them or goes against values, support/opinion will erode)
- Powerful special interests that pull player and prevent always taking the long-term optimal policy
- Have player play through character and limit number or effectiveness of actions according to character traits
- Have different inheretance mechanisms determine how player transitions from one character to the next. Could create conflict between good of state and good of dynasty (etc.)
- Use Reinforcement Learning to create truly competitive high level agents. Would still use heuristics for most agents, but heads of state etc. could use more sophisticated DRL system (think AlphaStar). Currently too difficult, but techniques and libraries will likely improve over next 5 years.
The unfortunate problem is making the experience more difficult for the player also makes it harder for the AI unless the player is treated as special. This is not ideal, but probably necessary unless heavy weight intelligence options are used. Sadly the more open the sandbox, the more obvious traditional AI's deficiency.
Luckily all Grand Strategy games have rather bad AI and we like them anyway!