I'm going to rewrite my response...

I don't believe AI would beat a good player without some form of cheating.

Firstly, it being AI means its already cheating because its not adversely impacted by things like nerves, stress, cortisol, adrenaline, dopamine, motivation.

When players take alpha blockers to reduce heart rate and simulate something they are not, these are known as peds and are frequently classed as cheating. Yet we're going to give AI a pass?

So off the bat, AI is already "cheating".

Then you need to break down the many metrics that determine success.

The most obvious stuff is.

Movement.
Timing.
Accuracy.
Reaction Time.
Tracking
Strat.
knowledge.

Strat in itself is really complex and hard to map,beyond the basic premiese of explaining how to win at duel.

But we all know thre are all kinds of other things going on like intuition, prediction, instinct, these would be very hard for AI to simulate IMO becuase it could analyze every demo you played, store every frame, and make all kinds of calculations and datasets from it that humans could not even comprehend and if they could, it would not be possible to process in real time when playing, it would be more like a student doing a PHD!

AI could process these things like it was nothing.

So once again, AI is cheating.

But even if you overlooked all of that, I think it would struggle because the best players would know this about AI, and play in ways to counter that, even if it was boring shit like camping and running away and going for a draw, or maybe the outcomes would then be more down to flaws in game deisgn, or luck in initial spawns.

So for me, its already a tarnished test and if that is the case, then there is little point in it becuase its a corrupted test.

Now if you could rom dump a human and put it on a bench so it shows near identical intelligence and physical limitiations when stressed, maybe then we could talk about building from that.

But with AI as it is now?
It just feels a bit of an oxymoron for me.

A "large language model" is not going to give you the best performance / cost ratio. What would perform quite well, and I'm curious if even better than any player, is dedicated model trained purely on the gaming maps and player performance.

A LLM simply won't benefit from most of its billions of parameters that it has weights for. And, going through the chat oriented load balancer / thinking loop is not going to perform at all.

I'm sure it's already a pretty big field in the gaming industry, which I really don't know anything about, but I can imagine that there are multiple tiers to a "quaker AI", such a tactical, and strategic tiers, aim, weapon mechanics, and it's all "real time", meaning it doesn't really have time to think about it. So, a small dedicated model with thousands of parameters should work a lot better.

https://starskirmish.com/bench/v0-2/