StarSkirmish Bench

StarSkirmish evaluates how strong of a StarCraft: Brood War bot each LLM can write given one hour of wall clock time. Models write Protoss bots in C++ against BWAPI 4.4.0, which are then evaluated against a roster of competitive human-written bots and demo bots to test long horizon reasoning and agentic coding. Version 0.2 updates the benchmark with flagship models from OpenAI, Anthropic, Google, and Meta, running in their respective first-party CLI harnesses at maximum reasoning settings. An LLM's score reflects its average win rate across multiple runs against the fixed roster of human-written and demo bots.