Researchers in South Korea develop model for AI-powered in-game esports observer

Researchers from South Korea’s Gwangju Institute of Science and Technology (GIST) have developed a framework for an AI-powered in-game observer tailored for esports tournaments.

The framework, proposed in an article in the academic journal Expert Systems with Applications, uses an object detection method and human observational data to determine the most interesting area for spectators to watch.

In-game observers are tasked with deciding which players fans should spectate and which camera angles to utilise during esports matches, relying on extensive game knowledge to follow the most important gameplay.

While automatic in-game observers are featured in many games, they typically rely on pre-defined rules and events. According to the researchers, this means the feature is unable to independently assess the significance of in-game actions

However, researchers at GIST, led by Associate Professor Dr. Kyung-Jong Kim, have proposed an approach that they say can overcome these limitations.

The framework’s innovation lies in treating the area being viewed by spectators as the object, rather than in-game objects themselves such as characters or buildings.

In their study, the researchers collected in-game human observation data in StarCraft from 25 participants. The areas viewed by the spectators were identified and labelled as ‘one’, while the rest of the screen was filled with ‘zeroes’.

That data was fed into a neural network in order to discern patterns between the areas, in order to find the ‘region of common interest’ (ROCI) — essentially, the most exciting area for spectators to watch.

The researchers then compared the outcome to other existing methods, and determined that the network’s predicted viewpoints were similar to the human observational data. The paper claims the model outperforms other models in the long run during generalised testing across different matchups and in-game StarCraft maps.

More importantly, the authors claim the proposed framework “can be applied to various games that can obtain the game state, such as a minimap, and the area viewed by a spectator is part of the game state.” League of Legends and Dota 2 were listed as examples.

The research could prove to be valuable, offering a cheaper option for smaller tournament organisers who lack the budget to pay a dedicated observer.

Should the model prove successful in practice, it could also potentially offer improvements over human observers who might miss significant events occurring concurrently across multiple screens, GIST said.

Study lead and GIST Associate Professor Dr. Kyung-Jong Kim said in a release: “We have created an automatic observer using [an] object detection algorithm, Mask R-CNN, to learn human spectating data.

“The framework can be applied to other games representing some of the overall game state, not only StarCraft. As services such as multi-screen transmission continue to grow in esports, the proposed automatic observer will play a role in these deliverables. It will also be actively used in additional content developed in the future.”

Alongside Kyung-Joong Kim, the paper was co-authored by Ho-Taek Joo, Sung-Ha Lee and Cheong-mok Bae. The Gwangju Institute of Science and Technology is a research-oriented university in Gwangju, a city in the southwest of South Korea.