Thousands tune into a livestream of Claude 3.7 Sonnet navigating the game on its own better than its predecessor. Anthropic tells PCMag it's a more effective way of measuring AI progress.

A new Twitch channel called ClaudePlaysPokemon uses Claude's LLM capabilities to see if the AI can become a true Pokemon Master.

AI researchers have used all sorts of video games, from Street Fighter to Pictionary, to test new models — often more for amusement than utility. But Anthropic said that Pokémon proved to be a useful benchmark for Claude 3.7 Sonnet, which can effectively “think” through the sorts of puzzles the game contains.

Anthropic’s AI Claude 3.7 Sonnet played Pokémon Red on Twitch, showcasing its decision-making. Could AI agents play games for humans someday?