Anthropic has released Claude 3.7 Sonnet, featuring a significant new capability: extended thinking mode. This update, described as Anthropic’s most intelligent model to date, allows users to toggle between standard responses and deeper cognitive processing for more complex questions. Developers can also set specific “thinking budgets” of up to 128K tokens to control precisely how long Claude spends analyzing problems.
Performance Improvements and Capabilities
The data shows Claude 3.7 Sonnet’s extended thinking provides measurable performance gains across multiple domains:
Testing reveals that Claude 3.7 Sonnet’s mathematical reasoning improves logarithmically with additional “thinking tokens.” Performance on the 2024 American Invitational Mathematics Examination demonstrates consistent improvement as the thinking capacity increases, allowing the model to work through complex problems with greater accuracy.
Claude 3.7 Sonnet demonstrates enhanced “action scaling” – the ability to iteratively call functions and respond to environmental changes until completing open-ended tasks. Performance on the OSWorld evaluation, which measures multimodal computer use skills, shows steady improvement over longer interaction periods compared to previous models. This capability allows Claude to maintain focus and accomplish objectives that require sustained interaction.
In complex task performance testing, Claude 3.7 Sonnet showed remarkable advancement. During Pokémon Red gameplay testing, previous Claude models failed at early stages, while Claude 3.7 Sonnet successfully defeated three Gym Leaders across tens of thousands of sustained interactions. The model demonstrated effective strategy refinement and hypothesis testing throughout extended gameplay, showcasing its ability to maintain context and improve its approach over time.
Scientific knowledge benchmarking revealed substantial improvements as well. Using parallel test-time compute with 256 independent samples and a 64k-token thinking budget, Claude 3.7 Sonnet achieved an 84.8% overall score on the GPQA evaluation, which features challenging questions in biology, chemistry, and physics. Most impressively, it scored 96.5% specifically on physics questions, demonstrating its ability to reason through complex scientific problems. The model further excels on SWE-bench Verified, which tests real-world software issue resolution, and TAU-bench, which evaluates AI agents on complex tasks. In graduate-level reasoning tasks, Claude 3.7 achieved 78.2% accuracy in extended thinking mode, outperforming several competitor models.
Safety and security remain priorities in this release. Comprehensive evaluation confirmed that current ASL-2 safety standards remain appropriate for the model. Defenses against prompt injection attacks during computer use have improved from 74% to 88% prevention rate with only a 0.5% false-positive rate. The model demonstrates a 45% reduction in unnecessary refusals compared to its predecessor, striking a better balance between safety and usefulness. Additionally, potentially harmful content in thought processes is encrypted for user safety, allowing the model to process information internally without exposing potentially problematic content.
The new model also demonstrates remarkable improvements in specific business applications. For Slack and Salesforce, it delivers 30% better summarization and 24% enhanced information retrieval. Notion users benefit from chat accuracy improving from 56% to 70%, while iGent AI reports 7x success on complex tasks and 2.5x fewer context errors. These metrics translate directly to improved workflow efficiency for enterprise users.
A New Paradigm for AI Assistance
Claude 3.7 Sonnet represents a significant advancement in AI reasoning capabilities by making the thinking process both more extensive and more transparent. The visible thought process allows users to understand and verify Claude’s reasoning, while the extended computational resources enable it to tackle more complex, open-ended tasks. Beyond gaming demonstrations, these improvements position Claude 3.7 Sonnet as a powerful foundation for developers building sophisticated AI agents for real-world applications.
