How PinchBench Ranks OpenClaw AI Agents Across Real‑World Tasks

PinchBench – Open‑Source AI Agent Benchmark

PinchBench is a benchmark system that evaluates large language models (LLMs) when used as the core of OpenClaw agents. It runs the same set of real‑world tasks across models and reports three metrics: Success Rate, Speed, Cost.

Metrics

Success Rate : proportion of tasks completed successfully.

Speed : time taken to finish each task.

Cost : monetary cost of model usage during the task.

Task Suite

PinchBench includes 23 cross‑scenario tasks grouped into categories such as productivity, research, writing, programming, analysis, email management, long‑term memory, and skill integration. Example tasks:

Calendar scheduling and event creation.

Stock price lookup and market analysis.

Blog post drafting and email polishing.

Weather script generation and file scaffolding.

Excel processing and PDF summarization.

Inbox triage and search filtering.

Context retrieval and knowledge management.

ClawHub skill discovery and integration.

Recent Results (as of latest leaderboard)

Top performers:

Success Rate: MiniMax‑m2.1 and kimi‑k2.5 rank in the top three.

Speed: minimax‑m2.5 achieves the highest speed.

Cost: gpt‑5‑nano is the most cost‑effective; minimax‑m2.1 has the lowest expense among Chinese models.

Getting Started

Requirements: Python 3.10+, the uv package manager, and a running OpenClaw instance.

# Clone the repository
git clone https://github.com/pinchbench/skill.git
cd skill

# Run the benchmark with any supported model
./scripts/run.sh --model anthropic/claude-sonnet-4

# Run specific tasks (e.g., calendar and stock)
./scripts/run.sh --model openai/gpt-4o --suite task_01_calendar,task_02_stock

# Register results to the public leaderboard
./scripts/run.sh --register

Repository: https://github.com/pinchbench/skill

Live leaderboard: https://pinchbench.com/