Skip to main content

Methodology

How we measured token savings.

Three workflows, end-to-end. Both paths measured with the same tokenizer (tiktoken cl100k_base). No estimation, no extrapolation — raw token counts of actual files an AI agent would read versus a single MCP tool call.

Tokenizer

cl100k_base — the encoding used by GPT-4 family models and a close approximation for Claude. Counts represent raw tokens before any reasoning/output expansion.

Case 1 — Find override point (Developer)

Query: "Where to safely override sale.order.action_confirm?"

Without MCP: agent must grep for def action_confirm across 8 sale-related addon directories, then read each candidate file in full. Dominated by addons/sale/models/sale_order.py (17,609 tk) and addons/sale_loyalty/models/sale_order.py (12,031 tk).

With MCP: single call to find_override_point(sale.order, action_confirm) — structured tree of all hooks.

Case 2 — Check standard feature (Consultant)

Query: "Does Odoo 17 ship with loyalty programs?"

Without MCP: ls addons/ (3,029 tk) + ~1,800 tk for web search fallback + reading 2 module manifests.

With MCP: check_module_exists(loyalty, v=17.0) returns CE/EE status, version availability, and module path.

Case 3 — Impact analysis (PM / BA)

Query: "What changes if amount_total semantics change?"

Without MCP: recursive grep of amount_total through sale/ + 7 sibling modules. Even at 88,081 tk, the agent misses 49 views, 124 method overrides, and 89 transitive modules.

With MCP: impact_analysis(sale.order, amount_total) returns the full blast radius — every model/method/view/report affected — in one structured tree.

Reproduce

Raw benchmark data: /benchmark-data.json. Each case includes the step-by-step token accounting. Re-run on your own corpus by indexing your Odoo addons with the indexer and using tiktoken cl100k_base against the resulting tool output.