Estimating the tokens (cost) of household LLM queries

[MaxGhenis]

We need to estimate the tokens of household LLM queries for understanding our costs of this service.

@anth-volk shared this sample. request for the household AI explanation, in this case for SNAP. This includes three levels of the computation tree.

I ran this through https://platform.openai.com/tokenizer, which counted 7,801 tokens.

I then ran this through various LLMs and again through the OpenAI tokenizer to count output tokens:

• Claude 3.5 Sonnet: 265
• GPT-4o: 564 tokens
• Deepseek R1: 414
• Gemini 2.0 Advanced (1.5 provided an unacceptably poor response): 974

To be conservative, let's price a prompt with 10,000 input tokens and 1,000 output tokens. Based on https://docsbot.ai/tools/gpt-openai-api-pricing-calculator (which looks accurate from my checks), here's how it'd cost by model in cents:

• Claude 3.5 Sonnet: 4.2
• GPT-4o: 3.3
• Deepseek R1: 0.73
• Gemini 1.5 Pro (2.0 Pro not available by API): 1.65

[richardchen874-sys]

I would estimate this at the “household query” level rather than only counting prompt tokens. If a query triggers retrieval, validation, or multiple model calls, the real cost can be very different from the first completion. I am testing an OpenAI-compatible multi-model API layer focused on official Chinese models, and policy/question-answering workloads are a good case for cheaper routing plus final synthesis.