Under the hood
How Auto picks a model
Every message you send on Auto goes to one of 13 models from 5 companies. This page explains exactly how that choice is made, which models win what and why, how we test it, and where it still gets things wrong. Every table and score below is computed from the same code that routes your messages.
13
models Auto chooses from
221
labelled test prompts
78.3%
field right on prompts it was never tuned on
0
times an easy task was judged hard, or a hard one easy
01The short version
No single model is best at everything, and the best model is rarely needed. A greeting doesn’t need a frontier model; a proof does. Auto does three things for every message, in a few milliseconds and without calling any AI:
- Reads the message: which field it belongs to (code, math, law, creative writing and 10 others), how hard it is, and whether it leans on creativity, reasoning or explanation.
- Chooses a tier: quick models for easy messages, everyday models for normal ones, top models for hard ones.
- Ranks the models in that tier by how good each is at that field and those needs, against what it costs, and sends the message to the winner. The rest become fallbacks if it fails.
You can always skip Auto and pick a model yourself. Every reply shows which model answered and how many tokens it used.
02Step 1: reading the message
The classifier is a set of rules, not a model, so it adds no cost and no delay. It produces three things.
Field. One of 14: General, Writing, Creative, Code, Computer science, Math, Science, Biology, Engineering, Humanities, Research, Law, Finance, Tutoring. Each field has signal phrases with weights: a decisive term like “derivative” or “haiku” counts 3, a strong one like “equation” or “lease” counts 2, and a weak one like “compare” counts 1. The field with the most points wins, and it needs at least 2 points to beat General. Weak words only count together, so “research a gift for my dad” stays General.
Everyday phrases are the hard part. “My cell phone battery dies” mentions cells and batteries; “solve a fight with my roommate” has “solve”; “a 3-5 day trip” looks like subtraction. The rules are written to ignore these, and the test set includes them on purpose (section 8).
Difficulty. Easy, medium or hard, from what is asked, not just length. Greetings, quick facts and short pieces like a haiku are easy. Proofs, designs, “step by step”, “in depth”, long outputs (“a 3000 word story”), and weighing several options are hard. Everything else is medium.
Needs. Whether the answer leans on creativity (stories, names, lyrics), reasoning (debugging, proofs, decisions) or explanation (teaching, “why”, “walk me through”). A message can have several, or none.
| Message | Field | Difficulty | Needs | Model |
|---|---|---|---|---|
| thanks, that helped a lot | General | easy | — | Gemini 3.8 Flash |
| What's the capital of Australia? | General | easy | — | Gemini 3.8 Flash |
| Why does this throw a TypeError? | Code | medium | reasoning, explanation | Claude Sonnet 5.5 |
| Write a poem for my grandma's 90th birthday | Creative | medium | creativity | Claude Sonnet 5.5 |
| Prove that the square root of 2 is irrational | Math | hard | reasoning | Gemini 4 Carbon |
| Is it legal to record a phone call in California? | Law | easy | — | Claude Sonnet 5.5 |
| My cell phone battery dies by noon, any tips? | General | medium | — | Claude Sonnet 5.5 |
| Design a rate limiter in TypeScript that works across multiple serverless instances, handling edge cases | Code | hard | reasoning | Claude Opus 5.5 |
03Step 2: choosing a tier
Models come in three tiers by price and capability. Difficulty picks the tier, and also how long the model is allowed to think.
| Difficulty | Tier | Thinking effort | Models Auto can pick |
|---|---|---|---|
| easy | Quick | low | DeepSeek Flash, Gemini 3.8 Flash, GPT-6 Luna, Claude Haiku 5.5 |
| medium | Everyday | medium | DeepSeek V4 Pro, Claude Sonnet 5.5, GPT-6.1 Sol, GPT-6.1 Terra, Gemini 4 Hydrogen |
| hard | Top | high | Grok 4.7, Gemini 4 Carbon, GPT-6 Astra, Claude Opus 5.5 |
Four rules can change the tier or the effort:
- Floors. Some fields never go below a tier because a cheap model’s mistakes cost too much there: Law → at least Everyday.
- Reasoning boost. A message that needs reasoning gets one more level of thinking (medium becomes high).
- Low credits. Below 200 credits left in the month, Auto uses Quick models only, so your last credits stretch further.
- Your plan. Free includes every tier except Claude Fable 5.2, within its monthly credits. If a tier has no model your plan or the server can reach, Auto steps down a tier, then up.
04Step 3: ranking the models
Inside the tier, every model gets a score for this message. The highest wins:
score = strength in the field (1–5, per model)
+ Σ traitWeight[need] × (trait[need] − 3) (for each need the message has)
− costWeight[difficulty] × log2(price ÷ cheapest price in the tier)- Strength: how good the model is at the field, 1 to 5, from the research in section 9.
- Traits: each model also has a 1–5 score for creativity, reasoning and explanation. When the message has a need, a model above 3 gains and one below loses. The weights are creativity 0.5, reasoning 0.25 and explanation 0.25: creative voice varies most between models, so it counts most.
- Price: the cost weight is 0.3 for easy and medium messages and 0.2 for hard ones. At 0.3, a model twice the price of the cheapest in its tier must be 0.3 points stronger to win. Hard messages weigh price less, because getting them right matters more than saving a fraction of a cent.
Two real messages, scored term by term:
Message
“Write a short story about a robot who learns to paint”
Read as Creative, medium, needs creativity → Everyday tier.
| Model | Strength | Traits | Price | Score |
|---|---|---|---|---|
| Claude Sonnet 5.5 picked | 5.0 | +1.00 | −0.30 | 5.70 |
| GPT-6.1 Sol | 4.0 | +0.00 | −0.30 | 3.70 |
| Gemini 4 Hydrogen | 3.5 | +0.00 | −0.18 | 3.32 |
| GPT-6.1 Terra | 3.5 | −0.25 | +0.00 | 3.25 |
| DeepSeek V4 Pro | 3.0 | −0.50 | −0.03 | 2.47 |
Message
“Explain quantum entanglement without hand-waving: what's actually correlated and why it can't send signals”
Read as Science, hard, needs explanation → Top tier.
| Model | Strength | Traits | Price | Score |
|---|---|---|---|---|
| Gemini 4 Carbon picked | 5.0 | +0.25 | −0.06 | 5.19 |
| Claude Opus 5.5 | 4.5 | +0.50 | −0.06 | 4.94 |
| GPT-6 Astra | 5.0 | +0.25 | −0.32 | 4.93 |
| Grok 4.7 | 4.0 | +0.25 | +0.00 | 4.25 |
Ties go to the cheaper model, then the stronger all-rounder. Claude Fable 5.2 is never picked by Auto: it costs about 2 times Claude Opus 5.5 for a small gain, so it’s there when you pin it.
05The models and their scores
Strength per field, 1 (weak) to 5 (strong). Half points mark cases where the evidence separates two models without a full point. Price is a blended cost per million tokens, assuming six input tokens for every output token, which is typical in chat.
| Model | Tier | $/M | General | Writing | Creative | Code | Computer science | Math | Science | Biology | Engineering | Humanities | Research | Law | Finance | Tutoring |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 LunaOpenAI | Quick | $0.16 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 2 | 2 | 2 | 3 |
| Claude Haiku 5.5Anthropic | Quick | $0.16 | 3 | 4 | 4 | 4 | 4 | 3 | 3 | 3 | 3 | 4 | 3 | 3 | 3 | 4 |
| DeepSeek FlashDeepSeek | Quick | $0.43 | 3 | 3 | 3 | 3 | 3 | 3 | 2 | 2 | 2 | 3 | 2 | 2 | 2 | 3 |
| Gemini 3.8 FlashGoogle | Quick | $1.18 | 4 | 3 | 3 | 3 | 3 | 4 | 4 | 4 | 3 | 3 | 4 | 3 | 3 | 4 |
| GPT-6.1 TerraOpenAI | Everyday | $1.57 | 3.5 | 3.5 | 3.5 | 4 | 4 | 4.5 | 3.5 | 3.5 | 4 | 3.5 | 4 | 3.5 | 3.5 | 3.5 |
| DeepSeek V4 ProDeepSeek | Everyday | $1.70 | 3 | 3 | 3 | 4 | 4 | 5 | 4 | 3 | 4 | 3 | 3 | 3 | 4 | 3 |
| Gemini 4 HydrogenGoogle | Everyday | $2.36 | 4 | 3.5 | 3.5 | 4 | 4 | 4.5 | 4.5 | 4.5 | 4 | 4 | 4.5 | 3.5 | 4 | 4.5 |
| Claude Sonnet 5.5Anthropic | Everyday | $3.14 | 4.5 | 5 | 5 | 5 | 5 | 4 | 4 | 4 | 4 | 5 | 4.5 | 4 | 4 | 5 |
| GPT-6.1 SolOpenAI | Everyday | $3.14 | 4 | 4 | 4 | 4.5 | 4.5 | 5 | 4 | 4 | 4.5 | 4 | 4.5 | 4 | 4 | 4 |
| Grok 4.7xAI | Top | $5.14 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 3 | 4 | 4 | 4 | 4.5 | 4 | 4 |
| Gemini 4 CarbonGoogle | Top | $6.29 | 4.5 | 4 | 4 | 5 | 5 | 5 | 5 | 5 | 5 | 4 | 5 | 4 | 4.5 | 4.5 |
| Claude Opus 5.5Anthropic | Top | $6.29 | 5 | 5 | 5 | 5 | 5 | 4.5 | 4.5 | 4.5 | 5 | 5 | 5 | 5 | 5 | 5 |
| Claude Fable 5.2Anthropic | Top | $13 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| GPT-6 AstraOpenAI | Top | $16 | 5 | 4 | 4 | 5 | 5 | 5 | 5 | 5 | 5 | 4 | 5 | 5 | 5 | 5 |
| Model | Creativity | Reasoning | Explanation | Reads images |
|---|---|---|---|---|
| GPT-6 Luna | 3 | 3 | 3 | Yes |
| Claude Haiku 5.5 | 4 | 3 | 4 | Yes |
| DeepSeek Flash | 2 | 3 | 3 | Yes |
| Gemini 3.8 Flash | 3 | 4 | 4 | Yes |
| GPT-6.1 Terra | 2.5 | 4.5 | 3.5 | Yes |
| DeepSeek V4 Pro | 2 | 4 | 3 | No |
| Gemini 4 Hydrogen | 3 | 4 | 4 | Yes |
| Claude Sonnet 5.5 | 5 | 4 | 5 | Yes |
| GPT-6.1 Sol | 3 | 5 | 4 | Yes |
| Grok 4.7 | 4 | 4 | 4 | Yes |
| Gemini 4 Carbon | 3.5 | 5 | 4 | Yes |
| Claude Opus 5.5 | 5 | 5 | 5 | Yes |
| Claude Fable 5.2 | 5 | 5 | 5 | Yes |
| GPT-6 Astra | 3 | 5 | 4 | Yes |
06Where messages actually go
All 221 test prompts, routed with every provider available and plenty of credits, grouped by how Auto read them. This is the clearest picture of what Auto does day to day.
| Field | Difficulty | Prompts | Picked |
|---|---|---|---|
| General | easy | 27 | Gemini 3.8 Flash (26), Claude Haiku 5.5 (1) |
| General | medium | 24 | Claude Sonnet 5.5 |
| General | hard | 2 | Claude Opus 5.5 |
| Writing | easy | 1 | Claude Haiku 5.5 |
| Writing | medium | 12 | Claude Sonnet 5.5 |
| Writing | hard | 2 | Claude Opus 5.5 |
| Creative | easy | 4 | Claude Haiku 5.5 |
| Creative | medium | 12 | Claude Sonnet 5.5 |
| Creative | hard | 2 | Claude Opus 5.5 |
| Code | easy | 1 | Claude Haiku 5.5 |
| Code | medium | 13 | Claude Sonnet 5.5 |
| Code | hard | 7 | Claude Opus 5.5 |
| Computer science | easy | 1 | Claude Haiku 5.5 |
| Computer science | medium | 6 | Claude Sonnet 5.5 |
| Computer science | hard | 3 | Claude Opus 5.5 |
| Math | easy | 6 | Gemini 3.8 Flash |
| Math | medium | 7 | DeepSeek V4 Pro |
| Math | hard | 4 | Gemini 4 Carbon |
| Science | easy | 1 | Gemini 3.8 Flash |
| Science | medium | 10 | Gemini 4 Hydrogen |
| Science | hard | 4 | Gemini 4 Carbon |
| Biology | easy | 2 | Gemini 3.8 Flash |
| Biology | medium | 8 | Gemini 4 Hydrogen |
| Biology | hard | 3 | Gemini 4 Carbon |
| Engineering | medium | 7 | GPT-6.1 Sol |
| Engineering | hard | 2 | Claude Opus 5.5 |
| Humanities | easy | 4 | Claude Haiku 5.5 |
| Humanities | medium | 9 | Claude Sonnet 5.5 |
| Humanities | hard | 2 | Claude Opus 5.5 |
| Research | medium | 5 | GPT-6.1 Sol (3), Gemini 4 Hydrogen (2) |
| Research | hard | 3 | Claude Opus 5.5 |
| Law | easy | 7 | Claude Sonnet 5.5 |
| Law | medium | 2 | Claude Sonnet 5.5 |
| Law | hard | 2 | Claude Opus 5.5 |
| Finance | easy | 2 | Claude Haiku 5.5 |
| Finance | medium | 7 | Claude Sonnet 5.5 (5), DeepSeek V4 Pro (2) |
| Finance | hard | 2 | Claude Opus 5.5 |
| Tutoring | medium | 5 | Claude Sonnet 5.5 |
Claude Sonnet 5.5 handles the largest share because most messages are medium difficulty and it leads the Everyday tier on independent tests. Models with 0% here (DeepSeek Flash, Grok 4.7, GPT-6 Luna, GPT-6.1 Terra, GPT-6 Astra, Claude Fable 5.2) are still available to pin and as fallbacks; on these prompts another model in their tier always scored higher.
07How we test it
We wrote 221 realistic prompts across every field and difficulty: greetings, homework, code with stack traces, proofs, legal questions, poems, and everyday messages full of misleading words. Each one is labelled with the field, difficulty and needs a careful person would give it. Where two fields are both reasonable (“explain photosynthesis to a 10 year old” is biology and tutoring), either counts as right.
The prompts are split in two:
- Tuning set (175): used to build and adjust the rules. Scores here are optimistic by design.
- Held-out set (46): written after the rules were built and never used to adjust them. This is the honest estimate of how Auto does on messages it has never seen. When a held-out prompt fails, the fix has to be general; then the prompt moves to the tuning set and a fresh one replaces it.
Beyond field and difficulty, we score the outcome that matters: did Auto pick the same model it would have picked with perfect labels? A wrong label often still lands on the right model.
| Set | Prompts | Field right | Difficulty right | Easy ↔ hard mistakes | Right model |
|---|---|---|---|---|---|
| Tuning | 175 | 98.9% | 94.9% | 0 | 96.6% |
| Held out (never tuned on) | 46 | 78.3% | 73.9% | 0 | 73.9% |
Before the rewrite (November 2026), the original classifier got the field right 68% of the time on the first 125 prompts and 44% on the first unseen set. One mid-tier model received 38% of all messages, including plain chat, because it had top scores and the lowest price in its tier. The rewrite replaced first-match keyword lists with weighted signals, added needs, rebuilt the model scores from research, and moved every routing number into one place.
The tests run on every change. They fail if accuracy drops, if any message is judged two levels off, or if a pick the research supports changes (for example, hard code must go to the strongest coder, not the cheapest top model).
08Where it still gets it wrong
Rules only know the words they were given. Every new batch of messages brings vocabulary they don’t have, which is why the held-out score sits around 80%. These are every held-out miss, unedited:
| Message | Read as | Should be | Model picked | Ideal model |
|---|---|---|---|---|
| What's the best way to learn to touch type? | General · easy | General or Tutoring · medium | Gemini 3.8 Flash | Claude Sonnet 5.5 |
| Fix this: `for i in range(10) print(i)` | General · medium | Code · easy | Claude Sonnet 5.5 | Claude Haiku 5.5 |
| How do I center a div with CSS? | Code · medium | Code · easy | Claude Sonnet 5.5 | Claude Haiku 5.5 |
| What's the difference between TCP and UDP? | General · medium | Computer science · medium | Claude Sonnet 5.5 | Claude Sonnet 5.5 |
| Explain how garbage collection works in the JVM, including generations and pause times | General · hard | Computer science or Code · hard | Claude Opus 5.5 | Claude Opus 5.5 |
| Convert 72 degrees Fahrenheit to Celsius | General · easy | Math or Science · easy | Gemini 3.8 Flash | Gemini 3.8 Flash |
| Find the eigenvalues of the matrix [[2,1],[1,2]] and explain what they mean geometrically | Math · medium | Math · hard | DeepSeek V4 Pro | Gemini 4 Carbon |
| What happens to the body during a fever? | General · easy | Biology · medium | Gemini 3.8 Flash | Gemini 4 Hydrogen |
| Design a water filtration system for a small village with no electricity, with trade-offs | Science · hard | Engineering · hard | Gemini 4 Carbon | Claude Opus 5.5 |
| Who was Cleopatra? | General · easy | Humanities · easy | Gemini 3.8 Flash | Claude Haiku 5.5 |
| What is existentialism in simple terms? | Humanities · easy | Humanities · medium | Claude Haiku 5.5 | Claude Sonnet 5.5 |
| What are the themes of To Kill a Mockingbird? | Humanities · easy | Humanities · medium | Claude Haiku 5.5 | Claude Sonnet 5.5 |
| Invent a new holiday and describe how people celebrate it | General · medium | Creative · medium | Claude Sonnet 5.5 | Claude Sonnet 5.5 |
| What does the evidence say about whether coffee is good for you? | General · easy | Research or Biology · medium | Gemini 3.8 Flash | Gemini 4 Hydrogen |
| Can my landlord enter my apartment without notice? | Law · easy | Law · medium | Claude Sonnet 5.5 | Claude Sonnet 5.5 |
| Do I need a lawyer to make a will? | Law · easy | Law · medium | Claude Sonnet 5.5 | Claude Sonnet 5.5 |
| Is it better to lease or buy a car? | Law · easy | Finance · medium | Claude Sonnet 5.5 | DeepSeek V4 Pro |
| What should I watch tonight if I liked Interstellar? | General · easy | General · medium | Gemini 3.8 Flash | Claude Sonnet 5.5 |
The pattern: most misses are short questions read as easy when they deserve a normal answer, so they go to a Quick model instead of an Everyday one. That costs some quality, never a big bill, and no hard task has gone to a Quick model. When a field is missed, the message usually lands on a strong all-rounder anyway. If an answer feels thin, pinning a model for that chat fixes it.
And the everyday traps it now handles, from the tuning set:
| Message | Read as | Model |
|---|---|---|
| How do I fix a rusty bike chain? | General · medium | Claude Sonnet 5.5 |
| My cell phone battery dies by noon, any tips? | General · medium | Claude Sonnet 5.5 |
| How do I solve a fight with my roommate about chores? | General · medium | Claude Sonnet 5.5 |
| Plan a 3-5 day trip to Portugal | General · medium | Claude Sonnet 5.5 |
| What's a good java place downtown to work from? | General · easy | Gemini 3.8 Flash |
| I have a bug bite that won't stop itching | General · medium | Claude Sonnet 5.5 |
| Give me a creative name for my python (the snake, not the language) | Creative · easy | Claude Haiku 5.5 |
09The research behind the scores
The scores in section 5 are judgments, not measurements. They come from model makers’ published results, independent indexes and public leaderboards, which often disagree, use different settings, and get replaced by new versions every few months. We use them as starting points and change them when better evidence arrives. Research snapshot: November 2026.
| Choice | Evidence |
|---|---|
| Quick factual questions → Gemini 3.8 Flash | Highest factual-recall accuracy of the Quick models on Artificial Analysis’s AA-Omniscience test. Comparison |
| Easy code and short writing → Claude Haiku 5.5 | Highest Artificial Analysis Intelligence Index of the Quick models, and ahead on agentic coding tests. Latent Space, TechVerdict |
| Everyday code, writing and explanations → Claude Sonnet 5.5 | Leads the Everyday tier on independent coding (Terminal-Bench) and knowledge-work (GDPval-AA) results. BenchLM, Codersera |
| Medium science and biology → Gemini 4 Hydrogen | Google’s fast Gemini 4 (new in November 2026). It scores 51 on the Artificial Analysis Intelligence Index, close to GPT-6.1 Sol (52), at a lower price, and keeps the Gemini line’s strong factual recall. Launch coverage |
| Medium math and engineering → DeepSeek V4 Pro or GPT-6.1 Sol | Strong reasoning and math at a lower cost per task. DeepSeek’s own math results are near the frontier; its coding scores are lower in independent rankings, so it’s scored lower for code. BenchLM, DeepSeek-V4 report |
| Hard code, writing, creative, research and law → Claude Opus 5.5 | Leads LMArena’s human-voted creative writing board and is reviewers’ most recommended model for prose; frontier coding. Fiction comparison, LMArena text |
| Hard science and math → Gemini 4 Carbon | Google’s strongest model (November 2026) and first on the Artificial Analysis Intelligence Index (60), built for hard science, math and debugging. It replaced Gemini 3.1 Pro. On research it ties Claude Opus 5.5 at the same price, and ties go to the stronger all-rounder. Index leaderboard, GPQA paper |
| Grok 4.7 and GPT-6.1 Terra as backups, not first choices | Grok 4.7 is a good price for a Top-tier model, but its independent intelligence index sits below the leaders, and it trails on long, hard coding. The Decoder. GPT-6.1 Terra (new in October 2026) costs half GPT-6.1 Sol but sits a step below it, so it steps in when another model is unavailable, and you can pick it yourself for cheaper everyday answers. OpenAI |
What benchmarks don’t measure well: explaining and tutoring, which we judge from model positioning and reviews; and creative quality, where AI-judged and human-voted rankings disagree. We favor human votes.
10What you control
- Pin a model. Pick any model in the menu under the message box instead of Auto. Pinned models skip all of the above, including the tier.
- See what answered. Each reply shows the model and the tokens it used; Usage breaks your credits down by model.
- Team mode. For big tasks, a planner splits the work and specialists on different models run in parallel. It uses the same scores, and prefers a different company for each teammate.
- Credits. Auto is not a way to spend more: a greeting on a Quick model costs a fraction of a credit, and the low-credit rule keeps the end of your month on Quick models.
11Privacy and providers
Routing runs on our server with simple rules. No AI reads your message to classify it, and the routing decision is logged with the field, difficulty and model (not used for training).
The chosen model’s company receives your message and the context needed to answer it: OpenAI, Anthropic, Google, DeepSeek, xAI. DeepSeek processes data in China; Auto can be set to leave DeepSeek out entirely. Full details are in the Privacy Policy.
12What’s next
- A model as the classifier. A small, fast model reading each message would understand vocabulary the rules have never seen. The rules would stay as a fallback. Target: well above today’s 78% on unseen prompts, for a fraction of a cent and a fraction of a second per message.
- Real-world numbers. Once there’s enough traffic, this page will show the real share of messages per model, the average cost per message compared with sending everything to a Top-tier model, and how often people re-ask with a pinned model (a sign Auto chose badly).
- Showing why. A small “why this model” note on each reply.
- Re-scoring. Scores are revisited when providers release new models, and every change is checked against the tests above.
Changelog
- November 2026. Weighted classifier with needs; model scores rebuilt from research; routing settings in one policy; the test suite and this page.
See Auto pick for yourself, soon.
Apier is invite-only for now. Join the waitlist and we’ll let you know when your spot opens.
Join the waitlist