Why the same model performs differently across providers: quantization moves accuracy by 29 points

DeepSeek-V4-Pro-0813, gpt-oss-120b, and GLM-5.2 each return different accuracy and speed numbers depending on host. On GLM-5.2, the Endpoint Accuracy Index spans 73 to 102 across 13 providers, a 29-point gap tied to quantization, retrieved 2026-08-21.

The provider column matters as much as the model name. Read the table once; the number will not change per provider you did not check.

Same model, different endpoint, different number

A model name identifies a set of trained weights. It does not identify a precision format, an output token ceiling, a prompt template, or a routing path, and each of those changes per provider. Artificial Analysis, an independent benchmarking site, tracks three such models across their hosting providers, retrieved 2026-08-21.

GLM-5.2's output speed ranges from 55.2 tokens per second on DeepInfra's FP4 endpoint to 360.4 on Databricks, a 6.5x spread as stated by the source. gpt-oss-120b ranges from 48.0 to 1,641.3 on Cerebras, a 34.2x spread.

DeepSeek-V4-Pro-0813 ranges from 52.8 to 168.3 tokens per second, and its time to first token ranges from 12.95 seconds on Baseten to 49.07 on DigitalOcean, lower is better. None of these three models changed. The endpoint did.

Model Provider comparison

Table note: unreported means the source had not published that figure as of the retrieval date; it does not mean a provider was excluded from ranking. Endpoint Accuracy Index values are Artificial Analysis's own composite, not an independent LLM Waves Research measurement.

The three rows cover 46 provider endpoints for models with otherwise identical weights. GLM-5.2's two lowest readings, 73 and 75, both run FP4, while every provider at 97 or above discloses no quantization or reports FP8 or NVFP4. DeepSeek-V4-Pro-0813's provider page publishes no accuracy column, insufficient data rather than parity.

Basis and method

This article aggregates figures published by Artificial Analysis and by an academic measurement study, per the LLM Waves methodology page; LLM Waves Research ran no benchmark suite here and tested no model directly. The only original work is the arithmetic on top of those figures, published inline with each number. No sentence uses "we tested" or "we measured": this is a Basis 2, aggregated, article.

Artificial Analysis provider pages for all three models, retrieved 2026-08-21, report output speed, time to first token, price, and, for two of three models, an Endpoint Accuracy Index from BFCL v4-500, HLE-250, and AA-LCR-25. These pages exclude system throughput under concurrent load, a different metric from per-request output speed.

A 2026 measurement study on arXiv, built on Q4 2025 logs from 29 providers, separately reports routing cut measured cost 37.8% on Qwen3-32B and raised measured throughput about 90% on DeepSeek-V3.2 versus each model's official endpoint; that study did not test the models above.

What decides the number you get

Four variables decide whether a hosted endpoint returns the reference number: the precision format a provider quantizes to, the output token ceiling it sets, the prompt scaffolding it injects, and the routing layer, if any, between a developer and the endpoint serving the request. Current per-provider standings sit on the LLM Waves model leaderboards, which move independently of this snapshot.

The provider variance index

Averaging each model's fastest-to-slowest speed ratio, unweighted across the three, gives a raw figure of 14.6x and a hardware-adjusted figure of 5.5x once gpt-oss-120b's non-GPU rows are excluded. Averaging the two models with a published accuracy index gives a 30-point mean spread; DeepSeek-V4-Pro-0813 is excluded, insufficient data rather than a zero. Both figures are LLM Waves Research's own arithmetic; the formula and excluded rows sit in the downloadable dataset below.

The chart below sets the three models side by side before the per-model breakdowns that follow.

One model's provider gap is five times wider than another's, and gpt-oss-120b's own gap narrows from 34.2x to 6.9x once wafer-scale and dataflow accelerators are set aside, a limitation of comparing across hardware classes rather than across models.

Quantization precision explains most of the accuracy gap

Quantization reduces the numeric precision a model runs at, commonly from a 16-bit format down to 8-bit (FP8) or 4-bit (FP4, NVFP4), to cut memory and raise throughput. Fewer bits per weight means less numeric headroom for computation, which shows up as a lower evaluation score rather than a visible error.

On the GLM-5.2 table, every FP4 endpoint scores at or below 75 on the Endpoint Accuracy Index, while the group at 97 to 102 discloses no quantization or reports FP8 or NVFP4. That pattern is a correlation across 13 disclosed rows, not a controlled experiment, since providers also differ in serving stack and batching policy.

The disclosure gap behind the index

The Endpoint Accuracy Index is a composite of BFCL v4-500, HLE-250, and AA-LCR-25, normalized so 100 represents the reference deployment; its component weights are not published, so the index reads as a ranking signal rather than a reproducible formula. Only 8 of GLM-5.2's 20 tracked providers label a precision format at all.

The chart below places every disclosed precision level next to its measured score.

Every provider below 90 on the index runs a reduced-precision format, FP4 in both cases; every provider at 97 or above either withholds that label or reports full precision, FP8, or NVFP4.

Tool calling and output limits move accuracy independent of quantization

Precision is not the only lever. On gpt-oss-120b, the Endpoint Accuracy Index spreads 31 points, 70 on Cloudflare to 101 on Amazon Bedrock, with no quantization disclosed for either.

A provider that caps output tokens mid-response can interrupt a multi-step tool call before completion, lowering the tool-calling component without changing a model weight. HLE-250 credits partial reasoning chains, so a lower output ceiling can truncate a chain before its answer token.

Provider routing changes price and throughput without changing the model

Routing through an aggregation layer, rather than a model's official endpoint, cut measured cost 37.8% on Qwen3-32B and raised measured throughput about 90% on DeepSeek-V3.2, per the arXiv study cited above.

The same study found OpenAI-compatible formatting supported by an average of 12.2 providers per model, against 3.6 for Anthropic-compatible formatting, a gap that decides which routing layers a model can reach.

Hidden prompt scaffolding and hardware class explain the rest of the gap

A request rarely reaches a hosted model as written. Providers inject role tokens, tool definitions, and cache markers into the token stream first, and that scaffolding differs by provider even on an identical visible request, a pattern documented in a November 2025 essay on LLM API architecture.

Hardware class explains the rest: gpt-oss-120b's 34.2x spread narrows to 6.9x once Cerebras and SambaNova, built on wafer-scale and dataflow chips, are excluded and the comparison restricted to GPU-hosted providers.

The chart below lines up every gpt-oss-120b provider by raw output speed.

Cerebras and SambaNova, the two non-GPU rows, account for most of the spread; GPU-hosted providers alone still range from 48 to 330 tokens per second, a 6.9x gap on hardware built for general use.

What we did not measure

This article did not run prompts against any of the 46 provider endpoints, did not test system throughput under concurrent load, did not verify quantization on the 12 GLM-5.2 endpoints disclosing none, and did not reproduce any Endpoint Accuracy Index score. Every figure above is Artificial Analysis's or the cited paper's own result, aggregated and, where stated, averaged by LLM Waves Research.

Which reader does this change the calculation for

For high volume: a 34.2x speed gap on gpt-oss-120b changes queue length at fixed cost more than a model upgrade would, provided the accuracy floor above still clears a project's threshold. For low latency: DeepSeek-V4-Pro-0813's time to first token spans 12.95 to 49.07 seconds across 8 providers, so provider choice sets the wait, not model choice.

For self-hosting: none of the variance above applies once weights run on infrastructure a team controls, the disqualifier here. For enterprise: an undisclosed precision format on 12 of 20 GLM-5.2 endpoints is a procurement question the public page cannot answer.

Does the same model always return the same answer on every provider?

No. The same published weights can be quantized to a different precision, capped at a different output length, and wrapped in a different hidden prompt template by each host; tracked data shows Endpoint Accuracy Index gaps of 29 to 31 points between the highest and lowest scoring provider across two tracked models.

Why is one provider so much cheaper for the same model?

Price differences of up to 6.8x on GLM-5.2 line up with quantization and hardware choice more than with margin. A provider running FP4 on commodity GPUs carries a lower compute cost per token than one running full precision on newer accelerators, and that difference passes through to the listed price.

Does a higher Endpoint Accuracy Index always mean a better provider?

Not for every workload. The index composites tool-calling, reasoning, and long-context evaluations; a provider scoring lower can still be the right choice for a workload that touches none of those, provided the output ceiling and TTFT for that specific provider are checked separately.

Can a provider serve a quantized model without disclosing the precision format?

That is a contract and disclosure question between a developer and a specific provider, not one this article can answer; 12 of GLM-5.2's 20 tracked providers publish no precision label at all, the reason the disclosure gap is treated as a named metric here.

The provider column needs rechecking more often than the model name does

Model names change on a release cycle measured in months. Provider configurations, precision format included, change without a version bump on a schedule no public page announces.

The comparison above is a snapshot, not a standing guarantee.

Read the live, continuously updated provider comparison rather than treating this snapshot as current beyond its stated dates.

Changelog 2026-08-21: Published. Three models, 46 provider endpoints, aggregated from Artificial Analysis and one arXiv measurement study. Ranking basis: output speed ratio, descending.