DeepSeek V4 Flash 0731's Score of 50, Audited

"DeepSeek V4 Flash 0731 (Reasoning, Max Effort) scores 50 on the Artificial Analysis Intelligence Index, placing it well above average among comparable models (median: 25)." That sentence is Artificial Analysis's own summary of DeepSeek's July 31 open-weights refresh, and 50 is the headline figure on the page.

A second set of claims travels alongside it. Unsloth's model page, quoted by a user on the NVIDIA developer forum on July 31, says 0731 delivers "the best performance for its size" and is "outperforming V4-Pro (Preview)."

Those are separate measurements, made by separate parties, under separate conditions. The conditions are where the useful information sits.

Artificial Analysis spent $72.02 running nine evaluations to produce the 50

The 50 is a composite. Artificial Analysis Intelligence Index v4.1 folds nine evaluations into one score: GDPval-AA v2 (agentic real-world work tasks), τ³-Banking (agentic tool use), Terminal-Bench v2.1 (agentic coding and terminal use), SciCode (coding), Humanity's Last Exam (reasoning and knowledge), GPQA Diamond (scientific reasoning), CritPt (physics reasoning), AA-Omniscience (knowledge) and AA-LCR (long context reasoning). Three of the nine carry the word "agentic" in Artificial Analysis's own description of them.

The run was done by Artificial Analysis, not DeepSeek, and that provenance is worth as much as the score. Vendor benchmark tables are chosen by the vendor; this one was chosen by a third party that runs the same nine evals against every model it tracks. Artificial Analysis also publishes what the evaluation cost it: $72.02 in tokens to put 0731 through the full index.

Two conditions attach to the number. First, it describes a specific configuration, the reasoning variant at maximum effort, and the page notes plainly that "a non-reasoning variant may also exist." Second, the model is currently available "through 1 API provider," so the score was measured through a single endpoint served by DeepSeek itself.

The companion ranking, #3 of 101, is a class ranking rather than an overall one. Artificial Analysis compares open-weights models only against other open-weights models in the same size bracket, where Large means anything above 150B parameters. The model page does not publish the roster of that 101-model class, so the comparison set is not inspectable from where the rank is displayed. The claim on offer is third among large open-weights models, and it says nothing about position among the 590 models the site tracks.

The 50 covers reasoning quality and Artificial Analysis has not measured speed

On the same page that reports the intelligence score, the Speed field reads N/A. Output tokens per second: unknown. In Artificial Analysis's own four-unit rating system, speed for this model is scored "Unknown out of 4 units."

So the case being made for 0731 as a coding-agent workhorse rests entirely on answer quality per dollar. Nobody has published a throughput figure for it. Anyone telling you it is fast is inferring that from the 13B active parameters, which is a reasonable guess about a mixture-of-experts model and still a guess.

The other boundary is modality. 0731 takes text and returns text. It does not accept image input, and Artificial Analysis states flatly that it is not multimodal. The 1M-token context window is real and is the largest practical feature here, roughly 1,500 A4 pages by the site's own conversion. Long codebases, yes. Screenshots of a failing UI, no.

The index does not charge the model for the 210 million tokens it spent

Getting to 50 took 0731 210M output tokens across the index. The median for comparable open-weights models is 100M. Artificial Analysis files this under Verbosity and ranks the model #36 of 101 on it, which is a polite way of saying it thinks out loud at roughly twice the typical length.

An intelligence index scores the answer. It does not deduct points for the route taken to get there. The same page therefore carries a separate metric, cost per Intelligence Index task, which Artificial Analysis computes from five price lines: input, cache hit, cache write, reasoning and answer tokens. Verbosity feeds that metric even though it never touches the score. And 0731's raw prices are competitive rather than category-leading: $0.14 per 1M input and $0.28 per 1M output put it #22 of 101, not #1.

There is a third exclusion, and it is the one most likely to bite a reader directly. The artifact that scored 50 is DeepSeek's own hosted endpoint running the reasoning variant at max effort, not any of the community quantizations linked in the forum thread. That thread opened on July 31 at 17:27 UTC with Unsloth's GGUF set. An NVFP4 conversion was posted on August 1 at 01:15, and at 17:56 the same day a user posted a working llama-server recipe running the UD-IQ2_M build at 262,144 context with --reasoning off. A two-bit quantization with reasoning disabled is a different configuration from the one Artificial Analysis measured, and no published score describes it.

Same caveat on Unsloth's comparison. "Outperforming V4-Pro" carries a parenthetical: Preview. The Flash model is 284B total with 13B active; V4-Pro is 1.6T with 49B active, roughly five times the total parameter count. Beating the preview build of a model that size is what the sentence claims. It makes no statement about shipped V4-Pro.

Cache hit pricing decides what a coding agent actually costs you

The most extreme number on the page belongs to cache hits: $0.003 per 1M tokens, a 98% discount, ranked #1 of 101 models in the class. The cheapness argument rests on that line, and whether it applies to you is a property of your workload rather than of the model.

Artificial Analysis quotes a blended rate of $0.06 per 1M tokens using a 7:2:1 ratio of cache hit to input to output. Seven parts of every request being cache hits describes an agent loop precisely: the same growing conversation prefix resubmitted turn after turn, with only the newest tokens uncached. If that is your shape, the blended figure is roughly honest and the model is remarkably cheap.

If your shape is different, the blend does not apply. One-shot classification over a stream of unrelated documents caches nothing, so you pay $0.14 on input. Long generated answers pay $0.28 on output, multiplied by a model that emits about twice the median token count. The per-token price is low. The per-task price depends on the verbosity figure sitting further down the same page.

One more budget item is structural: a single API provider means no failover and no price competition on that endpoint yet. Open weights make competition possible. They do not make it exist on the day of release.

Running it yourself costs 162GB before it costs a cent

The license is MIT, which permits commercial use, and the weights are on Hugging Face. That is the strongest thing about this release and the part no score captures.

The practical floor is storage and memory. Unsloth's card, as quoted in the NVIDIA thread, says the full-precision-lossless Q8 build (UD-Q8_K_XL) is 162GB "and only 7GB bigger than Q4 (UD-Q4_K_XL)." Check that second half against the actual file listing before you plan around it, because a 7GB gap between Q4 and Q8 is not what quantization normally does. The 162GB figure for Q8 is the one to budget against.

Tooling is still moving. A user reported on July 31 that multi-token prediction "is not yet compatible on the community vLLM," and the NVFP4 conversion followed at 01:15 UTC on August 1. GGUF is the format local front ends like Ollama and LM Studio read, though whether any specific build loads cleanly on release day is a per-release question, not a guarantee. Two days after launch the honest status is that the weights are out, several conversions exist, and the serving stack is being assembled in public.

The 50 would turn out to be wrong the moment a second provider serves 0731 and a re-run at that endpoint lands somewhere else. A score measured through one API is partly a measurement of that API.