These scores were compiled from the linked sources. These published scores were not independently tested here. Different prompts, precision, thinking budgets and tools can change results.
GPQA Diamond
40.4 / 100
Hard science questions. This is a science reasoning test, not an overall intelligence score.
Benchmark efficiency: 31.701 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.
Test settings: Final frozen candidate evaluation, BF16 on NVIDIA L40S. GPQA 198, HumanEval+ 164, IFEval 541; MMLU-Pro 11,332 untouched holdout questions.
Published result for the original model; this selected GGUF file was not separately benchmarked. Different prompts, precision and thinking settings can affect comparisons.
Writing Python that passes extra edge-case tests. EvalPlus pass@1 on all 164 problems.
Benchmark efficiency: 42.106 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.
Test settings: Final frozen candidate evaluation, BF16 on NVIDIA L40S. GPQA 198, HumanEval+ 164, IFEval 541; MMLU-Pro 11,332 untouched holdout questions.
Published result for the original model; this selected GGUF file was not separately benchmarked. Different prompts, precision and thinking settings can affect comparisons.
Following every instruction in a prompt. Strict prompt-level accuracy on the full 541-prompt set.
Benchmark efficiency: 56.568 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.
Test settings: Final frozen candidate evaluation, BF16 on NVIDIA L40S. GPQA 198, HumanEval+ 164, IFEval 541; MMLU-Pro 11,332 untouched holdout questions.
Published result for the original model; this selected GGUF file was not separately benchmarked. Different prompts, precision and thinking settings can affect comparisons.
Noema final untouched 11,332-question holdout. Development items excluded; separate from full-test scores.
Benchmark efficiency: 41.722 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.
Test settings: Final frozen candidate evaluation, BF16 on NVIDIA L40S. GPQA 198, HumanEval+ 164, IFEval 541; MMLU-Pro 11,332 untouched holdout questions.
Published result for the original model; this selected GGUF file was not separately benchmarked. Different prompts, precision and thinking settings can affect comparisons.
Context and version evidence: The repository and file name identify the selected model. Revision hashes are shown above when recorded. Tested context length, full hardware configuration and run date are not consistently recorded in this snapshot. Only settings explicitly stated in each benchmark record are known.
The 1.274 GB figure covers model weights in this file. It excludes the context cache, runtime, operating system and any separate multimodal files. It is not a measured RAM requirement or a token-generation speed.
These complete configurations pass the initial weight-size check for the selected 1.27 GB file. Allow extra memory for context and software. Check the model’s runtime notes and your inference software before buying.
Listing evidence: 2026-09-25. Amazon.com base price visible in an owner-supplied screenshot reviewed 25 Sep 2026; capture time not supplied. Dated reference, not a live quote. Excludes card signup offers, coupons, tax and shipping.
Listing evidence: 2026-10-03. Apple US launch price announced 22 September 2026: USD 1,699. Dated manufacturer reference, not an Amazon offer; tax and shipping may differ.
Listing evidence: 2026-09-24. GMKtec, EVO-X2 / 64GB / 1TB, recorded 22 Sep 2026; exact ASIN capacity identified 24 Sep. This is a dated reference, not a current Amazon price; tax, shipping and bundle contents may differ.
Memory matches are estimates, not tested compatibility or speed. As an Amazon Associate I earn from qualifying purchases.
Confirm model architecture and quantization support in your runner. The selected weights use 1.274 GB; this excludes context cache, runtime buffers and other programs.
These are the first listed tiers that pass our reserve policy, not measured minimum requirements. System RAM and VRAM remain separate. Use the memory tool to change the reserve for your context and runner.