Bonsai 27B (1-bit)

Writing Python that passes extra edge-case tests. EvalPlus pass@1 on all 164 problems.

Benchmark efficiency: 23.565 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: HumanEval+.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Following precise instructions. These cards report IFEval without consistently specifying prompt-level versus instruction-level scoring. Treat this view as approximate.

Benchmark efficiency: 20.8 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: IFEval.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Solving grade-school math word problems. Published answer accuracy; known small subsets are separate.

Benchmark efficiency: 24.399 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: GSM8K.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Competition math from AIME 2025. Published answer accuracy.

Benchmark efficiency: 23.334 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: AIME25.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Competition math from AIME 2026. Published answer accuracy.

Benchmark efficiency: 22.895 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: AIME26.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Basic Python programming with extra tests. EvalPlus pass@1 on the sanitized 378-problem set.

Benchmark efficiency: 20.928 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: MBPP+.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Math reasoning over all 500 problems. Published exact answer accuracy.

Benchmark efficiency: 25.766 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: MATH-500.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Revised MMLU knowledge test. Kept separate from original MMLU and MMLU-Pro.

Benchmark efficiency: 21.757 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: MMLU-Redux.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Published coding pass rates without a verified benchmark version. Do not compare these directly with the v6 view.

Benchmark efficiency: 20.087 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: LiveCodeBench.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Berkeley Function Calling Leaderboard version 3. Kept separate from version 4.

Benchmark efficiency: 18.594 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: BFCL v3.

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.

Instruction following: prompt-level loose accuracy. Separate from IFEval and other IFBench metrics.

Benchmark efficiency: 13.766 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher thinking-mode benchmark table for the compressed Bonsai model; not scores copied from its Qwen parent. Custom Prism llama.cpp kernels are required. LiveCodeBench version and IFEval aggregation are unspecified. Table row: IFBench (prompt-loose).

Published point estimate, not an independent head-to-head evaluation. Quantization and inference settings can change results.