H2O-Lightning 4B Native

Micro accuracy across 48 Easy, 72 Original and 111 Hard-public decisions using the typed-decision serving bundle. Not an official leaderboard rank.

Benchmark efficiency: 19.701 published score points per GB of selected weights. This ratio is not an inference-speed measure; check the test settings to see whether the score covers the selected quantization.

Test settings: Publisher-run test of the selected Q8_0 decision weights on all 231 public JevBench text questions through the llama.cpp decision endpoint with 40,960-token context and one slot. Q8_0 answered 204/231 correctly (88.31%); the publisher's BF16 llama.cpp run answered 205/231. The card reports the same top answers for native and shim deployments on 231/231 prompts when both use these weights and the documented decision contract.

Artifact-specific publisher result, rounded from 204/231; not independent, not the official JevBench composite leaderboard and not a general-chat score. The official repository provides a hash-checked launcher pinned to bootstrap commit 4e179b1c1147cfe55e1f3c47814853a5f1b85c0a and package commit 5eb7e563fa8e4eeaaca17b2230498dd112005159, serving typed decisions through POST /v1/decisions. Its current shim also exposes a tested OpenAI Decisions API route for openai Python 3.26 or later; it maps predicate, choice and score questions onto the same decision format as /v1/systemone, which remains available. The publisher says a score answer's confidence is the probability of its most likely level, usage.input_tokens counts the full input even when state is cut for reading, usage.processed_input_tokens is what vLLM read, and usage.output_tokens counts one label token per answered question. These are interface and accounting semantics, not new benchmark or speed evidence. Its optional text-only reasoned client adds up to 256 analysis tokens before repeating the decision; it is extra inference, not training, and rejects image inputs. In the publisher's small 72-case/120-question suite, BF16 improved from 98 to 120 correct questions; the Q8 comparison reused that suite and improved from 99 to 120, so this is not an independent or universal accuracy guarantee and is not used as a ranking score. On 4,074 held-out decision questions under standard llama.cpp plus the publisher shim versus BF16 vLLM, Q8_0, Q5_K_M and Q4_K_M matched BF16 headline accuracy within reported measurement error; Q5_K_M changed about 3% of top answers and Q4_K_M about 5%, including some confident cases. The exact stock Q5_K_M file is 3,074,987,168 bytes with SHA-256 4fa5ad9fc19c47acbf877b017e05d8281a95d4e4943a8b5e737e7610351e9737; the exact stock Q4_K_M file is 2,708,804,768 bytes with SHA-256 6356a49ed0812da07d772df752ed9f57bc3da6d0a10c7df19f5c7726af6943d9. The later same-question comparison measured the mixed IQ3_XXS file at 2,160,135,136 bytes and 2.5 points below BF16 (95% CI -3.5 to -1.5) with 88.6% top-answer agreement, while plain IQ3_XXS is 1,904,493,536 bytes and 3.5 points below BF16 (95% CI -4.6 to -2.6) with 87.3% agreement. Mixed IQ3_XXS scored 1.1 points above plain IQ3_XXS (95% CI +0.4 to +1.7); its 206/231 public-question result versus Q8_0's 204/231 is within noise and is not treated as superior accuracy. Q5_K_M, Q4_K_M and both IQ3_XXS files use standard llama.cpp plus the publisher shim; the selected native Q8_0 file still requires unmerged llama.cpp PR #30361. The selected language file excludes the separate vision projector, runtime overhead, KV/context memory and concurrency. No speed or hardware-compatibility claim is inferred. Apache-2.0 license.