The dequant fallback runs DT3 matmuls as fp16 weights through the f16
matmul pipelines, which default to fp16 accumulators when the device
supports fp16. DT3 weights are the sum of two fp16-scaled ternary
planes and are generally not fp16-representable, so the fallback
already pays one fp16 rounding on the weights; accumulating on top of
that in fp16 measurably hurts.
Force GGML_PREC_F32 for the DT3 fallback, which selects the f32acc
pipelines - the same numerics as the CUDA GEMM fallback (fp16 inputs,
fp32 compute).
Measured on Qwen2.5-3B DT3, wiki.test, 4 chunks, --no-mmap, AMD
Radeon 890M (RADV STRIX1), CPU reference 15.0608:
default batch, before 15.5562 (+3.29%)
default batch, after 15.4062 (+2.29%)
GGML_VK_DISABLE_F16=1 15.3348 (+1.82%, floor of this route)
The remaining gap is shared with the mul_mat_vec route (15.3302, which
does not move under GGML_VK_DISABLE_F16) and is under investigation
separately; DT3 dequantization itself is bit-exact vs the CPU
reference on real model tensors (214,695,936 elements, 0 mismatches).
Millaguie
ad6dd747d4