vulkan : add DT3 dequant, get_rows and scalar mul_mat_vec

Wires the dual-plane ternary type into the Vulkan backend through three
paths only:

- get_rows and the generic scalar mul_mat_vec use per-element decode in
  dequant_funcs.glsl: the byte and base-3 digit are located from the
  element index (regions qs[0..16), qs[16..24), qh[0..2)), the byte is
  multiplied by 3^n mod 256 and the top digit taken. w = d1*t1 + d2*t2
  is accumulated in fp32; both products are exact so the sum carries a
  single float rounding and reproduces the CPU reference bit by bit
  (verified: 0 mismatches over the 112-block synthetic test and over
  214,695,936 elements of real model tensors).
- larger matmuls fall back to dequant_dt3.comp (decode each byte once
  with q <- q*3 mod 256, fp32 sum, one rounding at the f16 write) plus
  the existing f16 matmul pipelines.

The qh bytes hold only 4 trits; their 5th base-3 digit is packing
padding that decodes to -1, so both decoders stop at 4 digits.

Deliberately NOT implemented, and declined instead of half-supported:
no coopmat/coopmat2/MMQ shaders are generated for DT3, and supports_op
answers false for MUL_MAT_ID (mul_mat_vec_id shaders are not generated
either). GET_ROWS and MUL_MAT answer true.

test-dt3-gpu accepts the Vulkan backend (and IGPU-type devices) and
passes on RADV STRIX1: dequant 0 mismatches, mul_mat n<=8 norm rel err
~1e-8, GEMM fallback bit-identical to an F16 GEMM on fp16-rounded
weights.
This commit is contained in:
Millaguie
2026-08-11 03:00:47 +02:00
parent c01c26b56e
commit 8ba4db150f
6 changed files with 165 additions and 13 deletions
+17 -9
View File
@@ -33,8 +33,12 @@
// random bytes: every byte value 0..255 must decode identically on both
// sides, including values >= 243 that never come out of the packer.
//
// DT3 is implemented for CUDA and HIP only. Without one of those backends the
// test is skipped and succeeds — an unsupported backend is not a failure.
// DT3 is implemented for CUDA, HIP and Vulkan. Without one of those backends
// the test is skipped and succeeds — an unsupported backend is not a failure.
// On Vulkan the n <= 8 path is the scalar mul_mat_vec shader (fp32 dot on
// exactly decoded weights, not an integer dot), and the larger-n path is
// dequantization to fp16 + the f16 matmul pipeline; both are judged by the
// same gates as the CUDA MMVQ/GEMM paths.
#include "ggml.h"
#include "ggml-alloc.h"
@@ -461,21 +465,25 @@ static void build_control_data(ggml_type type, std::vector<uint8_t> & data, std:
}
int main(void) {
// Only CUDA and HIP (which reports itself as "ROCm") implement DT3. Any
// other GPU backend is skipped rather than failed: Vulkan and SYCL answer
// supports_op == false for DT3, which is the correct answer for them and
// Only CUDA, HIP (which reports itself as "ROCm") and Vulkan implement
// DT3. Any other GPU backend is skipped rather than failed: SYCL answers
// supports_op == false for DT3, which is the correct answer for it and
// not a bug to report, and Metal answers true for almost any type but has
// no DT3 shader, so it would die in pipeline compilation mid-test. Picking
// the backend by name keeps this test honest on machines we do not have.
ggml_backend_t backend = nullptr;
for (size_t i = 0; i < ggml_backend_dev_count(); ++i) {
ggml_backend_dev_t dev = ggml_backend_dev_get(i);
if (ggml_backend_dev_type(dev) != GGML_BACKEND_DEVICE_TYPE_GPU) {
// IGPU is a distinct device type from GPU: an integrated Vulkan device
// with unified memory reports as IGPU, and accepting only GPU silently
// skipped the very hardware this backend is for.
const auto dt = ggml_backend_dev_type(dev);
if (dt != GGML_BACKEND_DEVICE_TYPE_GPU && dt != GGML_BACKEND_DEVICE_TYPE_IGPU) {
continue;
}
const char * name = ggml_backend_dev_name(dev);
if (strncmp(name, "CUDA", 4) != 0 && strncmp(name, "ROCm", 4) != 0) {
printf("skipping GPU backend %s: DT3 is only implemented for CUDA/HIP\n", name);
if (strncmp(name, "CUDA", 4) != 0 && strncmp(name, "ROCm", 4) != 0 && strncmp(name, "Vulkan", 6) != 0) {
printf("skipping GPU backend %s: DT3 is only implemented for CUDA/HIP/Vulkan\n", name);
continue;
}
backend = ggml_backend_dev_init(dev, nullptr);
@@ -483,7 +491,7 @@ int main(void) {
break;
}
if (backend == nullptr) {
printf("no CUDA/HIP backend available, skipping\n");
printf("no CUDA/HIP/Vulkan backend available, skipping\n");
return 0;
}