vulkan : add DT3 dequant, get_rows and scalar mul_mat_vec
Wires the dual-plane ternary type into the Vulkan backend through three paths only: - get_rows and the generic scalar mul_mat_vec use per-element decode in dequant_funcs.glsl: the byte and base-3 digit are located from the element index (regions qs[0..16), qs[16..24), qh[0..2)), the byte is multiplied by 3^n mod 256 and the top digit taken. w = d1*t1 + d2*t2 is accumulated in fp32; both products are exact so the sum carries a single float rounding and reproduces the CPU reference bit by bit (verified: 0 mismatches over the 112-block synthetic test and over 214,695,936 elements of real model tensors). - larger matmuls fall back to dequant_dt3.comp (decode each byte once with q <- q*3 mod 256, fp32 sum, one rounding at the f16 write) plus the existing f16 matmul pipelines. The qh bytes hold only 4 trits; their 5th base-3 digit is packing padding that decodes to -1, so both decoders stop at 4 digits. Deliberately NOT implemented, and declined instead of half-supported: no coopmat/coopmat2/MMQ shaders are generated for DT3, and supports_op answers false for MUL_MAT_ID (mul_mat_vec_id shaders are not generated either). GET_ROWS and MUL_MAT answer true. test-dt3-gpu accepts the Vulkan backend (and IGPU-type devices) and passes on RADV STRIX1: dequant 0 mismatches, mul_mat n<=8 norm rel err ~1e-8, GEMM fallback bit-identical to an F16 GEMM on fp16-rounded weights.
This commit is contained in:
+17
-9
@@ -33,8 +33,12 @@
|
||||
// random bytes: every byte value 0..255 must decode identically on both
|
||||
// sides, including values >= 243 that never come out of the packer.
|
||||
//
|
||||
// DT3 is implemented for CUDA and HIP only. Without one of those backends the
|
||||
// test is skipped and succeeds — an unsupported backend is not a failure.
|
||||
// DT3 is implemented for CUDA, HIP and Vulkan. Without one of those backends
|
||||
// the test is skipped and succeeds — an unsupported backend is not a failure.
|
||||
// On Vulkan the n <= 8 path is the scalar mul_mat_vec shader (fp32 dot on
|
||||
// exactly decoded weights, not an integer dot), and the larger-n path is
|
||||
// dequantization to fp16 + the f16 matmul pipeline; both are judged by the
|
||||
// same gates as the CUDA MMVQ/GEMM paths.
|
||||
|
||||
#include "ggml.h"
|
||||
#include "ggml-alloc.h"
|
||||
@@ -461,21 +465,25 @@ static void build_control_data(ggml_type type, std::vector<uint8_t> & data, std:
|
||||
}
|
||||
|
||||
int main(void) {
|
||||
// Only CUDA and HIP (which reports itself as "ROCm") implement DT3. Any
|
||||
// other GPU backend is skipped rather than failed: Vulkan and SYCL answer
|
||||
// supports_op == false for DT3, which is the correct answer for them and
|
||||
// Only CUDA, HIP (which reports itself as "ROCm") and Vulkan implement
|
||||
// DT3. Any other GPU backend is skipped rather than failed: SYCL answers
|
||||
// supports_op == false for DT3, which is the correct answer for it and
|
||||
// not a bug to report, and Metal answers true for almost any type but has
|
||||
// no DT3 shader, so it would die in pipeline compilation mid-test. Picking
|
||||
// the backend by name keeps this test honest on machines we do not have.
|
||||
ggml_backend_t backend = nullptr;
|
||||
for (size_t i = 0; i < ggml_backend_dev_count(); ++i) {
|
||||
ggml_backend_dev_t dev = ggml_backend_dev_get(i);
|
||||
if (ggml_backend_dev_type(dev) != GGML_BACKEND_DEVICE_TYPE_GPU) {
|
||||
// IGPU is a distinct device type from GPU: an integrated Vulkan device
|
||||
// with unified memory reports as IGPU, and accepting only GPU silently
|
||||
// skipped the very hardware this backend is for.
|
||||
const auto dt = ggml_backend_dev_type(dev);
|
||||
if (dt != GGML_BACKEND_DEVICE_TYPE_GPU && dt != GGML_BACKEND_DEVICE_TYPE_IGPU) {
|
||||
continue;
|
||||
}
|
||||
const char * name = ggml_backend_dev_name(dev);
|
||||
if (strncmp(name, "CUDA", 4) != 0 && strncmp(name, "ROCm", 4) != 0) {
|
||||
printf("skipping GPU backend %s: DT3 is only implemented for CUDA/HIP\n", name);
|
||||
if (strncmp(name, "CUDA", 4) != 0 && strncmp(name, "ROCm", 4) != 0 && strncmp(name, "Vulkan", 6) != 0) {
|
||||
printf("skipping GPU backend %s: DT3 is only implemented for CUDA/HIP/Vulkan\n", name);
|
||||
continue;
|
||||
}
|
||||
backend = ggml_backend_dev_init(dev, nullptr);
|
||||
@@ -483,7 +491,7 @@ int main(void) {
|
||||
break;
|
||||
}
|
||||
if (backend == nullptr) {
|
||||
printf("no CUDA/HIP backend available, skipping\n");
|
||||
printf("no CUDA/HIP/Vulkan backend available, skipping\n");
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user