millaguie/llama.cpp
1
0
Fork 0
Code Issues Pull Requests Actions 32 Packages Projects Releases Wiki Activity

tests : skip test-dt3-gpu on backends that do not implement DT3

Browse Source 
The test picked the first GPU device it found and treated an unsupported
op as a failure. Vulkan and SYCL answer supports_op == false for DT3,
which is the right answer for them and not a bug to report, so the test
went red on machines that were behaving correctly. Metal is worse: it
answers true for almost any type but has no DT3 shader, so the run died
in pipeline compilation halfway through.

Pick the backend by name instead — CUDA and HIP (which reports itself as
ROCm) are the only ones implementing DT3 — and skip everything else. A
supports_op failure on those two is still a real failure.

Also document that the n <= 8 gating mirrors MMVQ_MAX_BATCH_SIZE by hand
and goes stale silently if the MMVQ dispatch changes.
This commit is contained in:
Millaguie
2026-08-10 23:09:23 +02:00
parent 3c8fd43e7f
commit 470840e01e
1 changed files with 26 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test-dt3-gpu.cpp
+26 -6
View File
@@ -33,7 +33,8 @@
// random bytes: every byte value 0..255 must decode identically on both
// sides, including values >= 243 that never come out of the packer.
//
// Without a GPU backend the test is skipped and succeeds.
// DT3 is implemented for CUDA and HIP only. Without one of those backends the
// test is skipped and succeeds — an unsupported backend is not a failure.
#include "ggml.h"
#include "ggml-alloc.h"
@@ -324,6 +325,13 @@ static int test_mul_mat(ggml_backend_t backend, ggml_type type, const std::vecto
// hardware and on GGML_CUDA_CUBLAS_COMPUTE_TYPE) are cuBLAS's, not
// ours: for the strict type it is gated below by bit-identity with
// the same GEMM on an F16 tensor, and only reported here.
// This mirrors MMVQ_MAX_BATCH_SIZE (8) from ggml-cuda/mmvq.cu by hand,
// because the constant and the per-arch should_use_mmvq tables are not
// exported. If upstream raises the limit, or an architecture routes a
// larger batch through MMVQ, this gating goes stale silently: n = 16
// would take the MMVQ path but still be judged as the GEMM one, which
// only loosens the check, never tightens it. Whoever touches the MMVQ
// dispatch should revisit this line.
const bool is_mmvq = n <= 8;
const bool gated = is_mmvq || !strict;
const double err_gate = is_mmvq ? err.norm_rel : (err.norm_rel < err16.norm_rel ? err.norm_rel : err16.norm_rel);
@@ -453,17 +461,29 @@ static void build_control_data(ggml_type type, std::vector<uint8_t> & data, std:
}
int main(void) {
// Only CUDA and HIP (which reports itself as "ROCm") implement DT3. Any
// other GPU backend is skipped rather than failed: Vulkan and SYCL answer
// supports_op == false for DT3, which is the correct answer for them and
// not a bug to report, and Metal answers true for almost any type but has
// no DT3 shader, so it would die in pipeline compilation mid-test. Picking
// the backend by name keeps this test honest on machines we do not have.
ggml_backend_t backend = nullptr;
for (size_t i = 0; i < ggml_backend_dev_count(); ++i) {
ggml_backend_dev_t dev = ggml_backend_dev_get(i);
if (ggml_backend_dev_type(dev) == GGML_BACKEND_DEVICE_TYPE_GPU) {
backend = ggml_backend_dev_init(dev, nullptr);
printf("using GPU backend: %s\n", ggml_backend_dev_name(dev));
break;
if (ggml_backend_dev_type(dev) != GGML_BACKEND_DEVICE_TYPE_GPU) {
continue;
}
const char * name = ggml_backend_dev_name(dev);
if (strncmp(name, "CUDA", 4) != 0 && strncmp(name, "ROCm", 4) != 0) {
printf("skipping GPU backend %s: DT3 is only implemented for CUDA/HIP\n", name);
continue;
}
backend = ggml_backend_dev_init(dev, nullptr);
printf("using GPU backend: %s\n", name);
break;
}
if (backend == nullptr) {
printf("no GPU backend available, skipping\n");
printf("no CUDA/HIP backend available, skipping\n");
return 0;
}