llama : register the DT3 file type
Adds LLAMA_FTYPE_MOSTLY_DT3 at the end of the ftype enum, the loader name/guess mappings, the quantization fallbacks (same as the other ternary types), and the llama-quantize table entry. The table entry warns that the in-tree quantizer is only the reference one: DT3 models with the measured quality are produced by the external PTQTP pipeline.
This commit is contained in:
@@ -47,6 +47,7 @@ static const std::vector<quant_option> QUANT_OPTIONS = {
|
||||
{ "IQ1_M", LLAMA_FTYPE_MOSTLY_IQ1_M, " 1.75 bpw quantization", },
|
||||
{ "TQ1_0", LLAMA_FTYPE_MOSTLY_TQ1_0, " 1.69 bpw ternarization", },
|
||||
{ "TQ2_0", LLAMA_FTYPE_MOSTLY_TQ2_0, " 2.06 bpw ternarization", },
|
||||
{ "DT3", LLAMA_FTYPE_MOSTLY_DT3, " 3.5 bpw dual-ternary (REF quantizer only - use ternaria's PTQTP for quality)", },
|
||||
{ "Q2_K", LLAMA_FTYPE_MOSTLY_Q2_K, " 2.96G, +3.5199 ppl @ Llama-3-8B", },
|
||||
{ "Q2_K_S", LLAMA_FTYPE_MOSTLY_Q2_K_S, " 2.96G, +3.1836 ppl @ Llama-3-8B", },
|
||||
{ "IQ3_XXS", LLAMA_FTYPE_MOSTLY_IQ3_XXS, " 3.06 bpw quantization", },
|
||||
|
||||
Reference in New Issue
Block a user