millaguie/llama.cpp
1
0 Fork 0
Code Issues Pull Requests Actions 34 Packages Projects Releases Wiki Activity

model : add GroveMoE support (#15510)

Browse Source 
* add GroveMoE support

* remove constexpr that fails on certain compilers

* revert crude scalar div implementation, use cast

* build_attn_inp_kv_unified -> build_attn_inp_kv

* fix build_attn

* re-apply ffn_exps regex changes
This commit is contained in:
Sigbjørn Skjæret authored and GitHub committed 2025-09-25 19:50:28 +02:00
1 parent b05a9d650f
commit 835b2b915c
11 files changed
+451 -4
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

common/common.h
+1 -1
View File
@@ -738,7 +738,7 @@ const char * const LLM_KV_SPLIT_TENSORS_COUNT = "split.tensors.count";
// MoE utils
//
const char * const LLM_FFN_EXPS_REGEX = "\\.ffn_(up|down|gate)_exps";
const char * const LLM_FFN_EXPS_REGEX = "\\.ffn_(up|down|gate)_(ch|)exps";
static std::string llm_ffn_exps_block_regex(int idx) {
return string_format("blk\\.%d%s", idx, LLM_FFN_EXPS_REGEX);