server: refactor batch construction (#24843)

Browse Source

* server: refactor batch construction

* wip

* wip 2

* wip 3

* wip 4

* add abort_all_slots

* handle batch full more carefully

* fix assert

* rm debug log

* small nits

* (debug) add timings

* debug: force llama_synchronize for accurate timings

* address comments

* disable DEBUG_TIMINGS

...

This commit is contained in:

Xuan-Son Nguyen authored and GitHub committed 2026-06-21 14:16:11 +02:00

1 parent 0d135df48c

commit bddfd2b113

1 file changed

+583 -351

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

tools/server/server-context.cpp

+583 -351

File diff suppressed because it is too large. Load diff