Give two-session Qwen serving a bounded, opt-in decode grace for large cold prompts. Returning to normal interleaving at the deadline bounds the wait when the running reply takes longer than expected. Bead: nix-overlay-qfi
This commit is contained in:
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user