Fix compaction bug: use User role for summary to maintain alternation

The previous implementation added the summary as a System message, which
caused "Conversation must start with a user message" errors because the
first non-system message after compaction was Assistant (the preserved
last assistant message).

Fix: Change summary from System to User message, creating valid alternation:
[System Prompt] -> [Summary as USER] -> [Last Assistant] -> [Latest User]

This also prevents system message bloat across multiple compactions since
the summary is now part of the conversation flow and gets replaced on
each compaction.

Added test_second_compaction_no_bloat to verify no accumulation.
This commit is contained in:
Dhanji R. Prasanna
2026-01-26 15:24:04 +11:00
parent 712eca1904
commit 9de8e8cc76
3 changed files with 190 additions and 17 deletions

View File

@@ -292,9 +292,11 @@ Format this as a detailed but concise summary that can be used to resume the con
self.add_message(Message::new(MessageRole::System, stub_content));
}
// Add the summary
// Add the summary as a USER message (not System) to maintain proper alternation.
// This allows: [Summary as User] -> [Last Assistant] -> [Latest User]
// which is valid User/Assistant alternation.
self.add_message(Message::new(
MessageRole::System,
MessageRole::User,
format!("Previous conversation summary:\n\n{}", summary),
));