Commit Graph

15 Commits

Author SHA1 Message Date
Dhanji R. Prasanna
9754c4ee66 Fix code fence closing without trailing newline
When a code block ended without a trailing newline after the closing
\`\`\`, two bugs occurred in flush_incomplete():

1. The closing \`\`\` was included as part of the code block content
   (displayed with syntax highlighting)
2. The same \`\`\` was then emitted again as literal text because
   current_line was not cleared after being pushed to block_buffer

The fix:
- Check if current_line is the closing fence before adding to block_buffer
- Always clear current_line after processing in the CodeBlock case

Added two tests:
- test_code_fence_after_blank_line: code fence with trailing newline
- test_code_fence_no_trailing_newline: code fence without trailing newline
2026-01-11 19:34:46 +05:30
Dhanji R. Prasanna
2fbdac7aa9 Fix extra newlines before tool calls in JSON filter
The JSON tool call filter was outputting newlines immediately as they
were encountered. When the LLM output contained multiple newlines before
a tool call, each newline was output before the tool call JSON was
detected and suppressed, leaving orphaned blank lines in the output.

Changes:
- Add pending_newlines field to FilterState to buffer newlines at line start
- First newline after content is output immediately, subsequent ones buffered
- When tool call confirmed, pending_newlines cleared (suppressing extra blanks)
- When not a tool call, pending_newlines output with the buffer
- Add flush_json_tool_filter() to flush pending content at end of streaming
- Update tests to reflect new behavior
- Add tests for newline suppression behavior
2026-01-11 17:04:27 +05:30
Dhanji R. Prasanna
39918cf281 fix: process bold/italic/code formatting inside markdown headers
The format_header() function was not calling format_inline_content()
to process inline formatting like **bold**, *italic*, and `code`
within headers. This caused raw markdown markers to appear in output.

Added 4 tests to verify the fix:
- test_bold_inside_header
- test_italic_inside_header
- test_code_inside_header
- test_mixed_formatting_inside_header
2026-01-11 08:00:34 +08:00
Dhanji R. Prasanna
e731bc8217 Make remember tool instructions more imperative in system prompts
- Change 'call remember' to 'you MUST call remember' in native prompt
- Change 'IF you discovered' to 'ALWAYS...when you discovered'
- Add explicit list of trigger tools (code_search, rg, grep, find, read_file)
- Add reminder to Response Guidelines section
- Add remember tool and Project Memory section to non-native prompt
- Remove redundant console output from remember tool
- Fix test compilation errors (missing summary parameter, temporary borrow)
2026-01-11 06:49:45 +08:00
Dhanji R. Prasanna
777191b3cb Remove final_output tool - let summaries stream naturally
- Remove final_output from tool definitions, dispatch, and misc tools
- Update system prompts to request summaries as regular markdown text
- Remove print_final_output from UiWriter trait and all implementations
- Remove final_output handling from agent core logic
- Rename final_output_summary → summary in session continuation
- Delete final_output test files
- Update tool count tests (12→11, 27→26)

This allows LLM summaries to stream through the markdown formatter
for a more natural, responsive user experience instead of buffering
everything into a tool call.
2026-01-09 14:57:24 +11:00
Dhanji R. Prasanna
d96d8c1d90 Rewrite JSON tool call filter with clean state machine
Fixes bug where JSON tool calls were printed as text due to chunking issues.

Changes:
- Complete rewrite of filter_json.rs with 3-state machine:
  - Streaming: normal pass-through, watches for newline + whitespace + {
  - Buffering: confirms/denies tool pattern with ~20 char buffer
  - Suppressing: string-aware brace counting until balanced
- Character-by-character processing eliminates chunk boundary issues
- Proper handling of } inside JSON strings (was causing premature exit)
- Detects truncated JSON followed by complete JSON (LLM retry case)
- Removed regex dependency, simpler pattern matching
- Added 59 stress tests covering malformed JSON, partial patterns,
  streaming edge cases, adversarial inputs, and real-world patterns

All 86 filter_json tests pass.
2026-01-09 14:05:11 +11:00
Dhanji R. Prasanna
a72d5a650a Fix two markdown formatting bugs
Bug 1: Inline code after list bullets not detected
- After emitting a list bullet, at_line_start was not set to false
- This caused the next backtick to be treated as a potential code fence
- Fixed by setting at_line_start = false after emitting bullet

Bug 2: Code block closing on indented backticks
- Code blocks containing indented ``` (4+ spaces) were closing prematurely
- The .trim() check was too permissive
- Fixed by only allowing closing fence with <= 3 spaces indent (CommonMark spec)

Added tests for both edge cases.
2026-01-08 20:50:26 +11:00
Dhanji R. Prasanna
19a804e0be Add syntax highlighting for Racket, Elisp, and Scheme
Add language alias mapping in highlight_code() to map:
- racket, rkt -> lisp
- elisp, emacs-lisp -> lisp
- scheme -> lisp
- common-lisp, cl -> lisp
- shell, sh, zsh, dockerfile -> bash

Syntect's built-in Lisp syntax handles all Lisp-family languages well.
Added test to verify the aliases work correctly.
2026-01-08 20:35:34 +11:00
Dhanji R. Prasanna
df706308ca Unify final_output rendering with streaming markdown formatter
Replace the separate syntax_highlight module with the streaming markdown
formatter for final_output rendering. This:

- Removes special buffered rendering logic for final_output
- Uses the same StreamingMarkdownFormatter used for agent responses
- Removes the spinner animation (content renders immediately)
- Deletes the now-unused syntax_highlight.rs module
- Updates test to use the streaming formatter

Benefits:
- Consistent rendering across all markdown output
- Less code to maintain (removed ~250 lines)
- Same syntax highlighting via syntect (already in streaming formatter)
2026-01-08 20:30:44 +11:00
Dhanji R. Prasanna
347513b04c Add comprehensive stress tests for streaming markdown formatter
Add 10 stress tests covering:
- Nested formatting (bold in italic, italic in bold)
- Empty/minimal content edge cases
- Escape sequences and special characters
- Lists with complex inline formatting
- Links with various content types
- Tables with formatting in cells
- Code blocks (should not format contents)
- Mixed block elements (headers, quotes, rules)
- Nested lists (3+ levels, mixed types)
- Pathological/adversarial inputs (unbalanced delimiters, unicode, long lines)

All 45 tests pass.
2026-01-08 20:27:28 +11:00
Dhanji R. Prasanna
5d20da2609 Add 54 integration tests for CLI, tools, and message serialization
New test files:
- crates/g3-cli/tests/cli_integration_test.rs (14 tests)
  Blackbox CLI tests: help/version flags, argument validation,
  conflicting modes, flock mode requirements

- crates/g3-core/tests/tool_execution_test.rs (20 tests)
  Tool call structure tests and unified diff application:
  read_file, write_file, str_replace, shell, background_process,
  todo, final_output, code_search, take_screenshot

- crates/g3-providers/tests/message_serialization_test.rs (20 tests)
  Round-trip serialization tests for Message, MessageRole,
  CacheControl, and Tool types. Covers Unicode, special chars,
  and edge cases.

All tests follow blackbox/integration-first principles with
documentation of what they protect and intentionally do not assert.
2026-01-07 09:23:34 +11:00
Dhanji R. Prasanna
38fcaaf449 Add edge case tests for filter_json_tool_calls
- test_brace_inside_json_string_value: braces inside JSON strings
- test_multiple_braces_in_string: multiple braces in string values
- test_escaped_quotes_with_braces: escaped quotes with braces
- test_brace_in_string_across_chunks: streaming with braces in strings
- test_complex_nested_with_string_braces: nested JSON with string braces
- test_str_replace_with_diff_content: real-world str_replace case
- test_tool_call_after_other_content: tool call after other output
- test_tool_call_with_nested_tool_pattern_in_string: nested patterns

All 27 tests pass.
2025-12-22 13:30:57 +11:00
Dhanji R. Prasanna
3bc254962c clean up filter_json a bit (more to come) 2025-12-22 12:03:09 +11:00
Dhanji R. Prasanna
01a5284d6d Move fixed_filter_json from g3-core to g3-cli
Properly separates UI display concern from core library:
- fixed_filter_json module now lives in g3-cli (UI layer)
- UiWriter trait gains filter_json_tool_calls() and reset_json_filter() methods
- g3-core delegates filtering to UI layer via trait methods
- Different UiWriter implementations can choose their own filtering behavior
- ConsoleUiWriter filters JSON tool calls for clean terminal display
- MachineUiWriter/NullUiWriter use default pass-through

Benefits:
- Proper separation of concerns
- Core stays clean without display-specific logic
- Testability - filter can be tested independently in g3-cli
2025-12-22 10:32:21 +11:00
Jochen
0327a6dfdf make sure coach feedback is extracted. 2025-12-02 22:00:58 +11:00