g3/DESIGN.md

# G3 - AI Coding Agent - Design Document

## Overview

G3 is a **modular, composable AI coding agent** built in Rust that helps you complete tasks by writing and executing code. It provides a flexible architecture for interacting with various Large Language Model (LLM) providers while offering powerful code generation, file manipulation, and task automation capabilities.

The agent follows a **tool-first philosophy**: instead of just providing advice, G3 actively uses tools to read files, write code, execute commands, and complete tasks autonomously.

## Core Principles

1. **Tool-First Philosophy**: Solve problems by actively using tools rather than just providing advice
2. **Modular Architecture**: Clear separation of concerns across multiple Rust crates
3. **Provider Flexibility**: Support multiple LLM providers through a unified interface
4. **Modularity**: Clear separation of concerns
5. **Composability**: Components can be combined in different ways
6. **Performance**: Built in Rust for speed and reliability
7. **Context Intelligence**: Smart context window management with auto-summarization
8. **Error Resilience**: Robust error handling with automatic retry logic

## Project Structure

G3 is organized as a Rust workspace with the following crates:

```
g3/
├── src/main.rs                   # Main entry point (delegates to g3-cli)
├── crates/
│   ├── g3-cli/                   # Command-line interface, TUI, and retro mode
│   ├── g3-core/                  # Core agent engine, tools, and streaming logic
│   ├── g3-providers/             # LLM provider abstractions and implementations
│   ├── g3-config/                # Configuration management
│   └── g3-execution/             # Code execution engine
├── logs/                         # Session logs (auto-created)
├── README.md                     # Project documentation
└── DESIGN.md                     # This design document
```

## Architecture Overview

### High-Level Architecture

```
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   g3-cli        │    │   g3-core       │    │ g3-providers    │
│                 │    │                 │    │                 │
│ • CLI parsing   │◄──►│ • Agent engine  │◄──►│ • Anthropic     │
│ • Interactive   │    │ • Context mgmt  │    │ • Databricks    │
│ • Retro TUI     │    │ • Tool system   │    │ • Embedded      │
│ • Autonomous    │    │ • Streaming     │    │   (llama.cpp)   │
│   mode          │    │ • Task exec     │    │ • OAuth flow    │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         └───────────────────────┼───────────────────────┘
                                 │
                    ┌─────────────────┐    ┌─────────────────┐
                    │ g3-execution    │    │   g3-config     │
                    │                 │    │                 │
                    │ • Code exec     │    │ • TOML config   │
                    │ • Shell cmds    │    │ • Env overrides │
                    │ • Streaming     │    │ • Provider      │
                    │ • Error hdlg    │    │   settings      │
                    └─────────────────┘    └─────────────────┘
```

## Core Components

### 1. g3-core: Agent Engine

**Primary Responsibilities:**
- Main orchestration logic for handling conversations and task execution
- Context window management with intelligent token tracking
- Built-in tool system for file operations and command execution
- Streaming response parsing with real-time tool call detection
- Error handling with automatic retry logic

**Key Features:**
- **Context Window Intelligence**: Automatic monitoring with percentage-based tracking (80% capacity triggers auto-summarization)
- **Tool System**: Built-in tools for file operations (read, write, edit), shell commands, and structured output
- **Streaming Parser**: Real-time parsing of LLM responses with tool call detection and execution
- **Session Management**: Automatic session logging with detailed conversation history and token usage
- **Error Recovery**: Sophisticated error classification and retry logic for recoverable errors

**Available Tools:**
- `shell`: Execute shell commands with streaming output
- `read_file`: Read file contents with optional character range support
- `write_file`: Create or overwrite files with content
- `str_replace`: Apply unified diffs to files with precise editing
- `final_output`: Signal task completion with detailed summaries
- **Project Management**: Workspace handling, requirements.md processing for autonomous mode

### 2. g3-providers: LLM Provider Abstraction

**Primary Responsibilities:**
- Unified interface for multiple LLM providers
- Provider-specific optimizations and feature support
- OAuth authentication flows
- Streaming and non-streaming completion support

**Supported Providers:**
- **Anthropic**: Claude models via API with native tool calling support
- **Databricks**: Foundation Model APIs with OAuth and token-based authentication (default provider)
- **Embedded**: Local models via llama.cpp with GPU acceleration (Metal/CUDA)
- **Provider Registry**: Dynamic provider management and hot-swapping

**Key Features:**
- **Native Tool Calling**: Full support for structured tool calls where available
- **Fallback Parsing**: JSON tool call parsing for providers without native support
- **OAuth Integration**: Built-in OAuth flow for secure provider authentication
- **Context-Aware**: Provider-specific context length and token limit handling
- **Streaming Support**: Real-time response streaming with tool call detection

### 3. g3-cli: Command-Line Interface

**Primary Responsibilities:**
- Command-line argument parsing and validation
- Interactive terminal interface with history support
- Retro-style terminal UI (80s sci-fi inspired)
- Autonomous mode with coach-player feedback loops
- Session management and workspace handling

**Execution Modes:**
- **Single-shot**: Execute one task and exit
- **Interactive**: REPL-style conversation with the agent (default mode)
- **Autonomous**: Coach-player feedback loop for complex projects
- **Retro TUI**: Full-screen terminal interface with real-time updates

**Key Features:**
- **Multi-line Input**: Support for complex, multi-line prompts with backslash continuation
- **Context Progress**: Real-time display of token usage and context window status
- **Error Recovery**: Automatic retry logic for timeout and recoverable errors
- **History Management**: Persistent command history across sessions
- **Theme Support**: Customizable color themes for retro mode
- **Cancellation**: Ctrl+C support for graceful operation cancellation

### 4. g3-execution: Code Execution Engine

**Primary Responsibilities:**
- Safe execution of shell commands and scripts
- Streaming output capture and display
- Multi-language code execution support
- Error handling and result formatting

**Supported Execution:**
- **Bash/Shell**: Direct command execution with streaming output (primary use case)
- **Python**: Script execution via temporary files (legacy support)
- **JavaScript**: Node.js-based execution (legacy support)

**Key Features:**
- **Streaming Output**: Real-time command output display
- **Error Capture**: Comprehensive stderr and stdout handling
- **Exit Code Tracking**: Proper success/failure detection
- **Async Execution**: Non-blocking command execution
- **Output Formatting**: Clean, user-friendly result presentation

### 5. g3-config: Configuration Management

**Primary Responsibilities:**
- TOML-based configuration file management
- Environment variable overrides
- Provider-specific settings and credentials
- CLI argument integration

**Configuration Hierarchy:**
1. Default configuration (Databricks provider with OAuth)
2. Configuration files (`~/.config/g3/config.toml`, `./g3.toml`)
3. Environment variables (`G3_*`)
4. CLI arguments (highest priority)

**Key Features:**
- **Auto-generation**: Creates default configuration files if none exist
- **Provider Overrides**: Runtime provider and model selection
- **Validation**: Configuration validation with helpful error messages
- **Flexible Paths**: Support for shell expansion (`~`, environment variables)

## Advanced Features

### Context Window Management

G3 implements sophisticated context window management:

- **Automatic Monitoring**: Tracks token usage with percentage-based thresholds
- **Smart Summarization**: Auto-triggers at 80% capacity to prevent context overflow
- **Conversation Preservation**: Maintains conversation continuity through intelligent summaries
- **Provider-Specific Limits**: Adapts to different model context windows (4k to 200k+ tokens)
- **Cumulative Tracking**: Monitors total token usage across entire sessions

### Error Handling & Resilience

Comprehensive error handling system:

- **Error Classification**: Distinguishes between recoverable and non-recoverable errors
- **Automatic Retry**: Exponential backoff with jitter for rate limits, timeouts, and server errors
- **Detailed Logging**: Comprehensive error context including stack traces and session data
- **Error Persistence**: Saves detailed error logs to `logs/errors/` for analysis
- **Graceful Degradation**: Continues operation when possible, fails gracefully when not

### Session Management

Automatic session tracking and logging:

- **Session IDs**: Generated based on initial prompts for easy identification
- **Complete Logs**: Full conversation history, token usage, and timing data
- **JSON Format**: Structured logs for easy parsing and analysis
- **Automatic Cleanup**: Organized in `logs/` directory with timestamps
- **Status Tracking**: Records session completion status (completed, cancelled, error)

### Autonomous Mode

Advanced autonomous operation with coach-player feedback:

- **Requirements-Driven**: Reads `requirements.md` for project specifications
- **Dual-Agent System**: Separate player (implementation) and coach (review) agents
- **Iterative Improvement**: Multiple rounds of implementation and feedback
- **Progress Tracking**: Detailed reporting of turns, token usage, and final status
- **Workspace Management**: Automatic workspace setup and file organization

## Provider Comparison

| Feature | Anthropic | Databricks (Default) | Embedded |
|---------|-----------|------------|----------|
| **Cost** | Pay per token | Pay per token | Free after download |
| **Privacy** | Data sent to API | Data sent to API | Completely local |
| **Performance** | Very fast | Very fast | Depends on hardware |
| **Model Quality** | Excellent | Excellent | Good (varies by model) |
| **Offline Support** | No | No | Yes |
| **Setup Complexity** | API key only | OAuth or token | Model download required |
| **Context Window** | 200k tokens | Varies by model | 4k-32k tokens |
| **Tool Calling** | Native support | Native support | JSON fallback |
| **Hardware Requirements** | None | None | 4-16GB RAM, optional GPU |

## Configuration Examples

### Cloud-First Setup (Anthropic)
```toml
[providers]
default_provider = "anthropic"

[providers.anthropic]
api_key = "sk-ant-..."
model = "claude-3-5-sonnet-20241022"
max_tokens = 8192
temperature = 0.1
```

### Enterprise Setup (Databricks - Default)
```toml
[providers]
default_provider = "databricks"

[providers.databricks]
host = "https://your-workspace.cloud.databricks.com"
model = "databricks-claude-sonnet-4"
max_tokens = 32000
temperature = 0.1
use_oauth = true
```

### Privacy-First Setup (Local Models)
```toml
[providers]
default_provider = "embedded"

[providers.embedded]
model_path = "~/.cache/g3/models/qwen2.5-7b-instruct-q3_k_m.gguf"
model_type = "qwen"
context_length = 32768
max_tokens = 2048
temperature = 0.1
gpu_layers = 32
threads = 8
```

### Hybrid Setup
```toml
[providers]
default_provider = "embedded"

# Local model for most tasks
[providers.embedded]
model_path = "~/.cache/g3/models/codellama-7b-instruct.Q4_K_M.gguf"
model_type = "codellama"
context_length = 16384
gpu_layers = 32

# Cloud fallback for complex tasks
[providers.anthropic]
api_key = "sk-ant-..."
model = "claude-3-5-sonnet-20241022"
```

## Usage Examples

### Single-Shot Mode
```bash
g3 "implement a fibonacci function in Rust"
```

### Interactive Mode
```bash
g3
g3> read the README and suggest improvements
g3> implement the suggestions you made
```

### Autonomous Mode
```bash
g3 --autonomous --max-turns 10
# Reads requirements.md and implements iteratively
```

### Retro TUI Mode
```bash
g3 --retro --theme dracula
# Full-screen terminal interface
```

## Implementation Details

### Planned Features
- **Plugin System**: Custom tool and provider plugins
- **Web Interface**: Browser-based UI for remote access
- **Model Quantization**: Optimized local model deployment
- **Multi-Model Ensemble**: Combine multiple models for better results
- **Advanced Sandboxing**: Enhanced security for code execution
- **Collaborative Mode**: Multi-user sessions and shared workspaces

### Technical Improvements
- **Performance Optimization**: Faster streaming and tool execution
- **Memory Management**: Better handling of large contexts and files
- **Caching System**: Intelligent caching of model responses and computations
- **Monitoring**: Built-in metrics and performance monitoring
- **Testing**: Comprehensive test suite and CI/CD integration

## Development Guidelines

### Code Organization
- **Modular Design**: Each crate has a single, well-defined responsibility
- **Trait-Based**: Use traits for abstraction and testability
- **Error Handling**: Comprehensive error types with context
- **Documentation**: Inline docs and examples for all public APIs
- **Testing**: Unit tests, integration tests, and property-based testing

### Performance Considerations
- **Async-First**: All I/O operations are asynchronous (Tokio runtime)
- **Streaming**: Real-time response processing where possible
- **Memory Efficiency**: Careful memory management for large contexts
- **Caching**: Strategic caching of expensive operations
- **Profiling**: Regular performance profiling and optimization

This design document reflects the current state of G3 as a mature, production-ready AI coding agent with sophisticated architecture and comprehensive feature set.

## Current Implementation Status

### Fully Implemented
- ✅ **Core Agent Engine**: Complete with streaming, tool execution, and context management
- ✅ **Provider System**: Anthropic, Databricks, and Embedded providers with OAuth support
- ✅ **Tool System**: All 5 core tools (shell, read_file, write_file, str_replace, final_output)
- ✅ **CLI Interface**: Interactive mode, single-shot mode, retro TUI
- ✅ **Autonomous Mode**: Coach-player feedback loop with requirements.md processing
- ✅ **Configuration**: TOML-based config with environment overrides
- ✅ **Error Handling**: Comprehensive retry logic and error classification
- ✅ **Session Logging**: Automatic session tracking and JSON logs
- ✅ **Context Management**: Auto-summarization at 80% capacity

### Architecture Highlights
- **Workspace**: 5 crates with clear separation of concerns
- **Dependencies**: Modern Rust ecosystem (Tokio, Clap, Serde, etc.)
- **Streaming**: Real-time response processing with tool call detection
- **Cross-Platform**: Works on macOS, Linux, and Windows
- **GPU Support**: Metal acceleration for local models on macOS

### Key Files
- `src/main.rs`: main entry point delegating to g3-cli
- `crates/g3-core/src/lib.rs`: main agent implementation
- `crates/g3-cli/src/lib.rs`: CLI and interaction modes
- `crates/g3-providers/src/lib.rs`: provider trait and registry
- `crates/g3-config/src/lib.rs`: configuration management
- `crates/g3-execution/src/lib.rs`: code execution engine