386176899e
Vision tools removed: - extract_text (OCR from image files) - extract_text_with_boxes (OCR with bounding boxes) - vision_find_text (find text in app windows) - vision_click_text (find and click on text) - vision_click_near_text (click near text labels) macax tools removed: - macax_list_apps - macax_get_frontmost_app - macax_activate_app - macax_press_key - macax_type_text The LLM can now read images directly via read_image tool. take_screenshot is retained for capturing application windows. Files deleted: - crates/g3-core/src/tools/vision.rs - crates/g3-core/src/tools/macax.rs - docs/macax-tools.md Updated tool counts: 12 core + 15 webdriver = 27 total
54 lines
1.9 KiB
Rust
54 lines
1.9 KiB
Rust
//! Tool executor trait and context for tool execution.
|
|
|
|
use anyhow::Result;
|
|
use std::sync::Arc;
|
|
use tokio::sync::RwLock;
|
|
|
|
use crate::background_process::BackgroundProcessManager;
|
|
use crate::paths::{ensure_session_dir, get_session_todo_path, get_todo_path};
|
|
use crate::ui_writer::UiWriter;
|
|
use crate::webdriver_session::WebDriverSession;
|
|
use crate::ToolCall;
|
|
use g3_config::Config;
|
|
|
|
/// Context passed to tool executors containing shared state.
|
|
pub struct ToolContext<'a, W: UiWriter> {
|
|
pub config: &'a Config,
|
|
pub ui_writer: &'a W,
|
|
pub session_id: Option<&'a str>,
|
|
pub working_dir: Option<&'a str>,
|
|
pub computer_controller: Option<&'a Box<dyn g3_computer_control::ComputerController>>,
|
|
pub webdriver_session: &'a Arc<RwLock<Option<Arc<tokio::sync::Mutex<WebDriverSession>>>>>,
|
|
pub webdriver_process: &'a Arc<RwLock<Option<tokio::process::Child>>>,
|
|
pub background_process_manager: &'a Arc<BackgroundProcessManager>,
|
|
pub todo_content: &'a Arc<RwLock<String>>,
|
|
pub pending_images: &'a mut Vec<g3_providers::ImageContent>,
|
|
pub is_autonomous: bool,
|
|
pub requirements_sha: Option<&'a str>,
|
|
}
|
|
|
|
impl<'a, W: UiWriter> ToolContext<'a, W> {
|
|
/// Get the path to the TODO file (session-scoped or workspace).
|
|
pub fn get_todo_path(&self) -> std::path::PathBuf {
|
|
if let Some(session_id) = self.session_id {
|
|
let _ = ensure_session_dir(session_id);
|
|
get_session_todo_path(session_id)
|
|
} else {
|
|
get_todo_path()
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Trait for tool executors.
|
|
/// Each tool category implements this trait.
|
|
pub trait ToolExecutor<W: UiWriter> {
|
|
/// Execute a tool call and return the result.
|
|
/// Returns None if this executor doesn't handle the given tool.
|
|
fn execute<'a>(
|
|
tool_call: &'a ToolCall,
|
|
ctx: &'a mut ToolContext<'_, W>,
|
|
) -> impl std::future::Future<Output = Option<Result<String>>> + Send + 'a
|
|
where
|
|
W: 'a;
|
|
}
|