🎨 Modern Web Dashboard
🖱️ Intuitive drag-and-drop 📱 Responsive design 🎯 Visual progress tracking
*Beautiful web interface with streamlined workflow for all skill levels*
---
### 🎬 **Introduction Video**
*🎯 **Watch our complete introduction** - See how DeepCode transforms research papers and natural language into production-ready code*
---
> *"Where AI Agents Transform Ideas into Production-Ready Code"*
---
## 📑 Table of Contents
- [📰 News](#-news)
- [🚀 Key Features](#-key-features)
- [🏗️ Architecture](#️-architecture)
- [📊 Experimental Results](#-experimental-results)
- [🚀 Quick Start](#-quick-start)
- [🤖 nanobot Integration (Feishu Chatbot)](#-nanobot-integration-feishu-chatbot)
- [💡 Examples](#-examples)
- [🎬 Live Demonstrations](#-live-demonstrations)
- [⭐ Star History](#-star-history)
- [📄 License](#-license)
---
## 📰 News
🎉 **[2026-02] nanobot ✖️ DeepCode. Just chat naturally with openclaw/nanobot to handle your coding tasks:**
✦
- [nanobot](https://github.com/HKUDS/nanobot) nanobot now powers your agentic coding & engineering! 🤖💻
- Step away from your laptop — make vibe coding even more vibe! Code directly from your phone! 📱✨
- One-command deploy: `./nanobot/run_nanobot.sh` → **[Setup Guide →](#-nanobot-integration-feishu-chatbot)**
Feishu Bot in Action — Natural language → Full code generation with setup instructions
---
🎉 **[2026-02] New Web UI Experience Upgrade!**
- 🔄 **User-in-Loop Interaction**: Support real-time user interaction during workflows - AI asks clarifying questions directly in the chat
- 💬 **Inline Interaction Design**: Interaction prompts appear naturally within the chat flow for a seamless experience
- 🚀 **One-Click Launch**: Simply run `deepcode` to start the new UI (cross-platform: Windows/macOS/Linux)
- 🔧 **Improved Process Management**: Enhanced service start/stop mechanism with automatic port cleanup
- 📡 **WebSocket Real-time Communication**: Fixed message loss issues, ensuring proper interaction state synchronization
DeepCode New Web UI - Modern React-based Interface
---
🎉 **[2025-10-28] DeepCode Achieves SOTA on PaperBench!**
DeepCode sets new benchmarks on OpenAI's PaperBench Code-Dev across all categories:
- 🏆 **Surpasses Human Experts**: **75.9%** (DeepCode) vs Top Machine Learning PhDs 72.4% (+3.5%).
- 🥇 **Outperforms SOTA Commercial Code Agents**: **84.8%** (DeepCode) vs Leading Commercial Code Agents (+26.1%) (Cursor, Claude Code, and Codex).
- 🔬 **Advances Scientific Coding**: **73.5%** (DeepCode) vs PaperCoder 51.1% (+22.4%).
- 🚀 **Beats LLM Agents**: **73.5%** (DeepCode) vs best LLM frameworks 43.3% (+30.2%).
---
## 🚀 Key Features
🚀 Paper2Code
Automated Implementation of Complex Algorithms
Effortlessly converts complex algorithms from research papers into high-quality, production-ready code, accelerating algorithm reproduction.
🎨 Text2Web
Automated Front-End Web Development
Translates plain textual descriptions into fully functional, visually appealing front-end web code for rapid interface creation.
⚙️ Text2Backend
Automated Back-End Development
Generates efficient, scalable, and feature-rich back-end code from simple text inputs, streamlining server-side development.
---
## 📊 Experimental Results
We evaluate **DeepCode** on the [*PaperBench*](https://openai.com/index/paperbench/) benchmark (released by OpenAI), a rigorous testbed requiring AI agents to independently reproduce 20 ICML 2024 papers from scratch. The benchmark comprises 8,316 gradable components assessed using SimpleJudge with hierarchical weighting.
Our experiments compare DeepCode against four baseline categories: **(1) Human Experts**, **(2) State-of-the-Art Commercial Code Agents**, **(3) Scientific Code Agents**, and **(4) LLM-Based Agents**.
### ① 🧠 Human Expert Performance (Top Machine Learning PhD)
**DeepCode: 75.9% vs. Top Machine Learning PhD: 72.4% (+3.5%)**
DeepCode achieves **75.9%** on the 3-paper human evaluation subset, **surpassing the best-of-3 human expert baseline (72.4%) by +3.5 percentage points**. This demonstrates that our framework not only matches but exceeds expert-level code reproduction capabilities, representing a significant milestone in autonomous scientific software engineering.
### ② 💼 State-of-the-Art Commercial Code Agents
**DeepCode: 84.8% vs. Best Commercial Agent: 58.7% (+26.1%)**
On the 5-paper subset, DeepCode substantially outperforms leading commercial coding tools:
- Cursor: 58.4%
- Claude Code: 58.7%
- Codex: 40.0%
- **DeepCode: 84.8%**
This represents a **+26.1% improvement** over the leading commercial code agent. All commercial agents utilize Claude Sonnet 4.5 or GPT-5 Codex-high, highlighting that **DeepCode's superior architecture**—rather than base model capability—drives this performance gap.
### ③ 🔬 Scientific Code Agents
**DeepCode: 73.5% vs. PaperCoder: 51.1% (+22.4%)**
Compared to PaperCoder (**51.1%**), the state-of-the-art scientific code reproduction framework, DeepCode achieves **73.5%**, demonstrating a **+22.4% relative improvement**. This substantial margin validates our multi-module architecture combining planning, hierarchical task decomposition, code generation, and iterative debugging over simpler pipeline-based approaches.
### ④ 🤖 LLM-Based Agents
**DeepCode: 73.5% vs. Best LLM Agent: 43.3% (+30.2%)**
DeepCode significantly outperforms all tested LLM agents:
- Claude 3.5 Sonnet + IterativeAgent: 27.5%
- o1 + IterativeAgent (36 hours): 42.4%
- o1 BasicAgent: 43.3%
- **DeepCode: 73.5%**
The **+30.2% improvement** over the best-performing LLM agent demonstrates that sophisticated agent scaffolding, rather than extended inference time or larger models, is critical for complex code reproduction tasks.
---
### 🎯 **Autonomous Self-Orchestrating Multi-Agent Architecture**
**The Challenges**:
- 📄 **Implementation Complexity**: Converting academic papers and complex algorithms into working code requires significant technical effort and domain expertise
- 🔬 **Research Bottleneck**: Researchers spend valuable time implementing algorithms instead of focusing on their core research and discovery work
- ⏱️ **Development Delays**: Product teams experience long wait times between concept and testable prototypes, slowing down innovation cycles
- 🔄 **Repetitive Coding**: Developers repeatedly implement similar patterns and functionality instead of building on existing solutions
**DeepCode** addresses these workflow inefficiencies by providing reliable automation for common development tasks, streamlining your development workflow from concept to code.
```mermaid
flowchart LR
A["📄 Research Papers 💬 Text Prompts 🌐 URLs & Document 📎 Files: PDF, DOC, PPTX, TXT, HTML"] --> B["🧠 DeepCode Multi-Agent Engine"]
B --> C["🚀 Algorithm Implementation 🎨 Frontend Development ⚙️ Backend Development"]
style A fill:#ff6b6b,stroke:#c0392b,stroke-width:2px,color:#000
style B fill:#00d4ff,stroke:#0984e3,stroke-width:3px,color:#000
style C fill:#00b894,stroke:#00a085,stroke-width:2px,color:#000
```
---
## 🏗️ Architecture
### 📊 **System Overview**
**DeepCode** is an AI-powered development platform that automates code generation and implementation tasks. Our multi-agent system handles the complexity of translating requirements into functional, well-structured code, allowing you to focus on innovation rather than implementation details.
🎯 **Technical Capabilities**:
🧬 **Research-to-Production Pipeline**
Multi-modal document analysis engine that extracts algorithmic logic and mathematical models from academic papers. Generates optimized implementations with proper data structures while preserving computational complexity characteristics.
🪄 **Natural Language Code Synthesis**
Context-aware code generation using fine-tuned language models trained on curated code repositories. Maintains architectural consistency across modules while supporting multiple programming languages and frameworks.
⚡ **Automated Prototyping Engine**
Intelligent scaffolding system generating complete application structures including database schemas, API endpoints, and frontend components. Uses dependency analysis to ensure scalable architecture from initial generation.
💎 **Quality Assurance Automation**
Integrated static analysis with automated unit test generation and documentation synthesis. Employs AST analysis for code correctness and property-based testing for comprehensive coverage.
🔮 **CodeRAG Integration System**
Advanced retrieval-augmented generation combining semantic vector embeddings with graph-based dependency analysis. Automatically discovers optimal libraries and implementation patterns from large-scale code corpus.
---
### 🔧 **Core Techniques**
- 🧠 **Intelligent Orchestration Agent**: Central decision-making system that coordinates workflow phases and analyzes requirements. Employs dynamic planning algorithms to adapt execution strategies in real-time based on evolving project complexity. Dynamically selects optimal processing strategies for each implementation step.
- 💾 **Efficient Memory Mechanism**: Advanced context engineering system that manages large-scale code contexts efficiently. Implements hierarchical memory structures with intelligent compression for handling complex codebases. This component enables instant retrieval of implementation patterns and maintains semantic coherence across extended development sessions.
- 🔍 **Advanced CodeRAG System**: Global code comprehension engine that analyzes complex inter-dependencies across repositories. Performs cross-codebase relationship mapping to understand architectural patterns from a holistic perspective. This module leverages dependency graphs and semantic analysis to provide globally-aware code recommendations during implementation.
---
### 🤖 **Multi-Agent Architecture of DeepCode**:
- **🎯 Central Orchestrating Agent**: Orchestrates entire workflow execution and makes strategic decisions. Coordinates specialized agents based on input complexity analysis. Implements dynamic task planning and resource allocation algorithms.
- **📝 Intent Understanding Agent**: Performs deep semantic analysis of user requirements to decode complex intentions. Extracts functional specifications and technical constraints through advanced NLP processing. Transforms ambiguous human descriptions into precise, actionable development specifications with structured task decomposition.
- **📄 Document Parsing Agent**: Processes complex technical documents and research papers with advanced parsing capabilities. Extracts algorithms and methodologies using document understanding models. Converts academic concepts into practical implementation specifications through intelligent content analysis.
- **🏗️ Code Planning Agent**: Performs architectural design and technology stack optimization. Dynamic planning for adaptive development roadmaps. Enforces coding standards and generates modular structures through automated design pattern selection.
- **🔍 Code Reference Mining Agent**: Discovers relevant repositories and frameworks through intelligent search algorithms. Analyzes codebases for compatibility and integration potential. Provides recommendations based on similarity metrics and automated dependency analysis.
- **📚 Code Indexing Agent**: Builds comprehensive knowledge graphs of discovered codebases. Maintains semantic relationships between code components. Enables intelligent retrieval and cross-reference capabilities.
- **🧬 Code Generation Agent**: Synthesizes gathered information into executable code implementations. Creates functional interfaces and integrates discovered components. Generates comprehensive test suites and documentation for reproducibility.
---
#### 🛠️ **Implementation Tools Matrix**
**🔧 Powered by MCP (Model Context Protocol)**
DeepCode leverages the **Model Context Protocol (MCP)** standard to seamlessly integrate with various tools and services. This standardized approach ensures reliable communication between AI agents and external systems, enabling powerful automation capabilities.
##### 📡 **MCP Servers & Tools**
| 🛠️ **MCP Server** | 🔧 **Primary Function** | 💡 **Purpose & Capabilities** |
|-------------------|-------------------------|-------------------------------|
| **🔍 brave** | Web Search Engine | Real-time information retrieval via Brave Search API |
| **🌐 bocha-mcp** | Alternative Search | Secondary search option with independent API access |
| **📂 filesystem** | File System Operations | Local file and directory management, read/write operations |
| **🌐 fetch** | Web Content Retrieval | Fetch and extract content from URLs and web resources |
| **📥 github-downloader** | Repository Management | Clone and download GitHub repositories for analysis |
| **📋 file-downloader** | Document Processing | Download and convert files (PDF, DOCX, etc.) to Markdown |
| **⚡ command-executor** | System Commands | Execute bash/shell commands for environment management |
| **🧬 code-implementation** | Code Generation Hub | Comprehensive code reproduction with execution and testing |
| **📚 code-reference-indexer** | Smart Code Search | Intelligent indexing and search of code repositories |
| **📄 document-segmentation** | Smart Document Analysis | Intelligent document segmentation for large papers and technical documents |
##### 🔧 **Legacy Tool Functions** *(for reference)*
| 🛠️ **Function** | 🎯 **Usage Context** |
|-----------------|---------------------|
| **📄 read_code_mem** | Efficient code context retrieval from memory |
| **✍️ write_file** | Direct file content generation and modification |
| **🐍 execute_python** | Python code testing and validation |
| **📁 get_file_structure** | Project structure analysis and organization |
| **⚙️ set_workspace** | Dynamic workspace and environment configuration |
| **📊 get_operation_history** | Process monitoring and operation tracking |
---
🎛️ **Multi-Interface Framework**
RESTful API with CLI and web frontends featuring real-time code streaming, interactive debugging, and extensible plugin architecture for CI/CD integration.
**🚀 Multi-Agent Intelligent Pipeline:**
### 🌟 **Intelligence Processing Flow**
💡 INPUT LAYER
📄 Research Papers • 💬 Natural Language • 🌐 URLs • 📋 Requirements
🎯 CENTRAL ORCHESTRATION
Strategic Decision Making • Workflow Coordination • Agent Management
📝 TEXT ANALYSIS
Requirement Processing
📄 DOCUMENT ANALYSIS
Paper & Spec Processing
📋 REPRODUCTION PLANNING
Deep Paper Analysis • Code Requirements Parsing • Reproduction Strategy Development
⚡ OUTPUT DELIVERY
📦 Complete Codebase • 🧪 Test Suite • 📚 Documentation • 🚀 Deployment Ready
### 🔄 **Process Intelligence Features**
🎯 Adaptive Flow
Dynamic agent selection based on input complexity
🧠 Smart Coordination
Intelligent task distribution and parallel processing
🔍 Context Awareness
Deep understanding through CodeRAG integration
⚡ Quality Assurance
Automated testing and validation throughout
---
## 🚀 Quick Start
### 📋 **Prerequisites**
Before installing DeepCode, ensure you have the following:
| Requirement | Version | Purpose |
|-------------|---------|---------|
| **Python** | 3.9+ | Core runtime |
| **Node.js** | 18+ | New UI frontend |
| **npm** | 8+ | Package management |
```bash
# Check your versions
python --version # Should be 3.9+
node --version # Should be 18+
npm --version # Should be 8+
```
📥 Install Node.js (if not installed)
```bash
# macOS (using Homebrew)
brew install node
# Ubuntu/Debian
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs
# Windows
# Download from https://nodejs.org/
```
### 📦 **Step 1: Installation**
Choose one of the following installation methods:
#### ⚡ **Direct Installation (Recommended)**
```bash
# 🚀 Install DeepCode package directly
pip install deepcode-hku
# 🔑 Download configuration files
curl -O https://raw.githubusercontent.com/HKUDS/DeepCode/main/mcp_agent.config.yaml
curl -O https://raw.githubusercontent.com/HKUDS/DeepCode/main/mcp_agent.secrets.yaml
```
#### 🔧 **Development Installation (From Source)**
📂 Click to expand development installation options
##### 🔥 **Using UV (Recommended for Development)**
```bash
git clone https://github.com/HKUDS/DeepCode.git
cd DeepCode/
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv --python=3.13
source .venv/bin/activate # On Windows: .venv\Scripts\activate
uv pip install -r requirements.txt
# Install frontend dependencies
npm install --prefix new_ui/frontend
```
##### 🐍 **Using Traditional pip**
```bash
git clone https://github.com/HKUDS/DeepCode.git
cd DeepCode/
pip install -r requirements.txt
# Install frontend dependencies
npm install --prefix new_ui/frontend
```
### 🔧 **Step 2: Configuration**
> The following configuration applies to **all installation methods** (pip, UV, source, and Docker).
#### 🔑 API Keys *(required)*
Edit `mcp_agent.secrets.yaml` with your API keys:
```yaml
# At least ONE provider API key is required
openai:
api_key: "your_openai_api_key"
base_url: "https://openrouter.ai/api/v1" # Optional: for OpenRouter or custom endpoints
anthropic:
api_key: "your_anthropic_api_key" # For Claude models
google:
api_key: "your_google_api_key" # For Gemini models
```
#### 🤖 LLM Provider *(optional)*
Edit `mcp_agent.config.yaml` to choose your preferred LLM provider (line ~106):
```yaml
# Options: "google", "anthropic", "openai"
# If not set or unavailable, will automatically fallback to first available provider
llm_provider: "google"
```
#### 🔍 Search API Keys *(optional)*
Configure web search in `mcp_agent.config.yaml`:
```yaml
# For Brave Search (default) — set in brave.env section (line ~28)
brave:
env:
BRAVE_API_KEY: "your_brave_api_key_here"
# For Bocha-MCP (alternative) — set in bocha-mcp.env section (line ~74)
bocha-mcp:
env:
BOCHA_API_KEY: "your_bocha_api_key_here"
```
#### 📄 Document Segmentation *(optional)*
Control document processing in `mcp_agent.config.yaml`:
```yaml
document_segmentation:
enabled: true # true/false — whether to use intelligent document segmentation
size_threshold_chars: 50000 # Document size threshold to trigger segmentation
```
🪟 Windows Users: Additional MCP Server Configuration
If you're using Windows, you may need to configure MCP servers manually in `mcp_agent.config.yaml`:
```bash
# 1. Install MCP servers globally
npm i -g @modelcontextprotocol/server-brave-search
npm i -g @modelcontextprotocol/server-filesystem
# 2. Find your global node_modules path
npm -g root
```
Then update your `mcp_agent.config.yaml` to use absolute paths:
```yaml
mcp:
servers:
brave:
command: "node"
args: ["C:/Program Files/nodejs/node_modules/@modelcontextprotocol/server-brave-search/dist/index.js"]
filesystem:
command: "node"
args: ["C:/Program Files/nodejs/node_modules/@modelcontextprotocol/server-filesystem/dist/index.js", "."]
```
> **Note**: Replace the path with your actual global node_modules path from step 2.
🔍 Search Server Configuration (Optional)
DeepCode supports multiple search servers for web search functionality. You can configure your preferred option in `mcp_agent.config.yaml`:
```yaml
# Default search server configuration
# Options: "brave" or "bocha-mcp"
default_search_server: "brave"
```
**Available Options:**
- **🔍 Brave Search** (`"brave"`): Default option with high-quality search results. Requires `BRAVE_API_KEY`. Recommended for most users.
- **🌐 Bocha-MCP** (`"bocha-mcp"`): Alternative search server. Requires `BOCHA_API_KEY`. Uses local Python server implementation.
**Full MCP server configuration in mcp_agent.config.yaml:**
```yaml
# For Brave Search (default) - around line 28
brave:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-brave-search"]
env:
BRAVE_API_KEY: "your_brave_api_key_here"
# For Bocha-MCP (alternative) - around line 74
bocha-mcp:
command: "python"
args: ["tools/bocha_search_server.py"]
env:
PYTHONPATH: "."
BOCHA_API_KEY: "your_bocha_api_key_here"
```
> **💡 Tip**: Both search servers require API key configuration. Choose the one that best fits your API access and requirements.
### ⚡ **Step 3: Launch Application**
Choose your preferred launch method:
🐳 Docker (Recommended)
🚀 Local (deepcode command)
🛠️ Other Methods
No Python/Node needed — everything in container.
```bash
git clone https://github.com/HKUDS/DeepCode.git
cd DeepCode/
cp mcp_agent.secrets.yaml.example \
mcp_agent.secrets.yaml
# Edit secrets with your API keys
./deepcode_docker/run_docker.sh
# Access → http://localhost:8000
```
Auto-installs deps on first run.
```bash
deepcode
# Frontend → http://localhost:5173
# Backend → http://localhost:8000
# Ctrl+C to stop
```
Features: User-in-Loop, real-time progress, inline chat.
**[▶️ Watch Demo](https://www.youtube.com/watch?v=nFt5mLaMEac)**
*Intelligent image processing with background removal and enhancement*
#### 🌐 **Frontend Implementation**
**Complete Web Application**
**[▶️ Watch Demo](https://www.youtube.com/watch?v=78wx3dkTaAU)**
*Full-stack web development from concept to deployment*
### 🆕 **Recent Updates**
#### 📄 **Smart Document Segmentation (v1.2.0)**
- **Intelligent Processing**: Automatically handles large research papers and technical documents that exceed LLM token limits
- **Configurable Control**: Toggle segmentation via configuration with size-based thresholds
- **Semantic Analysis**: Advanced content understanding with algorithm, concept, and formula preservation
- **Backward Compatibility**: Seamlessly falls back to traditional processing for smaller documents
### 🚀 **Coming Soon**
We're continuously enhancing DeepCode with exciting new features:
#### 🔧 **Enhanced Code Reliability & Validation**
- **Automated Testing**: Comprehensive functionality testing with execution verification and error detection.
- **Code Quality Assurance**: Multi-level validation through static analysis, dynamic testing, and performance benchmarking.
- **Smart Debugging**: AI-powered error detection with automatic correction suggestions
#### 📊 **PaperBench Performance Showcase**
- **Benchmark Dashboard**: Comprehensive performance metrics on the PaperBench evaluation suite.
- **Accuracy Metrics**: Detailed comparison with state-of-the-art paper reproduction systems.
- **Success Analytics**: Statistical analysis across paper categories and complexity levels.
#### ⚡ **System-wide Optimizations**
- **Performance Boost**: Multi-threaded processing and optimized agent coordination for faster generation.
- **Enhanced Reasoning**: Advanced reasoning capabilities with improved context understanding.
- **Expanded Support**: Extended compatibility with additional programming languages and frameworks.
---
## ⭐ Star History
*Community Growth Trajectory*
---
### 🚀 **Ready to Transform Development?**
---
### 📖 **Citation**
If you find DeepCode useful in your research or applications, please kindly cite:
```
@misc{li2025deepcodeopenagenticcoding,
title={DeepCode: Open Agentic Coding},
author={Zongwei Li and Zhonghang Li and Zirui Guo and Xubin Ren and Chao Huang},
year={2025},
eprint={2512.07921},
archivePrefix={arXiv},
primaryClass={cs.SE},
url={https://arxiv.org/abs/2512.07921},
}
```
---
### 📄 **License**
**MIT License** - Copyright (c) 2025 Data Intelligence Lab, The University of Hong Kong
---
================================================
FILE: __init__.py
================================================
"""
DeepCode - AI Research Engine
🧬 Next-Generation AI Research Automation Platform
⚡ Transform research papers into working code automatically
"""
__version__ = "1.2.0"
__author__ = "DeepCode Team"
__url__ = "https://github.com/HKUDS/DeepCode"
__repo__ = "https://github.com/Jany-M/DeepCode/"
# Import main components for easy access
from utils import FileProcessor, DialogueLogger
__all__ = [
"FileProcessor",
"DialogueLogger",
"__version__",
"__author__",
"__url__",
]
================================================
FILE: cli/__init__.py
================================================
"""
CLI Module for DeepCode Agent
DeepCode智能体CLI模块
包含以下组件 / Contains the following components:
- cli_app: CLI应用主程序 / CLI application main program
- cli_interface: CLI界面组件 / CLI interface components
- cli_launcher: CLI启动器 / CLI launcher
"""
__version__ = "1.0.0"
__author__ = "DeepCode Team - Data Intelligence Lab @ HKU"
from .cli_app import main as cli_main
from .cli_interface import CLIInterface
from .cli_launcher import main as launcher_main
__all__ = ["cli_main", "CLIInterface", "launcher_main"]
================================================
FILE: cli/cli_app.py
================================================
#!/usr/bin/env python3
"""
DeepCode - CLI Application Main Program
深度代码 - CLI应用主程序
🧬 Open-Source Code Agent by Data Intelligence Lab @ HKU
⚡ Revolutionizing research reproducibility through collaborative AI
"""
import os
import sys
import asyncio
import time
import json
# 禁止生成.pyc文件
os.environ["PYTHONDONTWRITEBYTECODE"] = "1"
# 添加项目根目录到路径
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
if parent_dir not in sys.path:
sys.path.insert(0, parent_dir)
# 导入MCP应用和工作流
from cli.workflows import CLIWorkflowAdapter
from cli.cli_interface import CLIInterface, Colors
class CLIApp:
"""CLI应用主类 - 升级版智能体编排引擎"""
def __init__(self):
self.cli = CLIInterface()
self.workflow_adapter = CLIWorkflowAdapter(cli_interface=self.cli)
self.app = None # Will be initialized by workflow adapter
self.logger = None
self.context = None
# Document segmentation will be managed by CLI interface
async def initialize_mcp_app(self):
"""初始化MCP应用 - 使用工作流适配器"""
# Workflow adapter will handle MCP initialization
return await self.workflow_adapter.initialize_mcp_app()
async def cleanup_mcp_app(self):
"""清理MCP应用 - 使用工作流适配器"""
await self.workflow_adapter.cleanup_mcp_app()
async def process_requirement_analysis_non_interactive(self, initial_idea: str):
"""处理需求分析工作流(非交互式,用于命令行参数) (NEW: matching UI version)"""
try:
self.cli.print_separator()
self.cli.print_status(
"🧠 Starting requirement analysis workflow...", "info"
)
# Step 1: Generate guiding questions
self.cli.print_status(
"🤖 Generating AI-guided questions to refine your requirements...",
"processing",
)
questions_result = (
await self.workflow_adapter.execute_requirement_analysis_workflow(
user_input=initial_idea, analysis_mode="generate_questions"
)
)
if questions_result["status"] != "success":
self.cli.print_status(
f"❌ Failed to generate questions: {questions_result.get('error', 'Unknown error')}",
"error",
)
return questions_result
# Step 2: Display questions
questions_json = questions_result["result"]
self.cli.display_guiding_questions(questions_json)
# For non-interactive mode, we can't get user answers, so we provide a summary
self.cli.print_status(
"ℹ️ In non-interactive mode, using initial idea for implementation",
"info",
)
self.cli.print_status(
"💡 For guided analysis, please use interactive mode (python main_cli.py)",
"info",
)
# Proceed directly with the initial idea as the requirement
self.cli.print_status(
"🚀 Starting code implementation based on initial requirements...",
"processing",
)
implementation_result = await self.process_input(initial_idea, "chat")
return {
"status": "success",
"questions_generated": questions_result,
"implementation": implementation_result,
}
except Exception as e:
error_msg = str(e)
self.cli.print_error_box("Requirement Analysis Error", error_msg)
self.cli.print_status(
f"Error during requirement analysis: {error_msg}", "error"
)
return {"status": "error", "error": error_msg}
async def process_requirement_analysis(self):
"""处理需求分析工作流(交互式) (NEW: matching UI version)"""
try:
# Step 1: Get initial requirements from user
self.cli.print_separator()
self.cli.print_status(
"🧠 Starting requirement analysis workflow...", "info"
)
user_input = self.cli.get_requirement_analysis_input()
if not user_input:
self.cli.print_status("Requirement analysis cancelled", "warning")
return {"status": "cancelled"}
# Step 2: Generate guiding questions
self.cli.print_status(
"🤖 Generating AI-guided questions to refine your requirements...",
"processing",
)
questions_result = (
await self.workflow_adapter.execute_requirement_analysis_workflow(
user_input=user_input, analysis_mode="generate_questions"
)
)
if questions_result["status"] != "success":
self.cli.print_status(
f"❌ Failed to generate questions: {questions_result.get('error', 'Unknown error')}",
"error",
)
return questions_result
# Step 3: Display questions and get user answers
questions_json = questions_result["result"]
self.cli.display_guiding_questions(questions_json)
# Ask if user wants to answer the questions
proceed = (
input(
f"\n{Colors.BOLD}{Colors.YELLOW}Would you like to answer these questions? (y/n):{Colors.ENDC} "
)
.strip()
.lower()
)
if proceed != "y":
self.cli.print_status(
"You can still use the initial requirements for chat input",
"info",
)
return {"status": "partial", "initial_requirements": user_input}
user_answers = self.cli.get_question_answers(questions_json)
# Step 4: Generate requirement summary
self.cli.print_status(
"📄 Generating detailed requirement document...", "processing"
)
summary_result = (
await self.workflow_adapter.execute_requirement_analysis_workflow(
user_input=user_input,
analysis_mode="summarize_requirements",
user_answers=user_answers,
)
)
if summary_result["status"] != "success":
self.cli.print_status(
f"❌ Failed to generate summary: {summary_result.get('error', 'Unknown error')}",
"error",
)
return summary_result
# Step 5: Display requirement summary
requirement_summary = summary_result["result"]
should_proceed = self.cli.display_requirement_summary(requirement_summary)
if should_proceed:
# Step 6: Proceed with chat-based implementation
self.cli.print_status(
"🚀 Starting code implementation based on analyzed requirements...",
"processing",
)
implementation_result = await self.process_input(
requirement_summary, "chat"
)
return {
"status": "success",
"requirement_analysis": summary_result,
"implementation": implementation_result,
}
else:
self.cli.print_status(
"Requirement analysis completed. Implementation skipped.", "info"
)
return {
"status": "success",
"requirement_analysis": summary_result,
"implementation": None,
}
except Exception as e:
error_msg = str(e)
self.cli.print_error_box("Requirement Analysis Error", error_msg)
self.cli.print_status(
f"Error during requirement analysis: {error_msg}", "error"
)
return {"status": "error", "error": error_msg}
async def process_input(self, input_source: str, input_type: str):
"""处理输入源(URL或文件)- 使用升级版智能体编排引擎"""
try:
# Document segmentation configuration is managed by CLI interface
self.cli.print_separator()
self.cli.print_status(
"🚀 Starting intelligent agent orchestration...", "processing"
)
# 显示处理阶段(根据配置决定)
chat_mode = input_type == "chat"
self.cli.display_processing_stages(
0, self.cli.enable_indexing, chat_mode=chat_mode
)
# 使用工作流适配器进行处理
result = await self.workflow_adapter.process_input_with_orchestration(
input_source=input_source,
input_type=input_type,
enable_indexing=self.cli.enable_indexing,
)
if result["status"] == "success":
# 显示完成状态
if chat_mode:
final_stage = 4
else:
final_stage = 8 if self.cli.enable_indexing else 5
self.cli.display_processing_stages(
final_stage, self.cli.enable_indexing, chat_mode=chat_mode
)
self.cli.print_status(
"🎉 Agent orchestration completed successfully!", "complete"
)
# 显示结果
self.display_results(
result.get("analysis_result", ""),
result.get("download_result", ""),
result.get("repo_result", ""),
result.get("pipeline_mode", "comprehensive"),
)
else:
self.cli.print_status(
f"❌ Processing failed: {result.get('error', 'Unknown error')}",
"error",
)
# 添加到历史记录
self.cli.add_to_history(input_source, result)
return result
except Exception as e:
error_msg = str(e)
self.cli.print_error_box("Agent Orchestration Error", error_msg)
self.cli.print_status(f"Error during orchestration: {error_msg}", "error")
# 添加错误到历史记录
error_result = {"status": "error", "error": error_msg}
self.cli.add_to_history(input_source, error_result)
return error_result
def display_results(
self,
analysis_result: str,
download_result: str,
repo_result: str,
pipeline_mode: str = "comprehensive",
):
"""显示处理结果"""
self.cli.print_results_header()
# 显示流水线模式
if pipeline_mode == "chat":
mode_display = "💬 Chat Planning Mode"
elif pipeline_mode == "comprehensive":
mode_display = "🧠 Comprehensive Mode"
else:
mode_display = "⚡ Optimized Mode"
print(
f"{Colors.BOLD}{Colors.PURPLE}🤖 PIPELINE MODE: {mode_display}{Colors.ENDC}"
)
self.cli.print_separator("─", 79, Colors.PURPLE)
print(f"{Colors.BOLD}{Colors.OKCYAN}📊 ANALYSIS PHASE RESULTS:{Colors.ENDC}")
self.cli.print_separator("─", 79, Colors.CYAN)
# 尝试解析并格式化分析结果
try:
if analysis_result.strip().startswith("{"):
parsed_analysis = json.loads(analysis_result)
print(json.dumps(parsed_analysis, indent=2, ensure_ascii=False))
else:
print(
analysis_result[:1000] + "..."
if len(analysis_result) > 1000
else analysis_result
)
except Exception:
print(
analysis_result[:1000] + "..."
if len(analysis_result) > 1000
else analysis_result
)
print(f"\n{Colors.BOLD}{Colors.PURPLE}📥 DOWNLOAD PHASE RESULTS:{Colors.ENDC}")
self.cli.print_separator("─", 79, Colors.PURPLE)
print(
download_result[:1000] + "..."
if len(download_result) > 1000
else download_result
)
print(
f"\n{Colors.BOLD}{Colors.GREEN}⚙️ IMPLEMENTATION PHASE RESULTS:{Colors.ENDC}"
)
self.cli.print_separator("─", 79, Colors.GREEN)
print(repo_result[:1000] + "..." if len(repo_result) > 1000 else repo_result)
# 尝试提取生成的代码目录信息
if "Code generated in:" in repo_result:
code_dir = (
repo_result.split("Code generated in:")[-1].strip().split("\n")[0]
)
print(
f"\n{Colors.BOLD}{Colors.YELLOW}📁 Generated Code Directory: {Colors.ENDC}{code_dir}"
)
# 显示处理完成的工作流阶段
print(
f"\n{Colors.BOLD}{Colors.OKCYAN}🔄 COMPLETED WORKFLOW STAGES:{Colors.ENDC}"
)
if pipeline_mode == "chat":
stages = [
"🚀 Engine Initialization",
"💬 Requirements Analysis",
"🏗️ Workspace Setup",
"📝 Implementation Plan Generation",
"⚙️ Code Implementation",
]
else:
stages = [
"📄 Document Processing",
"🔍 Reference Analysis",
"📋 Plan Generation",
"📦 Repository Download",
"🗂️ Codebase Indexing",
"⚙️ Code Implementation",
]
for stage in stages:
print(f" ✅ {stage}")
self.cli.print_separator()
async def run_interactive_session(self):
"""运行交互式会话"""
# 清屏并显示启动界面
self.cli.clear_screen()
self.cli.print_logo()
self.cli.print_welcome_banner()
# 初始化MCP应用
await self.initialize_mcp_app()
try:
# 主交互循环
while self.cli.is_running:
self.cli.create_menu()
choice = self.cli.get_user_input()
if choice in ["q", "quit", "exit"]:
self.cli.print_goodbye()
break
elif choice in ["u", "url"]:
url = self.cli.get_url_input()
if url:
await self.process_input(url, "url")
elif choice in ["f", "file"]:
file_path = self.cli.upload_file_gui()
if file_path:
await self.process_input(f"file://{file_path}", "file")
elif choice in ["t", "chat", "text"]:
chat_input = self.cli.get_chat_input()
if chat_input:
await self.process_input(chat_input, "chat")
elif choice in ["r", "req", "requirement", "requirements"]:
# NEW: Requirement Analysis workflow
await self.process_requirement_analysis()
elif choice in ["h", "history"]:
self.cli.show_history()
elif choice in ["c", "config", "configure"]:
# Show configuration menu - all settings managed by CLI interface
self.cli.show_configuration_menu()
else:
self.cli.print_status(
"Invalid choice. Please select U, F, T, R, C, H, or Q.",
"warning",
)
# 询问是否继续
if self.cli.is_running and choice in [
"u",
"f",
"t",
"r",
"chat",
"text",
"req",
"requirement",
"requirements",
]:
if not self.cli.ask_continue():
self.cli.is_running = False
self.cli.print_status("Session ended by user", "info")
except KeyboardInterrupt:
print(f"\n{Colors.WARNING}⚠️ Process interrupted by user{Colors.ENDC}")
except Exception as e:
print(f"\n{Colors.FAIL}❌ Unexpected error: {str(e)}{Colors.ENDC}")
finally:
# 清理资源
await self.cleanup_mcp_app()
async def main():
"""主函数"""
start_time = time.time()
try:
# 创建并运行CLI应用
app = CLIApp()
await app.run_interactive_session()
except KeyboardInterrupt:
print(f"\n{Colors.WARNING}⚠️ Application interrupted by user{Colors.ENDC}")
except Exception as e:
print(f"\n{Colors.FAIL}❌ Application error: {str(e)}{Colors.ENDC}")
finally:
end_time = time.time()
print(
f"\n{Colors.BOLD}{Colors.CYAN}⏱️ Total runtime: {end_time - start_time:.2f} seconds{Colors.ENDC}"
)
# 清理缓存文件
print(f"{Colors.YELLOW}🧹 Cleaning up cache files...{Colors.ENDC}")
if os.name == "nt": # Windows
os.system(
"powershell -Command \"Get-ChildItem -Path . -Filter '__pycache__' -Recurse -Directory | Remove-Item -Recurse -Force\" 2>nul"
)
else: # Unix/Linux/macOS
os.system('find . -type d -name "__pycache__" -exec rm -r {} + 2>/dev/null')
print(
f"{Colors.OKGREEN}✨ Goodbye! Thanks for using DeepCode CLI! ✨{Colors.ENDC}"
)
if __name__ == "__main__":
asyncio.run(main())
================================================
FILE: cli/cli_interface.py
================================================
#!/usr/bin/env python3
"""
Enhanced CLI Interface Module for DeepCode
增强版CLI界面模块 - 专为DeepCode设计
"""
import os
import time
import platform
from typing import Optional
class Colors:
"""ANSI color codes for terminal styling"""
HEADER = "\033[95m"
OKBLUE = "\033[94m"
OKCYAN = "\033[96m"
OKGREEN = "\033[92m"
WARNING = "\033[93m"
FAIL = "\033[91m"
ENDC = "\033[0m"
BOLD = "\033[1m"
UNDERLINE = "\033[4m"
# Gradient colors
PURPLE = "\033[35m"
MAGENTA = "\033[95m"
BLUE = "\033[34m"
CYAN = "\033[36m"
GREEN = "\033[32m"
YELLOW = "\033[33m"
class CLIInterface:
"""Enhanced CLI interface with modern styling for DeepCode"""
def __init__(self):
self.uploaded_file = None
self.is_running = True
self.processing_history = []
self.enable_indexing = (
False # Default configuration (matching UI: fast mode by default)
)
# Load segmentation config from the same source as UI
self._load_segmentation_config()
# Initialize tkinter availability
self._init_tkinter()
def _load_segmentation_config(self):
"""Load segmentation configuration from mcp_agent.config.yaml"""
try:
from utils.llm_utils import get_document_segmentation_config
seg_config = get_document_segmentation_config()
self.segmentation_enabled = seg_config.get("enabled", True)
self.segmentation_threshold = seg_config.get("size_threshold_chars", 50000)
except Exception as e:
print(f"⚠️ Warning: Failed to load segmentation config: {e}")
# Fall back to defaults
self.segmentation_enabled = True
self.segmentation_threshold = 50000
def _save_segmentation_config(self):
"""Save segmentation configuration to mcp_agent.config.yaml"""
import yaml
import os
# Get the project root directory (where mcp_agent.config.yaml is located)
current_file = os.path.abspath(__file__)
cli_dir = os.path.dirname(current_file) # cli directory
project_root = os.path.dirname(cli_dir) # project root
config_path = os.path.join(project_root, "mcp_agent.config.yaml")
try:
# Read current config
with open(config_path, "r", encoding="utf-8") as f:
config = yaml.safe_load(f)
# Update document segmentation settings
if "document_segmentation" not in config:
config["document_segmentation"] = {}
config["document_segmentation"]["enabled"] = self.segmentation_enabled
config["document_segmentation"]["size_threshold_chars"] = (
self.segmentation_threshold
)
# Write updated config
with open(config_path, "w", encoding="utf-8") as f:
yaml.dump(config, f, default_flow_style=False, allow_unicode=True)
print(
f"{Colors.OKGREEN}✅ Document segmentation configuration updated{Colors.ENDC}"
)
except Exception as e:
print(
f"{Colors.WARNING}⚠️ Failed to update segmentation config: {str(e)}{Colors.ENDC}"
)
def _init_tkinter(self):
"""Initialize tkinter availability check"""
# Check tkinter availability for file dialogs
self.tkinter_available = True
try:
import tkinter as tk
# Test if tkinter can create a window
test_root = tk.Tk()
test_root.withdraw()
test_root.destroy()
except Exception:
self.tkinter_available = False
def clear_screen(self):
"""Clear terminal screen"""
os.system("cls" if os.name == "nt" else "clear")
def print_logo(self):
"""Print enhanced ASCII logo for DeepCode CLI"""
logo = f"""
{Colors.CYAN}╔═══════════════════════════════════════════════════════════════════════════════╗
║ ║
║ {Colors.BOLD}{Colors.MAGENTA}██████╗ ███████╗███████╗██████╗ ██████╗ ██████╗ ██████╗ ███████╗{Colors.CYAN} ║
║ {Colors.BOLD}{Colors.PURPLE}██╔══██╗██╔════╝██╔════╝██╔══██╗██╔════╝██╔═══██╗██╔══██╗██╔════╝{Colors.CYAN} ║
║ {Colors.BOLD}{Colors.BLUE}██║ ██║█████╗ █████╗ ██████╔╝██║ ██║ ██║██║ ██║█████╗ {Colors.CYAN} ║
║ {Colors.BOLD}{Colors.OKBLUE}██║ ██║██╔══╝ ██╔══╝ ██╔═══╝ ██║ ██║ ██║██║ ██║██╔══╝ {Colors.CYAN} ║
║ {Colors.BOLD}{Colors.OKCYAN}██████╔╝███████╗███████╗██║ ╚██████╗╚██████╔╝██████╔╝███████╗{Colors.CYAN} ║
║ {Colors.BOLD}{Colors.GREEN}╚═════╝ ╚══════╝╚══════╝╚═╝ ╚═════╝ ╚═════╝ ╚═════╝ ╚══════╝{Colors.CYAN} ║
║ ║
║ {Colors.BOLD}{Colors.GREEN}🧬 OPEN-SOURCE CODE AGENT • DATA INTELLIGENCE LAB @ HKU 🚀 {Colors.CYAN}║
║ {Colors.BOLD}{Colors.GREEN}⚡ REVOLUTIONIZING RESEARCH REPRODUCIBILITY ⚡ {Colors.CYAN}║
║ ║
╚═══════════════════════════════════════════════════════════════════════════════╝{Colors.ENDC}
"""
print(logo)
def print_welcome_banner(self):
"""Print enhanced welcome banner"""
banner = f"""
{Colors.BOLD}{Colors.CYAN}╔═══════════════════════════════════════════════════════════════════════════════╗
║ WELCOME TO DEEPCODE CLI ║
╠═══════════════════════════════════════════════════════════════════════════════╣
║ {Colors.YELLOW}Open-Source Code Agent | Data Intelligence Lab @ HKU | MIT License {Colors.CYAN}║
║ {Colors.GREEN}Status: Ready | Engine: Multi-Agent Architecture Initialized {Colors.CYAN}║
║ {Colors.PURPLE}Mission: Revolutionizing Research Reproducibility {Colors.CYAN}║
║ ║
║ {Colors.BOLD}{Colors.OKCYAN}💎 CORE CAPABILITIES:{Colors.ENDC} {Colors.CYAN}║
║ {Colors.BOLD}{Colors.OKCYAN}▶ Automated Paper-to-Code Reproduction {Colors.CYAN}║
║ {Colors.BOLD}{Colors.OKCYAN}▶ Collaborative Multi-Agent Architecture {Colors.CYAN}║
║ {Colors.BOLD}{Colors.OKCYAN}▶ Intelligent Code Implementation & Validation {Colors.CYAN}║
║ {Colors.BOLD}{Colors.OKCYAN}▶ Future Vision: One Sentence → Complete Codebase {Colors.CYAN}║
╚═══════════════════════════════════════════════════════════════════════════════╝{Colors.ENDC}
"""
print(banner)
def print_separator(self, char="═", length=79, color=Colors.CYAN):
"""Print a styled separator line"""
print(f"{color}{char * length}{Colors.ENDC}")
def print_status(self, message: str, status_type: str = "info"):
"""Print status message with appropriate styling"""
status_styles = {
"success": f"{Colors.OKGREEN}✅",
"error": f"{Colors.FAIL}❌",
"warning": f"{Colors.WARNING}⚠️ ",
"info": f"{Colors.OKBLUE}ℹ️ ",
"processing": f"{Colors.YELLOW}⏳",
"upload": f"{Colors.PURPLE}📁",
"download": f"{Colors.CYAN}📥",
"analysis": f"{Colors.MAGENTA}🔍",
"implementation": f"{Colors.GREEN}⚙️ ",
"complete": f"{Colors.OKGREEN}🎉",
}
icon = status_styles.get(status_type, status_styles["info"])
timestamp = time.strftime("%H:%M:%S")
print(
f"[{Colors.BOLD}{timestamp}{Colors.ENDC}] {icon} {Colors.BOLD}{message}{Colors.ENDC}"
)
def create_menu(self):
"""Create enhanced interactive menu"""
# Display current configuration
pipeline_mode = "🧠 COMPREHENSIVE" if self.enable_indexing else "⚡ OPTIMIZED"
index_status = "✅ Enabled" if self.enable_indexing else "🔶 Disabled"
segmentation_mode = (
"📄 SMART" if self.segmentation_enabled else "📋 TRADITIONAL"
)
menu = f"""
{Colors.BOLD}{Colors.CYAN}╔═══════════════════════════════════════════════════════════════════════════════╗
║ MAIN MENU ║
╠═══════════════════════════════════════════════════════════════════════════════╣
║ {Colors.OKGREEN}🌐 [U] Process URL {Colors.CYAN}│ {Colors.PURPLE}📁 [F] Upload File {Colors.CYAN}│ {Colors.MAGENTA}💬 [T] Chat Input{Colors.CYAN} ║
║ {Colors.BLUE}🧠 [R] Req. Analysis {Colors.CYAN}│ {Colors.OKCYAN}⚙️ [C] Configure {Colors.CYAN}│ {Colors.YELLOW}📊 [H] History{Colors.CYAN} ║
║ {Colors.FAIL}❌ [Q] Quit{Colors.CYAN} ║
║ ║
║ {Colors.BOLD}🤖 Current Pipeline Mode: {pipeline_mode}{Colors.CYAN} ║
║ {Colors.BOLD}🗂️ Codebase Indexing: {index_status}{Colors.CYAN} ║
║ {Colors.BOLD}📄 Document Processing: {segmentation_mode}{Colors.CYAN} ║
║ ║
║ {Colors.YELLOW}📝 URL Processing:{Colors.CYAN} ║
║ {Colors.YELLOW} ▶ Enter research paper URL (arXiv, IEEE, ACM, etc.) {Colors.CYAN}║
║ {Colors.YELLOW} ▶ Supports direct PDF links and academic paper pages {Colors.CYAN}║
║ ║
║ {Colors.PURPLE}📁 File Processing:{Colors.CYAN} ║
║ {Colors.PURPLE} ▶ Upload PDF, DOCX, PPTX, HTML, or TXT files {Colors.CYAN}║
║ {Colors.PURPLE} ▶ Intelligent file format detection and processing {Colors.CYAN}║
║ ║
║ {Colors.MAGENTA}💬 Chat Input:{Colors.CYAN} ║
║ {Colors.MAGENTA} ▶ Describe your coding requirements in natural language {Colors.CYAN}║
║ {Colors.MAGENTA} ▶ AI generates implementation plan and code automatically {Colors.CYAN}║
║ ║
║ {Colors.BLUE}🧠 Requirement Analysis (NEW):{Colors.CYAN} ║
║ {Colors.BLUE} ▶ Get AI-guided questions to refine your requirements {Colors.CYAN}║
║ {Colors.BLUE} ▶ Generate detailed requirement documents from your answers {Colors.CYAN}║
║ ║
║ {Colors.OKCYAN}🔄 Processing Pipeline:{Colors.CYAN} ║
║ {Colors.OKCYAN} ▶ Intelligent agent orchestration → Code synthesis {Colors.CYAN}║
║ {Colors.OKCYAN} ▶ Multi-agent coordination with progress tracking {Colors.CYAN}║
╚═══════════════════════════════════════════════════════════════════════════════╝{Colors.ENDC}
"""
print(menu)
def get_user_input(self):
"""Get user input with styled prompt"""
print(f"\n{Colors.BOLD}{Colors.OKCYAN}➤ Your choice: {Colors.ENDC}", end="")
return input().strip().lower()
def upload_file_gui(self) -> Optional[str]:
"""Enhanced file upload interface with better error handling"""
if not self.tkinter_available:
self.print_status(
"GUI file dialog not available - using manual input", "warning"
)
return self._get_manual_file_path()
def select_file():
try:
import tkinter as tk
from tkinter import filedialog
root = tk.Tk()
root.withdraw()
root.attributes("-topmost", True)
file_types = [
("Research Papers", "*.pdf;*.docx;*.doc"),
("PDF Files", "*.pdf"),
("Word Documents", "*.docx;*.doc"),
("PowerPoint Files", "*.pptx;*.ppt"),
("HTML Files", "*.html;*.htm"),
("Text Files", "*.txt;*.md"),
("All Files", "*.*"),
]
if platform.system() == "Darwin":
file_types = [
("Research Papers", ".pdf .docx .doc"),
("PDF Files", ".pdf"),
("Word Documents", ".docx .doc"),
("PowerPoint Files", ".pptx .ppt"),
("HTML Files", ".html .htm"),
("Text Files", ".txt .md"),
("All Files", ".*"),
]
file_path = filedialog.askopenfilename(
title="Select Research File - DeepCode CLI",
filetypes=file_types,
initialdir=os.getcwd(),
)
root.destroy()
return file_path
except Exception as e:
self.print_status(f"File dialog error: {str(e)}", "error")
return self._get_manual_file_path()
self.print_status("Opening file browser dialog...", "upload")
file_path = select_file()
if file_path:
self.print_status(
f"File selected: {os.path.basename(file_path)}", "success"
)
return file_path
else:
self.print_status("No file selected", "warning")
return None
def _get_manual_file_path(self) -> Optional[str]:
"""Get file path through manual input with validation"""
self.print_separator("─", 79, Colors.YELLOW)
print(f"{Colors.BOLD}{Colors.YELLOW}📁 Manual File Path Input{Colors.ENDC}")
print(
f"{Colors.CYAN}Please enter the full path to your research paper file:{Colors.ENDC}"
)
print(
f"{Colors.CYAN}Supported formats: PDF, DOCX, PPTX, HTML, TXT, MD{Colors.ENDC}"
)
self.print_separator("─", 79, Colors.YELLOW)
while True:
print(f"\n{Colors.BOLD}{Colors.OKCYAN}📂 File path: {Colors.ENDC}", end="")
file_path = input().strip()
if not file_path:
self.print_status(
"Empty path entered. Please try again or press Ctrl+C to cancel.",
"warning",
)
continue
file_path = os.path.expanduser(file_path)
file_path = os.path.abspath(file_path)
if not os.path.exists(file_path):
self.print_status(f"File not found: {file_path}", "error")
retry = (
input(f"{Colors.YELLOW}Try again? (y/n): {Colors.ENDC}")
.strip()
.lower()
)
if retry != "y":
return None
continue
if not os.path.isfile(file_path):
self.print_status(f"Path is not a file: {file_path}", "error")
continue
supported_extensions = {
".pdf",
".docx",
".doc",
".pptx",
".ppt",
".html",
".htm",
".txt",
".md",
}
file_ext = os.path.splitext(file_path)[1].lower()
if file_ext not in supported_extensions:
self.print_status(f"Unsupported file format: {file_ext}", "warning")
proceed = (
input(f"{Colors.YELLOW}Process anyway? (y/n): {Colors.ENDC}")
.strip()
.lower()
)
if proceed != "y":
continue
self.print_status(
f"File validated: {os.path.basename(file_path)}", "success"
)
return file_path
def get_url_input(self) -> str:
"""Enhanced URL input with validation"""
self.print_separator("─", 79, Colors.GREEN)
print(f"{Colors.BOLD}{Colors.GREEN}🌐 URL Input Interface{Colors.ENDC}")
print(
f"{Colors.CYAN}Enter a research paper URL from supported platforms:{Colors.ENDC}"
)
print(
f"{Colors.CYAN}• arXiv (arxiv.org) • IEEE Xplore (ieeexplore.ieee.org){Colors.ENDC}"
)
print(
f"{Colors.CYAN}• ACM Digital Library • SpringerLink • Nature • Science{Colors.ENDC}"
)
print(
f"{Colors.CYAN}• Direct PDF links • Academic publisher websites{Colors.ENDC}"
)
self.print_separator("─", 79, Colors.GREEN)
while True:
print(f"\n{Colors.BOLD}{Colors.OKCYAN}🔗 URL: {Colors.ENDC}", end="")
url = input().strip()
if not url:
self.print_status(
"Empty URL entered. Please try again or press Ctrl+C to cancel.",
"warning",
)
continue
if not url.startswith(("http://", "https://")):
self.print_status("URL must start with http:// or https://", "error")
retry = (
input(f"{Colors.YELLOW}Try again? (y/n): {Colors.ENDC}")
.strip()
.lower()
)
if retry != "y":
return ""
continue
academic_domains = [
"arxiv.org",
"ieeexplore.ieee.org",
"dl.acm.org",
"link.springer.com",
"nature.com",
"science.org",
"scholar.google.com",
"researchgate.net",
"semanticscholar.org",
]
is_academic = any(domain in url.lower() for domain in academic_domains)
if not is_academic and not url.lower().endswith(".pdf"):
self.print_status(
"URL doesn't appear to be from a known academic platform", "warning"
)
proceed = (
input(f"{Colors.YELLOW}Process anyway? (y/n): {Colors.ENDC}")
.strip()
.lower()
)
if proceed != "y":
continue
self.print_status(f"URL validated: {url}", "success")
return url
def get_chat_input(self) -> str:
"""Enhanced chat input interface for coding requirements"""
self.print_separator("─", 79, Colors.PURPLE)
print(f"{Colors.BOLD}{Colors.PURPLE}💬 Chat Input Interface{Colors.ENDC}")
print(
f"{Colors.CYAN}Describe your coding requirements in natural language.{Colors.ENDC}"
)
print(
f"{Colors.CYAN}Our AI will analyze your needs and generate a comprehensive implementation plan.{Colors.ENDC}"
)
self.print_separator("─", 79, Colors.PURPLE)
# Display examples to help users
print(f"\n{Colors.BOLD}{Colors.YELLOW}💡 Examples:{Colors.ENDC}")
print(f"{Colors.CYAN}Academic Research:{Colors.ENDC}")
print(
" • 'I need to implement a reinforcement learning algorithm for robotic control'"
)
print(
" • 'Create a neural network for image classification with attention mechanisms'"
)
print(f"{Colors.CYAN}Engineering Projects:{Colors.ENDC}")
print(
" • 'Develop a web application for project management with user authentication'"
)
print(" • 'Create a data visualization dashboard for sales analytics'")
print(f"{Colors.CYAN}Mixed Projects:{Colors.ENDC}")
print(
" • 'Implement a machine learning model with a web interface for real-time predictions'"
)
self.print_separator("─", 79, Colors.PURPLE)
print(
f"\n{Colors.BOLD}{Colors.OKCYAN}✏️ Enter your coding requirements below:{Colors.ENDC}"
)
print(
f"{Colors.YELLOW}(Type your description, press Enter twice when finished, or Ctrl+C to cancel){Colors.ENDC}"
)
lines = []
empty_line_count = 0
while True:
try:
if len(lines) == 0:
print(f"{Colors.BOLD}> {Colors.ENDC}", end="")
else:
print(f"{Colors.BOLD} {Colors.ENDC}", end="")
line = input()
if line.strip() == "":
empty_line_count += 1
if empty_line_count >= 2:
# Two consecutive empty lines means user finished input
break
lines.append("") # Keep empty line for formatting
else:
empty_line_count = 0
lines.append(line)
except KeyboardInterrupt:
print(f"\n{Colors.WARNING}Input cancelled by user{Colors.ENDC}")
return ""
# Join all lines and clean up
user_input = "\n".join(lines).strip()
if not user_input:
self.print_status("No input provided", "warning")
return ""
if len(user_input) < 20:
self.print_status(
"Input too short. Please provide more detailed requirements (at least 20 characters)",
"warning",
)
retry = (
input(f"{Colors.YELLOW}Try again? (y/n): {Colors.ENDC}").strip().lower()
)
if retry == "y":
return self.get_chat_input() # Recursive call for retry
return ""
# Display input summary
word_count = len(user_input.split())
char_count = len(user_input)
print(f"\n{Colors.BOLD}{Colors.GREEN}📋 Input Summary:{Colors.ENDC}")
print(f" • {Colors.CYAN}Word count: {word_count}{Colors.ENDC}")
print(f" • {Colors.CYAN}Character count: {char_count}{Colors.ENDC}")
# Show preview
preview = user_input[:200] + "..." if len(user_input) > 200 else user_input
print(f"\n{Colors.BOLD}{Colors.CYAN}📄 Preview:{Colors.ENDC}")
print(f"{Colors.YELLOW}{preview}{Colors.ENDC}")
# Confirm with user
confirm = (
input(
f"\n{Colors.BOLD}{Colors.OKCYAN}Proceed with this input? (y/n): {Colors.ENDC}"
)
.strip()
.lower()
)
if confirm != "y":
retry = (
input(f"{Colors.YELLOW}Edit input? (y/n): {Colors.ENDC}")
.strip()
.lower()
)
if retry == "y":
return self.get_chat_input() # Recursive call for retry
return ""
self.print_status(
f"Chat input captured: {word_count} words, {char_count} characters",
"success",
)
return user_input
def show_progress_bar(self, message: str, duration: float = 2.0):
"""Show animated progress bar"""
print(f"\n{Colors.BOLD}{Colors.CYAN}{message}{Colors.ENDC}")
bar_length = 50
for i in range(bar_length + 1):
percent = (i / bar_length) * 100
filled = "█" * i
empty = "░" * (bar_length - i)
print(
f"\r{Colors.OKGREEN}[{filled}{empty}] {percent:3.0f}%{Colors.ENDC}",
end="",
flush=True,
)
time.sleep(duration / bar_length)
print(f"\n{Colors.OKGREEN}✓ {message} completed{Colors.ENDC}")
def show_spinner(self, message: str, duration: float = 1.0):
"""Show spinner animation"""
spinner_chars = "⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏"
end_time = time.time() + duration
print(
f"{Colors.BOLD}{Colors.CYAN}{message}... {Colors.ENDC}", end="", flush=True
)
i = 0
while time.time() < end_time:
print(
f"\r{Colors.BOLD}{Colors.CYAN}{message}... {Colors.YELLOW}{spinner_chars[i % len(spinner_chars)]}{Colors.ENDC}",
end="",
flush=True,
)
time.sleep(0.1)
i += 1
print(
f"\r{Colors.BOLD}{Colors.CYAN}{message}... {Colors.OKGREEN}✓{Colors.ENDC}"
)
def display_processing_stages(
self,
current_stage: int = 0,
enable_indexing: bool = True,
chat_mode: bool = False,
):
"""Display processing pipeline stages with current progress"""
if chat_mode:
# Chat mode - simplified workflow for user requirements
stages = [
("🚀", "Initialize", "Setting up chat engine"),
("💬", "Planning", "Analyzing requirements"),
("🏗️", "Setup", "Creating workspace"),
("📝", "Save Plan", "Saving implementation plan"),
("⚙️", "Implement", "Generating code"),
]
pipeline_mode = "CHAT PLANNING"
elif enable_indexing:
# Full pipeline with all stages
stages = [
("🚀", "Initialize", "Setting up AI engine"),
("📊", "Analyze", "Analyzing research content"),
("📥", "Download", "Processing document"),
("📋", "Plan", "Generating code architecture"),
("🔍", "References", "Analyzing references"),
("📦", "Repos", "Downloading repositories"),
("🗂️", "Index", "Building code index"),
("⚙️", "Implement", "Implementing code"),
]
pipeline_mode = "COMPREHENSIVE"
else:
# Fast mode - skip indexing related stages
stages = [
("🚀", "Initialize", "Setting up AI engine"),
("📊", "Analyze", "Analyzing research content"),
("📥", "Download", "Processing document"),
("📋", "Plan", "Generating code architecture"),
("⚙️", "Implement", "Implementing code"),
]
pipeline_mode = "OPTIMIZED"
print(
f"\n{Colors.BOLD}{Colors.CYAN}📋 {pipeline_mode} PIPELINE STATUS{Colors.ENDC}"
)
self.print_separator("─", 79, Colors.CYAN)
for i, (icon, name, desc) in enumerate(stages):
if i < current_stage:
status = f"{Colors.OKGREEN}✓ COMPLETED{Colors.ENDC}"
elif i == current_stage:
status = f"{Colors.YELLOW}⏳ IN PROGRESS{Colors.ENDC}"
else:
status = f"{Colors.CYAN}⏸️ PENDING{Colors.ENDC}"
print(
f"{icon} {Colors.BOLD}{name:<12}{Colors.ENDC} │ {desc:<25} │ {status}"
)
self.print_separator("─", 79, Colors.CYAN)
def print_results_header(self):
"""Print results section header"""
header = f"""
{Colors.BOLD}{Colors.OKGREEN}╔═══════════════════════════════════════════════════════════════════════════════╗
║ PROCESSING RESULTS ║
╚═══════════════════════════════════════════════════════════════════════════════╝{Colors.ENDC}
"""
print(header)
def print_error_box(self, title: str, error_msg: str):
"""Print formatted error box"""
print(
f"\n{Colors.FAIL}╔══════════════════════════════════════════════════════════════╗"
)
print(f"║ {Colors.BOLD}ERROR: {title:<50}{Colors.FAIL} ║")
print("╠══════════════════════════════════════════════════════════════╣")
words = error_msg.split()
lines = []
current_line = ""
for word in words:
if len(current_line + word) <= 54:
current_line += word + " "
else:
lines.append(current_line.strip())
current_line = word + " "
if current_line:
lines.append(current_line.strip())
for line in lines:
print(f"║ {line:<56} ║")
print(
f"╚══════════════════════════════════════════════════════════════╝{Colors.ENDC}"
)
def cleanup_cache(self):
"""清理Python缓存文件 / Clean up Python cache files"""
try:
self.print_status("Cleaning up cache files...", "info")
# 清理__pycache__目录
os.system('find . -type d -name "__pycache__" -exec rm -r {} + 2>/dev/null')
# 清理.pyc文件
os.system('find . -name "*.pyc" -delete 2>/dev/null')
self.print_status("Cache cleanup completed", "success")
except Exception as e:
self.print_status(f"Cache cleanup failed: {e}", "warning")
def print_goodbye(self):
"""Print goodbye message"""
# 清理缓存文件
self.cleanup_cache()
goodbye = f"""
{Colors.BOLD}{Colors.CYAN}╔═══════════════════════════════════════════════════════════════════════════════╗
║ GOODBYE ║
╠═══════════════════════════════════════════════════════════════════════════════╣
║ {Colors.OKGREEN}🎉 Thank you for using DeepCode CLI! {Colors.CYAN}║
║ ║
║ {Colors.YELLOW}🧬 Join our community in revolutionizing research reproducibility {Colors.CYAN}║
║ {Colors.PURPLE}⚡ Together, we're building the future of automated code generation {Colors.CYAN}║
║ ║
║ {Colors.OKCYAN}💡 Questions? Contribute to our open-source mission at GitHub {Colors.CYAN}║
║ {Colors.GREEN}🧹 Cache files cleaned up for optimal performance {Colors.CYAN}║
║ ║
╚═══════════════════════════════════════════════════════════════════════════════╝{Colors.ENDC}
"""
print(goodbye)
def get_requirement_analysis_input(self) -> str:
"""Enhanced requirement analysis input interface (NEW: matching UI version)"""
self.print_separator("─", 79, Colors.BLUE)
print(
f"{Colors.BOLD}{Colors.BLUE}🧠 Requirement Analysis Interface{Colors.ENDC}"
)
print(
f"{Colors.CYAN}Describe your project idea or requirements briefly.{Colors.ENDC}"
)
print(
f"{Colors.CYAN}Our AI will generate guiding questions to help you refine your vision.{Colors.ENDC}"
)
self.print_separator("─", 79, Colors.BLUE)
# Display examples
print(f"\n{Colors.BOLD}{Colors.YELLOW}💡 Examples:{Colors.ENDC}")
print(
f"{Colors.CYAN} • 'I want to build a machine learning system for image recognition'{Colors.ENDC}"
)
print(
f"{Colors.CYAN} • 'Create a web app for project management with real-time collaboration'{Colors.ENDC}"
)
print(
f"{Colors.CYAN} • 'Develop a data analysis pipeline for financial forecasting'{Colors.ENDC}"
)
self.print_separator("─", 79, Colors.BLUE)
print(
f"\n{Colors.BOLD}{Colors.OKCYAN}✏️ Enter your initial requirements below:{Colors.ENDC}"
)
print(
f"{Colors.YELLOW}(Type your description, press Enter twice when finished, or Ctrl+C to cancel){Colors.ENDC}"
)
lines = []
empty_line_count = 0
while True:
try:
if len(lines) == 0:
print(f"{Colors.BOLD}> {Colors.ENDC}", end="")
else:
print(f"{Colors.BOLD} {Colors.ENDC}", end="")
line = input()
if line.strip() == "":
empty_line_count += 1
if empty_line_count >= 2:
break
lines.append("")
else:
empty_line_count = 0
lines.append(line)
except KeyboardInterrupt:
print(f"\n{Colors.WARNING}Input cancelled by user{Colors.ENDC}")
return ""
user_input = "\n".join(lines).strip()
if not user_input:
self.print_status("No input provided", "warning")
return ""
if len(user_input) < 20:
self.print_status(
"Input too short. Please provide more details (at least 20 characters)",
"warning",
)
retry = (
input(f"{Colors.YELLOW}Try again? (y/n): {Colors.ENDC}").strip().lower()
)
if retry == "y":
return self.get_requirement_analysis_input()
return ""
# Display input summary
word_count = len(user_input.split())
char_count = len(user_input)
print(f"\n{Colors.BOLD}{Colors.GREEN}📋 Input Summary:{Colors.ENDC}")
print(f" • {Colors.CYAN}Word count: {word_count}{Colors.ENDC}")
print(f" • {Colors.CYAN}Character count: {char_count}{Colors.ENDC}")
# Show preview
preview = user_input[:200] + "..." if len(user_input) > 200 else user_input
print(f"\n{Colors.BOLD}{Colors.CYAN}📄 Preview:{Colors.ENDC}")
print(f"{Colors.YELLOW}{preview}{Colors.ENDC}")
# Confirm
confirm = (
input(
f"\n{Colors.BOLD}{Colors.OKCYAN}Proceed with this input? (y/n): {Colors.ENDC}"
)
.strip()
.lower()
)
if confirm != "y":
retry = (
input(f"{Colors.YELLOW}Edit input? (y/n): {Colors.ENDC}")
.strip()
.lower()
)
if retry == "y":
return self.get_requirement_analysis_input()
return ""
self.print_status(
f"Requirement input captured: {word_count} words, {char_count} characters",
"success",
)
return user_input
def display_guiding_questions(self, questions_json: str):
"""Display AI-generated guiding questions (NEW: matching UI version)"""
import json
try:
questions = json.loads(questions_json)
self.print_separator("═", 79, Colors.GREEN)
print(
f"\n{Colors.BOLD}{Colors.GREEN}🤖 AI-Generated Guiding Questions{Colors.ENDC}"
)
print(
f"{Colors.CYAN}Please answer these questions to help refine your requirements:{Colors.ENDC}\n"
)
self.print_separator("─", 79, Colors.GREEN)
for i, q in enumerate(questions, 1):
print(
f"\n{Colors.BOLD}{Colors.YELLOW}Question {i}:{Colors.ENDC} {Colors.CYAN}{q}{Colors.ENDC}"
)
self.print_separator("═", 79, Colors.GREEN)
except json.JSONDecodeError:
self.print_status("Failed to parse questions", "error")
print(questions_json)
def get_question_answers(self, questions_json: str) -> dict:
"""Get user answers to guiding questions (NEW: matching UI version)"""
import json
try:
questions = json.loads(questions_json)
answers = {}
print(
f"\n{Colors.BOLD}{Colors.BLUE}📝 Answer the following questions:{Colors.ENDC}"
)
print(
f"{Colors.CYAN}(Type your answer and press Enter for each question){Colors.ENDC}\n"
)
for i, question in enumerate(questions, 1):
print(
f"\n{Colors.BOLD}{Colors.YELLOW}Q{i}:{Colors.ENDC} {Colors.CYAN}{question}{Colors.ENDC}"
)
print(f"{Colors.BOLD}{Colors.OKCYAN}Your answer:{Colors.ENDC} ", end="")
answer = input().strip()
answers[f"question_{i}"] = answer
if answer:
self.print_status(f"Answer {i} recorded", "success")
else:
self.print_status(f"Answer {i} left blank", "warning")
return answers
except json.JSONDecodeError:
self.print_status("Failed to parse questions", "error")
return {}
def display_requirement_summary(self, summary: str):
"""Display generated requirement document (NEW: matching UI version)"""
self.print_separator("═", 79, Colors.GREEN)
print(
f"\n{Colors.BOLD}{Colors.GREEN}📄 Generated Requirement Document{Colors.ENDC}\n"
)
self.print_separator("─", 79, Colors.GREEN)
print(f"{Colors.CYAN}{summary}{Colors.ENDC}")
self.print_separator("═", 79, Colors.GREEN)
# Ask if user wants to proceed with implementation
proceed = (
input(
f"\n{Colors.BOLD}{Colors.YELLOW}Would you like to proceed with code implementation based on these requirements? (y/n):{Colors.ENDC} "
)
.strip()
.lower()
)
return proceed == "y"
def ask_continue(self) -> bool:
"""Ask if user wants to continue with another paper"""
self.print_separator("─", 79, Colors.YELLOW)
print(f"\n{Colors.BOLD}{Colors.YELLOW}🔄 Process another paper?{Colors.ENDC}")
choice = input(f"{Colors.OKCYAN}Continue? (y/n): {Colors.ENDC}").strip().lower()
return choice in ["y", "yes", "1", "true"]
def add_to_history(self, input_source: str, result: dict):
"""Add processing result to history"""
entry = {
"timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
"input_source": input_source,
"status": result.get("status", "unknown"),
"result": result,
}
self.processing_history.append(entry)
def show_history(self):
"""Display processing history"""
if not self.processing_history:
self.print_status("No processing history available", "info")
return
print(f"\n{Colors.BOLD}{Colors.CYAN}📚 PROCESSING HISTORY{Colors.ENDC}")
self.print_separator("─", 79, Colors.CYAN)
for i, entry in enumerate(self.processing_history, 1):
status_icon = "✅" if entry["status"] == "success" else "❌"
source = entry["input_source"]
if len(source) > 50:
source = source[:47] + "..."
print(f"{i}. {status_icon} {entry['timestamp']} | {source}")
self.print_separator("─", 79, Colors.CYAN)
def show_configuration_menu(self):
"""Show configuration options menu"""
self.clear_screen()
# Get segmentation config status
segmentation_enabled = getattr(self, "segmentation_enabled", True)
segmentation_threshold = getattr(self, "segmentation_threshold", 50000)
print(f"""
{Colors.BOLD}{Colors.CYAN}╔═══════════════════════════════════════════════════════════════════════════════╗
║ CONFIGURATION MENU ║
╠═══════════════════════════════════════════════════════════════════════════════╣
║ ║
║ {Colors.BOLD}🤖 Agent Orchestration Engine Configuration{Colors.CYAN} ║
║ ║
║ {Colors.OKCYAN}[1] Pipeline Mode:{Colors.CYAN} ║
║ {Colors.BOLD}🧠 Comprehensive Mode{Colors.CYAN} - Full intelligence analysis (Default) ║
║ ✓ Research Analysis + Resource Processing ║
║ ✓ Reference Intelligence Discovery ║
║ ✓ Automated Repository Acquisition ║
║ ✓ Codebase Intelligence Orchestration ║
║ ✓ Intelligent Code Implementation Synthesis ║
║ ║
║ {Colors.BOLD}⚡ Optimized Mode{Colors.CYAN} - Fast processing (Skip indexing) ║
║ ✓ Research Analysis + Resource Processing ║
║ ✓ Code Architecture Synthesis ║
║ ✓ Intelligent Code Implementation Synthesis ║
║ ✗ Reference Intelligence Discovery (Skipped) ║
║ ✗ Repository Acquisition (Skipped) ║
║ ✗ Codebase Intelligence Orchestration (Skipped) ║
║ ║
║ {Colors.OKCYAN}[2] Document Processing:{Colors.CYAN} ║
║ {Colors.BOLD}📄 Smart Segmentation{Colors.CYAN} - Intelligent document analysis (Default) ║
║ ✓ Semantic boundary detection ║
║ ✓ Algorithm integrity preservation ║
║ ✓ Formula chain recognition ║
║ ✓ Adaptive character limits ║
║ ║
║ {Colors.BOLD}📋 Traditional Processing{Colors.CYAN} - Full document reading ║
║ ✓ Complete document analysis ║
║ ✗ Smart segmentation (Disabled) ║
║ ║
║ {Colors.YELLOW}Current Settings:{Colors.CYAN} ║
║ Pipeline: {'🧠 Comprehensive Mode' if self.enable_indexing else '⚡ Optimized Mode'} ║
║ Document: {'📄 Smart Segmentation' if segmentation_enabled else '📋 Traditional Processing'} ║
║ Threshold: {segmentation_threshold} characters ║
║ ║
║ {Colors.OKGREEN}[T] Toggle Pipeline {Colors.BLUE}[S] Toggle Segmentation {Colors.FAIL}[B] Back{Colors.CYAN} ║
╚═══════════════════════════════════════════════════════════════════════════════╝{Colors.ENDC}
""")
while True:
print(
f"\n{Colors.BOLD}{Colors.OKCYAN}➤ Configuration choice: {Colors.ENDC}",
end="",
)
choice = input().strip().lower()
if choice in ["t", "toggle"]:
self.enable_indexing = not self.enable_indexing
mode = "🧠 Comprehensive" if self.enable_indexing else "⚡ Optimized"
self.print_status(f"Pipeline mode switched to: {mode}", "success")
time.sleep(1)
self.show_configuration_menu()
return
elif choice in ["s", "segmentation"]:
current_state = getattr(self, "segmentation_enabled", True)
self.segmentation_enabled = not current_state
# Save the configuration to file
self._save_segmentation_config()
seg_mode = (
"📄 Smart Segmentation"
if self.segmentation_enabled
else "📋 Traditional Processing"
)
self.print_status(
f"Document processing switched to: {seg_mode}", "success"
)
time.sleep(1)
self.show_configuration_menu()
return
elif choice in ["b", "back"]:
return
else:
self.print_status(
"Invalid choice. Please enter 'T', 'S', or 'B'.", "warning"
)
================================================
FILE: cli/cli_launcher.py
================================================
#!/usr/bin/env python3
"""
DeepCode - CLI Research Engine Launcher
DeepCode - CLI研究引擎启动器
🧬 Open-Source Code Agent by Data Intelligence Lab @ HKU (CLI Edition)
⚡ Revolutionizing research reproducibility through collaborative AI via command line
"""
import sys
from pathlib import Path
def check_dependencies():
"""检查必要的依赖是否已安装 / Check if necessary dependencies are installed"""
import importlib.util
print("🔍 Checking CLI dependencies...")
missing_deps = []
# Check asyncio availability
if importlib.util.find_spec("asyncio") is not None:
print("✅ Asyncio is available")
else:
missing_deps.append("asyncio")
# Check PyYAML availability
if importlib.util.find_spec("yaml") is not None:
print("✅ PyYAML is installed")
else:
missing_deps.append("pyyaml")
# Check Tkinter availability
if importlib.util.find_spec("tkinter") is not None:
print("✅ Tkinter is available (for file dialogs)")
else:
print("⚠️ Tkinter not available - file dialogs will use manual input")
# Check for MCP agent dependencies
if importlib.util.find_spec("mcp_agent.app") is not None:
print("✅ MCP Agent framework is available")
else:
missing_deps.append("mcp-agent")
# Check for workflow dependencies
# 添加项目根目录到路径
current_dir = Path(__file__).parent
project_root = current_dir.parent
if str(project_root) not in sys.path:
sys.path.insert(0, str(project_root))
if importlib.util.find_spec("workflows.agent_orchestration_engine") is not None:
print("✅ Workflow modules are available")
else:
print("⚠️ Workflow modules may not be properly configured")
# Check for CLI components
if importlib.util.find_spec("cli.cli_app") is not None:
print("✅ CLI application components are available")
else:
print("❌ CLI application components missing")
missing_deps.append("cli-components")
if missing_deps:
print("\n❌ Missing dependencies:")
for dep in missing_deps:
print(f" - {dep}")
print("\nPlease install missing dependencies using:")
print(
f"pip install {' '.join([d for d in missing_deps if d != 'cli-components'])}"
)
if "cli-components" in missing_deps:
print(
"CLI components appear to be missing - please check the cli/ directory"
)
return False
print("✅ All CLI dependencies satisfied")
return True
def print_banner():
"""显示CLI启动横幅 / Display CLI startup banner"""
banner = """
╔══════════════════════════════════════════════════════════════╗
║ ║
║ 🧬 DeepCode - Open-Source Code Agent ║
║ ║
║ ⚡ DATA INTELLIGENCE LAB @ HKU ⚡ ║
║ ║
║ ║
║ ║
╚══════════════════════════════════════════════════════════════╝
"""
print(banner)
def main():
"""主函数 / Main function"""
print_banner()
# 检查依赖 / Check dependencies
if not check_dependencies():
print("\n🚨 Please install missing dependencies and try again.")
sys.exit(1)
# 获取当前脚本目录 / Get current script directory
current_dir = Path(__file__).parent
project_root = current_dir.parent
cli_app_path = current_dir / "cli_app.py"
# 检查cli_app.py是否存在 / Check if cli_app.py exists
if not cli_app_path.exists():
print(f"❌ CLI application file not found: {cli_app_path}")
print("Please ensure the cli/cli_app.py file exists.")
sys.exit(1)
print(f"\n📁 CLI App location: {cli_app_path}")
print("🖥️ Starting DeepCode CLI interface...")
print("🚀 Initializing command line application")
print("=" * 70)
print("💡 Tip: Follow the interactive prompts to process your research")
print("🛑 Press Ctrl+C to exit at any time")
print("=" * 70)
# 启动CLI应用 / Launch CLI application
try:
# 导入并运行CLI应用
if str(project_root) not in sys.path:
sys.path.insert(0, str(project_root)) # 添加项目根目录到路径
from cli.cli_app import main as cli_main
print("\n🎯 Launching CLI application...")
# 使用asyncio运行主函数
import asyncio
asyncio.run(cli_main())
except KeyboardInterrupt:
print("\n\n🛑 DeepCode CLI stopped by user")
print("Thank you for using DeepCode CLI! 🧬")
except ImportError as e:
print(f"\n❌ Failed to import CLI application: {e}")
print("Please check if all modules are properly installed.")
sys.exit(1)
except Exception as e:
print(f"\n❌ Unexpected error: {e}")
print("Please check your Python environment and try again.")
sys.exit(1)
if __name__ == "__main__":
main()
================================================
FILE: cli/main_cli.py
================================================
#!/usr/bin/env python3
"""
DeepCode CLI - Open-Source Code Agent
深度代码CLI - 开源代码智能体
🧬 Data Intelligence Lab @ HKU
⚡ Revolutionizing Research Reproducibility through Multi-Agent Architecture
"""
import os
import sys
import asyncio
import argparse
# 禁止生成.pyc文件
os.environ["PYTHONDONTWRITEBYTECODE"] = "1"
# 添加项目根目录到路径
current_dir = os.path.dirname(os.path.abspath(__file__))
parent_dir = os.path.dirname(current_dir)
if parent_dir not in sys.path:
sys.path.insert(0, parent_dir)
# 导入CLI应用
from cli.cli_app import CLIApp, Colors
def print_enhanced_banner():
"""显示增强版启动横幅"""
banner = f"""
{Colors.CYAN}╔══════════════════════════════════════════════════════════════════════════════╗
║ ║
║ {Colors.BOLD}{Colors.MAGENTA}🧬 DeepCode - Open-Source Code Agent{Colors.CYAN} ║
║ ║
║ {Colors.BOLD}{Colors.YELLOW}⚡ DATA INTELLIGENCE LAB @ HKU ⚡{Colors.CYAN} ║
║ ║
║ Revolutionizing research reproducibility through collaborative AI ║
║ Building the future where code is reproduced from natural language ║
║ ║
║ {Colors.BOLD}{Colors.GREEN}🤖 Key Features:{Colors.CYAN} ║
║ • Automated paper-to-code reproduction ║
║ • Multi-agent collaborative architecture ║
║ • Open-source and extensible design ║
║ • Join our growing research community ║
║ ║
╚══════════════════════════════════════════════════════════════════════════════╝{Colors.ENDC}
"""
print(banner)
def check_environment():
"""检查运行环境"""
print(f"{Colors.CYAN}🔍 Checking environment...{Colors.ENDC}")
# 检查Python版本
if sys.version_info < (3, 8):
print(
f"{Colors.FAIL}❌ Python 3.8+ required. Current: {sys.version}{Colors.ENDC}"
)
return False
print(f"{Colors.OKGREEN}✅ Python {sys.version.split()[0]} - OK{Colors.ENDC}")
# 检查必要模块
required_modules = [
("asyncio", "Async IO support"),
("pathlib", "Path handling"),
("typing", "Type hints"),
]
missing_modules = []
for module, desc in required_modules:
try:
__import__(module)
print(f"{Colors.OKGREEN}✅ {desc} - OK{Colors.ENDC}")
except ImportError:
missing_modules.append(module)
print(f"{Colors.FAIL}❌ {desc} - Missing{Colors.ENDC}")
if missing_modules:
print(
f"{Colors.FAIL}❌ Missing required modules: {', '.join(missing_modules)}{Colors.ENDC}"
)
return False
print(f"{Colors.OKGREEN}✅ Environment check passed{Colors.ENDC}")
return True
def parse_arguments():
"""解析命令行参数"""
parser = argparse.ArgumentParser(
description="DeepCode CLI - Open-Source Code Agent by Data Intelligence Lab @ HKU",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=f"""
{Colors.BOLD}Examples:{Colors.ENDC}
{Colors.CYAN}python main_cli.py{Colors.ENDC} # Interactive mode
{Colors.CYAN}python main_cli.py --file paper.pdf{Colors.ENDC} # Process file directly
{Colors.CYAN}python main_cli.py --url https://...{Colors.ENDC} # Process URL directly
{Colors.CYAN}python main_cli.py --chat "Build a web app..."{Colors.ENDC} # Process chat requirements
{Colors.CYAN}python main_cli.py --requirement "ML system for..."{Colors.ENDC} # Guided requirement analysis (NEW)
{Colors.CYAN}python main_cli.py --optimized{Colors.ENDC} # Use optimized mode
{Colors.CYAN}python main_cli.py --disable-segmentation{Colors.ENDC} # Disable document segmentation
{Colors.CYAN}python main_cli.py --segmentation-threshold 30000{Colors.ENDC} # Custom segmentation threshold
{Colors.BOLD}Pipeline Modes:{Colors.ENDC}
{Colors.GREEN}Comprehensive{Colors.ENDC}: Full intelligence analysis with indexing
{Colors.YELLOW}Optimized{Colors.ENDC}: Fast processing without indexing
{Colors.BLUE}Requirement Analysis{Colors.ENDC}: Guided Q&A to refine requirements (NEW)
{Colors.BOLD}Document Processing:{Colors.ENDC}
{Colors.BLUE}Smart Segmentation{Colors.ENDC}: Intelligent document segmentation for large papers
{Colors.MAGENTA}Supported Formats{Colors.ENDC}: PDF, DOCX, DOC, PPT, PPTX, XLS, XLSX, HTML, TXT, MD
""",
)
parser.add_argument(
"--file", "-f", type=str, help="Process a specific file (PDF, DOCX, TXT, etc.)"
)
parser.add_argument(
"--url", "-u", type=str, help="Process a research paper from URL"
)
parser.add_argument(
"--chat",
"-t",
type=str,
help="Process coding requirements via chat input (provide requirements as argument)",
)
parser.add_argument(
"--requirement",
"-r",
type=str,
help="Process requirements via guided analysis (provide initial idea as argument)",
)
parser.add_argument(
"--optimized",
"-o",
action="store_true",
help="Use optimized mode (skip indexing for faster processing)",
)
parser.add_argument(
"--disable-segmentation",
action="store_true",
help="Disable intelligent document segmentation (use traditional full-document processing)",
)
parser.add_argument(
"--segmentation-threshold",
type=int,
default=50000,
help="Document size threshold (characters) to trigger segmentation (default: 50000)",
)
parser.add_argument(
"--verbose", "-v", action="store_true", help="Enable verbose output"
)
return parser.parse_args()
async def run_direct_processing(app: CLIApp, input_source: str, input_type: str):
"""直接处理模式(非交互式)"""
try:
print(
f"\n{Colors.BOLD}{Colors.CYAN}🚀 Starting direct processing mode...{Colors.ENDC}"
)
print(f"{Colors.CYAN}Input: {input_source}{Colors.ENDC}")
print(f"{Colors.CYAN}Type: {input_type}{Colors.ENDC}")
print(
f"{Colors.CYAN}Mode: {'🧠 Comprehensive' if app.cli.enable_indexing else '⚡ Optimized'}{Colors.ENDC}"
)
# 初始化应用
init_result = await app.initialize_mcp_app()
if init_result["status"] != "success":
print(
f"{Colors.FAIL}❌ Initialization failed: {init_result['message']}{Colors.ENDC}"
)
return False
# 处理输入
result = await app.process_input(input_source, input_type)
if result["status"] == "success":
print(
f"\n{Colors.BOLD}{Colors.OKGREEN}🎉 Processing completed successfully!{Colors.ENDC}"
)
return True
else:
print(
f"\n{Colors.BOLD}{Colors.FAIL}❌ Processing failed: {result.get('error', 'Unknown error')}{Colors.ENDC}"
)
return False
except Exception as e:
print(f"\n{Colors.FAIL}❌ Direct processing error: {str(e)}{Colors.ENDC}")
return False
finally:
await app.cleanup_mcp_app()
async def run_requirement_analysis(app: CLIApp, initial_idea: str):
"""需求分析模式(非交互式) - NEW: matching UI version"""
try:
print(
f"\n{Colors.BOLD}{Colors.BLUE}🧠 Starting requirement analysis mode...{Colors.ENDC}"
)
print(f"{Colors.CYAN}Initial Idea: {initial_idea}{Colors.ENDC}")
# 初始化应用
init_result = await app.initialize_mcp_app()
if init_result["status"] != "success":
print(
f"{Colors.FAIL}❌ Initialization failed: {init_result['message']}{Colors.ENDC}"
)
return False
# 执行需求分析工作流
result = await app.process_requirement_analysis_non_interactive(initial_idea)
if result["status"] == "success":
print(
f"\n{Colors.BOLD}{Colors.OKGREEN}🎉 Requirement analysis completed successfully!{Colors.ENDC}"
)
return True
else:
print(
f"\n{Colors.BOLD}{Colors.FAIL}❌ Requirement analysis failed: {result.get('error', 'Unknown error')}{Colors.ENDC}"
)
return False
except Exception as e:
print(f"\n{Colors.FAIL}❌ Requirement analysis error: {str(e)}{Colors.ENDC}")
return False
finally:
await app.cleanup_mcp_app()
async def main():
"""主函数"""
# 解析命令行参数
args = parse_arguments()
# 显示横幅
print_enhanced_banner()
# 检查环境
if not check_environment():
print(
f"\n{Colors.FAIL}🚨 Environment check failed. Please fix the issues and try again.{Colors.ENDC}"
)
sys.exit(1)
try:
# 创建CLI应用
app = CLIApp()
# 设置配置 - 默认禁用索引功能以加快处理速度
if args.optimized:
app.cli.enable_indexing = False
print(
f"\n{Colors.YELLOW}⚡ Optimized mode enabled - indexing disabled{Colors.ENDC}"
)
else:
# 默认也禁用索引功能
app.cli.enable_indexing = False
print(
f"\n{Colors.YELLOW}⚡ Fast mode enabled - indexing disabled by default{Colors.ENDC}"
)
# Configure document segmentation settings
if hasattr(args, "disable_segmentation") and args.disable_segmentation:
print(
f"\n{Colors.MAGENTA}📄 Document segmentation disabled - using traditional processing{Colors.ENDC}"
)
app.cli.segmentation_enabled = False
app.cli.segmentation_threshold = args.segmentation_threshold
app.cli._save_segmentation_config()
else:
print(
f"\n{Colors.BLUE}📄 Smart document segmentation enabled (threshold: {args.segmentation_threshold} chars){Colors.ENDC}"
)
app.cli.segmentation_enabled = True
app.cli.segmentation_threshold = args.segmentation_threshold
app.cli._save_segmentation_config()
# 检查是否为直接处理模式
if args.file or args.url or args.chat or args.requirement:
if args.file:
# 验证文件存在
if not os.path.exists(args.file):
print(f"{Colors.FAIL}❌ File not found: {args.file}{Colors.ENDC}")
sys.exit(1)
# 使用 file:// 前缀保持与交互模式一致,确保文件被复制而非移动
file_url = f"file://{os.path.abspath(args.file)}"
success = await run_direct_processing(app, file_url, "file")
elif args.url:
success = await run_direct_processing(app, args.url, "url")
elif args.chat:
# 验证chat输入长度
if len(args.chat.strip()) < 20:
print(
f"{Colors.FAIL}❌ Chat input too short. Please provide more detailed requirements (at least 20 characters){Colors.ENDC}"
)
sys.exit(1)
success = await run_direct_processing(app, args.chat, "chat")
elif args.requirement:
# NEW: Requirement analysis mode
# 验证需求输入长度
if len(args.requirement.strip()) < 10:
print(
f"{Colors.FAIL}❌ Requirement input too short. Please provide more details (at least 10 characters){Colors.ENDC}"
)
sys.exit(1)
success = await run_requirement_analysis(app, args.requirement)
sys.exit(0 if success else 1)
else:
# 交互式模式
print(f"\n{Colors.CYAN}🎮 Starting interactive mode...{Colors.ENDC}")
await app.run_interactive_session()
except KeyboardInterrupt:
print(f"\n{Colors.WARNING}⚠️ Application interrupted by user{Colors.ENDC}")
sys.exit(1)
except Exception as e:
print(f"\n{Colors.FAIL}❌ Application errors: {str(e)}{Colors.ENDC}")
sys.exit(1)
if __name__ == "__main__":
asyncio.run(main())
================================================
FILE: cli/workflows/__init__.py
================================================
"""
CLI-specific Workflow Adapters
CLI专用工作流适配器
This module provides CLI-optimized versions of workflow components that are
specifically adapted for command-line interface usage patterns.
"""
from .cli_workflow_adapter import CLIWorkflowAdapter
__all__ = ["CLIWorkflowAdapter"]
================================================
FILE: cli/workflows/cli_workflow_adapter.py
================================================
"""
CLI Workflow Adapter for Agent Orchestration Engine
CLI工作流适配器 - 智能体编排引擎
This adapter provides CLI-optimized interface to the latest agent orchestration engine,
with enhanced progress reporting, error handling, and CLI-specific optimizations.
Version: 2.1 (Updated to match UI version - Added Requirement Analysis)
Changes:
- Default enable_indexing=False for faster processing (matching UI defaults)
- Mode-aware progress callback with detailed stage mapping
- Chat pipeline now accepts enable_indexing parameter
- Improved error handling and resource management
- Enhanced progress display for different modes (fast/comprehensive/chat)
- NEW: Added requirement analysis workflow support
"""
import os
from typing import Callable, Dict, Any
from mcp_agent.app import MCPApp
class CLIWorkflowAdapter:
"""
CLI-optimized workflow adapter for the intelligent agent orchestration engine.
This adapter provides:
- Enhanced CLI progress reporting
- Optimized error handling for CLI environments
- Streamlined interface for command-line usage
- Integration with the latest agent orchestration engine
"""
def __init__(self, cli_interface=None):
"""
Initialize CLI workflow adapter.
Args:
cli_interface: CLI interface instance for progress reporting
"""
self.cli_interface = cli_interface
self.app = None
self.logger = None
self.context = None
async def initialize_mcp_app(self) -> Dict[str, Any]:
"""
Initialize MCP application for CLI usage (improved version matching UI).
Returns:
dict: Initialization result
"""
try:
if self.cli_interface:
self.cli_interface.show_spinner(
"🚀 Initializing Agent Orchestration Engine", 2.0
)
# Initialize MCP application using async context manager (matching UI pattern)
self.app = MCPApp(name="cli_agent_orchestration")
self.app_context = self.app.run()
agent_app = await self.app_context.__aenter__()
self.logger = agent_app.logger
self.context = agent_app.context
# Configure filesystem access
self.context.config.mcp.servers["filesystem"].args.extend([os.getcwd()])
if self.cli_interface:
self.cli_interface.print_status(
"🧠 Agent Orchestration Engine initialized successfully", "success"
)
return {
"status": "success",
"message": "MCP application initialized successfully",
}
except Exception as e:
error_msg = f"Failed to initialize MCP application: {str(e)}"
if self.cli_interface:
self.cli_interface.print_status(error_msg, "error")
return {"status": "error", "message": error_msg}
async def cleanup_mcp_app(self):
"""
Clean up MCP application resources.
"""
if hasattr(self, "app_context"):
try:
await self.app_context.__aexit__(None, None, None)
if self.cli_interface:
self.cli_interface.print_status(
"🧹 Resources cleaned up successfully", "info"
)
except Exception as e:
if self.cli_interface:
self.cli_interface.print_status(
f"⚠️ Cleanup warning: {str(e)}", "warning"
)
def create_cli_progress_callback(self, enable_indexing: bool = True) -> Callable:
"""
Create CLI-optimized progress callback function with mode-aware stage mapping.
This matches the UI version's detailed progress mapping logic.
Args:
enable_indexing: Whether indexing is enabled (affects stage mapping)
Returns:
Callable: Progress callback function
"""
def progress_callback(progress: int, message: str):
if self.cli_interface:
# Mode-aware stage mapping (matching UI version logic)
if enable_indexing:
# Full workflow mapping: Initialize -> Analyze -> Download -> Plan -> References -> Repos -> Index -> Implement
if progress <= 5:
stage = 0 # Initialize
elif progress <= 10:
stage = 1 # Analyze
elif progress <= 25:
stage = 2 # Download
elif progress <= 40:
stage = 3 # Plan
elif progress <= 50:
stage = 4 # References
elif progress <= 60:
stage = 5 # Repos
elif progress <= 70:
stage = 6 # Index
elif progress <= 85:
stage = 7 # Implement
else:
stage = 8 # Complete
else:
# Fast mode mapping: Initialize -> Analyze -> Download -> Plan -> Implement
if progress <= 5:
stage = 0 # Initialize
elif progress <= 10:
stage = 1 # Analyze
elif progress <= 25:
stage = 2 # Download
elif progress <= 40:
stage = 3 # Plan
elif progress <= 85:
stage = 4 # Implement (skip References, Repos, Index)
else:
stage = 4 # Complete
self.cli_interface.display_processing_stages(stage, enable_indexing)
# Display status message
self.cli_interface.print_status(message, "processing")
return progress_callback
async def execute_full_pipeline(
self, input_source: str, enable_indexing: bool = False
) -> Dict[str, Any]:
"""
Execute the complete intelligent multi-agent research orchestration pipeline.
Updated to match UI version: default enable_indexing=False for faster processing.
Args:
input_source: Research input source (file path, URL, or preprocessed analysis)
enable_indexing: Whether to enable advanced intelligence analysis (default: False)
Returns:
dict: Comprehensive pipeline execution result
"""
try:
# Import the latest agent orchestration engine
from workflows.agent_orchestration_engine import (
execute_multi_agent_research_pipeline,
)
# Create CLI progress callback with mode awareness
progress_callback = self.create_cli_progress_callback(enable_indexing)
# Display pipeline start
if self.cli_interface:
if enable_indexing:
mode_msg = "🧠 comprehensive (with indexing)"
else:
mode_msg = "⚡ fast (indexing disabled)"
self.cli_interface.print_status(
f"🚀 Starting {mode_msg} agent orchestration pipeline...",
"processing",
)
self.cli_interface.display_processing_stages(0, enable_indexing)
# Execute the pipeline
result = await execute_multi_agent_research_pipeline(
input_source=input_source,
logger=self.logger,
progress_callback=progress_callback,
enable_indexing=enable_indexing,
)
# Display completion
if self.cli_interface:
final_stage = 8 if enable_indexing else 4
self.cli_interface.display_processing_stages(
final_stage, enable_indexing
)
self.cli_interface.print_status(
"🎉 Agent orchestration pipeline completed successfully!",
"complete",
)
return {
"status": "success",
"result": result,
"pipeline_mode": "comprehensive" if enable_indexing else "optimized",
}
except Exception as e:
error_msg = f"Pipeline execution failed: {str(e)}"
if self.cli_interface:
self.cli_interface.print_status(error_msg, "error")
return {
"status": "error",
"error": error_msg,
"pipeline_mode": "comprehensive" if enable_indexing else "optimized",
}
async def execute_requirement_analysis_workflow(
self, user_input: str, analysis_mode: str, user_answers: Dict[str, str] = None
) -> Dict[str, Any]:
"""
Execute requirement analysis workflow (NEW: matching UI version).
This workflow helps users refine their requirements through guided questions
and intelligent analysis before starting code implementation.
Args:
user_input: User's initial requirements or description
analysis_mode: Analysis mode ("generate_questions" or "summarize_requirements")
user_answers: Dictionary of user answers to guiding questions (for summarize mode)
Returns:
dict: Analysis result with questions or requirement summary
"""
try:
# Import the requirement analysis workflow
from workflows.agent_orchestration_engine import (
execute_requirement_analysis_workflow,
)
# Create CLI progress callback
def analysis_progress_callback(progress: int, message: str):
if self.cli_interface:
self.cli_interface.print_status(message, "processing")
# Display workflow start
if self.cli_interface:
if analysis_mode == "generate_questions":
self.cli_interface.print_status(
"🤖 Generating guiding questions for your requirements...",
"processing",
)
else:
self.cli_interface.print_status(
"📄 Analyzing and summarizing your detailed requirements...",
"processing",
)
# Execute the requirement analysis workflow
result = await execute_requirement_analysis_workflow(
user_input=user_input,
analysis_mode=analysis_mode,
user_answers=user_answers,
logger=self.logger,
progress_callback=analysis_progress_callback,
)
# Display completion
if self.cli_interface:
if result["status"] == "success":
if analysis_mode == "generate_questions":
self.cli_interface.print_status(
"✅ Guiding questions generated successfully!", "success"
)
else:
self.cli_interface.print_status(
"✅ Requirements analysis completed successfully!",
"success",
)
else:
self.cli_interface.print_status(
f"❌ Analysis failed: {result.get('error', 'Unknown error')}",
"error",
)
return result
except Exception as e:
error_msg = f"Requirement analysis workflow failed: {str(e)}"
if self.cli_interface:
self.cli_interface.print_status(error_msg, "error")
return {"status": "error", "error": error_msg}
async def execute_chat_pipeline(
self, user_input: str, enable_indexing: bool = False
) -> Dict[str, Any]:
"""
Execute the chat-based planning and implementation pipeline.
Updated to match UI version: accepts enable_indexing parameter.
Args:
user_input: User's coding requirements and description
enable_indexing: Whether to enable indexing for enhanced code understanding (default: False)
Returns:
dict: Chat pipeline execution result
"""
try:
# Import the chat-based pipeline
from workflows.agent_orchestration_engine import (
execute_chat_based_planning_pipeline,
)
# Create CLI progress callback for chat mode
def chat_progress_callback(progress: int, message: str):
if self.cli_interface:
# Map progress to CLI stages for chat mode (matching UI logic)
if progress <= 5:
stage = 0 # Initialize
elif progress <= 30:
stage = 1 # Planning
elif progress <= 50:
stage = 2 # Setup
elif progress <= 70:
stage = 3 # Save Plan
else:
stage = 4 # Implement
self.cli_interface.display_processing_stages(stage, chat_mode=True)
# Display status message
self.cli_interface.print_status(message, "processing")
# Display pipeline start
if self.cli_interface:
indexing_note = (
" (with indexing)" if enable_indexing else " (fast mode)"
)
self.cli_interface.print_status(
f"🚀 Starting chat-based planning pipeline{indexing_note}...",
"processing",
)
self.cli_interface.display_processing_stages(0, chat_mode=True)
# Execute the chat pipeline with configurable indexing
result = await execute_chat_based_planning_pipeline(
user_input=user_input,
logger=self.logger,
progress_callback=chat_progress_callback,
enable_indexing=enable_indexing, # Pass through enable_indexing parameter
)
# Display completion
if self.cli_interface:
self.cli_interface.display_processing_stages(4, chat_mode=True)
self.cli_interface.print_status(
"🎉 Chat-based planning pipeline completed successfully!",
"complete",
)
return {"status": "success", "result": result, "pipeline_mode": "chat"}
except Exception as e:
error_msg = f"Chat pipeline execution failed: {str(e)}"
if self.cli_interface:
self.cli_interface.print_status(error_msg, "error")
return {"status": "error", "error": error_msg, "pipeline_mode": "chat"}
async def process_input_with_orchestration(
self, input_source: str, input_type: str, enable_indexing: bool = False
) -> Dict[str, Any]:
"""
Process input using the intelligent agent orchestration engine.
This is the main CLI interface to the latest agent orchestration capabilities.
Updated to match UI version: default enable_indexing=False.
Args:
input_source: Input source (file path, URL, or chat input)
input_type: Type of input ('file', 'url', or 'chat')
enable_indexing: Whether to enable advanced intelligence analysis (default: False)
Returns:
dict: Processing result with status and details
"""
pipeline_result = None
try:
# Initialize MCP app
init_result = await self.initialize_mcp_app()
if init_result["status"] != "success":
return init_result
# Process file:// URLs for traditional file/URL inputs
if input_source.startswith("file://"):
file_path = input_source[7:]
if os.name == "nt" and file_path.startswith("/"):
file_path = file_path.lstrip("/")
input_source = file_path
# Execute appropriate pipeline based on input type
if input_type == "chat":
# Use chat-based planning pipeline for user requirements
# Pass enable_indexing to chat pipeline as well
pipeline_result = await self.execute_chat_pipeline(
input_source, enable_indexing=enable_indexing
)
else:
# Use traditional multi-agent research pipeline for files/URLs
pipeline_result = await self.execute_full_pipeline(
input_source, enable_indexing=enable_indexing
)
return {
"status": pipeline_result["status"],
"analysis_result": "Integrated into agent orchestration pipeline",
"download_result": "Integrated into agent orchestration pipeline",
"repo_result": pipeline_result.get("result", ""),
"pipeline_mode": pipeline_result.get("pipeline_mode", "comprehensive"),
"error": pipeline_result.get("error"),
}
except Exception as e:
error_msg = f"Error during orchestrated processing: {str(e)}"
if self.cli_interface:
self.cli_interface.print_status(error_msg, "error")
return {
"status": "error",
"error": error_msg,
"analysis_result": "",
"download_result": "",
"repo_result": "",
"pipeline_mode": "comprehensive" if enable_indexing else "optimized",
}
finally:
# Clean up resources
await self.cleanup_mcp_app()
================================================
FILE: config/mcp_tool_definitions.py
================================================
"""
MCP工具定义配置模块
MCP Tool Definitions Configuration Module
将工具定义从主程序逻辑中分离,提供标准化的工具定义格式
Separate tool definitions from main program logic, providing standardized tool definition format
支持的工具类型:
- 文件操作工具 (File Operations)
- 代码执行工具 (Code Execution)
- 搜索工具 (Search Tools)
- 项目结构工具 (Project Structure Tools)
"""
from typing import Dict, List, Any
class MCPToolDefinitions:
"""MCP工具定义管理器"""
@staticmethod
def get_code_implementation_tools() -> List[Dict[str, Any]]:
"""
获取代码实现相关的工具定义
Get tool definitions for code implementation
"""
return [
# MCPToolDefinitions._get_read_file_tool(),
# MCPToolDefinitions._get_read_multiple_files_tool(),
# MCPToolDefinitions._get_read_code_mem_tool(),
MCPToolDefinitions._get_write_file_tool(),
# MCPToolDefinitions._get_write_multiple_files_tool(),
# MCPToolDefinitions._get_execute_python_tool(),
# MCPToolDefinitions._get_execute_bash_tool(),
]
@staticmethod
def _get_read_file_tool() -> Dict[str, Any]:
"""读取文件工具定义"""
return {
"name": "read_file",
"description": "Read file content, supports specifying line number range",
"input_schema": {
"type": "object",
"properties": {
"file_path": {
"type": "string",
"description": "File path, relative to workspace",
},
"start_line": {
"type": "integer",
"description": "Start line number (starting from 1, optional)",
},
"end_line": {
"type": "integer",
"description": "End line number (starting from 1, optional)",
},
},
"required": ["file_path"],
},
}
@staticmethod
def _get_read_multiple_files_tool() -> Dict[str, Any]:
"""批量读取多个文件工具定义"""
return {
"name": "read_multiple_files",
"description": "Read multiple files in a single operation (for batch reading)",
"input_schema": {
"type": "object",
"properties": {
"file_requests": {
"type": "string",
"description": 'JSON string with file requests, e.g., \'{"file1.py": {}, "file2.py": {"start_line": 1, "end_line": 10}}\' or simple array \'["file1.py", "file2.py"]\'',
},
"max_files": {
"type": "integer",
"description": "Maximum number of files to read in one operation",
"default": 5,
"minimum": 1,
"maximum": 10,
},
},
"required": ["file_requests"],
},
}
@staticmethod
def _get_read_code_mem_tool() -> Dict[str, Any]:
"""Read code memory tool definition - reads from implement_code_summary.md"""
return {
"name": "read_code_mem",
"description": "Check if file summaries exist in implement_code_summary.md for multiple files in a single call. Returns summaries for all requested files if available.",
"input_schema": {
"type": "object",
"properties": {
"file_paths": {
"type": "array",
"items": {"type": "string"},
"description": "List of file paths to check for summary information in implement_code_summary.md",
}
},
"required": ["file_paths"],
},
}
@staticmethod
def _get_write_file_tool() -> Dict[str, Any]:
"""写入文件工具定义"""
return {
"name": "write_file",
"description": "Write content to file",
"input_schema": {
"type": "object",
"properties": {
"file_path": {
"type": "string",
"description": "File path, relative to workspace",
},
"content": {
"type": "string",
"description": "Content to write to file",
},
"create_dirs": {
"type": "boolean",
"description": "Whether to create directories if they don't exist",
"default": True,
},
"create_backup": {
"type": "boolean",
"description": "Whether to create backup file if file already exists",
"default": False,
},
},
"required": ["file_path", "content"],
},
}
@staticmethod
def _get_write_multiple_files_tool() -> Dict[str, Any]:
"""批量写入多个文件工具定义"""
return {
"name": "write_multiple_files",
"description": "Write multiple files in a single operation (for batch implementation)",
"input_schema": {
"type": "object",
"properties": {
"file_implementations": {
"type": "string",
"description": 'JSON string mapping file paths to content, e.g., \'{"file1.py": "content1", "file2.py": "content2"}\'',
},
"create_dirs": {
"type": "boolean",
"description": "Whether to create directories if they don't exist",
"default": True,
},
"create_backup": {
"type": "boolean",
"description": "Whether to create backup files if they already exist",
"default": False,
},
"max_files": {
"type": "integer",
"description": "Maximum number of files to write in one operation",
"default": 5,
"minimum": 1,
"maximum": 10,
},
},
"required": ["file_implementations"],
},
}
@staticmethod
def _get_execute_python_tool() -> Dict[str, Any]:
"""Python执行工具定义"""
return {
"name": "execute_python",
"description": "Execute Python code and return output",
"input_schema": {
"type": "object",
"properties": {
"code": {"type": "string", "description": "Python code to execute"},
"timeout": {
"type": "integer",
"description": "Timeout in seconds",
"default": 30,
},
},
"required": ["code"],
},
}
@staticmethod
def _get_execute_bash_tool() -> Dict[str, Any]:
"""Bash执行工具定义"""
return {
"name": "execute_bash",
"description": "Execute bash command",
"input_schema": {
"type": "object",
"properties": {
"command": {
"type": "string",
"description": "Bash command to execute",
},
"timeout": {
"type": "integer",
"description": "Timeout in seconds",
"default": 30,
},
},
"required": ["command"],
},
}
@staticmethod
def _get_file_structure_tool() -> Dict[str, Any]:
"""文件结构获取工具定义"""
return {
"name": "get_file_structure",
"description": "Get directory file structure",
"input_schema": {
"type": "object",
"properties": {
"directory": {
"type": "string",
"description": "Directory path, relative to workspace",
"default": ".",
},
"max_depth": {
"type": "integer",
"description": "Maximum traversal depth",
"default": 5,
},
},
},
}
@staticmethod
def _get_search_code_references_tool() -> Dict[str, Any]:
"""统一代码参考搜索工具定义 - 合并了三个步骤为一个工具"""
return {
"name": "search_code_references",
"description": "UNIFIED TOOL: Search relevant reference code from index files. Combines directory setup, index loading, and searching in a single call.",
"input_schema": {
"type": "object",
"properties": {
"indexes_path": {
"type": "string",
"description": "Path to the indexes directory containing JSON index files",
},
"target_file": {
"type": "string",
"description": "Target file path to be implemented",
},
"keywords": {
"type": "string",
"description": "Search keywords, comma-separated",
"default": "",
},
"max_results": {
"type": "integer",
"description": "Maximum number of results to return",
"default": 10,
},
},
"required": ["indexes_path", "target_file"],
},
}
@staticmethod
def _get_get_indexes_overview_tool() -> Dict[str, Any]:
"""获取索引概览工具定义"""
return {
"name": "get_indexes_overview",
"description": "Get overview of all available reference code index information from specified directory",
"input_schema": {
"type": "object",
"properties": {
"indexes_path": {
"type": "string",
"description": "Path to the indexes directory containing JSON index files",
}
},
"required": ["indexes_path"],
},
}
@staticmethod
def _get_set_workspace_tool() -> Dict[str, Any]:
"""Set workspace directory tool definition"""
return {
"name": "set_workspace",
"description": "Set the workspace directory for file operations",
"input_schema": {
"type": "object",
"properties": {
"workspace_path": {
"type": "string",
"description": "Directory path for the workspace",
}
},
"required": ["workspace_path"],
},
}
# @staticmethod
# def _get_set_indexes_directory_tool() -> Dict[str, Any]:
# """Set indexes directory tool definition - DEPRECATED: Use unified search_code_references instead"""
# return {
# "name": "set_indexes_directory",
# "description": "Set the directory path for code reference indexes",
# "input_schema": {
# "type": "object",
# "properties": {
# "indexes_path": {
# "type": "string",
# "description": "Directory path containing index JSON files"
# }
# },
# "required": ["indexes_path"]
# }
# }
@staticmethod
def get_available_tool_sets() -> Dict[str, str]:
"""
获取可用的工具集合
Get available tool sets
"""
return {
"code_implementation": "代码实现相关工具集 / Code implementation tool set",
# 可以在这里添加更多工具集
# "data_analysis": "数据分析工具集 / Data analysis tool set",
# "web_scraping": "网页爬取工具集 / Web scraping tool set",
}
@staticmethod
def get_tool_set(tool_set_name: str) -> List[Dict[str, Any]]:
"""
根据名称获取特定的工具集
Get specific tool set by name
"""
tool_sets = {
"code_implementation": MCPToolDefinitions.get_code_implementation_tools(),
}
return tool_sets.get(tool_set_name, [])
@staticmethod
def get_all_tools() -> List[Dict[str, Any]]:
"""
获取所有可用工具
Get all available tools
"""
all_tools = []
for tool_set_name in MCPToolDefinitions.get_available_tool_sets().keys():
all_tools.extend(MCPToolDefinitions.get_tool_set(tool_set_name))
return all_tools
# 便捷访问函数
def get_mcp_tools(tool_set: str = "code_implementation") -> List[Dict[str, Any]]:
"""
便捷函数:获取MCP工具定义
Convenience function: Get MCP tool definitions
Args:
tool_set: 工具集名称 (默认: "code_implementation")
Returns:
工具定义列表
"""
return MCPToolDefinitions.get_tool_set(tool_set)
================================================
FILE: config/mcp_tool_definitions_index.py
================================================
"""
MCP工具定义配置模块
MCP Tool Definitions Configuration Module
将工具定义从主程序逻辑中分离,提供标准化的工具定义格式
Separate tool definitions from main program logic, providing standardized tool definition format
支持的工具类型:
- 文件操作工具 (File Operations)
- 代码执行工具 (Code Execution)
- 搜索工具 (Search Tools)
- 项目结构工具 (Project Structure Tools)
"""
from typing import Dict, List, Any
class MCPToolDefinitions:
"""MCP工具定义管理器"""
@staticmethod
def get_code_implementation_tools() -> List[Dict[str, Any]]:
"""
获取代码实现相关的工具定义
Get tool definitions for code implementation
"""
return [
# MCPToolDefinitions._get_read_file_tool(),
# MCPToolDefinitions._get_read_multiple_files_tool(),
# MCPToolDefinitions._get_read_code_mem_tool(),
MCPToolDefinitions._get_write_file_tool(),
# MCPToolDefinitions._get_write_multiple_files_tool(),
# MCPToolDefinitions._get_execute_python_tool(),
# MCPToolDefinitions._get_execute_bash_tool(),
MCPToolDefinitions._get_search_code_references_tool(),
# MCPToolDefinitions._get_search_code_tool(),
# MCPToolDefinitions._get_file_structure_tool(),
# MCPToolDefinitions._get_set_workspace_tool(),
# MCPToolDefinitions._get_operation_history_tool(),
]
@staticmethod
def get_code_evaluation_tools() -> List[Dict[str, Any]]:
"""
获取代码评估相关的工具定义
Get tool definitions for code evaluation
"""
return [
MCPToolDefinitions._get_analyze_repo_structure_tool(),
MCPToolDefinitions._get_detect_dependencies_tool(),
MCPToolDefinitions._get_assess_code_quality_tool(),
MCPToolDefinitions._get_evaluate_documentation_tool(),
MCPToolDefinitions._get_check_reproduction_readiness_tool(),
MCPToolDefinitions._get_generate_evaluation_summary_tool(),
MCPToolDefinitions._get_detect_empty_files_tool(),
MCPToolDefinitions._get_detect_missing_files_tool(),
MCPToolDefinitions._get_generate_code_revision_report_tool(),
]
@staticmethod
def _get_read_file_tool() -> Dict[str, Any]:
"""读取文件工具定义"""
return {
"name": "read_file",
"description": "Read file content, supports specifying line number range",
"input_schema": {
"type": "object",
"properties": {
"file_path": {
"type": "string",
"description": "File path, relative to workspace",
},
"start_line": {
"type": "integer",
"description": "Start line number (starting from 1, optional)",
},
"end_line": {
"type": "integer",
"description": "End line number (starting from 1, optional)",
},
},
"required": ["file_path"],
},
}
@staticmethod
def _get_read_multiple_files_tool() -> Dict[str, Any]:
"""批量读取多个文件工具定义"""
return {
"name": "read_multiple_files",
"description": "Read multiple files in a single operation (for batch reading)",
"input_schema": {
"type": "object",
"properties": {
"file_requests": {
"type": "string",
"description": 'JSON string with file requests, e.g., \'{"file1.py": {}, "file2.py": {"start_line": 1, "end_line": 10}}\' or simple array \'["file1.py", "file2.py"]\'',
},
"max_files": {
"type": "integer",
"description": "Maximum number of files to read in one operation",
"default": 5,
"minimum": 1,
"maximum": 10,
},
},
"required": ["file_requests"],
},
}
@staticmethod
def _get_read_code_mem_tool() -> Dict[str, Any]:
"""Read code memory tool definition - reads from implement_code_summary.md"""
return {
"name": "read_code_mem",
"description": "Check if file summaries exist in implement_code_summary.md for multiple files in a single call. Returns summaries for all requested files if available.",
"input_schema": {
"type": "object",
"properties": {
"file_paths": {
"type": "array",
"items": {"type": "string"},
"description": "List of file paths to check for summary information in implement_code_summary.md",
}
},
"required": ["file_paths"],
},
}
@staticmethod
def _get_write_file_tool() -> Dict[str, Any]:
"""写入文件工具定义"""
return {
"name": "write_file",
"description": "Write content to file",
"input_schema": {
"type": "object",
"properties": {
"file_path": {
"type": "string",
"description": "File path, relative to workspace",
},
"content": {
"type": "string",
"description": "Content to write to file",
},
"create_dirs": {
"type": "boolean",
"description": "Whether to create directories if they don't exist",
"default": True,
},
"create_backup": {
"type": "boolean",
"description": "Whether to create backup file if file already exists",
"default": False,
},
},
"required": ["file_path", "content"],
},
}
@staticmethod
def _get_write_multiple_files_tool() -> Dict[str, Any]:
"""批量写入多个文件工具定义"""
return {
"name": "write_multiple_files",
"description": "Write multiple files in a single operation (for batch implementation)",
"input_schema": {
"type": "object",
"properties": {
"file_implementations": {
"type": "string",
"description": 'JSON string mapping file paths to content, e.g., \'{"file1.py": "content1", "file2.py": "content2"}\'',
},
"create_dirs": {
"type": "boolean",
"description": "Whether to create directories if they don't exist",
"default": True,
},
"create_backup": {
"type": "boolean",
"description": "Whether to create backup files if they already exist",
"default": False,
},
"max_files": {
"type": "integer",
"description": "Maximum number of files to write in one operation",
"default": 5,
"minimum": 1,
"maximum": 10,
},
},
"required": ["file_implementations"],
},
}
@staticmethod
def _get_execute_python_tool() -> Dict[str, Any]:
"""Python执行工具定义"""
return {
"name": "execute_python",
"description": "Execute Python code and return output",
"input_schema": {
"type": "object",
"properties": {
"code": {"type": "string", "description": "Python code to execute"},
"timeout": {
"type": "integer",
"description": "Timeout in seconds",
"default": 30,
},
},
"required": ["code"],
},
}
@staticmethod
def _get_execute_bash_tool() -> Dict[str, Any]:
"""Bash执行工具定义"""
return {
"name": "execute_bash",
"description": "Execute bash command",
"input_schema": {
"type": "object",
"properties": {
"command": {
"type": "string",
"description": "Bash command to execute",
},
"timeout": {
"type": "integer",
"description": "Timeout in seconds",
"default": 30,
},
},
"required": ["command"],
},
}
@staticmethod
def _get_file_structure_tool() -> Dict[str, Any]:
"""文件结构获取工具定义"""
return {
"name": "get_file_structure",
"description": "Get directory file structure",
"input_schema": {
"type": "object",
"properties": {
"directory": {
"type": "string",
"description": "Directory path, relative to workspace",
"default": ".",
},
"max_depth": {
"type": "integer",
"description": "Maximum traversal depth",
"default": 5,
},
},
},
}
@staticmethod
def _get_search_code_references_tool() -> Dict[str, Any]:
"""统一代码参考搜索工具定义 - 合并了三个步骤为一个工具"""
return {
"name": "search_code_references",
"description": "UNIFIED TOOL: Search relevant reference code from index files. Combines directory setup, index loading, and searching in a single call.",
"input_schema": {
"type": "object",
"properties": {
"indexes_path": {
"type": "string",
"description": "Path to the indexes directory containing JSON index files",
},
"target_file": {
"type": "string",
"description": "Target file path to be implemented",
},
"keywords": {
"type": "string",
"description": "Search keywords, comma-separated",
"default": "",
},
"max_results": {
"type": "integer",
"description": "Maximum number of results to return",
"default": 10,
},
},
"required": ["indexes_path", "target_file"],
},
}
@staticmethod
def _get_search_code_tool() -> Dict[str, Any]:
"""代码搜索工具定义 - 在当前代码库中搜索模式"""
return {
"name": "search_code",
"description": "Search patterns in code files within the current repository",
"input_schema": {
"type": "object",
"properties": {
"pattern": {
"type": "string",
"description": "Search pattern",
},
"file_pattern": {
"type": "string",
"description": "File pattern (e.g., '*.py')",
"default": "*.py",
},
"use_regex": {
"type": "boolean",
"description": "Whether to use regular expressions",
"default": False,
},
"search_directory": {
"type": "string",
"description": "Specify search directory (optional)",
},
},
"required": ["pattern"],
},
}
@staticmethod
def _get_operation_history_tool() -> Dict[str, Any]:
"""操作历史工具定义"""
return {
"name": "get_operation_history",
"description": "Get operation history",
"input_schema": {
"type": "object",
"properties": {
"last_n": {
"type": "integer",
"description": "Return the last N operations",
"default": 10,
},
},
},
}
@staticmethod
def _get_get_indexes_overview_tool() -> Dict[str, Any]:
"""获取索引概览工具定义"""
return {
"name": "get_indexes_overview",
"description": "Get overview of all available reference code index information from specified directory",
"input_schema": {
"type": "object",
"properties": {
"indexes_path": {
"type": "string",
"description": "Path to the indexes directory containing JSON index files",
}
},
"required": ["indexes_path"],
},
}
@staticmethod
def _get_set_workspace_tool() -> Dict[str, Any]:
"""Set workspace directory tool definition"""
return {
"name": "set_workspace",
"description": "Set the workspace directory for file operations",
"input_schema": {
"type": "object",
"properties": {
"workspace_path": {
"type": "string",
"description": "Directory path for the workspace",
}
},
"required": ["workspace_path"],
},
}
# @staticmethod
# def _get_set_indexes_directory_tool() -> Dict[str, Any]:
# """Set indexes directory tool definition - DEPRECATED: Use unified search_code_references instead"""
# return {
# "name": "set_indexes_directory",
# "description": "Set the directory path for code reference indexes",
# "input_schema": {
# "type": "object",
# "properties": {
# "indexes_path": {
# "type": "string",
# "description": "Directory path containing index JSON files"
# }
# },
# "required": ["indexes_path"]
# }
# }
# Code evaluation tool definitions
@staticmethod
def _get_analyze_repo_structure_tool() -> Dict[str, Any]:
return {
"name": "analyze_repo_structure",
"description": "Perform comprehensive repository structure analysis",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository to analyze",
}
},
"required": ["repo_path"],
},
}
@staticmethod
def _get_detect_dependencies_tool() -> Dict[str, Any]:
return {
"name": "detect_dependencies",
"description": "Detect and analyze project dependencies across multiple languages",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository",
}
},
"required": ["repo_path"],
},
}
@staticmethod
def _get_assess_code_quality_tool() -> Dict[str, Any]:
return {
"name": "assess_code_quality",
"description": "Assess code quality metrics and identify potential issues",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository",
}
},
"required": ["repo_path"],
},
}
@staticmethod
def _get_evaluate_documentation_tool() -> Dict[str, Any]:
return {
"name": "evaluate_documentation",
"description": "Evaluate documentation completeness and quality",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository",
},
"docs_path": {
"type": "string",
"description": "Optional path to external documentation",
},
},
"required": ["repo_path"],
},
}
@staticmethod
def _get_check_reproduction_readiness_tool() -> Dict[str, Any]:
return {
"name": "check_reproduction_readiness",
"description": "Assess repository readiness for reproduction and validation",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository",
},
"docs_path": {
"type": "string",
"description": "Optional path to reproduction documentation",
},
},
"required": ["repo_path"],
},
}
@staticmethod
def _get_generate_evaluation_summary_tool() -> Dict[str, Any]:
return {
"name": "generate_evaluation_summary",
"description": "Generate comprehensive evaluation summary combining all analysis results",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository",
},
"docs_path": {
"type": "string",
"description": "Optional path to reproduction documentation",
},
},
"required": ["repo_path"],
},
}
@staticmethod
def _get_detect_empty_files_tool() -> Dict[str, Any]:
return {
"name": "detect_empty_files",
"description": "Detect empty files in the repository that may need implementation",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository to analyze",
}
},
"required": ["repo_path"],
},
}
@staticmethod
def _get_detect_missing_files_tool() -> Dict[str, Any]:
return {
"name": "detect_missing_files",
"description": "Detect missing essential files like main programs, tests, requirements, etc.",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository to analyze",
}
},
"required": ["repo_path"],
},
}
@staticmethod
def _get_generate_code_revision_report_tool() -> Dict[str, Any]:
return {
"name": "generate_code_revision_report",
"description": "Generate comprehensive code revision report combining empty files, missing files, and quality analysis",
"input_schema": {
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Path to the repository to analyze",
},
"docs_path": {
"type": "string",
"description": "Optional path to documentation",
},
},
"required": ["repo_path"],
},
}
@staticmethod
def get_available_tool_sets() -> Dict[str, str]:
"""
获取可用的工具集合
Get available tool sets
"""
return {
"code_implementation": "代码实现相关工具集 / Code implementation tool set",
"code_evaluation": "代码评估相关工具集 / Code evaluation tool set",
# 可以在这里添加更多工具集
# "data_analysis": "数据分析工具集 / Data analysis tool set",
# "web_scraping": "网页爬取工具集 / Web scraping tool set",
}
@staticmethod
def get_tool_set(tool_set_name: str) -> List[Dict[str, Any]]:
"""
根据名称获取特定的工具集
Get specific tool set by name
"""
tool_sets = {
"code_implementation": MCPToolDefinitions.get_code_implementation_tools(),
"code_evaluation": MCPToolDefinitions.get_code_evaluation_tools(),
}
return tool_sets.get(tool_set_name, [])
@staticmethod
def get_all_tools() -> List[Dict[str, Any]]:
"""
获取所有可用工具
Get all available tools
"""
all_tools = []
for tool_set_name in MCPToolDefinitions.get_available_tool_sets().keys():
all_tools.extend(MCPToolDefinitions.get_tool_set(tool_set_name))
return all_tools
# 便捷访问函数
def get_mcp_tools(tool_set: str = "code_implementation") -> List[Dict[str, Any]]:
"""
便捷函数:获取MCP工具定义
Convenience function: Get MCP tool definitions
Args:
tool_set: 工具集名称 (默认: "code_implementation")
Returns:
工具定义列表
"""
return MCPToolDefinitions.get_tool_set(tool_set)
================================================
FILE: deepcode.py
================================================
#!/usr/bin/env python3
"""
DeepCode - AI Research Engine Launcher
🧬 Next-Generation AI Research Automation Platform
⚡ Transform research papers into working code automatically
Cross-platform support: Windows, macOS, Linux
"""
import os
import sys
import subprocess
import signal
import platform
import socket
import time
from pathlib import Path
# Global process references for cleanup
_backend_process = None
_frontend_process = None
def get_platform():
"""Get current platform"""
system = platform.system().lower()
if system == "darwin":
return "macos"
elif system == "windows":
return "windows"
else:
return "linux"
def check_dependencies():
"""Check if necessary dependencies are installed for new UI"""
import importlib.util
import shutil
print("🔍 Checking dependencies...")
missing_deps = []
missing_system_deps = []
# Check FastAPI availability (for backend)
if importlib.util.find_spec("fastapi") is not None:
print("✅ FastAPI is installed")
else:
missing_deps.append("fastapi>=0.104.0")
# Check uvicorn availability (for backend server)
if importlib.util.find_spec("uvicorn") is not None:
print("✅ Uvicorn is installed")
else:
missing_deps.append("uvicorn>=0.24.0")
# Check PyYAML availability
if importlib.util.find_spec("yaml") is not None:
print("✅ PyYAML is installed")
else:
missing_deps.append("pyyaml>=6.0")
# Check pydantic-settings availability
if importlib.util.find_spec("pydantic_settings") is not None:
print("✅ Pydantic-settings is installed")
else:
missing_deps.append("pydantic-settings>=2.0.0")
# Check Node.js availability (for frontend)
node_cmd = "node.exe" if get_platform() == "windows" else "node"
if shutil.which(node_cmd) or shutil.which("node"):
try:
result = subprocess.run(
["node", "--version"],
capture_output=True,
text=True,
timeout=5,
shell=(get_platform() == "windows"),
)
if result.returncode == 0:
print(f"✅ Node.js is installed ({result.stdout.strip()})")
except Exception:
missing_system_deps.append("Node.js")
else:
missing_system_deps.append("Node.js")
print("❌ Node.js not found (required for frontend)")
# Check npm availability
npm_cmd = "npm.cmd" if get_platform() == "windows" else "npm"
if shutil.which(npm_cmd) or shutil.which("npm"):
print("✅ npm is available")
else:
missing_system_deps.append("npm")
print("❌ npm not found (required for frontend)")
# Display missing dependencies
if missing_deps or missing_system_deps:
print("\n📋 Dependency Status:")
if missing_deps:
print("❌ Missing Python dependencies:")
for dep in missing_deps:
print(f" - {dep}")
print(f"\nInstall with: pip install {' '.join(missing_deps)}")
if missing_system_deps:
print("\n❌ Missing system dependencies:")
for dep in missing_system_deps:
print(f" - {dep}")
print("\nInstall Node.js:")
print(" - Windows/macOS: https://nodejs.org/")
print(" - macOS: brew install node")
print(" - Ubuntu/Debian: sudo apt-get install nodejs npm")
# Fail if critical dependencies are missing
if missing_deps or missing_system_deps:
return False
else:
print("✅ All dependencies satisfied")
return True
def is_port_in_use(port: int) -> bool:
"""Check if a port is in use (cross-platform)"""
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
return s.connect_ex(("localhost", port)) == 0
def kill_process_on_port(port: int):
"""Kill process using a specific port (cross-platform)"""
current_platform = get_platform()
try:
if current_platform == "windows":
# Windows: use netstat and taskkill
result = subprocess.run(
f"netstat -ano | findstr :{port}",
capture_output=True,
text=True,
shell=True,
)
if result.stdout:
for line in result.stdout.strip().split("\n"):
parts = line.split()
if len(parts) >= 5:
pid = parts[-1]
if pid.isdigit():
subprocess.run(
f"taskkill /F /PID {pid}",
shell=True,
capture_output=True,
)
print(f" ✓ Killed process on port {port} (PID: {pid})")
else:
# macOS/Linux: use lsof
result = subprocess.run(
f"lsof -ti :{port}", capture_output=True, text=True, shell=True
)
if result.stdout:
pids = result.stdout.strip().split("\n")
for pid in pids:
if pid.isdigit():
os.kill(int(pid), signal.SIGKILL)
print(f" ✓ Killed process on port {port} (PID: {pid})")
except Exception as e:
print(f" ⚠️ Could not kill process on port {port}: {e}")
def cleanup_ports():
"""Clean up ports 8000 and 5173 if in use"""
for port in [8000, 5173]:
if is_port_in_use(port):
print(f"⚠️ Port {port} is in use, cleaning up...")
kill_process_on_port(port)
time.sleep(1)
def install_backend_deps():
"""Install backend dependencies if needed"""
import importlib.util
if importlib.util.find_spec("fastapi") is None:
print("📦 Installing backend dependencies...")
deps = [
"fastapi",
"uvicorn",
"pydantic-settings",
"python-multipart",
"aiofiles",
"websockets",
"pyyaml",
]
subprocess.run(
[sys.executable, "-m", "pip", "install", "-q"] + deps, check=True
)
print("✅ Backend dependencies installed")
def install_frontend_deps(frontend_dir: Path):
"""Install frontend dependencies if needed"""
node_modules = frontend_dir / "node_modules"
if not node_modules.exists():
print("📦 Installing frontend dependencies (first run)...")
npm_cmd = "npm.cmd" if get_platform() == "windows" else "npm"
subprocess.run(
[npm_cmd, "install"],
cwd=frontend_dir,
check=True,
shell=(get_platform() == "windows"),
)
print("✅ Frontend dependencies installed")
def start_backend(backend_dir: Path):
"""Start the backend server"""
global _backend_process
print("🔧 Starting backend server...")
# Use shell=True on Windows for proper command handling
if get_platform() == "windows":
_backend_process = subprocess.Popen(
f'"{sys.executable}" -m uvicorn main:app --host 0.0.0.0 --port 8000 --reload',
cwd=backend_dir,
shell=True,
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
)
else:
_backend_process = subprocess.Popen(
[
sys.executable,
"-m",
"uvicorn",
"main:app",
"--host",
"0.0.0.0",
"--port",
"8000",
"--reload",
],
cwd=backend_dir,
start_new_session=True, # Create new process group
)
# Wait for backend to start
time.sleep(2)
if _backend_process.poll() is None:
print("✅ Backend started: http://localhost:8000")
return True
else:
print("❌ Backend failed to start")
return False
def start_frontend(frontend_dir: Path):
"""Start the frontend dev server"""
global _frontend_process
print("🎨 Starting frontend server...")
npm_cmd = "npm.cmd" if get_platform() == "windows" else "npm"
if get_platform() == "windows":
_frontend_process = subprocess.Popen(
f"{npm_cmd} run dev",
cwd=frontend_dir,
shell=True,
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
)
else:
_frontend_process = subprocess.Popen(
[npm_cmd, "run", "dev"],
cwd=frontend_dir,
start_new_session=True, # Create new process group
)
# Wait for frontend to start
time.sleep(3)
if _frontend_process.poll() is None:
print("✅ Frontend started: http://localhost:5173")
return True
else:
print("❌ Frontend failed to start")
return False
def cleanup_processes():
"""Clean up running processes"""
global _backend_process, _frontend_process
print("\n🛑 Stopping services...")
for name, proc in [("Backend", _backend_process), ("Frontend", _frontend_process)]:
if proc and proc.poll() is None:
try:
if get_platform() == "windows":
# Windows: use taskkill with /T to kill tree
subprocess.run(
f"taskkill /F /T /PID {proc.pid}",
shell=True,
capture_output=True,
)
else:
# Unix: kill the process group
try:
os.killpg(os.getpgid(proc.pid), signal.SIGTERM)
proc.wait(timeout=5)
except Exception:
os.killpg(os.getpgid(proc.pid), signal.SIGKILL)
print(f" ✓ {name} stopped")
except Exception:
# Fallback: try direct terminate
try:
proc.terminate()
proc.wait(timeout=3)
print(f" ✓ {name} stopped")
except Exception:
try:
proc.kill()
print(f" ✓ {name} killed")
except Exception:
print(f" ⚠️ Could not stop {name}")
# Also clean up any orphaned processes on ports
time.sleep(0.5)
for port in [8000, 5173]:
if is_port_in_use(port):
kill_process_on_port(port)
print("✅ All services stopped")
def cleanup_cache():
"""Clean up Python cache files"""
try:
print("🧹 Cleaning up cache files...")
# Clean up __pycache__ directories
os.system('find . -type d -name "__pycache__" -exec rm -r {} + 2>/dev/null')
# Clean up .pyc files
os.system('find . -name "*.pyc" -delete 2>/dev/null')
print("✅ Cache cleanup completed")
except Exception as e:
print(f"⚠️ Cache cleanup failed: {e}")
def print_banner():
"""Display startup banner"""
banner = """
╔══════════════════════════════════════════════════════════════╗
║ ║
║ 🧬 DeepCode - AI Research Engine ║
║ ║
║ ⚡ NEURAL • AUTONOMOUS • REVOLUTIONARY ⚡ ║
║ ║
║ Transform research papers into working code ║
║ Next-generation AI automation platform ║
║ ║
╚══════════════════════════════════════════════════════════════╝
"""
print(banner)
def launch_classic_ui():
"""Launch classic Streamlit UI"""
import importlib.util
print("🌐 Launching Classic Streamlit UI...")
# Check if Streamlit is installed
if importlib.util.find_spec("streamlit") is None:
print("❌ Streamlit is not installed.")
print("Install with: pip install streamlit")
sys.exit(1)
current_dir = Path(__file__).parent
streamlit_app_path = current_dir / "ui" / "streamlit_app.py"
if not streamlit_app_path.exists():
print(f"❌ Streamlit app not found: {streamlit_app_path}")
sys.exit(1)
print(f"📁 UI App: {streamlit_app_path}")
print("🚀 Launching on http://localhost:8501")
print("=" * 70)
try:
cmd = [
sys.executable,
"-m",
"streamlit",
"run",
str(streamlit_app_path),
"--server.port",
"8501",
"--server.address",
"localhost",
"--browser.gatherUsageStats",
"false",
]
subprocess.run(cmd, check=True)
except KeyboardInterrupt:
print("\n\n🛑 Streamlit server stopped by user")
except Exception as e:
print(f"\n❌ Error: {e}")
sys.exit(1)
def _check_docker_prerequisites():
"""Check Docker prerequisites and config files. Returns (current_dir, compose_file, compose_args)."""
import shutil
current_dir = Path(__file__).parent
compose_file = current_dir / "deepcode_docker" / "docker-compose.yml"
if not compose_file.exists():
print("❌ deepcode_docker/docker-compose.yml not found")
print(" Make sure you are running from the DeepCode project root.")
sys.exit(1)
# Check Docker is installed
if not shutil.which("docker"):
print("❌ Docker not found. Please install Docker Desktop first.")
print(" https://www.docker.com/products/docker-desktop")
sys.exit(1)
# Check Docker daemon is running
result = subprocess.run(["docker", "info"], capture_output=True, text=True)
if result.returncode != 0:
print("❌ Docker is installed but not running.")
print(" Please start Docker Desktop and try again.")
sys.exit(1)
# Check/create secrets file
secrets_file = current_dir / "mcp_agent.secrets.yaml"
if not secrets_file.exists():
example = current_dir / "mcp_agent.secrets.yaml.example"
if example.exists():
print("⚠️ mcp_agent.secrets.yaml not found.")
print(" Creating from template...")
import shutil as sh
sh.copy2(example, secrets_file)
print(f" ✅ Created {secrets_file}")
print("")
print(" ⚠️ Please edit mcp_agent.secrets.yaml and fill in your API keys:")
print(f" {secrets_file}")
print("")
print(
" At least ONE LLM provider key is required (OpenAI/Anthropic/Google)."
)
print(" Then run 'deepcode' again.")
sys.exit(0)
else:
print(
"❌ mcp_agent.secrets.yaml not found. Please create it with your API keys."
)
sys.exit(1)
# Check config file
config_file = current_dir / "mcp_agent.config.yaml"
if not config_file.exists():
print("❌ mcp_agent.config.yaml not found.")
print(" This file should be in the project root.")
sys.exit(1)
# Ensure data directories exist
for d in ["deepcode_lab", "uploads", "logs"]:
(current_dir / d).mkdir(exist_ok=True)
os.chdir(current_dir)
compose_args = ["docker", "compose", "-f", str(compose_file)]
return current_dir, compose_file, compose_args
def launch_docker():
"""Launch DeepCode via Docker"""
current_dir, compose_file, compose_args = _check_docker_prerequisites()
print("🐳 Starting DeepCode with Docker...")
print("=" * 50)
try:
# Check if image exists (auto-build on first run)
result = subprocess.run(
compose_args + ["images", "-q"], capture_output=True, text=True
)
if not result.stdout.strip():
print(
"📦 First run detected — building Docker image (may take a few minutes)..."
)
subprocess.run(compose_args + ["build"], check=True)
# Start (if already running, docker compose will detect and skip)
subprocess.run(compose_args + ["up", "-d"], check=True)
print("")
print("=" * 50)
print("✅ DeepCode is running!")
print("")
print(" 🌐 Open: http://localhost:8000")
print(" 📚 Docs: http://localhost:8000/docs")
print("")
print(" 📋 View logs: docker logs deepcode -f")
print(
" 🛑 Stop: docker compose -f deepcode_docker/docker-compose.yml down"
)
print("=" * 50)
except subprocess.CalledProcessError as e:
print(f"\n❌ Docker failed: {e}")
sys.exit(1)
except KeyboardInterrupt:
print("\n🛑 Cancelled")
def launch_docker_cli():
"""Launch DeepCode CLI inside Docker container"""
current_dir, compose_file, compose_args = _check_docker_prerequisites()
print("🖥️ Starting DeepCode CLI in Docker...")
print("=" * 50)
try:
# Check if image exists (auto-build on first run)
result = subprocess.run(
compose_args + ["images", "-q"], capture_output=True, text=True
)
if not result.stdout.strip():
print(
"📦 First run detected — building Docker image (may take a few minutes)..."
)
subprocess.run(compose_args + ["build"], check=True)
# Run CLI interactively
subprocess.run(
compose_args + ["run", "--rm", "-it", "deepcode", "cli"], check=True
)
except subprocess.CalledProcessError as e:
print(f"\n❌ Docker failed: {e}")
sys.exit(1)
except KeyboardInterrupt:
print("\n🛑 Cancelled")
def launch_paper_test(paper_name: str, fast_mode: bool = False):
"""Launch paper testing mode"""
try:
print("\n🧪 Launching Paper Test Mode")
print(f"📄 Paper: {paper_name}")
print(f"⚡ Fast mode: {'enabled' if fast_mode else 'disabled'}")
print("=" * 60)
# Run the test setup
setup_cmd = [sys.executable, "test_paper.py", paper_name]
if fast_mode:
setup_cmd.append("--fast")
result = subprocess.run(setup_cmd, check=True)
if result.returncode == 0:
print("\n✅ Paper test setup completed successfully!")
print("📁 Files are ready in deepcode_lab/papers/")
print("\n💡 Next steps:")
print(" 1. Install MCP dependencies: pip install -r requirements.txt")
print(
f" 2. Run full pipeline: python -m workflows.paper_test_engine --paper {paper_name}"
+ (" --fast" if fast_mode else "")
)
except subprocess.CalledProcessError as e:
print(f"\n❌ Paper test setup failed: {e}")
sys.exit(1)
except Exception as e:
print(f"\n❌ Unexpected error: {e}")
sys.exit(1)
def main():
"""Main function"""
# Parse command line arguments
if len(sys.argv) > 1:
if sys.argv[1] == "test" and len(sys.argv) >= 3:
# Paper testing mode: python deepcode.py test rice [--fast]
paper_name = sys.argv[2]
fast_mode = "--fast" in sys.argv or "-f" in sys.argv
print_banner()
launch_paper_test(paper_name, fast_mode)
return
elif sys.argv[1] == "--local":
# Launch locally (without Docker) — fall through to local launch below
print_banner()
pass
elif sys.argv[1] == "--docker":
# Explicit Docker launch (same as default)
print_banner()
launch_docker()
return
elif sys.argv[1] == "--cli":
# Launch CLI inside Docker container
print_banner()
launch_docker_cli()
return
elif sys.argv[1] == "--classic":
# Launch classic Streamlit UI
print_banner()
launch_classic_ui()
return
elif sys.argv[1] in ["--help", "-h", "help"]:
print_banner()
print("""
🔧 Usage:
deepcode - Launch via Docker (default, recommended)
deepcode --docker - Same as above (launch via Docker)
deepcode --cli - Launch interactive CLI in Docker
deepcode --local - Launch locally (requires Python + Node.js)
deepcode test - Test paper reproduction
deepcode test --fast - Test paper (fast mode)
deepcode --classic - Launch classic Streamlit UI
📄 Examples:
deepcode - Start with Docker (one command)
deepcode --cli - Interactive CLI in Docker
deepcode --local - Start the new UI locally
deepcode test rice - Test RICE paper reproduction
deepcode test rice --fast - Test RICE paper (fast mode)
🌐 New UI Features:
• User-in-Loop interaction
• Real-time progress tracking
• Inline chat interaction
• Modern React-based interface
📁 Available papers:""")
# List available papers
papers_dir = "papers"
if os.path.exists(papers_dir):
for item in os.listdir(papers_dir):
item_path = os.path.join(papers_dir, item)
if os.path.isdir(item_path):
paper_md = os.path.join(item_path, "paper.md")
addendum_md = os.path.join(item_path, "addendum.md")
status = "✅" if os.path.exists(paper_md) else "❌"
addendum_status = "📄" if os.path.exists(addendum_md) else "➖"
print(f" {status} {item} {addendum_status}")
print(
"\n Legend: ✅ = paper.md exists, 📄 = addendum.md exists, ➖ = no addendum"
)
return
else:
# Unknown argument — show help hint
print(f"Unknown option: {sys.argv[1]}")
print("Run 'deepcode --help' for usage information.")
sys.exit(1)
else:
# Default (no arguments) → Docker
print_banner()
launch_docker()
return
# --- Local launch (only reached via --local) ---
# Show platform info
current_platform = get_platform()
print(f"🖥️ Platform: {current_platform.capitalize()}")
# Check dependencies
if not check_dependencies():
print("\n🚨 Please install missing dependencies and try again.")
sys.exit(1)
# Get paths
current_dir = Path(__file__).parent
new_ui_dir = current_dir / "new_ui"
backend_dir = new_ui_dir / "backend"
frontend_dir = new_ui_dir / "frontend"
# Check if new_ui directory exists
if not new_ui_dir.exists():
print(f"❌ New UI directory not found: {new_ui_dir}")
sys.exit(1)
print("\n🚀 Starting DeepCode New UI...")
print("=" * 70)
print("🎨 Frontend: http://localhost:5173")
print("🔧 Backend: http://localhost:8000")
print("📚 API Docs: http://localhost:8000/docs")
print("=" * 70)
print("💡 Tip: Keep this terminal open while using the application")
print("🛑 Press Ctrl+C to stop all services")
print("=" * 70)
try:
# Clean up ports if in use
cleanup_ports()
# Install dependencies if needed
install_backend_deps()
install_frontend_deps(frontend_dir)
# Start services
if not start_backend(backend_dir):
print("❌ Failed to start backend")
sys.exit(1)
if not start_frontend(frontend_dir):
print("❌ Failed to start frontend")
cleanup_processes()
sys.exit(1)
print("\n" + "=" * 70)
print("╔════════════════════════════════════════╗")
print("║ 🎉 DeepCode New UI is running! ║")
print("╠════════════════════════════════════════╣")
print("║ ║")
print("║ 🌐 Frontend: http://localhost:5173 ║")
print("║ 🔧 Backend: http://localhost:8000 ║")
print("║ 📚 API Docs: http://localhost:8000/docs║")
print("║ ║")
print("║ Press Ctrl+C to stop all services ║")
print("╚════════════════════════════════════════╝")
print("=" * 70 + "\n")
# Wait for processes
while True:
# Check if processes are still running
if _backend_process and _backend_process.poll() is not None:
print("⚠️ Backend process exited unexpectedly")
break
if _frontend_process and _frontend_process.poll() is not None:
print("⚠️ Frontend process exited unexpectedly")
break
time.sleep(1)
except KeyboardInterrupt:
print("\n")
except Exception as e:
print(f"\n❌ Unexpected error: {e}")
finally:
cleanup_processes()
cleanup_cache()
print("Thank you for using DeepCode! 🧬")
if __name__ == "__main__":
main()
================================================
FILE: deepcode_docker/.dockerignore
================================================
# Git
.git
.gitignore
# Node
new_ui/frontend/node_modules
new_ui/frontend/dist
# Python
__pycache__
*.pyc
*.pyo
*.egg-info
.eggs
dist
build
# Virtual environments
.venv
venv
env
# IDE
.vscode
.idea
.cursor
*.swp
*.swo
# Runtime data
deepcode_lab
uploads
logs
*.log
# Docker
deepcode_docker/Dockerfile
deepcode_docker/docker-compose.yml
deepcode_docker/.dockerignore
deepcode_docker/run_docker.sh
# Documentation
assets
*.md
LICENSE
================================================
FILE: deepcode_docker/Dockerfile
================================================
# =============================================================
# DeepCode - Docker Build
# Multi-stage: Frontend build → Final image with Python + Node
# =============================================================
# ------ Stage 1: Build frontend static assets ------
FROM node:18-alpine AS frontend-builder
WORKDIR /build
COPY new_ui/frontend/package*.json ./
RUN npm ci --no-audit --no-fund
COPY new_ui/frontend/ ./
RUN npm run build
# ------ Stage 2: Final image ------
FROM python:3.10-slim
# Metadata
LABEL maintainer="DeepCode Team"
LABEL description="DeepCode - AI Research Engine"
LABEL version="1.0"
# Environment
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
DEEPCODE_ENV=docker \
DEEPCODE_HOST=0.0.0.0 \
DEEPCODE_PORT=8000
# Install system dependencies:
# - git: for git clone operations in workflows
# - nodejs/npm/npx: for MCP servers (brave-search, filesystem, fetch)
# - curl: for health checks
RUN apt-get update && \
apt-get install -y --no-install-recommends \
git \
curl \
ca-certificates && \
# Install Node.js 18 via official binary (includes npm + npx)
ARCH=$(dpkg --print-architecture) && \
if [ "$ARCH" = "arm64" ]; then NODE_ARCH="arm64"; else NODE_ARCH="x64"; fi && \
curl -fsSL https://nodejs.org/dist/v18.20.8/node-v18.20.8-linux-${NODE_ARCH}.tar.gz \
| tar -xz -C /usr/local --strip-components=1 && \
# Install uv (Python package installer, used by mcp-server-fetch)
pip install --no-cache-dir uv && \
# Cleanup
apt-get clean && \
rm -rf /var/lib/apt/lists/* && \
# Verify
node --version && npm --version && npx --version
WORKDIR /app
# Install Python dependencies first (cache layer)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Pre-install npx MCP server packages (avoid download at runtime)
RUN npx -y @modelcontextprotocol/server-brave-search --help 2>/dev/null || true && \
npx -y @modelcontextprotocol/server-filesystem --help 2>/dev/null || true
# Copy project source code
COPY __init__.py setup.py deepcode.py ./
COPY config/ ./config/
COPY prompts/ ./prompts/
COPY schema/ ./schema/
COPY tools/ ./tools/
COPY utils/ ./utils/
COPY workflows/ ./workflows/
COPY cli/ ./cli/
COPY ui/ ./ui/
COPY new_ui/backend/ ./new_ui/backend/
# Copy frontend build output from Stage 1
COPY --from=frontend-builder /build/dist ./new_ui/frontend/dist
# Create runtime directories
RUN mkdir -p deepcode_lab uploads logs
# Copy entrypoint script
COPY deepcode_docker/docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=10s --start-period=15s --retries=3 \
CMD curl -f http://localhost:8000/health || exit 1
ENTRYPOINT ["/docker-entrypoint.sh"]
================================================
FILE: deepcode_docker/docker-compose.yml
================================================
services:
deepcode:
build:
context: ..
dockerfile: deepcode_docker/Dockerfile
container_name: deepcode
ports:
- "8000:8000"
volumes:
# Configuration (required)
- ../mcp_agent.config.yaml:/app/mcp_agent.config.yaml:ro
- ../mcp_agent.secrets.yaml:/app/mcp_agent.secrets.yaml:ro
# Persistent data
- ../deepcode_lab:/app/deepcode_lab
- ../uploads:/app/uploads
- ../logs:/app/logs
environment:
- DEEPCODE_ENV=docker
- DEEPCODE_PORT=8000
restart: unless-stopped
nanobot:
build:
context: ..
dockerfile: nanobot/Dockerfile
container_name: nanobot
ports:
- "18790:18790"
volumes:
# nanobot configuration (飞书/Telegram token 等)
- ../nanobot_config.json:/root/.nanobot/config.json:ro
# Persistent workspace data
- nanobot-workspace:/root/.nanobot/workspace
- nanobot-sessions:/root/.nanobot/sessions
# Shared with DeepCode: nanobot can access generated code
- ../deepcode_lab:/app/deepcode_lab
environment:
- NANOBOT_ENV=docker
# Internal API URL for nanobot -> DeepCode communication
- DEEPCODE_API_URL=http://deepcode:8000
depends_on:
- deepcode
restart: unless-stopped
volumes:
nanobot-workspace:
nanobot-sessions:
================================================
FILE: deepcode_docker/docker-entrypoint.sh
================================================
#!/bin/bash
set -e
echo "============================================"
echo " DeepCode - AI Research Engine (Docker)"
echo "============================================"
# ------ Validate configuration ------
if [ ! -f "mcp_agent.config.yaml" ]; then
echo "⚠️ mcp_agent.config.yaml not found, using default config"
fi
if [ ! -f "mcp_agent.secrets.yaml" ]; then
echo ""
echo "❌ ERROR: mcp_agent.secrets.yaml not found!"
echo ""
echo "Please mount your secrets file:"
echo " docker run -v ./mcp_agent.secrets.yaml:/app/mcp_agent.secrets.yaml ..."
echo ""
echo "Or use docker-compose with the provided template."
echo ""
exit 1
fi
# ------ Ensure directories exist ------
mkdir -p deepcode_lab uploads logs
# ------ CLI mode: launch interactive CLI ------
if [ "$1" = "cli" ]; then
shift
echo ""
echo "🖥️ Starting DeepCode CLI..."
echo "============================================"
echo ""
exec python cli/main_cli.py "$@"
fi
# ------ Web mode (default): start backend + frontend ------
echo ""
echo "🚀 Starting DeepCode..."
echo " API: http://localhost:${DEEPCODE_PORT:-8000}"
echo " Docs: http://localhost:${DEEPCODE_PORT:-8000}/docs"
echo "============================================"
echo ""
exec python -m uvicorn new_ui.backend.main:app \
--host "${DEEPCODE_HOST:-0.0.0.0}" \
--port "${DEEPCODE_PORT:-8000}" \
--workers 1 \
--log-level info
================================================
FILE: deepcode_docker/run_docker.sh
================================================
#!/bin/bash
# DeepCode Docker 一键启动脚本
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
COMPOSE_FILE="$SCRIPT_DIR/docker-compose.yml"
# 颜色定义
RED='\033[0;31m'
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[1;33m'
CYAN='\033[0;36m'
NC='\033[0m'
# docker compose wrapper — always use the correct compose file
dc() {
docker compose -f "$COMPOSE_FILE" "$@"
}
echo ""
echo "╔════════════════════════════════════════╗"
echo "║ DeepCode - Docker 启动脚本 ║"
echo "╚════════════════════════════════════════╝"
echo ""
# ============ 检查 Docker 环境 ============
check_docker() {
if ! command -v docker &> /dev/null; then
echo -e "${RED}❌ 未检测到 Docker,请先安装 Docker Desktop${NC}"
echo " 下载地址: https://www.docker.com/products/docker-desktop"
exit 1
fi
if ! docker info &> /dev/null 2>&1; then
echo -e "${RED}❌ Docker 服务未运行,请先启动 Docker Desktop${NC}"
exit 1
fi
echo -e "${GREEN}✓ Docker 环境正常${NC}"
}
# ============ 检查配置文件 ============
check_config() {
if [ ! -f "$PROJECT_ROOT/mcp_agent.config.yaml" ]; then
echo -e "${RED}❌ 缺少 mcp_agent.config.yaml 配置文件${NC}"
exit 1
fi
echo -e "${GREEN}✓ mcp_agent.config.yaml 已找到${NC}"
if [ ! -f "$PROJECT_ROOT/mcp_agent.secrets.yaml" ]; then
if [ -f "$PROJECT_ROOT/mcp_agent.secrets.yaml.example" ]; then
echo -e "${YELLOW}⚠ 未找到 mcp_agent.secrets.yaml${NC}"
echo -e "${YELLOW} 正在从模板创建...${NC}"
cp "$PROJECT_ROOT/mcp_agent.secrets.yaml.example" "$PROJECT_ROOT/mcp_agent.secrets.yaml"
echo -e "${YELLOW} ⚡ 请编辑 mcp_agent.secrets.yaml 填入你的 API Key,然后重新运行此脚本${NC}"
exit 1
else
echo -e "${RED}❌ 缺少 mcp_agent.secrets.yaml,且未找到模板文件${NC}"
exit 1
fi
fi
echo -e "${GREEN}✓ mcp_agent.secrets.yaml 已找到${NC}"
}
# ============ 创建必要目录 ============
ensure_dirs() {
mkdir -p "$PROJECT_ROOT/deepcode_lab" "$PROJECT_ROOT/uploads" "$PROJECT_ROOT/logs"
echo -e "${GREEN}✓ 数据目录已就绪 (deepcode_lab/, uploads/, logs/)${NC}"
}
# ============ 解析命令行参数 ============
ACTION="up"
BUILD_FLAG=""
DETACH_FLAG=""
usage() {
echo "用法: $0 [选项]"
echo ""
echo "选项:"
echo " --build 强制重新构建镜像"
echo " -d, --detach 后台运行(不占用终端)"
echo " stop 停止容器"
echo " restart 重启容器"
echo " logs 查看容器日志"
echo " status 查看容器状态"
echo " cli 在 Docker 容器内启动交互式 CLI"
echo " clean 停止并删除容器和镜像"
echo " -h, --help 显示帮助信息"
echo ""
echo "示例:"
echo " $0 # 构建并启动(首次会自动构建)"
echo " $0 --build # 强制重新构建后启动"
echo " $0 -d # 后台启动"
echo " $0 stop # 停止服务"
echo " $0 logs # 查看实时日志"
echo " $0 cli # 启动交互式 CLI"
echo " $0 clean # 完全清理"
}
while [[ $# -gt 0 ]]; do
case $1 in
--build)
BUILD_FLAG="--build"
shift
;;
-d|--detach)
DETACH_FLAG="-d"
shift
;;
stop)
ACTION="stop"
shift
;;
restart)
ACTION="restart"
shift
;;
logs)
ACTION="logs"
shift
;;
status)
ACTION="status"
shift
;;
clean)
ACTION="clean"
shift
;;
cli)
ACTION="cli"
shift
break # Remaining args passed to CLI
;;
-h|--help)
usage
exit 0
;;
*)
echo -e "${RED}未知参数: $1${NC}"
usage
exit 1
;;
esac
done
# ============ 执行操作 ============
case $ACTION in
up)
check_docker
check_config
ensure_dirs
echo ""
echo -e "${BLUE}🐳 启动 DeepCode Docker 容器...${NC}"
# 检查镜像是否存在,首次运行自动构建
if [ -z "$BUILD_FLAG" ]; then
if ! docker images | grep -q "deepcode"; then
echo -e "${YELLOW}⚡ 首次运行,自动构建镜像(可能需要几分钟)...${NC}"
BUILD_FLAG="--build"
fi
fi
dc up $BUILD_FLAG $DETACH_FLAG
if [ -n "$DETACH_FLAG" ]; then
# 后台模式,等待容器启动后显示信息
echo ""
echo -e "${YELLOW}⏳ 等待服务启动...${NC}"
for i in $(seq 1 30); do
if curl -sf http://localhost:8000/health > /dev/null 2>&1; then
echo ""
echo "╔════════════════════════════════════════╗"
echo -e "║ ${GREEN}DeepCode 已启动! (Docker)${NC} ║"
echo "╠════════════════════════════════════════╣"
echo "║ ║"
echo "║ 🌐 访问: http://localhost:8000 ║"
echo "║ 📚 API: http://localhost:8000/docs ║"
echo "║ ║"
echo "║ 查看日志: $0 logs ║"
echo "║ 停止服务: $0 stop ║"
echo "╚════════════════════════════════════════╝"
echo ""
exit 0
fi
sleep 2
done
echo -e "${YELLOW}⚠ 服务仍在启动中,请稍后访问 http://localhost:8000${NC}"
echo -e " 使用 ${CYAN}$0 logs${NC} 查看启动日志"
fi
;;
stop)
check_docker
echo -e "${BLUE}🛑 停止 DeepCode 容器...${NC}"
dc down
echo -e "${GREEN}✓ 服务已停止${NC}"
;;
restart)
check_docker
echo -e "${BLUE}🔄 重启 DeepCode 容器...${NC}"
dc down
dc up -d $BUILD_FLAG
echo -e "${GREEN}✓ 服务已重启${NC}"
echo -e " 访问: http://localhost:8000"
;;
logs)
check_docker
echo -e "${BLUE}📋 DeepCode 容器日志 (Ctrl+C 退出):${NC}"
echo ""
dc logs -f
;;
status)
check_docker
echo -e "${BLUE}📊 DeepCode 容器状态:${NC}"
echo ""
dc ps
echo ""
# 检查健康状态
if curl -sf http://localhost:8000/health > /dev/null 2>&1; then
echo -e "${GREEN}✓ 服务运行正常 (http://localhost:8000)${NC}"
else
echo -e "${YELLOW}⚠ 服务未响应或未启动${NC}"
fi
;;
cli)
check_docker
check_config
ensure_dirs
echo ""
echo -e "${BLUE}🖥️ 启动 DeepCode CLI (Docker)...${NC}"
echo ""
dc run --rm -it deepcode cli "$@"
;;
clean)
check_docker
echo -e "${YELLOW}⚠ 即将停止并删除 DeepCode 容器和镜像${NC}"
echo -e "${YELLOW} (数据目录 deepcode_lab/, uploads/, logs/ 不会被删除)${NC}"
read -p "确认? [y/N] " confirm
if [[ "$confirm" =~ ^[Yy]$ ]]; then
dc down --rmi local --remove-orphans
echo -e "${GREEN}✓ 已清理完成${NC}"
else
echo "已取消"
fi
;;
esac
================================================
FILE: mcp_agent.config.yaml
================================================
$schema: ./schema/mcp-agent.config.schema.json
anthropic: null
default_search_server: filesystem
document_segmentation:
enabled: false
size_threshold_chars: 50000
execution_engine: asyncio
logger:
level: info
path_settings:
path_pattern: logs/mcp-agent-{unique_id}.jsonl
timestamp_format: '%Y%m%d_%H%M%S'
unique_id: timestamp
progress_display: false
transports:
- console
- file
mcp:
servers:
bocha-mcp:
args:
- tools/bocha_search_server.py
command: python
env:
BOCHA_API_KEY: ''
PYTHONPATH: .
brave:
# macos and linux should use this
args:
- -y
- '@modelcontextprotocol/server-brave-search'
command: npx
# windows should use this
# args:
# # please use the correct path for your system
# - C:/Users/LEGION/AppData/Roaming/npm/node_modules/@modelcontextprotocol/server-brave-search/dist/index.js
# command: node
env:
BRAVE_API_KEY: ''
filesystem:
# macos and linux should use this
# Note: "No valid root directories" warning is harmless - connection still works
args:
- -y
- '@modelcontextprotocol/server-filesystem'
- .
- ./deepcode_lab
command: npx
# windows should use this
# args:
# # please use the correct path for your system
# - C:/Users/LEGION/AppData/Roaming/npm/node_modules/@modelcontextprotocol/server-filesystem/dist/index.js
# - .
# command: node
code-implementation:
args:
- tools/code_implementation_server.py
command: python
description: Paper code reproduction tool server - provides file operations,
code execution, search and other functions
env:
PYTHONPATH: .
code-reference-indexer:
args:
- tools/code_reference_indexer.py
command: python
description: Code reference indexer server - Provides intelligent code reference
search from indexed repositories
env:
PYTHONPATH: .
command-executor:
args:
- tools/command_executor.py
command: python
env:
PYTHONPATH: .
document-segmentation:
args:
- tools/document_segmentation_server.py
command: python
description: Document segmentation server - Provides intelligent document analysis
and segmented reading to optimize token usage
env:
PYTHONPATH: .
fetch:
args:
- mcp-server-fetch
command: uvx
file-downloader:
args:
- tools/pdf_downloader.py
command: python
env:
PYTHONPATH: .
github-downloader:
args:
- tools/git_command.py
command: python
env:
PYTHONPATH: .
# LLM Provider Priority (选择使用哪个LLM / Choose which LLM to use)
# Options: "anthropic", "google", "openai"
# If not set or provider unavailable, will fallback to first available provider
llm_provider: "openai" # 设置为 "google", "anthropic", 或 "openai"
#openrouter can be used here and openai professional key
openai:
base_max_tokens: 40000
default_model: "google/gemini-3-flash-preview"
planning_model: "google/gemini-3-flash-preview"
implementation_model: "google/gemini-3-flash-preview"
reasoning_effort: low # Only for thinking models
max_tokens_policy: adaptive
retry_max_tokens: 32768
# Provider configurations
# default_model is used by mcp_agent for planning/analysis phases
# implementation_model is used by code_implementation_workflow for code generation
google:
default_model: "gemini-3-pro-preview"
planning_model: "gemini-3-pro-preview"
implementation_model: "gemini-2.5-flash"
anthropic:
default_model: "claude-sonnet-4.5"
planning_model: "claude-sonnet-4.5"
implementation_model: "claude-sonnet-3.5"
planning_mode: traditional
================================================
FILE: mcp_agent.secrets.yaml.example
================================================
# =============================================================
# DeepCode - API Keys Configuration
# =============================================================
# Copy this file to mcp_agent.secrets.yaml and fill in your keys.
#
# At least ONE LLM provider API key is required.
# Config file takes priority over environment variables.
# =============================================================
# OpenAI / OpenRouter
openai:
api_key: ""
# For OpenRouter (recommended - access multiple models via one key):
# base_url: "https://openrouter.ai/api/v1"
# Anthropic (Claude)
anthropic:
api_key: ""
# Google (Gemini)
google:
api_key: ""
================================================
FILE: nanobot/.dockerignore
================================================
__pycache__
*.pyc
*.pyo
*.pyd
*.egg-info
dist/
build/
.git
.env
.assets
node_modules/
bridge/dist/
workspace/
================================================
FILE: nanobot/.gitignore
================================================
.assets
.env
*.pyc
dist/
build/
docs/
*.egg-info/
*.egg
*.pyc
*.pyo
*.pyd
*.pyw
*.pyz
*.pywz
*.pyzz
.venv/
__pycache__/
poetry.lock
.pytest_cache/
tests/
botpy.log
================================================
FILE: nanobot/COMMUNICATION.md
================================================
We provide QR codes for joining the HKUDS discussion groups on **WeChat** and **Feishu**.
You can join by scanning the QR codes below:
================================================
FILE: nanobot/Dockerfile
================================================
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim
# Install Node.js 20 for the WhatsApp bridge
RUN apt-get update && \
apt-get install -y --no-install-recommends curl ca-certificates gnupg git && \
mkdir -p /etc/apt/keyrings && \
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_20.x nodistro main" > /etc/apt/sources.list.d/nodesource.list && \
apt-get update && \
apt-get install -y --no-install-recommends nodejs && \
apt-get purge -y gnupg && \
apt-get autoremove -y && \
rm -rf /var/lib/apt/lists/*
WORKDIR /app
# Install Python dependencies first (cached layer)
# Note: build context is DeepCode root, so paths start with nanobot/
COPY nanobot/pyproject.toml nanobot/README.md nanobot/LICENSE ./
RUN mkdir -p nanobot bridge && touch nanobot/__init__.py && \
uv pip install --system --no-cache . && \
rm -rf nanobot bridge
# Copy the full source and install
COPY nanobot/nanobot/ nanobot/
COPY nanobot/bridge/ bridge/
RUN uv pip install --system --no-cache .
# Build the WhatsApp bridge
WORKDIR /app/bridge
RUN npm install && npm run build
WORKDIR /app
# Create config directory
RUN mkdir -p /root/.nanobot
# Gateway default port
EXPOSE 18790
ENTRYPOINT ["nanobot"]
CMD ["gateway"]
================================================
FILE: nanobot/LICENSE
================================================
MIT License
Copyright (c) 2025 nanobot contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: nanobot/README.md
================================================
nanobot: Ultra-Lightweight Personal AI Assistant
🐈 **nanobot** is an **ultra-lightweight** personal AI assistant inspired by [Clawdbot](https://github.com/openclaw/openclaw)
⚡️ Delivers core agent functionality in just **~4,000** lines of code — **99% smaller** than Clawdbot's 430k+ lines.
📏 Real-time line count: **3,510 lines** (run `bash core_agent_lines.sh` to verify anytime)
## 📢 News
- **2026-02-09** 💬 Added Slack, Email, and QQ support — nanobot now supports multiple chat platforms!
- **2026-02-08** 🔧 Refactored Providers—adding a new LLM provider now takes just 2 simple steps! Check [here](#providers).
- **2026-02-07** 🚀 Released v0.1.3.post5 with Qwen support & several key improvements! Check [here](https://github.com/HKUDS/nanobot/releases/tag/v0.1.3.post5) for details.
- **2026-02-06** ✨ Added Moonshot/Kimi provider, Discord integration, and enhanced security hardening!
- **2026-02-05** ✨ Added Feishu channel, DeepSeek provider, and enhanced scheduled tasks support!
- **2026-02-04** 🚀 Released v0.1.3.post4 with multi-provider & Docker support! Check [here](https://github.com/HKUDS/nanobot/releases/tag/v0.1.3.post4) for details.
- **2026-02-03** ⚡ Integrated vLLM for local LLM support and improved natural language task scheduling!
- **2026-02-02** 🎉 nanobot officially launched! Welcome to try 🐈 nanobot!
## Key Features of nanobot:
🪶 **Ultra-Lightweight**: Just ~4,000 lines of core agent code — 99% smaller than Clawdbot.
🔬 **Research-Ready**: Clean, readable code that's easy to understand, modify, and extend for research.
⚡️ **Lightning Fast**: Minimal footprint means faster startup, lower resource usage, and quicker iterations.
💎 **Easy-to-Use**: One-click to deploy and you're ready to go.
## 🏗️ Architecture
## ✨ Features
📈 24/7 Real-Time Market Analysis
🚀 Full-Stack Software Engineer
📅 Smart Daily Routine Manager
📚 Personal Knowledge Assistant
Discovery • Insights • Trends
Develop • Deploy • Scale
Schedule • Automate • Organize
Learn • Memory • Reasoning
## 📦 Install
**Install from source** (latest features, recommended for development)
```bash
git clone https://github.com/HKUDS/nanobot.git
cd nanobot
pip install -e .
```
**Install with [uv](https://github.com/astral-sh/uv)** (stable, fast)
```bash
uv tool install nanobot-ai
```
**Install from PyPI** (stable)
```bash
pip install nanobot-ai
```
## 🚀 Quick Start
> [!TIP]
> Set your API key in `~/.nanobot/config.json`.
> Get API keys: [OpenRouter](https://openrouter.ai/keys) (Global) · [DashScope](https://dashscope.console.aliyun.com) (Qwen) · [Brave Search](https://brave.com/search/api/) (optional, for web search)
**1. Initialize**
```bash
nanobot onboard
```
**2. Configure** (`~/.nanobot/config.json`)
For OpenRouter - recommended for global users:
```json
{
"providers": {
"openrouter": {
"apiKey": "sk-or-v1-xxx"
}
},
"agents": {
"defaults": {
"model": "anthropic/claude-opus-4-5"
}
}
}
```
**3. Chat**
```bash
nanobot agent -m "What is 2+2?"
```
That's it! You have a working AI assistant in 2 minutes.
## 🖥️ Local Models (vLLM)
Run nanobot with your own local models using vLLM or any OpenAI-compatible server.
**1. Start your vLLM server**
```bash
vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000
```
**2. Configure** (`~/.nanobot/config.json`)
```json
{
"providers": {
"vllm": {
"apiKey": "dummy",
"apiBase": "http://localhost:8000/v1"
}
},
"agents": {
"defaults": {
"model": "meta-llama/Llama-3.1-8B-Instruct"
}
}
}
```
**3. Chat**
```bash
nanobot agent -m "Hello from my local LLM!"
```
> [!TIP]
> The `apiKey` can be any non-empty string for local servers that don't require authentication.
## 💬 Chat Apps
Talk to your nanobot through Telegram, Discord, WhatsApp, Feishu, DingTalk, Slack, Email, or QQ — anytime, anywhere.
| Channel | Setup |
|---------|-------|
| **Telegram** | Easy (just a token) |
| **Discord** | Easy (bot token + intents) |
| **WhatsApp** | Medium (scan QR) |
| **Feishu** | Medium (app credentials) |
| **DingTalk** | Medium (app credentials) |
| **Slack** | Medium (bot + app tokens) |
| **Email** | Medium (IMAP/SMTP credentials) |
| **QQ** | Easy (app credentials) |
Telegram (Recommended)
**1. Create a bot**
- Open Telegram, search `@BotFather`
- Send `/newbot`, follow prompts
- Copy the token
**2. Configure**
```json
{
"channels": {
"telegram": {
"enabled": true,
"token": "YOUR_BOT_TOKEN",
"allowFrom": ["YOUR_USER_ID"]
}
}
}
```
> You can find your **User ID** in Telegram settings. It is shown as `@yourUserId`.
> Copy this value **without the `@` symbol** and paste it into the config file.
**3. Run**
```bash
nanobot gateway
```
Discord
**1. Create a bot**
- Go to https://discord.com/developers/applications
- Create an application → Bot → Add Bot
- Copy the bot token
**2. Enable intents**
- In the Bot settings, enable **MESSAGE CONTENT INTENT**
- (Optional) Enable **SERVER MEMBERS INTENT** if you plan to use allow lists based on member data
**3. Get your User ID**
- Discord Settings → Advanced → enable **Developer Mode**
- Right-click your avatar → **Copy User ID**
**4. Configure**
```json
{
"channels": {
"discord": {
"enabled": true,
"token": "YOUR_BOT_TOKEN",
"allowFrom": ["YOUR_USER_ID"]
}
}
}
```
**5. Invite the bot**
- OAuth2 → URL Generator
- Scopes: `bot`
- Bot Permissions: `Send Messages`, `Read Message History`
- Open the generated invite URL and add the bot to your server
**6. Run**
```bash
nanobot gateway
```
WhatsApp
Requires **Node.js ≥18**.
**1. Link device**
```bash
nanobot channels login
# Scan QR with WhatsApp → Settings → Linked Devices
```
**2. Configure**
```json
{
"channels": {
"whatsapp": {
"enabled": true,
"allowFrom": ["+1234567890"]
}
}
}
```
**3. Run** (two terminals)
```bash
# Terminal 1
nanobot channels login
# Terminal 2
nanobot gateway
```
Feishu (飞书)
Uses **WebSocket** long connection — no public IP required.
**1. Create a Feishu bot**
- Visit [Feishu Open Platform](https://open.feishu.cn/app)
- Create a new app → Enable **Bot** capability
- **Permissions**: Add `im:message` (send messages)
- **Events**: Add `im.message.receive_v1` (receive messages)
- Select **Long Connection** mode (requires running nanobot first to establish connection)
- Get **App ID** and **App Secret** from "Credentials & Basic Info"
- Publish the app
**2. Configure**
```json
{
"channels": {
"feishu": {
"enabled": true,
"appId": "cli_xxx",
"appSecret": "xxx",
"encryptKey": "",
"verificationToken": "",
"allowFrom": []
}
}
}
```
> `encryptKey` and `verificationToken` are optional for Long Connection mode.
> `allowFrom`: Leave empty to allow all users, or add `["ou_xxx"]` to restrict access.
**3. Run**
```bash
nanobot gateway
```
> [!TIP]
> Feishu uses WebSocket to receive messages — no webhook or public IP needed!
QQ (QQ私聊)
Uses **botpy SDK** with WebSocket — no public IP required.
**1. Create a QQ bot**
- Visit [QQ Open Platform](https://q.qq.com)
- Create a new bot application
- Get **AppID** and **Secret** from "Developer Settings"
**2. Configure**
```json
{
"channels": {
"qq": {
"enabled": true,
"appId": "YOUR_APP_ID",
"secret": "YOUR_APP_SECRET",
"allowFrom": []
}
}
}
```
> `allowFrom`: Leave empty for public access, or add user openids to restrict access.
> Example: `"allowFrom": ["user_openid_1", "user_openid_2"]`
**3. Run**
```bash
nanobot gateway
```
> [!TIP]
> QQ bot currently supports **private messages only**. Group chat support coming soon!
DingTalk (钉钉)
Uses **Stream Mode** — no public IP required.
**1. Create a DingTalk bot**
- Visit [DingTalk Open Platform](https://open-dev.dingtalk.com/)
- Create a new app -> Add **Robot** capability
- **Configuration**:
- Toggle **Stream Mode** ON
- **Permissions**: Add necessary permissions for sending messages
- Get **AppKey** (Client ID) and **AppSecret** (Client Secret) from "Credentials"
- Publish the app
**2. Configure**
```json
{
"channels": {
"dingtalk": {
"enabled": true,
"clientId": "YOUR_APP_KEY",
"clientSecret": "YOUR_APP_SECRET",
"allowFrom": []
}
}
}
```
> `allowFrom`: Leave empty to allow all users, or add `["staffId"]` to restrict access.
**3. Run**
```bash
nanobot gateway
```
Slack
Uses **Socket Mode** — no public URL required.
**1. Create a Slack app**
- Go to [Slack API](https://api.slack.com/apps) → Create New App
- **OAuth & Permissions**: Add bot scopes: `chat:write`, `reactions:write`, `app_mentions:read`
- Install to your workspace and copy the **Bot Token** (`xoxb-...`)
- **Socket Mode**: Enable it and generate an **App-Level Token** (`xapp-...`) with `connections:write` scope
- **Event Subscriptions**: Subscribe to `message.im`, `message.channels`, `app_mention`
**2. Configure**
```json
{
"channels": {
"slack": {
"enabled": true,
"botToken": "xoxb-...",
"appToken": "xapp-...",
"groupPolicy": "mention"
}
}
}
```
> `groupPolicy`: `"mention"` (respond only when @mentioned), `"open"` (respond to all messages), or `"allowlist"` (restrict to specific channels).
> DM policy defaults to open. Set `"dm": {"enabled": false}` to disable DMs.
**3. Run**
```bash
nanobot gateway
```
Email
Give nanobot its own email account. It polls **IMAP** for incoming mail and replies via **SMTP** — like a personal email assistant.
**1. Get credentials (Gmail example)**
- Create a dedicated Gmail account for your bot (e.g. `my-nanobot@gmail.com`)
- Enable 2-Step Verification → Create an [App Password](https://myaccount.google.com/apppasswords)
- Use this app password for both IMAP and SMTP
**2. Configure**
> - `consentGranted` must be `true` to allow mailbox access. This is a safety gate — set `false` to fully disable.
> - `allowFrom`: Leave empty to accept emails from anyone, or restrict to specific senders.
> - `smtpUseTls` and `smtpUseSsl` default to `true` / `false` respectively, which is correct for Gmail (port 587 + STARTTLS). No need to set them explicitly.
> - Set `"autoReplyEnabled": false` if you only want to read/analyze emails without sending automatic replies.
```json
{
"channels": {
"email": {
"enabled": true,
"consentGranted": true,
"imapHost": "imap.gmail.com",
"imapPort": 993,
"imapUsername": "my-nanobot@gmail.com",
"imapPassword": "your-app-password",
"smtpHost": "smtp.gmail.com",
"smtpPort": 587,
"smtpUsername": "my-nanobot@gmail.com",
"smtpPassword": "your-app-password",
"fromAddress": "my-nanobot@gmail.com",
"allowFrom": ["your-real-email@gmail.com"]
}
}
}
```
**3. Run**
```bash
nanobot gateway
```
## ⚙️ Configuration
Config file: `~/.nanobot/config.json`
### Providers
> [!TIP]
> - **Groq** provides free voice transcription via Whisper. If configured, Telegram voice messages will be automatically transcribed.
> - **Zhipu Coding Plan**: If you're on Zhipu's coding plan, set `"apiBase": "https://open.bigmodel.cn/api/coding/paas/v4"` in your zhipu provider config.
| Provider | Purpose | Get API Key |
|----------|---------|-------------|
| `openrouter` | LLM (recommended, access to all models) | [openrouter.ai](https://openrouter.ai) |
| `anthropic` | LLM (Claude direct) | [console.anthropic.com](https://console.anthropic.com) |
| `openai` | LLM (GPT direct) | [platform.openai.com](https://platform.openai.com) |
| `deepseek` | LLM (DeepSeek direct) | [platform.deepseek.com](https://platform.deepseek.com) |
| `groq` | LLM + **Voice transcription** (Whisper) | [console.groq.com](https://console.groq.com) |
| `gemini` | LLM (Gemini direct) | [aistudio.google.com](https://aistudio.google.com) |
| `aihubmix` | LLM (API gateway, access to all models) | [aihubmix.com](https://aihubmix.com) |
| `dashscope` | LLM (Qwen) | [dashscope.console.aliyun.com](https://dashscope.console.aliyun.com) |
| `moonshot` | LLM (Moonshot/Kimi) | [platform.moonshot.cn](https://platform.moonshot.cn) |
| `zhipu` | LLM (Zhipu GLM) | [open.bigmodel.cn](https://open.bigmodel.cn) |
| `vllm` | LLM (local, any OpenAI-compatible server) | — |
Adding a New Provider (Developer Guide)
nanobot uses a **Provider Registry** (`nanobot/providers/registry.py`) as the single source of truth.
Adding a new provider only takes **2 steps** — no if-elif chains to touch.
**Step 1.** Add a `ProviderSpec` entry to `PROVIDERS` in `nanobot/providers/registry.py`:
```python
ProviderSpec(
name="myprovider", # config field name
keywords=("myprovider", "mymodel"), # model-name keywords for auto-matching
env_key="MYPROVIDER_API_KEY", # env var for LiteLLM
display_name="My Provider", # shown in `nanobot status`
litellm_prefix="myprovider", # auto-prefix: model → myprovider/model
skip_prefixes=("myprovider/",), # don't double-prefix
)
```
**Step 2.** Add a field to `ProvidersConfig` in `nanobot/config/schema.py`:
```python
class ProvidersConfig(BaseModel):
...
myprovider: ProviderConfig = ProviderConfig()
```
That's it! Environment variables, model prefixing, config matching, and `nanobot status` display will all work automatically.
**Common `ProviderSpec` options:**
| Field | Description | Example |
|-------|-------------|---------|
| `litellm_prefix` | Auto-prefix model names for LiteLLM | `"dashscope"` → `dashscope/qwen-max` |
| `skip_prefixes` | Don't prefix if model already starts with these | `("dashscope/", "openrouter/")` |
| `env_extras` | Additional env vars to set | `(("ZHIPUAI_API_KEY", "{api_key}"),)` |
| `model_overrides` | Per-model parameter overrides | `(("kimi-k2.5", {"temperature": 1.0}),)` |
| `is_gateway` | Can route any model (like OpenRouter) | `True` |
| `detect_by_key_prefix` | Detect gateway by API key prefix | `"sk-or-"` |
| `detect_by_base_keyword` | Detect gateway by API base URL | `"openrouter"` |
| `strip_model_prefix` | Strip existing prefix before re-prefixing | `True` (for AiHubMix) |
### Security
> For production deployments, set `"restrictToWorkspace": true` in your config to sandbox the agent.
| Option | Default | Description |
|--------|---------|-------------|
| `tools.restrictToWorkspace` | `false` | When `true`, restricts **all** agent tools (shell, file read/write/edit, list) to the workspace directory. Prevents path traversal and out-of-scope access. |
| `channels.*.allowFrom` | `[]` (allow all) | Whitelist of user IDs. Empty = allow everyone; non-empty = only listed users can interact. |
## CLI Reference
| Command | Description |
|---------|-------------|
| `nanobot onboard` | Initialize config & workspace |
| `nanobot agent -m "..."` | Chat with the agent |
| `nanobot agent` | Interactive chat mode |
| `nanobot agent --no-markdown` | Show plain-text replies |
| `nanobot agent --logs` | Show runtime logs during chat |
| `nanobot gateway` | Start the gateway |
| `nanobot status` | Show status |
| `nanobot channels login` | Link WhatsApp (scan QR) |
| `nanobot channels status` | Show channel status |
Interactive mode exits: `exit`, `quit`, `/exit`, `/quit`, `:q`, or `Ctrl+D`.
Scheduled Tasks (Cron)
```bash
# Add a job
nanobot cron add --name "daily" --message "Good morning!" --cron "0 9 * * *"
nanobot cron add --name "hourly" --message "Check status" --every 3600
# List jobs
nanobot cron list
# Remove a job
nanobot cron remove
```
## 🐳 Docker
> [!TIP]
> The `-v ~/.nanobot:/root/.nanobot` flag mounts your local config directory into the container, so your config and workspace persist across container restarts.
Build and run nanobot in a container:
```bash
# Build the image
docker build -t nanobot .
# Initialize config (first time only)
docker run -v ~/.nanobot:/root/.nanobot --rm nanobot onboard
# Edit config on host to add API keys
vim ~/.nanobot/config.json
# Run gateway (connects to Telegram/WhatsApp)
docker run -v ~/.nanobot:/root/.nanobot -p 18790:18790 nanobot gateway
# Or run a single command
docker run -v ~/.nanobot:/root/.nanobot --rm nanobot agent -m "Hello!"
docker run -v ~/.nanobot:/root/.nanobot --rm nanobot status
```
## 📁 Project Structure
```
nanobot/
├── agent/ # 🧠 Core agent logic
│ ├── loop.py # Agent loop (LLM ↔ tool execution)
│ ├── context.py # Prompt builder
│ ├── memory.py # Persistent memory
│ ├── skills.py # Skills loader
│ ├── subagent.py # Background task execution
│ └── tools/ # Built-in tools (incl. spawn)
├── skills/ # 🎯 Bundled skills (github, weather, tmux...)
├── channels/ # 📱 WhatsApp integration
├── bus/ # 🚌 Message routing
├── cron/ # ⏰ Scheduled tasks
├── heartbeat/ # 💓 Proactive wake-up
├── providers/ # 🤖 LLM providers (OpenRouter, etc.)
├── session/ # 💬 Conversation sessions
├── config/ # ⚙️ Configuration
└── cli/ # 🖥️ Commands
```
## 🤝 Contribute & Roadmap
PRs welcome! The codebase is intentionally small and readable. 🤗
**Roadmap** — Pick an item and [open a PR](https://github.com/HKUDS/nanobot/pulls)!
- [x] **Voice Transcription** — Support for Groq Whisper (Issue #13)
- [ ] **Multi-modal** — See and hear (images, voice, video)
- [ ] **Long-term memory** — Never forget important context
- [ ] **Better reasoning** — Multi-step planning and reflection
- [ ] **More integrations** — Calendar and more
- [ ] **Self-improvement** — Learn from feedback and mistakes
### Contributors
## ⭐ Star History
Thanks for visiting ✨ nanobot!
nanobot is for educational, research, and technical exchange purposes only
================================================
FILE: nanobot/SECURITY.md
================================================
# Security Policy
## Reporting a Vulnerability
If you discover a security vulnerability in nanobot, please report it by:
1. **DO NOT** open a public GitHub issue
2. Create a private security advisory on GitHub or contact the repository maintainers
3. Include:
- Description of the vulnerability
- Steps to reproduce
- Potential impact
- Suggested fix (if any)
We aim to respond to security reports within 48 hours.
## Security Best Practices
### 1. API Key Management
**CRITICAL**: Never commit API keys to version control.
```bash
# ✅ Good: Store in config file with restricted permissions
chmod 600 ~/.nanobot/config.json
# ❌ Bad: Hardcoding keys in code or committing them
```
**Recommendations:**
- Store API keys in `~/.nanobot/config.json` with file permissions set to `0600`
- Consider using environment variables for sensitive keys
- Use OS keyring/credential manager for production deployments
- Rotate API keys regularly
- Use separate API keys for development and production
### 2. Channel Access Control
**IMPORTANT**: Always configure `allowFrom` lists for production use.
```json
{
"channels": {
"telegram": {
"enabled": true,
"token": "YOUR_BOT_TOKEN",
"allowFrom": ["123456789", "987654321"]
},
"whatsapp": {
"enabled": true,
"allowFrom": ["+1234567890"]
}
}
}
```
**Security Notes:**
- Empty `allowFrom` list will **ALLOW ALL** users (open by default for personal use)
- Get your Telegram user ID from `@userinfobot`
- Use full phone numbers with country code for WhatsApp
- Review access logs regularly for unauthorized access attempts
### 3. Shell Command Execution
The `exec` tool can execute shell commands. While dangerous command patterns are blocked, you should:
- ✅ Review all tool usage in agent logs
- ✅ Understand what commands the agent is running
- ✅ Use a dedicated user account with limited privileges
- ✅ Never run nanobot as root
- ❌ Don't disable security checks
- ❌ Don't run on systems with sensitive data without careful review
**Blocked patterns:**
- `rm -rf /` - Root filesystem deletion
- Fork bombs
- Filesystem formatting (`mkfs.*`)
- Raw disk writes
- Other destructive operations
### 4. File System Access
File operations have path traversal protection, but:
- ✅ Run nanobot with a dedicated user account
- ✅ Use filesystem permissions to protect sensitive directories
- ✅ Regularly audit file operations in logs
- ❌ Don't give unrestricted access to sensitive files
### 5. Network Security
**API Calls:**
- All external API calls use HTTPS by default
- Timeouts are configured to prevent hanging requests
- Consider using a firewall to restrict outbound connections if needed
**WhatsApp Bridge:**
- The bridge runs on `localhost:3001` by default
- If exposing to network, use proper authentication and TLS
- Keep authentication data in `~/.nanobot/whatsapp-auth` secure (mode 0700)
### 6. Dependency Security
**Critical**: Keep dependencies updated!
```bash
# Check for vulnerable dependencies
pip install pip-audit
pip-audit
# Update to latest secure versions
pip install --upgrade nanobot-ai
```
For Node.js dependencies (WhatsApp bridge):
```bash
cd bridge
npm audit
npm audit fix
```
**Important Notes:**
- Keep `litellm` updated to the latest version for security fixes
- We've updated `ws` to `>=8.17.1` to fix DoS vulnerability
- Run `pip-audit` or `npm audit` regularly
- Subscribe to security advisories for nanobot and its dependencies
### 7. Production Deployment
For production use:
1. **Isolate the Environment**
```bash
# Run in a container or VM
docker run --rm -it python:3.11
pip install nanobot-ai
```
2. **Use a Dedicated User**
```bash
sudo useradd -m -s /bin/bash nanobot
sudo -u nanobot nanobot gateway
```
3. **Set Proper Permissions**
```bash
chmod 700 ~/.nanobot
chmod 600 ~/.nanobot/config.json
chmod 700 ~/.nanobot/whatsapp-auth
```
4. **Enable Logging**
```bash
# Configure log monitoring
tail -f ~/.nanobot/logs/nanobot.log
```
5. **Use Rate Limiting**
- Configure rate limits on your API providers
- Monitor usage for anomalies
- Set spending limits on LLM APIs
6. **Regular Updates**
```bash
# Check for updates weekly
pip install --upgrade nanobot-ai
```
### 8. Development vs Production
**Development:**
- Use separate API keys
- Test with non-sensitive data
- Enable verbose logging
- Use a test Telegram bot
**Production:**
- Use dedicated API keys with spending limits
- Restrict file system access
- Enable audit logging
- Regular security reviews
- Monitor for unusual activity
### 9. Data Privacy
- **Logs may contain sensitive information** - secure log files appropriately
- **LLM providers see your prompts** - review their privacy policies
- **Chat history is stored locally** - protect the `~/.nanobot` directory
- **API keys are in plain text** - use OS keyring for production
### 10. Incident Response
If you suspect a security breach:
1. **Immediately revoke compromised API keys**
2. **Review logs for unauthorized access**
```bash
grep "Access denied" ~/.nanobot/logs/nanobot.log
```
3. **Check for unexpected file modifications**
4. **Rotate all credentials**
5. **Update to latest version**
6. **Report the incident** to maintainers
## Security Features
### Built-in Security Controls
✅ **Input Validation**
- Path traversal protection on file operations
- Dangerous command pattern detection
- Input length limits on HTTP requests
✅ **Authentication**
- Allow-list based access control
- Failed authentication attempt logging
- Open by default (configure allowFrom for production use)
✅ **Resource Protection**
- Command execution timeouts (60s default)
- Output truncation (10KB limit)
- HTTP request timeouts (10-30s)
✅ **Secure Communication**
- HTTPS for all external API calls
- TLS for Telegram API
- WebSocket security for WhatsApp bridge
## Known Limitations
⚠️ **Current Security Limitations:**
1. **No Rate Limiting** - Users can send unlimited messages (add your own if needed)
2. **Plain Text Config** - API keys stored in plain text (use keyring for production)
3. **No Session Management** - No automatic session expiry
4. **Limited Command Filtering** - Only blocks obvious dangerous patterns
5. **No Audit Trail** - Limited security event logging (enhance as needed)
## Security Checklist
Before deploying nanobot:
- [ ] API keys stored securely (not in code)
- [ ] Config file permissions set to 0600
- [ ] `allowFrom` lists configured for all channels
- [ ] Running as non-root user
- [ ] File system permissions properly restricted
- [ ] Dependencies updated to latest secure versions
- [ ] Logs monitored for security events
- [ ] Rate limits configured on API providers
- [ ] Backup and disaster recovery plan in place
- [ ] Security review of custom skills/tools
## Updates
**Last Updated**: 2026-02-03
For the latest security updates and announcements, check:
- GitHub Security Advisories: https://github.com/HKUDS/nanobot/security/advisories
- Release Notes: https://github.com/HKUDS/nanobot/releases
## License
See LICENSE file for details.
================================================
FILE: nanobot/bridge/package.json
================================================
{
"name": "nanobot-whatsapp-bridge",
"version": "0.1.0",
"description": "WhatsApp bridge for nanobot using Baileys",
"type": "module",
"main": "dist/index.js",
"scripts": {
"build": "tsc",
"start": "node dist/index.js",
"dev": "tsc && node dist/index.js"
},
"dependencies": {
"@whiskeysockets/baileys": "7.0.0-rc.9",
"ws": "^8.17.1",
"qrcode-terminal": "^0.12.0",
"pino": "^9.0.0"
},
"devDependencies": {
"@types/node": "^20.14.0",
"@types/ws": "^8.5.10",
"typescript": "^5.4.0"
},
"engines": {
"node": ">=20.0.0"
}
}
================================================
FILE: nanobot/bridge/src/index.ts
================================================
#!/usr/bin/env node
/**
* nanobot WhatsApp Bridge
*
* This bridge connects WhatsApp Web to nanobot's Python backend
* via WebSocket. It handles authentication, message forwarding,
* and reconnection logic.
*
* Usage:
* npm run build && npm start
*
* Or with custom settings:
* BRIDGE_PORT=3001 AUTH_DIR=~/.nanobot/whatsapp npm start
*/
// Polyfill crypto for Baileys in ESM
import { webcrypto } from 'crypto';
if (!globalThis.crypto) {
(globalThis as any).crypto = webcrypto;
}
import { BridgeServer } from './server.js';
import { homedir } from 'os';
import { join } from 'path';
const PORT = parseInt(process.env.BRIDGE_PORT || '3001', 10);
const AUTH_DIR = process.env.AUTH_DIR || join(homedir(), '.nanobot', 'whatsapp-auth');
console.log('🐈 nanobot WhatsApp Bridge');
console.log('========================\n');
const server = new BridgeServer(PORT, AUTH_DIR);
// Handle graceful shutdown
process.on('SIGINT', async () => {
console.log('\n\nShutting down...');
await server.stop();
process.exit(0);
});
process.on('SIGTERM', async () => {
await server.stop();
process.exit(0);
});
// Start the server
server.start().catch((error) => {
console.error('Failed to start bridge:', error);
process.exit(1);
});
================================================
FILE: nanobot/bridge/src/server.ts
================================================
/**
* WebSocket server for Python-Node.js bridge communication.
*/
import { WebSocketServer, WebSocket } from 'ws';
import { WhatsAppClient, InboundMessage } from './whatsapp.js';
interface SendCommand {
type: 'send';
to: string;
text: string;
}
interface BridgeMessage {
type: 'message' | 'status' | 'qr' | 'error';
[key: string]: unknown;
}
export class BridgeServer {
private wss: WebSocketServer | null = null;
private wa: WhatsAppClient | null = null;
private clients: Set = new Set();
constructor(private port: number, private authDir: string) {}
async start(): Promise {
// Create WebSocket server
this.wss = new WebSocketServer({ port: this.port });
console.log(`🌉 Bridge server listening on ws://localhost:${this.port}`);
// Initialize WhatsApp client
this.wa = new WhatsAppClient({
authDir: this.authDir,
onMessage: (msg) => this.broadcast({ type: 'message', ...msg }),
onQR: (qr) => this.broadcast({ type: 'qr', qr }),
onStatus: (status) => this.broadcast({ type: 'status', status }),
});
// Handle WebSocket connections
this.wss.on('connection', (ws) => {
console.log('🔗 Python client connected');
this.clients.add(ws);
ws.on('message', async (data) => {
try {
const cmd = JSON.parse(data.toString()) as SendCommand;
await this.handleCommand(cmd);
ws.send(JSON.stringify({ type: 'sent', to: cmd.to }));
} catch (error) {
console.error('Error handling command:', error);
ws.send(JSON.stringify({ type: 'error', error: String(error) }));
}
});
ws.on('close', () => {
console.log('🔌 Python client disconnected');
this.clients.delete(ws);
});
ws.on('error', (error) => {
console.error('WebSocket error:', error);
this.clients.delete(ws);
});
});
// Connect to WhatsApp
await this.wa.connect();
}
private async handleCommand(cmd: SendCommand): Promise {
if (cmd.type === 'send' && this.wa) {
await this.wa.sendMessage(cmd.to, cmd.text);
}
}
private broadcast(msg: BridgeMessage): void {
const data = JSON.stringify(msg);
for (const client of this.clients) {
if (client.readyState === WebSocket.OPEN) {
client.send(data);
}
}
}
async stop(): Promise {
// Close all client connections
for (const client of this.clients) {
client.close();
}
this.clients.clear();
// Close WebSocket server
if (this.wss) {
this.wss.close();
this.wss = null;
}
// Disconnect WhatsApp
if (this.wa) {
await this.wa.disconnect();
this.wa = null;
}
}
}
================================================
FILE: nanobot/bridge/src/types.d.ts
================================================
declare module 'qrcode-terminal' {
export function generate(text: string, options?: { small?: boolean }): void;
}
================================================
FILE: nanobot/bridge/src/whatsapp.ts
================================================
/**
* WhatsApp client wrapper using Baileys.
* Based on OpenClaw's working implementation.
*/
/* eslint-disable @typescript-eslint/no-explicit-any */
import makeWASocket, {
DisconnectReason,
useMultiFileAuthState,
fetchLatestBaileysVersion,
makeCacheableSignalKeyStore,
} from '@whiskeysockets/baileys';
import { Boom } from '@hapi/boom';
import qrcode from 'qrcode-terminal';
import pino from 'pino';
const VERSION = '0.1.0';
export interface InboundMessage {
id: string;
sender: string;
pn: string;
content: string;
timestamp: number;
isGroup: boolean;
}
export interface WhatsAppClientOptions {
authDir: string;
onMessage: (msg: InboundMessage) => void;
onQR: (qr: string) => void;
onStatus: (status: string) => void;
}
export class WhatsAppClient {
private sock: any = null;
private options: WhatsAppClientOptions;
private reconnecting = false;
constructor(options: WhatsAppClientOptions) {
this.options = options;
}
async connect(): Promise {
const logger = pino({ level: 'silent' });
const { state, saveCreds } = await useMultiFileAuthState(this.options.authDir);
const { version } = await fetchLatestBaileysVersion();
console.log(`Using Baileys version: ${version.join('.')}`);
// Create socket following OpenClaw's pattern
this.sock = makeWASocket({
auth: {
creds: state.creds,
keys: makeCacheableSignalKeyStore(state.keys, logger),
},
version,
logger,
printQRInTerminal: false,
browser: ['nanobot', 'cli', VERSION],
syncFullHistory: false,
markOnlineOnConnect: false,
});
// Handle WebSocket errors
if (this.sock.ws && typeof this.sock.ws.on === 'function') {
this.sock.ws.on('error', (err: Error) => {
console.error('WebSocket error:', err.message);
});
}
// Handle connection updates
this.sock.ev.on('connection.update', async (update: any) => {
const { connection, lastDisconnect, qr } = update;
if (qr) {
// Display QR code in terminal
console.log('\n📱 Scan this QR code with WhatsApp (Linked Devices):\n');
qrcode.generate(qr, { small: true });
this.options.onQR(qr);
}
if (connection === 'close') {
const statusCode = (lastDisconnect?.error as Boom)?.output?.statusCode;
const shouldReconnect = statusCode !== DisconnectReason.loggedOut;
console.log(`Connection closed. Status: ${statusCode}, Will reconnect: ${shouldReconnect}`);
this.options.onStatus('disconnected');
if (shouldReconnect && !this.reconnecting) {
this.reconnecting = true;
console.log('Reconnecting in 5 seconds...');
setTimeout(() => {
this.reconnecting = false;
this.connect();
}, 5000);
}
} else if (connection === 'open') {
console.log('✅ Connected to WhatsApp');
this.options.onStatus('connected');
}
});
// Save credentials on update
this.sock.ev.on('creds.update', saveCreds);
// Handle incoming messages
this.sock.ev.on('messages.upsert', async ({ messages, type }: { messages: any[]; type: string }) => {
if (type !== 'notify') return;
for (const msg of messages) {
// Skip own messages
if (msg.key.fromMe) continue;
// Skip status updates
if (msg.key.remoteJid === 'status@broadcast') continue;
const content = this.extractMessageContent(msg);
if (!content) continue;
const isGroup = msg.key.remoteJid?.endsWith('@g.us') || false;
this.options.onMessage({
id: msg.key.id || '',
sender: msg.key.remoteJid || '',
pn: msg.key.remoteJidAlt || '',
content,
timestamp: msg.messageTimestamp as number,
isGroup,
});
}
});
}
private extractMessageContent(msg: any): string | null {
const message = msg.message;
if (!message) return null;
// Text message
if (message.conversation) {
return message.conversation;
}
// Extended text (reply, link preview)
if (message.extendedTextMessage?.text) {
return message.extendedTextMessage.text;
}
// Image with caption
if (message.imageMessage?.caption) {
return `[Image] ${message.imageMessage.caption}`;
}
// Video with caption
if (message.videoMessage?.caption) {
return `[Video] ${message.videoMessage.caption}`;
}
// Document with caption
if (message.documentMessage?.caption) {
return `[Document] ${message.documentMessage.caption}`;
}
// Voice/Audio message
if (message.audioMessage) {
return `[Voice Message]`;
}
return null;
}
async sendMessage(to: string, text: string): Promise {
if (!this.sock) {
throw new Error('Not connected');
}
await this.sock.sendMessage(to, { text });
}
async disconnect(): Promise {
if (this.sock) {
this.sock.end(undefined);
this.sock = null;
}
}
}
================================================
FILE: nanobot/bridge/tsconfig.json
================================================
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "node",
"esModuleInterop": true,
"strict": true,
"skipLibCheck": true,
"outDir": "./dist",
"rootDir": "./src",
"declaration": true,
"resolveJsonModule": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
================================================
FILE: nanobot/core_agent_lines.sh
================================================
#!/bin/bash
# Count core agent lines (excluding channels/, cli/, providers/ adapters)
cd "$(dirname "$0")" || exit 1
echo "nanobot core agent line count"
echo "================================"
echo ""
for dir in agent agent/tools bus config cron heartbeat session utils; do
count=$(find "nanobot/$dir" -maxdepth 1 -name "*.py" -exec cat {} + | wc -l)
printf " %-16s %5s lines\n" "$dir/" "$count"
done
root=$(cat nanobot/__init__.py nanobot/__main__.py | wc -l)
printf " %-16s %5s lines\n" "(root)" "$root"
echo ""
total=$(find nanobot -name "*.py" ! -path "*/channels/*" ! -path "*/cli/*" ! -path "*/providers/*" | xargs cat | wc -l)
echo " Core total: $total lines"
echo ""
echo " (excludes: channels/, cli/, providers/)"
================================================
FILE: nanobot/nanobot/__init__.py
================================================
"""
nanobot - A lightweight AI agent framework
"""
__version__ = "0.1.0"
__logo__ = "🐈"
================================================
FILE: nanobot/nanobot/__main__.py
================================================
"""
Entry point for running nanobot as a module: python -m nanobot
"""
from nanobot.cli.commands import app
if __name__ == "__main__":
app()
================================================
FILE: nanobot/nanobot/agent/__init__.py
================================================
"""Agent core module."""
from nanobot.agent.context import ContextBuilder
from nanobot.agent.loop import AgentLoop
from nanobot.agent.memory import MemoryStore
from nanobot.agent.skills import SkillsLoader
__all__ = ["AgentLoop", "ContextBuilder", "MemoryStore", "SkillsLoader"]
================================================
FILE: nanobot/nanobot/agent/context.py
================================================
"""Context builder for assembling agent prompts."""
import base64
import mimetypes
import platform
from pathlib import Path
from typing import Any
from nanobot.agent.memory import MemoryStore
from nanobot.agent.skills import SkillsLoader
class ContextBuilder:
"""
Builds the context (system prompt + messages) for the agent.
Assembles bootstrap files, memory, skills, and conversation history
into a coherent prompt for the LLM.
"""
BOOTSTRAP_FILES = ["AGENTS.md", "SOUL.md", "USER.md", "TOOLS.md", "IDENTITY.md"]
def __init__(self, workspace: Path):
self.workspace = workspace
self.memory = MemoryStore(workspace)
self.skills = SkillsLoader(workspace)
def build_system_prompt(self, skill_names: list[str] | None = None) -> str:
"""
Build the system prompt from bootstrap files, memory, and skills.
Args:
skill_names: Optional list of skills to include.
Returns:
Complete system prompt.
"""
parts = []
# Core identity
parts.append(self._get_identity())
# Bootstrap files
bootstrap = self._load_bootstrap_files()
if bootstrap:
parts.append(bootstrap)
# Memory context
memory = self.memory.get_memory_context()
if memory:
parts.append(f"# Memory\n\n{memory}")
# Skills - progressive loading
# 1. Always-loaded skills: include full content
always_skills = self.skills.get_always_skills()
if always_skills:
always_content = self.skills.load_skills_for_context(always_skills)
if always_content:
parts.append(f"# Active Skills\n\n{always_content}")
# 2. Available skills: only show summary (agent uses read_file to load)
skills_summary = self.skills.build_skills_summary()
if skills_summary:
parts.append(f"""# Skills
The following skills extend your capabilities. To use a skill, read its SKILL.md file using the read_file tool.
Skills with available="false" need dependencies installed first - you can try installing them with apt/brew.
{skills_summary}""")
return "\n\n---\n\n".join(parts)
def _get_identity(self) -> str:
"""Get the core identity section."""
from datetime import datetime
now = datetime.now().strftime("%Y-%m-%d %H:%M (%A)")
workspace_path = str(self.workspace.expanduser().resolve())
system = platform.system()
runtime = f"{'macOS' if system == 'Darwin' else system} {platform.machine()}, Python {platform.python_version()}"
return f"""# nanobot 🐈
You are nanobot, a helpful AI assistant. You have access to tools that allow you to:
- Read, write, and edit files
- Execute shell commands
- Search the web and fetch web pages
- Send messages to users on chat channels
- Spawn subagents for complex background tasks
## Current Time
{now}
## Runtime
{runtime}
## Workspace
Your workspace is at: {workspace_path}
- Memory files: {workspace_path}/memory/MEMORY.md
- Daily notes: {workspace_path}/memory/YYYY-MM-DD.md
- Custom skills: {workspace_path}/skills/{{skill-name}}/SKILL.md
IMPORTANT: When responding to direct questions or conversations, reply directly with your text response.
Only use the 'message' tool when you need to send a message to a specific chat channel (like WhatsApp).
For normal conversation, just respond with text - do not call the message tool.
Always be helpful, accurate, and concise. When using tools, explain what you're doing.
When remembering something, write to {workspace_path}/memory/MEMORY.md"""
def _load_bootstrap_files(self) -> str:
"""Load all bootstrap files from workspace."""
parts = []
for filename in self.BOOTSTRAP_FILES:
file_path = self.workspace / filename
if file_path.exists():
content = file_path.read_text(encoding="utf-8")
parts.append(f"## {filename}\n\n{content}")
return "\n\n".join(parts) if parts else ""
def build_messages(
self,
history: list[dict[str, Any]],
current_message: str,
skill_names: list[str] | None = None,
media: list[str] | None = None,
channel: str | None = None,
chat_id: str | None = None,
) -> list[dict[str, Any]]:
"""
Build the complete message list for an LLM call.
Args:
history: Previous conversation messages.
current_message: The new user message.
skill_names: Optional skills to include.
media: Optional list of local file paths for images/media.
channel: Current channel (telegram, feishu, etc.).
chat_id: Current chat/user ID.
Returns:
List of messages including system prompt.
"""
messages = []
# System prompt
system_prompt = self.build_system_prompt(skill_names)
if channel and chat_id:
system_prompt += f"\n\n## Current Session\nChannel: {channel}\nChat ID: {chat_id}"
messages.append({"role": "system", "content": system_prompt})
# History
messages.extend(history)
# Current message (with optional image attachments)
user_content = self._build_user_content(current_message, media)
messages.append({"role": "user", "content": user_content})
return messages
def _build_user_content(self, text: str, media: list[str] | None) -> str | list[dict[str, Any]]:
"""Build user message content with optional base64-encoded images."""
if not media:
return text
images = []
for path in media:
p = Path(path)
mime, _ = mimetypes.guess_type(path)
if not p.is_file() or not mime or not mime.startswith("image/"):
continue
b64 = base64.b64encode(p.read_bytes()).decode()
images.append({"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}})
if not images:
return text
return images + [{"type": "text", "text": text}]
def add_tool_result(
self, messages: list[dict[str, Any]], tool_call_id: str, tool_name: str, result: str
) -> list[dict[str, Any]]:
"""
Add a tool result to the message list.
Args:
messages: Current message list.
tool_call_id: ID of the tool call.
tool_name: Name of the tool.
result: Tool execution result.
Returns:
Updated message list.
"""
messages.append(
{"role": "tool", "tool_call_id": tool_call_id, "name": tool_name, "content": result}
)
return messages
def add_assistant_message(
self,
messages: list[dict[str, Any]],
content: str | None,
tool_calls: list[dict[str, Any]] | None = None,
reasoning_content: str | None = None,
) -> list[dict[str, Any]]:
"""
Add an assistant message to the message list.
Args:
messages: Current message list.
content: Message content.
tool_calls: Optional tool calls.
reasoning_content: Thinking output (Kimi, DeepSeek-R1, etc.).
Returns:
Updated message list.
"""
msg: dict[str, Any] = {"role": "assistant", "content": content or ""}
if tool_calls:
msg["tool_calls"] = tool_calls
# Thinking models reject history without this
if reasoning_content:
msg["reasoning_content"] = reasoning_content
messages.append(msg)
return messages
================================================
FILE: nanobot/nanobot/agent/loop.py
================================================
"""Agent loop: the core processing engine."""
from __future__ import annotations
import asyncio
import json
import os
from pathlib import Path
from typing import TYPE_CHECKING
from loguru import logger
from nanobot.agent.context import ContextBuilder
from nanobot.agent.subagent import SubagentManager
from nanobot.agent.tools.cron import CronTool
from nanobot.agent.tools.filesystem import EditFileTool, ListDirTool, ReadFileTool, WriteFileTool
from nanobot.agent.tools.message import MessageTool
from nanobot.agent.tools.registry import ToolRegistry
from nanobot.agent.tools.shell import ExecTool
from nanobot.agent.tools.spawn import SpawnTool
from nanobot.agent.tools.web import WebFetchTool, WebSearchTool
from nanobot.bus.events import InboundMessage, OutboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.providers.base import LLMProvider
from nanobot.session.manager import SessionManager
if TYPE_CHECKING:
from nanobot.config.schema import ExecToolConfig
from nanobot.cron.service import CronService
class AgentLoop:
"""
The agent loop is the core processing engine.
It:
1. Receives messages from the bus
2. Builds context with history, memory, skills
3. Calls the LLM
4. Executes tool calls
5. Sends responses back
"""
def __init__(
self,
bus: MessageBus,
provider: LLMProvider,
workspace: Path,
model: str | None = None,
max_iterations: int = 20,
brave_api_key: str | None = None,
exec_config: ExecToolConfig | None = None,
cron_service: CronService | None = None,
restrict_to_workspace: bool = False,
session_manager: SessionManager | None = None,
):
from nanobot.config.schema import ExecToolConfig
self.bus = bus
self.provider = provider
self.workspace = workspace
self.model = model or provider.get_default_model()
self.max_iterations = max_iterations
self.brave_api_key = brave_api_key
self.exec_config = exec_config or ExecToolConfig()
self.cron_service = cron_service
self.restrict_to_workspace = restrict_to_workspace
self.context = ContextBuilder(workspace)
self.sessions = session_manager or SessionManager(workspace)
self.tools = ToolRegistry()
self.subagents = SubagentManager(
provider=provider,
workspace=workspace,
bus=bus,
model=self.model,
brave_api_key=brave_api_key,
exec_config=self.exec_config,
restrict_to_workspace=restrict_to_workspace,
)
self._running = False
self._register_default_tools()
def _register_default_tools(self) -> None:
"""Register the default set of tools."""
# File tools (restrict to workspace if configured)
allowed_dir = self.workspace if self.restrict_to_workspace else None
self.tools.register(ReadFileTool(allowed_dir=allowed_dir))
self.tools.register(WriteFileTool(allowed_dir=allowed_dir))
self.tools.register(EditFileTool(allowed_dir=allowed_dir))
self.tools.register(ListDirTool(allowed_dir=allowed_dir))
# Shell tool
self.tools.register(
ExecTool(
working_dir=str(self.workspace),
timeout=self.exec_config.timeout,
restrict_to_workspace=self.restrict_to_workspace,
)
)
# Web tools
self.tools.register(WebSearchTool(api_key=self.brave_api_key))
self.tools.register(WebFetchTool())
# Message tool
message_tool = MessageTool(send_callback=self.bus.publish_outbound)
self.tools.register(message_tool)
# Spawn tool (for subagents)
spawn_tool = SpawnTool(manager=self.subagents)
self.tools.register(spawn_tool)
# Cron tool (for scheduling)
if self.cron_service:
self.tools.register(CronTool(self.cron_service))
# DeepCode tools (conditionally loaded when DEEPCODE_API_URL is set)
deepcode_url = os.environ.get("DEEPCODE_API_URL")
if deepcode_url:
from nanobot.agent.tools.deepcode import create_all_tools
for tool in create_all_tools(api_url=deepcode_url):
self.tools.register(tool)
logger.info(f"DeepCode tools registered (API: {deepcode_url})")
async def run(self) -> None:
"""Run the agent loop, processing messages from the bus."""
self._running = True
logger.info("Agent loop started")
while self._running:
try:
# Wait for next message
msg = await asyncio.wait_for(self.bus.consume_inbound(), timeout=1.0)
# Process it
try:
response = await self._process_message(msg)
if response:
await self.bus.publish_outbound(response)
except Exception as e:
logger.error(f"Error processing message: {e}")
# Send error response
await self.bus.publish_outbound(
OutboundMessage(
channel=msg.channel,
chat_id=msg.chat_id,
content=f"Sorry, I encountered an error: {str(e)}",
)
)
except asyncio.TimeoutError:
continue
def stop(self) -> None:
"""Stop the agent loop."""
self._running = False
logger.info("Agent loop stopping")
async def _process_message(self, msg: InboundMessage) -> OutboundMessage | None:
"""
Process a single inbound message.
Args:
msg: The inbound message to process.
Returns:
The response message, or None if no response needed.
"""
# Handle system messages (subagent announces)
# The chat_id contains the original "channel:chat_id" to route back to
if msg.channel == "system":
return await self._process_system_message(msg)
preview = msg.content[:80] + "..." if len(msg.content) > 80 else msg.content
logger.info(f"Processing message from {msg.channel}:{msg.sender_id}: {preview}")
# Get or create session
session = self.sessions.get_or_create(msg.session_key)
# Update tool contexts
message_tool = self.tools.get("message")
if isinstance(message_tool, MessageTool):
message_tool.set_context(msg.channel, msg.chat_id)
spawn_tool = self.tools.get("spawn")
if isinstance(spawn_tool, SpawnTool):
spawn_tool.set_context(msg.channel, msg.chat_id)
cron_tool = self.tools.get("cron")
if isinstance(cron_tool, CronTool):
cron_tool.set_context(msg.channel, msg.chat_id)
# Build initial messages (use get_history for LLM-formatted messages)
messages = self.context.build_messages(
history=session.get_history(),
current_message=msg.content,
media=msg.media if msg.media else None,
channel=msg.channel,
chat_id=msg.chat_id,
)
# Agent loop
iteration = 0
final_content = None
while iteration < self.max_iterations:
iteration += 1
# Call LLM
response = await self.provider.chat(
messages=messages, tools=self.tools.get_definitions(), model=self.model
)
# Handle tool calls
if response.has_tool_calls:
# Add assistant message with tool calls
tool_call_dicts = [
{
"id": tc.id,
"type": "function",
"function": {
"name": tc.name,
"arguments": json.dumps(tc.arguments), # Must be JSON string
},
}
for tc in response.tool_calls
]
messages = self.context.add_assistant_message(
messages,
response.content,
tool_call_dicts,
reasoning_content=response.reasoning_content,
)
# Execute tools
for tool_call in response.tool_calls:
args_str = json.dumps(tool_call.arguments, ensure_ascii=False)
logger.info(f"Tool call: {tool_call.name}({args_str[:200]})")
result = await self.tools.execute(tool_call.name, tool_call.arguments)
messages = self.context.add_tool_result(
messages, tool_call.id, tool_call.name, result
)
else:
# No tool calls, we're done
final_content = response.content
break
if final_content is None:
final_content = "I've completed processing but have no response to give."
# Log response preview
preview = final_content[:120] + "..." if len(final_content) > 120 else final_content
logger.info(f"Response to {msg.channel}:{msg.sender_id}: {preview}")
# Save to session
session.add_message("user", msg.content)
session.add_message("assistant", final_content)
self.sessions.save(session)
return OutboundMessage(
channel=msg.channel,
chat_id=msg.chat_id,
content=final_content,
metadata=msg.metadata
or {}, # Pass through for channel-specific needs (e.g. Slack thread_ts)
)
async def _process_system_message(self, msg: InboundMessage) -> OutboundMessage | None:
"""
Process a system message (e.g., subagent announce).
The chat_id field contains "original_channel:original_chat_id" to route
the response back to the correct destination.
"""
logger.info(f"Processing system message from {msg.sender_id}")
# Parse origin from chat_id (format: "channel:chat_id")
if ":" in msg.chat_id:
parts = msg.chat_id.split(":", 1)
origin_channel = parts[0]
origin_chat_id = parts[1]
else:
# Fallback
origin_channel = "cli"
origin_chat_id = msg.chat_id
# Use the origin session for context
session_key = f"{origin_channel}:{origin_chat_id}"
session = self.sessions.get_or_create(session_key)
# Update tool contexts
message_tool = self.tools.get("message")
if isinstance(message_tool, MessageTool):
message_tool.set_context(origin_channel, origin_chat_id)
spawn_tool = self.tools.get("spawn")
if isinstance(spawn_tool, SpawnTool):
spawn_tool.set_context(origin_channel, origin_chat_id)
cron_tool = self.tools.get("cron")
if isinstance(cron_tool, CronTool):
cron_tool.set_context(origin_channel, origin_chat_id)
# Build messages with the announce content
messages = self.context.build_messages(
history=session.get_history(),
current_message=msg.content,
channel=origin_channel,
chat_id=origin_chat_id,
)
# Agent loop (limited for announce handling)
iteration = 0
final_content = None
while iteration < self.max_iterations:
iteration += 1
response = await self.provider.chat(
messages=messages, tools=self.tools.get_definitions(), model=self.model
)
if response.has_tool_calls:
tool_call_dicts = [
{
"id": tc.id,
"type": "function",
"function": {"name": tc.name, "arguments": json.dumps(tc.arguments)},
}
for tc in response.tool_calls
]
messages = self.context.add_assistant_message(
messages,
response.content,
tool_call_dicts,
reasoning_content=response.reasoning_content,
)
for tool_call in response.tool_calls:
args_str = json.dumps(tool_call.arguments, ensure_ascii=False)
logger.info(f"Tool call: {tool_call.name}({args_str[:200]})")
result = await self.tools.execute(tool_call.name, tool_call.arguments)
messages = self.context.add_tool_result(
messages, tool_call.id, tool_call.name, result
)
else:
final_content = response.content
break
if final_content is None:
final_content = "Background task completed."
# Save to session (mark as system message in history)
session.add_message("user", f"[System: {msg.sender_id}] {msg.content}")
session.add_message("assistant", final_content)
self.sessions.save(session)
return OutboundMessage(
channel=origin_channel, chat_id=origin_chat_id, content=final_content
)
async def process_direct(
self,
content: str,
session_key: str = "cli:direct",
channel: str = "cli",
chat_id: str = "direct",
) -> str:
"""
Process a message directly (for CLI or cron usage).
Args:
content: The message content.
session_key: Session identifier.
channel: Source channel (for context).
chat_id: Source chat ID (for context).
Returns:
The agent's response.
"""
msg = InboundMessage(channel=channel, sender_id="user", chat_id=chat_id, content=content)
response = await self._process_message(msg)
return response.content if response else ""
================================================
FILE: nanobot/nanobot/agent/memory.py
================================================
"""Memory system for persistent agent memory."""
from datetime import datetime
from pathlib import Path
from nanobot.utils.helpers import ensure_dir, today_date
class MemoryStore:
"""
Memory system for the agent.
Supports daily notes (memory/YYYY-MM-DD.md) and long-term memory (MEMORY.md).
"""
def __init__(self, workspace: Path):
self.workspace = workspace
self.memory_dir = ensure_dir(workspace / "memory")
self.memory_file = self.memory_dir / "MEMORY.md"
def get_today_file(self) -> Path:
"""Get path to today's memory file."""
return self.memory_dir / f"{today_date()}.md"
def read_today(self) -> str:
"""Read today's memory notes."""
today_file = self.get_today_file()
if today_file.exists():
return today_file.read_text(encoding="utf-8")
return ""
def append_today(self, content: str) -> None:
"""Append content to today's memory notes."""
today_file = self.get_today_file()
if today_file.exists():
existing = today_file.read_text(encoding="utf-8")
content = existing + "\n" + content
else:
# Add header for new day
header = f"# {today_date()}\n\n"
content = header + content
today_file.write_text(content, encoding="utf-8")
def read_long_term(self) -> str:
"""Read long-term memory (MEMORY.md)."""
if self.memory_file.exists():
return self.memory_file.read_text(encoding="utf-8")
return ""
def write_long_term(self, content: str) -> None:
"""Write to long-term memory (MEMORY.md)."""
self.memory_file.write_text(content, encoding="utf-8")
def get_recent_memories(self, days: int = 7) -> str:
"""
Get memories from the last N days.
Args:
days: Number of days to look back.
Returns:
Combined memory content.
"""
from datetime import timedelta
memories = []
today = datetime.now().date()
for i in range(days):
date = today - timedelta(days=i)
date_str = date.strftime("%Y-%m-%d")
file_path = self.memory_dir / f"{date_str}.md"
if file_path.exists():
content = file_path.read_text(encoding="utf-8")
memories.append(content)
return "\n\n---\n\n".join(memories)
def list_memory_files(self) -> list[Path]:
"""List all memory files sorted by date (newest first)."""
if not self.memory_dir.exists():
return []
files = list(self.memory_dir.glob("????-??-??.md"))
return sorted(files, reverse=True)
def get_memory_context(self) -> str:
"""
Get memory context for the agent.
Returns:
Formatted memory context including long-term and recent memories.
"""
parts = []
# Long-term memory
long_term = self.read_long_term()
if long_term:
parts.append("## Long-term Memory\n" + long_term)
# Today's notes
today = self.read_today()
if today:
parts.append("## Today's Notes\n" + today)
return "\n\n".join(parts) if parts else ""
================================================
FILE: nanobot/nanobot/agent/skills.py
================================================
"""Skills loader for agent capabilities."""
import json
import os
import re
import shutil
from pathlib import Path
# Default builtin skills directory (relative to this file)
BUILTIN_SKILLS_DIR = Path(__file__).parent.parent / "skills"
class SkillsLoader:
"""
Loader for agent skills.
Skills are markdown files (SKILL.md) that teach the agent how to use
specific tools or perform certain tasks.
"""
def __init__(self, workspace: Path, builtin_skills_dir: Path | None = None):
self.workspace = workspace
self.workspace_skills = workspace / "skills"
self.builtin_skills = builtin_skills_dir or BUILTIN_SKILLS_DIR
def list_skills(self, filter_unavailable: bool = True) -> list[dict[str, str]]:
"""
List all available skills.
Args:
filter_unavailable: If True, filter out skills with unmet requirements.
Returns:
List of skill info dicts with 'name', 'path', 'source'.
"""
skills = []
# Workspace skills (highest priority)
if self.workspace_skills.exists():
for skill_dir in self.workspace_skills.iterdir():
if skill_dir.is_dir():
skill_file = skill_dir / "SKILL.md"
if skill_file.exists():
skills.append(
{"name": skill_dir.name, "path": str(skill_file), "source": "workspace"}
)
# Built-in skills
if self.builtin_skills and self.builtin_skills.exists():
for skill_dir in self.builtin_skills.iterdir():
if skill_dir.is_dir():
skill_file = skill_dir / "SKILL.md"
if skill_file.exists() and not any(s["name"] == skill_dir.name for s in skills):
skills.append(
{"name": skill_dir.name, "path": str(skill_file), "source": "builtin"}
)
# Filter by requirements
if filter_unavailable:
return [s for s in skills if self._check_requirements(self._get_skill_meta(s["name"]))]
return skills
def load_skill(self, name: str) -> str | None:
"""
Load a skill by name.
Args:
name: Skill name (directory name).
Returns:
Skill content or None if not found.
"""
# Check workspace first
workspace_skill = self.workspace_skills / name / "SKILL.md"
if workspace_skill.exists():
return workspace_skill.read_text(encoding="utf-8")
# Check built-in
if self.builtin_skills:
builtin_skill = self.builtin_skills / name / "SKILL.md"
if builtin_skill.exists():
return builtin_skill.read_text(encoding="utf-8")
return None
def load_skills_for_context(self, skill_names: list[str]) -> str:
"""
Load specific skills for inclusion in agent context.
Args:
skill_names: List of skill names to load.
Returns:
Formatted skills content.
"""
parts = []
for name in skill_names:
content = self.load_skill(name)
if content:
content = self._strip_frontmatter(content)
parts.append(f"### Skill: {name}\n\n{content}")
return "\n\n---\n\n".join(parts) if parts else ""
def build_skills_summary(self) -> str:
"""
Build a summary of all skills (name, description, path, availability).
This is used for progressive loading - the agent can read the full
skill content using read_file when needed.
Returns:
XML-formatted skills summary.
"""
all_skills = self.list_skills(filter_unavailable=False)
if not all_skills:
return ""
def escape_xml(s: str) -> str:
return s.replace("&", "&").replace("<", "<").replace(">", ">")
lines = [""]
for s in all_skills:
name = escape_xml(s["name"])
path = s["path"]
desc = escape_xml(self._get_skill_description(s["name"]))
skill_meta = self._get_skill_meta(s["name"])
available = self._check_requirements(skill_meta)
lines.append(f' ')
lines.append(f" {name}")
lines.append(f" {desc}")
lines.append(f" {path}")
# Show missing requirements for unavailable skills
if not available:
missing = self._get_missing_requirements(skill_meta)
if missing:
lines.append(f" {escape_xml(missing)}")
lines.append(" ")
lines.append("")
return "\n".join(lines)
def _get_missing_requirements(self, skill_meta: dict) -> str:
"""Get a description of missing requirements."""
missing = []
requires = skill_meta.get("requires", {})
for b in requires.get("bins", []):
if not shutil.which(b):
missing.append(f"CLI: {b}")
for env in requires.get("env", []):
if not os.environ.get(env):
missing.append(f"ENV: {env}")
return ", ".join(missing)
def _get_skill_description(self, name: str) -> str:
"""Get the description of a skill from its frontmatter."""
meta = self.get_skill_metadata(name)
if meta and meta.get("description"):
return meta["description"]
return name # Fallback to skill name
def _strip_frontmatter(self, content: str) -> str:
"""Remove YAML frontmatter from markdown content."""
if content.startswith("---"):
match = re.match(r"^---\n.*?\n---\n", content, re.DOTALL)
if match:
return content[match.end() :].strip()
return content
def _parse_nanobot_metadata(self, raw: str) -> dict:
"""Parse nanobot metadata JSON from frontmatter."""
try:
data = json.loads(raw)
return data.get("nanobot", {}) if isinstance(data, dict) else {}
except (json.JSONDecodeError, TypeError):
return {}
def _check_requirements(self, skill_meta: dict) -> bool:
"""Check if skill requirements are met (bins, env vars)."""
requires = skill_meta.get("requires", {})
for b in requires.get("bins", []):
if not shutil.which(b):
return False
for env in requires.get("env", []):
if not os.environ.get(env):
return False
return True
def _get_skill_meta(self, name: str) -> dict:
"""Get nanobot metadata for a skill (cached in frontmatter)."""
meta = self.get_skill_metadata(name) or {}
return self._parse_nanobot_metadata(meta.get("metadata", ""))
def get_always_skills(self) -> list[str]:
"""Get skills marked as always=true that meet requirements."""
result = []
for s in self.list_skills(filter_unavailable=True):
meta = self.get_skill_metadata(s["name"]) or {}
skill_meta = self._parse_nanobot_metadata(meta.get("metadata", ""))
if skill_meta.get("always") or meta.get("always"):
result.append(s["name"])
return result
def get_skill_metadata(self, name: str) -> dict | None:
"""
Get metadata from a skill's frontmatter.
Args:
name: Skill name.
Returns:
Metadata dict or None.
"""
content = self.load_skill(name)
if not content:
return None
if content.startswith("---"):
match = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
if match:
# Simple YAML parsing
metadata = {}
for line in match.group(1).split("\n"):
if ":" in line:
key, value = line.split(":", 1)
metadata[key.strip()] = value.strip().strip("\"'")
return metadata
return None
================================================
FILE: nanobot/nanobot/agent/subagent.py
================================================
"""Subagent manager for background task execution."""
from __future__ import annotations
import asyncio
import json
import uuid
from pathlib import Path
from typing import TYPE_CHECKING, Any
from loguru import logger
from nanobot.agent.tools.filesystem import ListDirTool, ReadFileTool, WriteFileTool
from nanobot.agent.tools.registry import ToolRegistry
from nanobot.agent.tools.shell import ExecTool
from nanobot.agent.tools.web import WebFetchTool, WebSearchTool
from nanobot.bus.events import InboundMessage
from nanobot.bus.queue import MessageBus
from nanobot.providers.base import LLMProvider
if TYPE_CHECKING:
from nanobot.config.schema import ExecToolConfig
class SubagentManager:
"""
Manages background subagent execution.
Subagents are lightweight agent instances that run in the background
to handle specific tasks. They share the same LLM provider but have
isolated context and a focused system prompt.
"""
def __init__(
self,
provider: LLMProvider,
workspace: Path,
bus: MessageBus,
model: str | None = None,
brave_api_key: str | None = None,
exec_config: ExecToolConfig | None = None,
restrict_to_workspace: bool = False,
):
from nanobot.config.schema import ExecToolConfig
self.provider = provider
self.workspace = workspace
self.bus = bus
self.model = model or provider.get_default_model()
self.brave_api_key = brave_api_key
self.exec_config = exec_config or ExecToolConfig()
self.restrict_to_workspace = restrict_to_workspace
self._running_tasks: dict[str, asyncio.Task[None]] = {}
async def spawn(
self,
task: str,
label: str | None = None,
origin_channel: str = "cli",
origin_chat_id: str = "direct",
) -> str:
"""
Spawn a subagent to execute a task in the background.
Args:
task: The task description for the subagent.
label: Optional human-readable label for the task.
origin_channel: The channel to announce results to.
origin_chat_id: The chat ID to announce results to.
Returns:
Status message indicating the subagent was started.
"""
task_id = str(uuid.uuid4())[:8]
display_label = label or task[:30] + ("..." if len(task) > 30 else "")
origin = {
"channel": origin_channel,
"chat_id": origin_chat_id,
}
# Create background task
bg_task = asyncio.create_task(self._run_subagent(task_id, task, display_label, origin))
self._running_tasks[task_id] = bg_task
# Cleanup when done
bg_task.add_done_callback(lambda _: self._running_tasks.pop(task_id, None))
logger.info(f"Spawned subagent [{task_id}]: {display_label}")
return f"Subagent [{display_label}] started (id: {task_id}). I'll notify you when it completes."
async def _run_subagent(
self,
task_id: str,
task: str,
label: str,
origin: dict[str, str],
) -> None:
"""Execute the subagent task and announce the result."""
logger.info(f"Subagent [{task_id}] starting task: {label}")
try:
# Build subagent tools (no message tool, no spawn tool)
tools = ToolRegistry()
allowed_dir = self.workspace if self.restrict_to_workspace else None
tools.register(ReadFileTool(allowed_dir=allowed_dir))
tools.register(WriteFileTool(allowed_dir=allowed_dir))
tools.register(ListDirTool(allowed_dir=allowed_dir))
tools.register(
ExecTool(
working_dir=str(self.workspace),
timeout=self.exec_config.timeout,
restrict_to_workspace=self.restrict_to_workspace,
)
)
tools.register(WebSearchTool(api_key=self.brave_api_key))
tools.register(WebFetchTool())
# Build messages with subagent-specific prompt
system_prompt = self._build_subagent_prompt(task)
messages: list[dict[str, Any]] = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": task},
]
# Run agent loop (limited iterations)
max_iterations = 15
iteration = 0
final_result: str | None = None
while iteration < max_iterations:
iteration += 1
response = await self.provider.chat(
messages=messages,
tools=tools.get_definitions(),
model=self.model,
)
if response.has_tool_calls:
# Add assistant message with tool calls
tool_call_dicts = [
{
"id": tc.id,
"type": "function",
"function": {
"name": tc.name,
"arguments": json.dumps(tc.arguments),
},
}
for tc in response.tool_calls
]
messages.append(
{
"role": "assistant",
"content": response.content or "",
"tool_calls": tool_call_dicts,
}
)
# Execute tools
for tool_call in response.tool_calls:
args_str = json.dumps(tool_call.arguments)
logger.debug(
f"Subagent [{task_id}] executing: {tool_call.name} with arguments: {args_str}"
)
result = await tools.execute(tool_call.name, tool_call.arguments)
messages.append(
{
"role": "tool",
"tool_call_id": tool_call.id,
"name": tool_call.name,
"content": result,
}
)
else:
final_result = response.content
break
if final_result is None:
final_result = "Task completed but no final response was generated."
logger.info(f"Subagent [{task_id}] completed successfully")
await self._announce_result(task_id, label, task, final_result, origin, "ok")
except Exception as e:
error_msg = f"Error: {str(e)}"
logger.error(f"Subagent [{task_id}] failed: {e}")
await self._announce_result(task_id, label, task, error_msg, origin, "error")
async def _announce_result(
self,
task_id: str,
label: str,
task: str,
result: str,
origin: dict[str, str],
status: str,
) -> None:
"""Announce the subagent result to the main agent via the message bus."""
status_text = "completed successfully" if status == "ok" else "failed"
announce_content = f"""[Subagent '{label}' {status_text}]
Task: {task}
Result:
{result}
Summarize this naturally for the user. Keep it brief (1-2 sentences). Do not mention technical details like "subagent" or task IDs."""
# Inject as system message to trigger main agent
msg = InboundMessage(
channel="system",
sender_id="subagent",
chat_id=f"{origin['channel']}:{origin['chat_id']}",
content=announce_content,
)
await self.bus.publish_inbound(msg)
logger.debug(
f"Subagent [{task_id}] announced result to {origin['channel']}:{origin['chat_id']}"
)
def _build_subagent_prompt(self, task: str) -> str:
"""Build a focused system prompt for the subagent."""
return f"""# Subagent
You are a subagent spawned by the main agent to complete a specific task.
## Your Task
{task}
## Rules
1. Stay focused - complete only the assigned task, nothing else
2. Your final response will be reported back to the main agent
3. Do not initiate conversations or take on side tasks
4. Be concise but informative in your findings
## What You Can Do
- Read and write files in the workspace
- Execute shell commands
- Search the web and fetch web pages
- Complete the task thoroughly
## What You Cannot Do
- Send messages directly to users (no message tool available)
- Spawn other subagents
- Access the main agent's conversation history
## Workspace
Your workspace is at: {self.workspace}
When you have completed the task, provide a clear summary of your findings or actions."""
def get_running_count(self) -> int:
"""Return the number of currently running subagents."""
return len(self._running_tasks)
================================================
FILE: nanobot/nanobot/agent/tools/__init__.py
================================================
"""Agent tools module."""
from nanobot.agent.tools.base import Tool
from nanobot.agent.tools.registry import ToolRegistry
__all__ = ["Tool", "ToolRegistry"]
================================================
FILE: nanobot/nanobot/agent/tools/base.py
================================================
"""Base class for agent tools."""
from abc import ABC, abstractmethod
from typing import Any
class Tool(ABC):
"""
Abstract base class for agent tools.
Tools are capabilities that the agent can use to interact with
the environment, such as reading files, executing commands, etc.
"""
_TYPE_MAP = {
"string": str,
"integer": int,
"number": (int, float),
"boolean": bool,
"array": list,
"object": dict,
}
@property
@abstractmethod
def name(self) -> str:
"""Tool name used in function calls."""
pass
@property
@abstractmethod
def description(self) -> str:
"""Description of what the tool does."""
pass
@property
@abstractmethod
def parameters(self) -> dict[str, Any]:
"""JSON Schema for tool parameters."""
pass
@abstractmethod
async def execute(self, **kwargs: Any) -> str:
"""
Execute the tool with given parameters.
Args:
**kwargs: Tool-specific parameters.
Returns:
String result of the tool execution.
"""
pass
def validate_params(self, params: dict[str, Any]) -> list[str]:
"""Validate tool parameters against JSON schema. Returns error list (empty if valid)."""
schema = self.parameters or {}
if schema.get("type", "object") != "object":
raise ValueError(f"Schema must be object type, got {schema.get('type')!r}")
return self._validate(params, {**schema, "type": "object"}, "")
def _validate(self, val: Any, schema: dict[str, Any], path: str) -> list[str]:
t, label = schema.get("type"), path or "parameter"
if t in self._TYPE_MAP and not isinstance(val, self._TYPE_MAP[t]):
return [f"{label} should be {t}"]
errors = []
if "enum" in schema and val not in schema["enum"]:
errors.append(f"{label} must be one of {schema['enum']}")
if t in ("integer", "number"):
if "minimum" in schema and val < schema["minimum"]:
errors.append(f"{label} must be >= {schema['minimum']}")
if "maximum" in schema and val > schema["maximum"]:
errors.append(f"{label} must be <= {schema['maximum']}")
if t == "string":
if "minLength" in schema and len(val) < schema["minLength"]:
errors.append(f"{label} must be at least {schema['minLength']} chars")
if "maxLength" in schema and len(val) > schema["maxLength"]:
errors.append(f"{label} must be at most {schema['maxLength']} chars")
if t == "object":
props = schema.get("properties", {})
for k in schema.get("required", []):
if k not in val:
errors.append(f"missing required {path + '.' + k if path else k}")
for k, v in val.items():
if k in props:
errors.extend(self._validate(v, props[k], path + "." + k if path else k))
if t == "array" and "items" in schema:
for i, item in enumerate(val):
errors.extend(
self._validate(item, schema["items"], f"{path}[{i}]" if path else f"[{i}]")
)
return errors
def to_schema(self) -> dict[str, Any]:
"""Convert tool to OpenAI function schema format."""
return {
"type": "function",
"function": {
"name": self.name,
"description": self.description,
"parameters": self.parameters,
},
}
================================================
FILE: nanobot/nanobot/agent/tools/cron.py
================================================
"""Cron tool for scheduling reminders and tasks."""
from typing import Any
from nanobot.agent.tools.base import Tool
from nanobot.cron.service import CronService
from nanobot.cron.types import CronSchedule
class CronTool(Tool):
"""Tool to schedule reminders and recurring tasks."""
def __init__(self, cron_service: CronService):
self._cron = cron_service
self._channel = ""
self._chat_id = ""
def set_context(self, channel: str, chat_id: str) -> None:
"""Set the current session context for delivery."""
self._channel = channel
self._chat_id = chat_id
@property
def name(self) -> str:
return "cron"
@property
def description(self) -> str:
return "Schedule reminders and recurring tasks. Actions: add, list, remove."
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"action": {
"type": "string",
"enum": ["add", "list", "remove"],
"description": "Action to perform",
},
"message": {"type": "string", "description": "Reminder message (for add)"},
"every_seconds": {
"type": "integer",
"description": "Interval in seconds (for recurring tasks)",
},
"cron_expr": {
"type": "string",
"description": "Cron expression like '0 9 * * *' (for scheduled tasks)",
},
"job_id": {"type": "string", "description": "Job ID (for remove)"},
},
"required": ["action"],
}
async def execute(
self,
action: str,
message: str = "",
every_seconds: int | None = None,
cron_expr: str | None = None,
job_id: str | None = None,
**kwargs: Any,
) -> str:
if action == "add":
return self._add_job(message, every_seconds, cron_expr)
elif action == "list":
return self._list_jobs()
elif action == "remove":
return self._remove_job(job_id)
return f"Unknown action: {action}"
def _add_job(self, message: str, every_seconds: int | None, cron_expr: str | None) -> str:
if not message:
return "Error: message is required for add"
if not self._channel or not self._chat_id:
return "Error: no session context (channel/chat_id)"
# Build schedule
if every_seconds:
schedule = CronSchedule(kind="every", every_ms=every_seconds * 1000)
elif cron_expr:
schedule = CronSchedule(kind="cron", expr=cron_expr)
else:
return "Error: either every_seconds or cron_expr is required"
job = self._cron.add_job(
name=message[:30],
schedule=schedule,
message=message,
deliver=True,
channel=self._channel,
to=self._chat_id,
)
return f"Created job '{job.name}' (id: {job.id})"
def _list_jobs(self) -> str:
jobs = self._cron.list_jobs()
if not jobs:
return "No scheduled jobs."
lines = [f"- {j.name} (id: {j.id}, {j.schedule.kind})" for j in jobs]
return "Scheduled jobs:\n" + "\n".join(lines)
def _remove_job(self, job_id: str | None) -> str:
if not job_id:
return "Error: job_id is required for remove"
if self._cron.remove_job(job_id):
return f"Removed job {job_id}"
return f"Job {job_id} not found"
================================================
FILE: nanobot/nanobot/agent/tools/deepcode.py
================================================
"""
DeepCode integration tools for nanobot.
These tools allow nanobot to interact with the DeepCode backend API
for paper-to-code reproduction, chat-based code generation, and task management.
Communication: HTTP requests to DeepCode's FastAPI backend.
In Docker Compose: nanobot -> http://deepcode:8000/api/v1/...
"""
import os
from typing import Any
import httpx
from nanobot.agent.tools.base import Tool
def _get_deepcode_url() -> str:
"""Get DeepCode API base URL from environment."""
return os.environ.get("DEEPCODE_API_URL", "http://deepcode:8000")
class DeepCodePaper2CodeTool(Tool):
"""Submit a paper (URL or file path) to DeepCode for automatic code reproduction."""
def __init__(self, api_url: str | None = None):
self._api_url = api_url or _get_deepcode_url()
@property
def name(self) -> str:
return "deepcode_paper2code"
@property
def description(self) -> str:
return (
"Submit a research paper to DeepCode for automatic code reproduction. "
"Accepts a paper URL (e.g. arxiv link) or a local file path. "
"Returns a task ID for tracking progress. "
"The code generation process runs in the background and may take 10-60 minutes. "
"Use deepcode_status to check progress."
)
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"input_source": {
"type": "string",
"description": "Paper URL (e.g. https://arxiv.org/abs/...) or local file path",
},
"input_type": {
"type": "string",
"enum": ["url", "file"],
"description": "Type of input: 'url' for web links, 'file' for local files",
},
"enable_indexing": {
"type": "boolean",
"description": "Enable code reference indexing for enhanced quality (slower but better). Default: false",
},
},
"required": ["input_source", "input_type"],
}
async def execute(
self,
input_source: str,
input_type: str = "url",
enable_indexing: bool = False,
**kwargs: Any,
) -> str:
try:
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.post(
f"{self._api_url}/api/v1/workflows/paper-to-code",
json={
"input_source": input_source,
"input_type": input_type,
"enable_indexing": enable_indexing,
},
)
response.raise_for_status()
data = response.json()
task_id = data.get("task_id", "unknown")
return (
f"Paper-to-code task submitted successfully!\n"
f"Task ID: {task_id}\n"
f"Status: {data.get('status', 'started')}\n"
f"Input: {input_source}\n"
f"Indexing: {'enabled' if enable_indexing else 'disabled (fast mode)'}\n\n"
f"The code generation is running in the background. "
f"Use deepcode_status with task_id='{task_id}' to check progress."
)
except httpx.ConnectError:
return "Error: Cannot connect to DeepCode backend. Is the DeepCode service running?"
except httpx.HTTPStatusError as e:
return (
f"Error: DeepCode API returned status {e.response.status_code}: {e.response.text}"
)
except Exception as e:
return f"Error submitting paper to DeepCode: {str(e)}"
class DeepCodeChat2CodeTool(Tool):
"""Submit text requirements to DeepCode for code generation."""
def __init__(self, api_url: str | None = None):
self._api_url = api_url or _get_deepcode_url()
@property
def name(self) -> str:
return "deepcode_chat2code"
@property
def description(self) -> str:
return (
"Submit coding requirements to DeepCode for automatic code generation. "
"Provide a text description of what you want to build (e.g. web app, algorithm, backend service). "
"DeepCode will generate a complete implementation. "
"Returns a task ID for tracking progress."
)
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"requirements": {
"type": "string",
"description": "Detailed description of coding requirements",
},
"enable_indexing": {
"type": "boolean",
"description": "Enable code reference indexing for enhanced quality. Default: false",
},
},
"required": ["requirements"],
}
async def execute(
self,
requirements: str,
enable_indexing: bool = False,
**kwargs: Any,
) -> str:
try:
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.post(
f"{self._api_url}/api/v1/workflows/chat-planning",
json={
"requirements": requirements,
"enable_indexing": enable_indexing,
},
)
response.raise_for_status()
data = response.json()
task_id = data.get("task_id", "unknown")
return (
f"Chat-to-code task submitted successfully!\n"
f"Task ID: {task_id}\n"
f"Status: {data.get('status', 'started')}\n"
f"Requirements: {requirements[:200]}{'...' if len(requirements) > 200 else ''}\n\n"
f"The code generation is running in the background. "
f"Use deepcode_status with task_id='{task_id}' to check progress."
)
except httpx.ConnectError:
return "Error: Cannot connect to DeepCode backend. Is the DeepCode service running?"
except httpx.HTTPStatusError as e:
return (
f"Error: DeepCode API returned status {e.response.status_code}: {e.response.text}"
)
except Exception as e:
return f"Error submitting requirements to DeepCode: {str(e)}"
class DeepCodeStatusTool(Tool):
"""Check the status and progress of a DeepCode task."""
def __init__(self, api_url: str | None = None):
self._api_url = api_url or _get_deepcode_url()
@property
def name(self) -> str:
return "deepcode_status"
@property
def description(self) -> str:
return (
"Check the status and progress of a DeepCode code generation task. "
"Provide the task_id returned by deepcode_paper2code or deepcode_chat2code. "
"Returns current status, progress percentage, and result when complete."
)
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"task_id": {
"type": "string",
"description": "The task ID to check status for",
},
},
"required": ["task_id"],
}
async def execute(self, task_id: str, **kwargs: Any) -> str:
try:
async with httpx.AsyncClient(timeout=15.0) as client:
response = await client.get(f"{self._api_url}/api/v1/workflows/status/{task_id}")
response.raise_for_status()
data = response.json()
status = data.get("status", "unknown")
progress = data.get("progress", 0)
message = data.get("message", "")
result = data.get("result")
error = data.get("error")
lines = [
f"Task ID: {task_id}",
f"Status: {status}",
f"Progress: {progress}%",
]
if message:
lines.append(f"Message: {message}")
if status == "completed" and result:
lines.append(f"\nResult:\n{result}")
elif status == "error" and error:
lines.append(f"\nError: {error}")
elif status == "waiting_for_input":
interaction = data.get("pending_interaction")
if interaction:
lines.append("\nWaiting for user input:")
lines.append(f" Type: {interaction.get('type', 'unknown')}")
lines.append(f" Title: {interaction.get('title', '')}")
lines.append(f" Description: {interaction.get('description', '')}")
return "\n".join(lines)
except httpx.ConnectError:
return "Error: Cannot connect to DeepCode backend. Is the DeepCode service running?"
except httpx.HTTPStatusError as e:
if e.response.status_code == 404:
return f"Error: Task '{task_id}' not found. It may have expired."
return (
f"Error: DeepCode API returned status {e.response.status_code}: {e.response.text}"
)
except Exception as e:
return f"Error checking task status: {str(e)}"
class DeepCodeListTasksTool(Tool):
"""List active and recent DeepCode tasks."""
def __init__(self, api_url: str | None = None):
self._api_url = api_url or _get_deepcode_url()
@property
def name(self) -> str:
return "deepcode_list_tasks"
@property
def description(self) -> str:
return (
"List all active and recent DeepCode code generation tasks. "
"Shows task IDs, status, progress, and results summary."
)
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"limit": {
"type": "integer",
"description": "Maximum number of recent tasks to show. Default: 10",
"minimum": 1,
"maximum": 50,
},
},
}
async def execute(self, limit: int = 10, **kwargs: Any) -> str:
try:
async with httpx.AsyncClient(timeout=15.0) as client:
# Fetch active tasks
active_resp = await client.get(f"{self._api_url}/api/v1/workflows/active")
active_resp.raise_for_status()
active_data = active_resp.json()
# Fetch recent tasks
recent_resp = await client.get(
f"{self._api_url}/api/v1/workflows/recent",
params={"limit": limit},
)
recent_resp.raise_for_status()
recent_data = recent_resp.json()
lines = []
# Active tasks
active_tasks = active_data.get("tasks", [])
if active_tasks:
lines.append(f"=== Active Tasks ({len(active_tasks)}) ===")
for task in active_tasks:
lines.append(
f" [{task.get('status', '?')}] {task.get('task_id', '?')} "
f"- {task.get('progress', 0)}% - {task.get('message', '')}"
)
lines.append("")
# Recent tasks
recent_tasks = recent_data.get("tasks", [])
if recent_tasks:
lines.append(f"=== Recent Tasks ({len(recent_tasks)}) ===")
for task in recent_tasks:
status_icon = {
"completed": "done",
"error": "error",
"running": "running",
"cancelled": "cancelled",
}.get(task.get("status", ""), "?")
lines.append(
f" [{status_icon}] {task.get('task_id', '?')} "
f"- {task.get('status', '?')} - {task.get('message', '')}"
)
if not lines:
return "No DeepCode tasks found."
return "\n".join(lines)
except httpx.ConnectError:
return "Error: Cannot connect to DeepCode backend. Is the DeepCode service running?"
except Exception as e:
return f"Error listing tasks: {str(e)}"
class DeepCodeCancelTool(Tool):
"""Cancel a running DeepCode task."""
def __init__(self, api_url: str | None = None):
self._api_url = api_url or _get_deepcode_url()
@property
def name(self) -> str:
return "deepcode_cancel"
@property
def description(self) -> str:
return "Cancel a running DeepCode code generation task."
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"task_id": {
"type": "string",
"description": "The task ID to cancel",
},
},
"required": ["task_id"],
}
async def execute(self, task_id: str, **kwargs: Any) -> str:
try:
async with httpx.AsyncClient(timeout=15.0) as client:
response = await client.post(f"{self._api_url}/api/v1/workflows/cancel/{task_id}")
response.raise_for_status()
return f"Task '{task_id}' has been cancelled successfully."
except httpx.ConnectError:
return "Error: Cannot connect to DeepCode backend. Is the DeepCode service running?"
except httpx.HTTPStatusError as e:
if e.response.status_code == 400:
return f"Error: Task '{task_id}' not found or cannot be cancelled."
return (
f"Error: DeepCode API returned status {e.response.status_code}: {e.response.text}"
)
except Exception as e:
return f"Error cancelling task: {str(e)}"
class DeepCodeRespondTool(Tool):
"""Respond to a DeepCode User-in-Loop interaction request."""
def __init__(self, api_url: str | None = None):
self._api_url = api_url or _get_deepcode_url()
@property
def name(self) -> str:
return "deepcode_respond"
@property
def description(self) -> str:
return (
"Respond to a DeepCode User-in-Loop interaction. "
"When a DeepCode task is waiting for user input (e.g. requirement clarification, "
"plan review), use this tool to submit the user's response. "
"First check deepcode_status to see the pending interaction details."
)
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"task_id": {
"type": "string",
"description": "The task ID that is waiting for input",
},
"action": {
"type": "string",
"enum": ["submit", "confirm", "modify", "skip", "cancel"],
"description": "User's action: submit answers, confirm plan, modify, skip, or cancel",
},
"data": {
"type": "object",
"description": "Response data (e.g. answers to questions, modification feedback)",
},
"skipped": {
"type": "boolean",
"description": "Whether the user chose to skip this interaction. Default: false",
},
},
"required": ["task_id", "action"],
}
async def execute(
self,
task_id: str,
action: str,
data: dict | None = None,
skipped: bool = False,
**kwargs: Any,
) -> str:
try:
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.post(
f"{self._api_url}/api/v1/workflows/respond/{task_id}",
json={
"action": action,
"data": data or {},
"skipped": skipped,
},
)
response.raise_for_status()
response.json() # validate JSON response
return (
f"Response submitted successfully!\n"
f"Task ID: {task_id}\n"
f"Action: {action}\n"
f"The workflow will now continue."
)
except httpx.ConnectError:
return "Error: Cannot connect to DeepCode backend. Is the DeepCode service running?"
except httpx.HTTPStatusError as e:
if e.response.status_code == 400:
detail = e.response.json().get("detail", "Unknown error")
return f"Error: {detail}"
return (
f"Error: DeepCode API returned status {e.response.status_code}: {e.response.text}"
)
except Exception as e:
return f"Error responding to interaction: {str(e)}"
# ============================================================
# Helper: create all DeepCode tools at once
# ============================================================
def create_all_tools(api_url: str | None = None) -> list[Tool]:
"""
Create all DeepCode tools with the given API URL.
Usage in AgentLoop._register_default_tools():
deepcode_url = os.environ.get("DEEPCODE_API_URL")
if deepcode_url:
from nanobot.agent.tools.deepcode import create_all_tools
for tool in create_all_tools(api_url=deepcode_url):
self.tools.register(tool)
"""
url = api_url or _get_deepcode_url()
return [
DeepCodePaper2CodeTool(api_url=url),
DeepCodeChat2CodeTool(api_url=url),
DeepCodeStatusTool(api_url=url),
DeepCodeListTasksTool(api_url=url),
DeepCodeCancelTool(api_url=url),
DeepCodeRespondTool(api_url=url),
]
================================================
FILE: nanobot/nanobot/agent/tools/filesystem.py
================================================
"""File system tools: read, write, edit."""
from pathlib import Path
from typing import Any
from nanobot.agent.tools.base import Tool
def _resolve_path(path: str, allowed_dir: Path | None = None) -> Path:
"""Resolve path and optionally enforce directory restriction."""
resolved = Path(path).expanduser().resolve()
if allowed_dir and not str(resolved).startswith(str(allowed_dir.resolve())):
raise PermissionError(f"Path {path} is outside allowed directory {allowed_dir}")
return resolved
class ReadFileTool(Tool):
"""Tool to read file contents."""
def __init__(self, allowed_dir: Path | None = None):
self._allowed_dir = allowed_dir
@property
def name(self) -> str:
return "read_file"
@property
def description(self) -> str:
return "Read the contents of a file at the given path."
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {"path": {"type": "string", "description": "The file path to read"}},
"required": ["path"],
}
async def execute(self, path: str, **kwargs: Any) -> str:
try:
file_path = _resolve_path(path, self._allowed_dir)
if not file_path.exists():
return f"Error: File not found: {path}"
if not file_path.is_file():
return f"Error: Not a file: {path}"
content = file_path.read_text(encoding="utf-8")
return content
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error reading file: {str(e)}"
class WriteFileTool(Tool):
"""Tool to write content to a file."""
def __init__(self, allowed_dir: Path | None = None):
self._allowed_dir = allowed_dir
@property
def name(self) -> str:
return "write_file"
@property
def description(self) -> str:
return "Write content to a file at the given path. Creates parent directories if needed."
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"path": {"type": "string", "description": "The file path to write to"},
"content": {"type": "string", "description": "The content to write"},
},
"required": ["path", "content"],
}
async def execute(self, path: str, content: str, **kwargs: Any) -> str:
try:
file_path = _resolve_path(path, self._allowed_dir)
file_path.parent.mkdir(parents=True, exist_ok=True)
file_path.write_text(content, encoding="utf-8")
return f"Successfully wrote {len(content)} bytes to {path}"
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error writing file: {str(e)}"
class EditFileTool(Tool):
"""Tool to edit a file by replacing text."""
def __init__(self, allowed_dir: Path | None = None):
self._allowed_dir = allowed_dir
@property
def name(self) -> str:
return "edit_file"
@property
def description(self) -> str:
return "Edit a file by replacing old_text with new_text. The old_text must exist exactly in the file."
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"path": {"type": "string", "description": "The file path to edit"},
"old_text": {"type": "string", "description": "The exact text to find and replace"},
"new_text": {"type": "string", "description": "The text to replace with"},
},
"required": ["path", "old_text", "new_text"],
}
async def execute(self, path: str, old_text: str, new_text: str, **kwargs: Any) -> str:
try:
file_path = _resolve_path(path, self._allowed_dir)
if not file_path.exists():
return f"Error: File not found: {path}"
content = file_path.read_text(encoding="utf-8")
if old_text not in content:
return "Error: old_text not found in file. Make sure it matches exactly."
# Count occurrences
count = content.count(old_text)
if count > 1:
return f"Warning: old_text appears {count} times. Please provide more context to make it unique."
new_content = content.replace(old_text, new_text, 1)
file_path.write_text(new_content, encoding="utf-8")
return f"Successfully edited {path}"
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error editing file: {str(e)}"
class ListDirTool(Tool):
"""Tool to list directory contents."""
def __init__(self, allowed_dir: Path | None = None):
self._allowed_dir = allowed_dir
@property
def name(self) -> str:
return "list_dir"
@property
def description(self) -> str:
return "List the contents of a directory."
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {"path": {"type": "string", "description": "The directory path to list"}},
"required": ["path"],
}
async def execute(self, path: str, **kwargs: Any) -> str:
try:
dir_path = _resolve_path(path, self._allowed_dir)
if not dir_path.exists():
return f"Error: Directory not found: {path}"
if not dir_path.is_dir():
return f"Error: Not a directory: {path}"
items = []
for item in sorted(dir_path.iterdir()):
prefix = "📁 " if item.is_dir() else "📄 "
items.append(f"{prefix}{item.name}")
if not items:
return f"Directory {path} is empty"
return "\n".join(items)
except PermissionError as e:
return f"Error: {e}"
except Exception as e:
return f"Error listing directory: {str(e)}"
================================================
FILE: nanobot/nanobot/agent/tools/message.py
================================================
"""Message tool for sending messages to users."""
from typing import Any, Awaitable, Callable
from nanobot.agent.tools.base import Tool
from nanobot.bus.events import OutboundMessage
class MessageTool(Tool):
"""Tool to send messages to users on chat channels."""
def __init__(
self,
send_callback: Callable[[OutboundMessage], Awaitable[None]] | None = None,
default_channel: str = "",
default_chat_id: str = "",
):
self._send_callback = send_callback
self._default_channel = default_channel
self._default_chat_id = default_chat_id
def set_context(self, channel: str, chat_id: str) -> None:
"""Set the current message context."""
self._default_channel = channel
self._default_chat_id = chat_id
def set_send_callback(self, callback: Callable[[OutboundMessage], Awaitable[None]]) -> None:
"""Set the callback for sending messages."""
self._send_callback = callback
@property
def name(self) -> str:
return "message"
@property
def description(self) -> str:
return "Send a message to the user. Use this when you want to communicate something."
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"content": {"type": "string", "description": "The message content to send"},
"channel": {
"type": "string",
"description": "Optional: target channel (telegram, discord, etc.)",
},
"chat_id": {"type": "string", "description": "Optional: target chat/user ID"},
},
"required": ["content"],
}
async def execute(
self, content: str, channel: str | None = None, chat_id: str | None = None, **kwargs: Any
) -> str:
channel = channel or self._default_channel
chat_id = chat_id or self._default_chat_id
if not channel or not chat_id:
return "Error: No target channel/chat specified"
if not self._send_callback:
return "Error: Message sending not configured"
msg = OutboundMessage(channel=channel, chat_id=chat_id, content=content)
try:
await self._send_callback(msg)
return f"Message sent to {channel}:{chat_id}"
except Exception as e:
return f"Error sending message: {str(e)}"
================================================
FILE: nanobot/nanobot/agent/tools/registry.py
================================================
"""Tool registry for dynamic tool management."""
from typing import Any
from nanobot.agent.tools.base import Tool
class ToolRegistry:
"""
Registry for agent tools.
Allows dynamic registration and execution of tools.
"""
def __init__(self):
self._tools: dict[str, Tool] = {}
def register(self, tool: Tool) -> None:
"""Register a tool."""
self._tools[tool.name] = tool
def unregister(self, name: str) -> None:
"""Unregister a tool by name."""
self._tools.pop(name, None)
def get(self, name: str) -> Tool | None:
"""Get a tool by name."""
return self._tools.get(name)
def has(self, name: str) -> bool:
"""Check if a tool is registered."""
return name in self._tools
def get_definitions(self) -> list[dict[str, Any]]:
"""Get all tool definitions in OpenAI format."""
return [tool.to_schema() for tool in self._tools.values()]
async def execute(self, name: str, params: dict[str, Any]) -> str:
"""
Execute a tool by name with given parameters.
Args:
name: Tool name.
params: Tool parameters.
Returns:
Tool execution result as string.
Raises:
KeyError: If tool not found.
"""
tool = self._tools.get(name)
if not tool:
return f"Error: Tool '{name}' not found"
try:
errors = tool.validate_params(params)
if errors:
return f"Error: Invalid parameters for tool '{name}': " + "; ".join(errors)
return await tool.execute(**params)
except Exception as e:
return f"Error executing {name}: {str(e)}"
@property
def tool_names(self) -> list[str]:
"""Get list of registered tool names."""
return list(self._tools.keys())
def __len__(self) -> int:
return len(self._tools)
def __contains__(self, name: str) -> bool:
return name in self._tools
================================================
FILE: nanobot/nanobot/agent/tools/shell.py
================================================
"""Shell execution tool."""
import asyncio
import os
import re
from pathlib import Path
from typing import Any
from nanobot.agent.tools.base import Tool
class ExecTool(Tool):
"""Tool to execute shell commands."""
def __init__(
self,
timeout: int = 60,
working_dir: str | None = None,
deny_patterns: list[str] | None = None,
allow_patterns: list[str] | None = None,
restrict_to_workspace: bool = False,
):
self.timeout = timeout
self.working_dir = working_dir
self.deny_patterns = deny_patterns or [
r"\brm\s+-[rf]{1,2}\b", # rm -r, rm -rf, rm -fr
r"\bdel\s+/[fq]\b", # del /f, del /q
r"\brmdir\s+/s\b", # rmdir /s
r"\b(format|mkfs|diskpart)\b", # disk operations
r"\bdd\s+if=", # dd
r">\s*/dev/sd", # write to disk
r"\b(shutdown|reboot|poweroff)\b", # system power
r":\(\)\s*\{.*\};\s*:", # fork bomb
]
self.allow_patterns = allow_patterns or []
self.restrict_to_workspace = restrict_to_workspace
@property
def name(self) -> str:
return "exec"
@property
def description(self) -> str:
return "Execute a shell command and return its output. Use with caution."
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"command": {"type": "string", "description": "The shell command to execute"},
"working_dir": {
"type": "string",
"description": "Optional working directory for the command",
},
},
"required": ["command"],
}
async def execute(self, command: str, working_dir: str | None = None, **kwargs: Any) -> str:
cwd = working_dir or self.working_dir or os.getcwd()
guard_error = self._guard_command(command, cwd)
if guard_error:
return guard_error
try:
process = await asyncio.create_subprocess_shell(
command,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=cwd,
)
try:
stdout, stderr = await asyncio.wait_for(process.communicate(), timeout=self.timeout)
except asyncio.TimeoutError:
process.kill()
return f"Error: Command timed out after {self.timeout} seconds"
output_parts = []
if stdout:
output_parts.append(stdout.decode("utf-8", errors="replace"))
if stderr:
stderr_text = stderr.decode("utf-8", errors="replace")
if stderr_text.strip():
output_parts.append(f"STDERR:\n{stderr_text}")
if process.returncode != 0:
output_parts.append(f"\nExit code: {process.returncode}")
result = "\n".join(output_parts) if output_parts else "(no output)"
# Truncate very long output
max_len = 10000
if len(result) > max_len:
result = result[:max_len] + f"\n... (truncated, {len(result) - max_len} more chars)"
return result
except Exception as e:
return f"Error executing command: {str(e)}"
def _guard_command(self, command: str, cwd: str) -> str | None:
"""Best-effort safety guard for potentially destructive commands."""
cmd = command.strip()
lower = cmd.lower()
for pattern in self.deny_patterns:
if re.search(pattern, lower):
return "Error: Command blocked by safety guard (dangerous pattern detected)"
if self.allow_patterns:
if not any(re.search(p, lower) for p in self.allow_patterns):
return "Error: Command blocked by safety guard (not in allowlist)"
if self.restrict_to_workspace:
if "..\\" in cmd or "../" in cmd:
return "Error: Command blocked by safety guard (path traversal detected)"
cwd_path = Path(cwd).resolve()
win_paths = re.findall(r"[A-Za-z]:\\[^\\\"']+", cmd)
posix_paths = re.findall(r"/[^\s\"']+", cmd)
for raw in win_paths + posix_paths:
try:
p = Path(raw).resolve()
except Exception:
continue
if cwd_path not in p.parents and p != cwd_path:
return "Error: Command blocked by safety guard (path outside working dir)"
return None
================================================
FILE: nanobot/nanobot/agent/tools/spawn.py
================================================
"""Spawn tool for creating background subagents."""
from typing import TYPE_CHECKING, Any
from nanobot.agent.tools.base import Tool
if TYPE_CHECKING:
from nanobot.agent.subagent import SubagentManager
class SpawnTool(Tool):
"""
Tool to spawn a subagent for background task execution.
The subagent runs asynchronously and announces its result back
to the main agent when complete.
"""
def __init__(self, manager: "SubagentManager"):
self._manager = manager
self._origin_channel = "cli"
self._origin_chat_id = "direct"
def set_context(self, channel: str, chat_id: str) -> None:
"""Set the origin context for subagent announcements."""
self._origin_channel = channel
self._origin_chat_id = chat_id
@property
def name(self) -> str:
return "spawn"
@property
def description(self) -> str:
return (
"Spawn a subagent to handle a task in the background. "
"Use this for complex or time-consuming tasks that can run independently. "
"The subagent will complete the task and report back when done."
)
@property
def parameters(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"task": {
"type": "string",
"description": "The task for the subagent to complete",
},
"label": {
"type": "string",
"description": "Optional short label for the task (for display)",
},
},
"required": ["task"],
}
async def execute(self, task: str, label: str | None = None, **kwargs: Any) -> str:
"""Spawn a subagent to execute the given task."""
return await self._manager.spawn(
task=task,
label=label,
origin_channel=self._origin_channel,
origin_chat_id=self._origin_chat_id,
)
================================================
FILE: nanobot/nanobot/agent/tools/web.py
================================================
"""Web tools: web_search and web_fetch."""
import html
import json
import os
import re
from typing import Any
from urllib.parse import urlparse
import httpx
from nanobot.agent.tools.base import Tool
# Shared constants
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7_2) AppleWebKit/537.36"
MAX_REDIRECTS = 5 # Limit redirects to prevent DoS attacks
def _strip_tags(text: str) -> str:
"""Remove HTML tags and decode entities."""
text = re.sub(r"