Repository: kroll-software/babyagi4all
Branch: main
Commit: 899cfb0d87c7
Files: 6
Total size: 14.9 KB
Directory structure:
gitextract_4kue40tb/
├── .gitattributes
├── .gitignore
├── LICENSE
├── README.md
├── babyagi.py
└── requirements.txt
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitattributes
================================================
*.py text eol=lf
================================================
FILE: .gitignore
================================================
__pycache__/
*.py[cod]
*$py.class
.env
env/
.venv
*venv/
.vscode/
.idea/
models
llama/
# for node
chroma/
node_modules/
.DS_Store
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2023 by Kroll Software-Entwicklung
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# BabyAGI4All
A small autonomous AI agent based on [BabyAGI](https://github.com/yoheinakajima/babyagi) by Yohei Nakajima.
</br>
Runs on CPU with the [GPT4All](https://github.com/nomic-ai/gpt4all) model by Nomic AI.
</br>
100% open source, 100% local, no API-keys needed.
</br>
# Installation:
1. Clone this repository
2. Install the requirements: *pip install -r requirements.txt*
3. Download a model file (see below)
4. Copy the file *.env.example* to *.env*
4. Edit the model-path and other preferences in the file *.env*
## Model Downloads
The following model files have been tested successfully:
* *gpt4all-lora-quantized-ggml.bin*
* *ggml-wizardLM-7B.q4_2.bin*
* *ggml-vicuna-7b-1.1-q4_2.bin*
Some of these model files can be downloaded from [here](https://github.com/nomic-ai/gpt4all-chat#manual-download-of-models).
</br>
</br>
Then run *python babyagi.py*
</br>
Have fun!
</br>
================================================
FILE: babyagi.py
================================================
import os
import time
import logging
from collections import deque
from typing import Dict, List
import importlib
import chromadb
from dotenv import load_dotenv
from chromadb.api.types import Documents, EmbeddingFunction, Embeddings
from chromadb.utils.embedding_functions import InstructorEmbeddingFunction
from llama_cpp import Llama
# Load default environment variables (.env)
load_dotenv()
# Engine configuration
LLM_MODEL = "GPT4All"
# Table config
RESULTS_STORE_NAME = os.getenv("RESULTS_STORE_NAME", os.getenv("TABLE_NAME", ""))
assert RESULTS_STORE_NAME, "\033[91m\033[1m" + "RESULTS_STORE_NAME environment variable is missing from .env" + "\033[0m\033[0m"
# Run configuration
INSTANCE_NAME = os.getenv("INSTANCE_NAME", os.getenv("BABY_NAME", "BabyAGI"))
COOPERATIVE_MODE = "none"
JOIN_EXISTING_OBJECTIVE = False
# Goal configuation
OBJECTIVE = os.getenv("OBJECTIVE", "")
INITIAL_TASK = os.getenv("INITIAL_TASK", os.getenv("FIRST_TASK", ""))
# Model configuration
TEMPERATURE = float(os.getenv("TEMPERATURE", 0.2))
VERBOSE = (os.getenv("VERBOSE", "false").lower() == "true")
# Extensions support begin
def can_import(module_name):
try:
importlib.import_module(module_name)
return True
except ImportError:
return False
print("\033[95m\033[1m"+"\n*****CONFIGURATION*****\n"+"\033[0m\033[0m")
print(f"Name : {INSTANCE_NAME}")
print(f"Mode : {'alone' if COOPERATIVE_MODE in ['n', 'none'] else 'local' if COOPERATIVE_MODE in ['l', 'local'] else 'distributed' if COOPERATIVE_MODE in ['d', 'distributed'] else 'undefined'}")
print(f"LLM : {LLM_MODEL}")
# Check if we know what we are doing
assert OBJECTIVE, "\033[91m\033[1m" + "OBJECTIVE environment variable is missing from .env" + "\033[0m\033[0m"
assert INITIAL_TASK, "\033[91m\033[1m" + "INITIAL_TASK environment variable is missing from .env" + "\033[0m\033[0m"
MODEL_PATH = os.getenv("MODEL_PATH", "models/gpt4all-lora-quantized-ggml.bin")
print(f"GPT4All : {MODEL_PATH}" + "\n")
assert os.path.exists(MODEL_PATH), "\033[91m\033[1m" + f"Model can't be found." + "\033[0m\033[0m"
#CTX_MAX = 2048
#CTX_MAX = 8192
CTX_MAX = 16384
#THREADS_NUM = 16
THREADS_NUM = 4
llm = Llama(
model_path=MODEL_PATH,
n_ctx=CTX_MAX, n_threads=THREADS_NUM,
use_mlock=True,
verbose=False,
)
print("\033[94m\033[1m" + "\n*****OBJECTIVE*****\n" + "\033[0m\033[0m")
print(f"{OBJECTIVE}")
if not JOIN_EXISTING_OBJECTIVE: print("\033[93m\033[1m" + "\nInitial task:" + "\033[0m\033[0m" + f" {INITIAL_TASK}")
else: print("\033[93m\033[1m" + f"\nJoining to help the objective" + "\033[0m\033[0m")
# Results storage using local ChromaDB
class DefaultResultsStorage:
def __init__(self):
logging.getLogger('chromadb').setLevel(logging.ERROR)
# Create Chroma collection
chroma_persist_dir = "chroma"
chroma_client = chromadb.Client(
settings=chromadb.config.Settings(
chroma_db_impl="duckdb+parquet",
persist_directory=chroma_persist_dir,
)
)
metric = "cosine"
embedding_function = InstructorEmbeddingFunction()
self.collection = chroma_client.get_or_create_collection(
name=RESULTS_STORE_NAME,
metadata={"hnsw:space": metric},
embedding_function=embedding_function,
)
def add(self, task: Dict, result: Dict, result_id: str, vector: List):
embeddings = self.collection._embedding_function([vector])
if (len(self.collection.get(ids=[result_id], include=[])["ids"]) > 0): # Check if the result already exists
self.collection.update(
ids=result_id,
embeddings=embeddings,
documents=vector,
metadatas={"task": task["task_name"], "result": result},
)
else:
self.collection.add(
ids=result_id,
embeddings=embeddings,
documents=vector,
metadatas={"task": task["task_name"], "result": result},
)
def query(self, query: str, top_results_num: int) -> List[dict]:
count: int = self.collection.count()
if count == 0:
return []
results = self.collection.query(
query_texts=query,
n_results=min(top_results_num, count),
include=["metadatas"]
)
tasks = []
count = len(results["ids"][0])
for i in range(count):
resultidstr = results["ids"][0][i]
id = int(resultidstr[7:])
item = results["metadatas"][0][i]
task = {'task_id': id, 'task_name': item["task"]}
tasks.append(task)
return tasks
# Initialize results storage
results_storage = DefaultResultsStorage()
# Task storage supporting only a single instance of BabyAGI
class SingleTaskListStorage:
def __init__(self):
self.tasks = deque([])
self.task_id_counter = 0
def append(self, task: Dict):
self.tasks.append(task)
def replace(self, tasks: List[Dict]):
self.tasks = deque(tasks)
def popleft(self):
return self.tasks.popleft()
def is_empty(self):
return False if self.tasks else True
def next_task_id(self):
self.task_id_counter += 1
return self.task_id_counter
def get_task_names(self):
return [t["task_name"] for t in self.tasks]
# Initialize tasks storage
tasks_storage = SingleTaskListStorage()
def gpt_call(prompt: str, temperature: float = TEMPERATURE, max_tokens: int = 256):
result = llm(prompt[:CTX_MAX], echo=True, temperature=temperature, max_tokens=max_tokens)
return result['choices'][0]['text'][len(prompt):].strip()
def strip_numbered_list(nl: List[str]) -> List[str]:
result_list = []
filter_chars = ['#', '(', ')', '[', ']', '.', ':', ' ']
for line in nl:
line = line.strip()
if len(line) > 0:
parts = line.split(" ", 1)
if len(parts) == 2:
left_part = ''.join(x for x in parts[0] if not x in filter_chars)
if left_part.isnumeric():
result_list.append(parts[1].strip())
else:
result_list.append(line)
else:
result_list.append(line)
# filter result_list
result_list = [line for line in result_list if len(line) > 3]
# remove duplicates
result_list = list(set(result_list))
return result_list
def fix_prompt(prompt: str) -> str:
lines = prompt.split("\n") if "\n" in prompt else [prompt]
return "\n".join([line.strip() for line in lines])
def task_creation_agent(
objective: str, result: Dict, task_description: str, task_list: List[str]
):
prompt = f"""
Your objective: {objective}\n
Take into account these previously completed tasks but don't repeat them: {task_list}.\n
The last completed task has the result: {result["data"]}.\n
Develop a task list based on the result.\n
Response:"""
prompt = fix_prompt(prompt)
response = gpt_call(prompt)
pos = response.find("1")
if (pos > 0):
response = response[pos - 1:]
if response == '':
print("\n*** Empty Response from task_creation_agent***")
new_tasks_list = result["data"].split("\n") if len(result) > 0 else [response]
else:
new_tasks = response.split("\n") if "\n" in response else [response]
new_tasks_list = strip_numbered_list(new_tasks)
return [{"task_name": task_name} for task_name in (t for t in new_tasks_list if not t == '')]
def prioritization_agent():
task_names = tasks_storage.get_task_names()
next_task_id = tasks_storage.next_task_id()
prompt = f"""
Please prioritize, summarize and consolidate the following tasks: {task_names}.\n
Consider the ultimate objective: {OBJECTIVE}.\n
Return the result as a numbered list.
"""
prompt = fix_prompt(prompt)
response = gpt_call(prompt)
pos = response.find("1")
if (pos > 0):
response = response[pos - 1:]
new_tasks = response.split("\n") if "\n" in response else [response]
new_tasks = strip_numbered_list(new_tasks)
new_tasks_list = []
i = 0
for task_string in new_tasks:
new_tasks_list.append({"task_id": i + next_task_id, "task_name": task_string})
i += 1
if len(new_tasks_list) > 0:
tasks_storage.replace(new_tasks_list)
# Execute a task based on the objective and five previous tasks
def execution_agent(objective: str, task: str) -> str:
"""
Executes a task based on the given objective and previous context.
Args:
objective (str): The objective or goal for the AI to perform the task.
task (str): The task to be executed by the AI.
Returns:
str: The response generated by the AI for the given task.
"""
context = context_agent(query=objective, top_results_num=5)
context_list = [t['task_name'] for t in context if t['task_name'] != INITIAL_TASK]
#context_list = [t['task_name'] for t in context]
# remove duplicates
context_list = list(set(context_list))
if VERBOSE and len(context_list) > 0:
print("\n*******RELEVANT CONTEXT******\n")
print(context_list)
if task == INITIAL_TASK:
prompt = f"""
You are an AI who performs one task based on the following objective: {objective}.\n
Your task: {task}\nResponse:"""
else:
prompt = f"""
Your objective: {objective}.\n
Take into account these previously completed tasks but don't repeat them: {context_list}.\n
Your task: {task}\n
Response:"""
#Give an advice how to achieve your task!\n
prompt = fix_prompt(prompt)
result = gpt_call(prompt)
pos = result.find("1")
if (pos > 0):
result = result[pos - 1:]
return result
# Get the top n completed tasks for the objective
def context_agent(query: str, top_results_num: int):
"""
Retrieves context for a given query from an index of tasks.
Args:
query (str): The query or objective for retrieving context.
top_results_num (int): The number of top results to retrieve.
Returns:
list: A list of tasks as context for the given query, sorted by relevance.
"""
results = results_storage.query(query=query, top_results_num=top_results_num)
#print("\n***** RESULTS *****")
#print(results)
return results
# Add the initial task if starting new objective
if not JOIN_EXISTING_OBJECTIVE:
initial_task = {
"task_id": tasks_storage.next_task_id(),
"task_name": INITIAL_TASK
}
tasks_storage.append(initial_task)
def main ():
while True:
# As long as there are tasks in the storage...
if not tasks_storage.is_empty():
# Print the task list
print("\033[95m\033[1m" + "\n*****TASK LIST*****\n" + "\033[0m\033[0m")
for t in tasks_storage.get_task_names():
print(" • "+t)
# Step 1: Pull the first incomplete task
task = tasks_storage.popleft()
print("\033[92m\033[1m" + "\n*****NEXT TASK*****\n" + "\033[0m\033[0m")
print(task['task_name'])
# Send to execution function to complete the task based on the context
result = execution_agent(OBJECTIVE, task["task_name"])
print("\033[93m\033[1m" + "\n*****TASK RESULT*****\n" + "\033[0m\033[0m")
print(result)
# Step 2: Enrich result and store in the results storage
# This is where you should enrich the result if needed
enriched_result = {
"data": result
}
# extract the actual result from the dictionary
# since we don't do enrichment currently
vector = enriched_result["data"]
result_id = f"result_{task['task_id']}"
results_storage.add(task, result, result_id, vector)
# Step 3: Create new tasks and reprioritize task list
# only the main instance in cooperative mode does that
new_tasks = task_creation_agent(
OBJECTIVE,
enriched_result,
task["task_name"],
tasks_storage.get_task_names(),
)
for new_task in new_tasks:
if not new_task['task_name'] == '':
new_task.update({"task_id": tasks_storage.next_task_id()})
tasks_storage.append(new_task)
if not JOIN_EXISTING_OBJECTIVE: prioritization_agent()
# Sleep a bit before checking the task list again
time.sleep(5)
else:
print ("Ready, no more tasks.")
if __name__ == "__main__":
main()
================================================
FILE: requirements.txt
================================================
argparse==1.4.0
chromadb==0.3.21
pre-commit>=3.2.0
python-dotenv==1.0.0
InstructorEmbedding>=1.0.0
llama-cpp-python==0.1.43
gitextract_4kue40tb/ ├── .gitattributes ├── .gitignore ├── LICENSE ├── README.md ├── babyagi.py └── requirements.txt
SYMBOL INDEX (21 symbols across 1 files)
FILE: babyagi.py
function can_import (line 39) | def can_import(module_name):
class DefaultResultsStorage (line 80) | class DefaultResultsStorage:
method __init__ (line 81) | def __init__(self):
method add (line 100) | def add(self, task: Dict, result: Dict, result_id: str, vector: List):
method query (line 118) | def query(self, query: str, top_results_num: int) -> List[dict]:
class SingleTaskListStorage (line 142) | class SingleTaskListStorage:
method __init__ (line 143) | def __init__(self):
method append (line 147) | def append(self, task: Dict):
method replace (line 150) | def replace(self, tasks: List[Dict]):
method popleft (line 153) | def popleft(self):
method is_empty (line 156) | def is_empty(self):
method next_task_id (line 159) | def next_task_id(self):
method get_task_names (line 163) | def get_task_names(self):
function gpt_call (line 170) | def gpt_call(prompt: str, temperature: float = TEMPERATURE, max_tokens: ...
function strip_numbered_list (line 174) | def strip_numbered_list(nl: List[str]) -> List[str]:
function fix_prompt (line 198) | def fix_prompt(prompt: str) -> str:
function task_creation_agent (line 202) | def task_creation_agent(
function prioritization_agent (line 229) | def prioritization_agent():
function execution_agent (line 259) | def execution_agent(objective: str, task: str) -> str:
function context_agent (line 307) | def context_agent(query: str, top_results_num: int):
function main (line 332) | def main ():
Condensed preview — 6 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (16K chars).
[
{
"path": ".gitattributes",
"chars": 16,
"preview": "*.py text eol=lf"
},
{
"path": ".gitignore",
"chars": 133,
"preview": "__pycache__/\n*.py[cod]\n*$py.class\n\n.env\nenv/\n.venv\n*venv/\n\n.vscode/\n.idea/\n\nmodels\nllama/\n\n# for node\nchroma/\nnode_modul"
},
{
"path": "LICENSE",
"chars": 1086,
"preview": "MIT License\n\nCopyright (c) 2023 by Kroll Software-Entwicklung\n\nPermission is hereby granted, free of charge, to any pers"
},
{
"path": "README.md",
"chars": 897,
"preview": "# BabyAGI4All\n\nA small autonomous AI agent based on [BabyAGI](https://github.com/yoheinakajima/babyagi) by Yohei Nakajim"
},
{
"path": "babyagi.py",
"chars": 12974,
"preview": "import os\nimport time\nimport logging\nfrom collections import deque\nfrom typing import Dict, List\nimport importlib\nimport"
},
{
"path": "requirements.txt",
"chars": 123,
"preview": "argparse==1.4.0\nchromadb==0.3.21\npre-commit>=3.2.0\npython-dotenv==1.0.0\nInstructorEmbedding>=1.0.0\nllama-cpp-python==0.1"
}
]
About this extraction
This page contains the full source code of the kroll-software/babyagi4all GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 6 files (14.9 KB), approximately 3.9k tokens, and a symbol index with 21 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.