Showing preview only (777K chars total). Download the full file or copy to clipboard to get everything.
Repository: sigoden/aichat
Branch: main
Commit: 82976d349ad9
Files: 79
Total size: 747.0 KB
Directory structure:
gitextract_9_1bw_kt/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ └── feature_request.md
│ └── workflows/
│ ├── ci.yaml
│ └── release.yaml
├── .gitignore
├── Argcfile.sh
├── Cargo.toml
├── LICENSE-APACHE
├── LICENSE-MIT
├── README.md
├── assets/
│ ├── arena.html
│ ├── playground.html
│ └── roles/
│ ├── %code%.md
│ ├── %create-prompt%.md
│ ├── %create-title%.md
│ ├── %explain-shell%.md
│ ├── %functions%.md
│ └── %shell%.md
├── config.agent.example.yaml
├── config.example.yaml
├── models.yaml
├── scripts/
│ ├── completions/
│ │ ├── aichat.bash
│ │ ├── aichat.fish
│ │ ├── aichat.nu
│ │ ├── aichat.ps1
│ │ └── aichat.zsh
│ └── shell-integration/
│ ├── integration.bash
│ ├── integration.fish
│ ├── integration.nu
│ ├── integration.ps1
│ └── integration.zsh
└── src/
├── cli.rs
├── client/
│ ├── access_token.rs
│ ├── azure_openai.rs
│ ├── bedrock.rs
│ ├── claude.rs
│ ├── cohere.rs
│ ├── common.rs
│ ├── gemini.rs
│ ├── macros.rs
│ ├── message.rs
│ ├── mod.rs
│ ├── model.rs
│ ├── openai.rs
│ ├── openai_compatible.rs
│ ├── stream.rs
│ └── vertexai.rs
├── config/
│ ├── agent.rs
│ ├── input.rs
│ ├── mod.rs
│ ├── role.rs
│ └── session.rs
├── function.rs
├── main.rs
├── rag/
│ ├── mod.rs
│ ├── serde_vectors.rs
│ └── splitter/
│ ├── language.rs
│ └── mod.rs
├── render/
│ ├── markdown.rs
│ ├── mod.rs
│ └── stream.rs
├── repl/
│ ├── completer.rs
│ ├── highlighter.rs
│ ├── mod.rs
│ └── prompt.rs
├── serve.rs
└── utils/
├── abort_signal.rs
├── clipboard.rs
├── command.rs
├── crypto.rs
├── html_to_md.rs
├── input.rs
├── loader.rs
├── mod.rs
├── path.rs
├── render_prompt.rs
├── request.rs
├── spinner.rs
└── variables.rs
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/ISSUE_TEMPLATE/bug_report.md
================================================
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
<!-- Your issue may already be reported! Please search for it before creating one. -->
**Describe the bug**
<!-- A clear and concise description of what the bug is. -->
**To Reproduce**
<!-- Steps to reproduce the behavior, including any relevant code snippets. -->
**Expected behavior**
<!-- A clear and concise description of what you expected to happen. -->
**Logs**
<!-- If applicable, Attach relevant log outputs that can help diagnose the issue, see https://github.com/sigoden/aichat/wiki/FAQ#how-to-log-or-debug for logging. -->
**Screenshots**
<!-- If applicable, add screenshots to help explain your problem. -->
**Configuration**
<!-- Please run `aichat --info` and paste the output -->
**Environment (please complete the following information):**
- os version: [e.g. Ubuntu 20.04]
- aichat version: [e.g. 0.9.0]
- terminal version: [e.g. GNOME Terminal 3.44.0]
**Additional context**
<!-- Add any other context about the problem here. -->
================================================
FILE: .github/ISSUE_TEMPLATE/feature_request.md
================================================
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''
---
<!-- Your issue may already be reported! Please search for it before creating one. -->
**Is your feature request related to a problem? Please describe.**
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
**Describe the solution you'd like**
<!-- A clear and concise description of what you want to happen. -->
**Describe alternatives you've considered**
<!-- A clear and concise description of any alternative solutions or features you've considered. -->
**Additional context**
<!-- Add any other context or screenshots about the feature request here. -->
================================================
FILE: .github/workflows/ci.yaml
================================================
name: CI
on:
pull_request:
branches:
- '*'
push:
branches:
- main
defaults:
run:
shell: bash
jobs:
all:
name: All
strategy:
matrix:
os:
- ubuntu-latest
- macos-latest
- windows-latest
runs-on: ${{matrix.os}}
env:
RUSTFLAGS: --deny warnings
steps:
- uses: actions/checkout@v4
- name: Install Rust Toolchain Components
uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Test
run: cargo test --all
- name: Clippy
run: cargo clippy --all --all-targets -- -D warnings
- name: Format
run: cargo fmt --all --check
================================================
FILE: .github/workflows/release.yaml
================================================
name: Release
on:
push:
tags:
- v[0-9]+.[0-9]+.[0-9]+*
jobs:
release:
name: Publish to GitHub Release
permissions:
contents: write
outputs:
rc: ${{ steps.check-tag.outputs.rc }}
strategy:
matrix:
include:
- target: aarch64-unknown-linux-musl
os: ubuntu-latest
use-cross: true
cargo-flags: ""
- target: aarch64-apple-darwin
os: macos-latest
use-cross: true
cargo-flags: ""
- target: aarch64-pc-windows-msvc
os: windows-latest
use-cross: true
cargo-flags: ""
- target: x86_64-apple-darwin
os: macos-latest
cargo-flags: ""
- target: x86_64-pc-windows-msvc
os: windows-latest
cargo-flags: ""
- target: x86_64-unknown-linux-musl
os: ubuntu-latest
use-cross: true
cargo-flags: ""
- target: i686-unknown-linux-musl
os: ubuntu-latest
use-cross: true
cargo-flags: ""
- target: i686-pc-windows-msvc
os: windows-latest
use-cross: true
cargo-flags: ""
- target: armv7-unknown-linux-musleabihf
os: ubuntu-latest
use-cross: true
cargo-flags: ""
- target: arm-unknown-linux-musleabihf
os: ubuntu-latest
use-cross: true
cargo-flags: ""
runs-on: ${{matrix.os}}
env:
BUILD_CMD: cargo
steps:
- uses: actions/checkout@v4
- name: Check Tag
id: check-tag
shell: bash
run: |
ver=${GITHUB_REF##*/}
echo "version=$ver" >> $GITHUB_OUTPUT
if [[ "$ver" =~ [0-9]+.[0-9]+.[0-9]+$ ]]; then
echo "rc=false" >> $GITHUB_OUTPUT
else
echo "rc=true" >> $GITHUB_OUTPUT
fi
- name: Install Rust Toolchain Components
uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}
- name: Install cross
if: matrix.use-cross
uses: taiki-e/install-action@v2
with:
tool: cross
- name: Overwrite build command env variable
if: matrix.use-cross
shell: bash
run: echo "BUILD_CMD=cross" >> $GITHUB_ENV
- name: Show Version Information (Rust, cargo, GCC)
shell: bash
run: |
gcc --version || true
rustup -V
rustup toolchain list
rustup default
cargo -V
rustc -V
- name: Build
shell: bash
run: $BUILD_CMD build --locked --release --target=${{ matrix.target }} ${{ matrix.cargo-flags }}
- name: Build Archive
shell: bash
id: package
env:
target: ${{ matrix.target }}
version: ${{ steps.check-tag.outputs.version }}
run: |
set -euxo pipefail
bin=${GITHUB_REPOSITORY##*/}
dist_dir=`pwd`/dist
name=$bin-$version-$target
executable=target/$target/release/$bin
if [[ "$RUNNER_OS" == "Windows" ]]; then
executable=$executable.exe
fi
mkdir $dist_dir
cp $executable $dist_dir
cd $dist_dir
if [[ "$RUNNER_OS" == "Windows" ]]; then
archive=$dist_dir/$name.zip
7z a $archive *
echo "archive=dist/$name.zip" >> $GITHUB_OUTPUT
else
archive=$dist_dir/$name.tar.gz
tar -czf $archive *
echo "archive=dist/$name.tar.gz" >> $GITHUB_OUTPUT
fi
- name: Publish Archive
uses: softprops/action-gh-release@v2
if: ${{ startsWith(github.ref, 'refs/tags/') }}
with:
draft: false
files: ${{ steps.package.outputs.archive }}
prerelease: ${{ steps.check-tag.outputs.rc == 'true' }}
publish-crate:
name: Publish to crates.io
if: ${{ needs.release.outputs.rc == 'false' }}
runs-on: ubuntu-latest
needs: release
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: Publish
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CRATES_IO_API_TOKEN }}
run: cargo publish
================================================
FILE: .gitignore
================================================
/target
/tmp
/.env
*.log
================================================
FILE: Argcfile.sh
================================================
#!/usr/bin/env bash
set -e
# @meta dotenv
# @env DRY_RUN Dry run mode
# @cmd Test configuration initialization
# @env AICHAT_CONFIG_DIR=tmp/test-init-config
# @arg args~
test-init-config() {
unset OPENAI_API_KEY
mkdir -p "$AICHAT_CONFIG_DIR"
config_file="$AICHAT_CONFIG_DIR/config.yaml"
if [[ -f "$config_file" ]]; then
rm -f "$config_file"
fi
cargo run -- "$@"
}
# @cmd Test running without configuration file
# @env AICHAT_PROVIDER!
# @env AICHAT_CONFIG_DIR=tmp/test-provider-env
# @arg args~
test-no-config() {
mkdir -p "$AICHAT_CONFIG_DIR"
rm -rf "$AICHAT_CONFIG_DIR/config.yaml"
cargo run -- "$@"
}
# @cmd Test function calling
# @option -m --model[?`_choice_model`]
# @option -p --preset[=weather|multi-weathers]
# @flag -S --no-stream
# @arg text~
test-function-calling() {
args=(--role %functions%)
if [[ -n "$argc_model" ]]; then
args+=("--model" "$argc_model")
fi
if [[ -n "$argc_no_stream" ]]; then
args+=("-S")
fi
if [[ -z "$argc_text" ]]; then
case "$argc_preset" in
multi-weathers)
text="what is the weather in London and Pairs?"
;;
weather|*)
text="what is the weather in London?"
;;
esac
else
text="${argc_text[*]}"
fi
cargo run -- "${args[@]}" "$text"
}
# @cmd Test clients
# @arg clients+[`_choice_client`]
test-clients() {
for c in "${argc_clients[@]}"; do
echo "### $c stream"
aichat -m "$c" 1 + 2 = ?
echo "### $c non-stream"
aichat -m "$c" -S 1 + 2 = ?
done
}
# @cmd Test proxy server
# @option -m --model[?`_choice_model`]
# @flag -S --no-stream
# @arg text~
test-server() {
args=()
if [[ -n "$argc_no_stream" ]]; then
args+=("-S")
fi
argc chat-llm "${args[@]}" \
--api-base http://localhost:8000/v1 \
--model "${argc_model:-default}" \
"$@"
}
# @cmd Chat with any LLM api
# @flag -S --no-stream
# @arg provider_model![?`_choice_provider_model`]
# @arg text~
chat() {
if [[ "$argc_provider_model" == *':'* ]]; then
model="${argc_provider_model##*:}"
argc_provider="${argc_provider_model%:*}"
else
argc_provider="${argc_provider_model}"
fi
for provider_config in "${OPENAI_COMPATIBLE_PROVIDERS[@]}"; do
if [[ "$argc_provider" == "${provider_config%%,*}" ]]; then
_retrieve_api_base
break
fi
done
if [[ -n "$api_base" ]]; then
env_prefix="$(echo "$argc_provider" | tr '[:lower:]' '[:upper:]')"
api_key_env="${env_prefix}_API_KEY"
api_key="${!api_key_env}"
if [[ -z "$model" ]]; then
model="$(echo "$provider_config" | cut -d, -f2)"
fi
if [[ -z "$model" ]]; then
model_env="${env_prefix}_MODEL"
model="${!model_env}"
fi
argc chat-openai-compatible \
--api-base "$api_base" \
--api-key "$api_key" \
--model "$model" \
"${argc_text[@]}"
else
argc chat-$argc_provider "${argc_text[@]}"
fi
}
# @cmd List models by openai-compatible api
# @flag --name-only Print model name only
# @arg provider![`_choice_provider`]
models() {
for provider_config in "${OPENAI_COMPATIBLE_PROVIDERS[@]}"; do
if [[ "$argc_provider" == "${provider_config%%,*}" ]]; then
_retrieve_api_base
break
fi
done
if [[ -n "$api_base" ]]; then
env_prefix="$(echo "$argc_provider" | tr '[:lower:]' '[:upper:]')"
api_key_env="${env_prefix}_API_KEY"
api_key="${!api_key_env}"
jq_args=()
if [[ -n "$argc_name_only" ]]; then
case "$argc_provider" in
cloudflare)
jq_args+=(-r '.result[].name')
;;
github)
jq_args+=(-r '.[].name')
;;
*)
jq_args+=(-r '.data[].id')
;;
esac
fi
_openai_compatible_models | jq "${jq_args[@]}"
else
if ! cat "$0" | grep -q "^models-$argc_provider"; then
_die "error: provider '$argc_provider' does not have a models api"
fi
cli_args=()
if [[ -n "$argc_name_only" ]]; then
cli_args+=(--name-only)
fi
argc models-$argc_provider "${cli_args[@]}"
fi
}
# @cmd Chat with openai-compatible api
# @option --api-base! $$
# @option --api-key! $$
# @option -m --model! $$
# @flag -S --no-stream
# @arg text~
chat-openai-compatible() {
_wrapper curl -i "$argc_api_base/chat/completions" \
-X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $argc_api_key" \
-d "$(_build_body openai "$@")"
}
# @cmd List models by openai-compatible api
# @option --api-base! $$
# @option --api-key! $$
# @flag --name-only Print model name only
models-openai-compatible() {
jq_args=()
if [[ -n "$argc_name_only" ]]; then
jq_args+=(-r '.data[].id')
fi
_openai_compatible_models | jq "${jq_args[@]}"
}
# @cmd Chat with azure-openai api
# @option --api-url! $$
# @option --api-key! $$
# @option -m --model! $$
# @flag -S --no-stream
# @arg text~
chat-azure-openai() {
_wrapper curl -i "$argc_api_url" \
-X POST \
-H "Content-Type: application/json" \
-H "api-key: $argc_api_key" \
-d "$(_build_body openai "$@")"
}
# @cmd Chat with gemini api
# @env GEMINI_API_KEY!
# @option -m --model=gemini-1.5-pro-latest $GEMINI_MODEL
# @flag -S --no-stream
# @arg text~
chat-gemini() {
method="streamGenerateContent"
if [[ -n "$argc_no_stream" ]]; then
method="generateContent"
fi
_wrapper curl -i "https://generativelanguage.googleapis.com/v1beta/models/${argc_model}:${method}?key=${GEMINI_API_KEY}" \
-i -X POST \
-H 'Content-Type: application/json' \
-d "$(_build_body gemini "$@")"
}
# @cmd List gemini models
# @env GEMINI_API_KEY!
# @flag --name-only Print model name only
models-gemini() {
jq_args=()
if [[ -n "$argc_name_only" ]]; then
jq_args+=(-r '.models[].name')
fi
_wrapper curl -fsSL "https://generativelanguage.googleapis.com/v1beta/models?key=${GEMINI_API_KEY}" \
-H 'Content-Type: application/json' \
| jq "${jq_args[@]}"
}
# @cmd Chat with claude api
# @env CLAUDE_API_KEY!
# @option -m --model=claude-3-haiku-20240307 $CLAUDE_MODEL
# @flag -S --no-stream
# @arg text~
chat-claude() {
_wrapper curl -i https://api.anthropic.com/v1/messages \
-X POST \
-H 'content-type: application/json' \
-H 'anthropic-version: 2023-06-01' \
-H 'anthropic-beta: tools-2024-05-16' \
-H "x-api-key: $CLAUDE_API_KEY" \
-d "$(_build_body claude "$@")"
}
# @cmd List claude models
# @env CLAUDE_API_KEY!
# @flag --name-only Print model name only
models-claude() {
jq_args=()
if [[ -n "$argc_name_only" ]]; then
jq_args+=(-r '.data[].id')
fi
_wrapper curl -fsSL "https://api.anthropic.com/v1/models" \
-H 'Content-Type: application/json' \
-H 'anthropic-version: 2023-06-01' \
-H "x-api-key: $CLAUDE_API_KEY" \
| jq "${jq_args[@]}"
}
# @cmd Chat with cohere api
# @env COHERE_API_KEY!
# @option -m --model=command-r-08-2024 $COHERE_MODEL
# @flag -S --no-stream
# @arg text~
chat-cohere() {
_wrapper curl -i https://api.cohere.ai/v2/chat \
-X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $COHERE_API_KEY" \
-d "$(_build_body cohere "$@")"
}
# @cmd List cohere models
# @env COHERE_API_KEY!
# @flag --name-only Print model name only
models-cohere() {
jq_args=()
if [[ -n "$argc_name_only" ]]; then
jq_args+=(-r '.models[].name')
fi
_wrapper curl -fsSL https://api.cohere.ai/v1/models \
-H "Authorization: Bearer $COHERE_API_KEY" \
| jq "${jq_args[@]}"
}
# @cmd Chat with vertexai api
# @env require-tools gcloud
# @env VERTEXAI_PROJECT_ID!
# @env VERTEXAI_LOCATION!
# @option -m --model=gemini-1.5-flash-002 $VERTEXAI_GEMINI_MODEL
# @flag -S --no-stream
# @arg text~
chat-vertexai() {
api_key="$(gcloud auth print-access-token)"
func="streamGenerateContent"
if [[ -n "$argc_no_stream" ]]; then
func="generateContent"
fi
url=https://$VERTEXAI_LOCATION-aiplatform.googleapis.com/v1/projects/$VERTEXAI_PROJECT_ID/locations/$VERTEXAI_LOCATION/publishers/google/models/$argc_model:$func
_wrapper curl -i $url \
-X POST \
-H "Authorization: Bearer $api_key" \
-H 'Content-Type: application/json' \
-d "$(_build_body vertexai "$@")"
}
_argc_before() {
OPENAI_COMPATIBLE_PROVIDERS=( \
openai,gpt-4o-mini,https://api.openai.com/v1 \
ai21,jamba-1.5-mini,https://api.ai21.com/studio/v1 \
cloudflare,@cf/meta/llama-3.1-8b-instruct,https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/ai/v1 \
deepinfra,meta-llama/Meta-Llama-3.1-8B-Instruct,https://api.deepinfra.com/v1/openai \
deepseek,deepseek-chat,https://api.deepseek.com \
ernie,ernie-4.0-turbo-8k-latest,https://qianfan.baidubce.com/v2 \
github,gpt-4o-mini,https://models.inference.ai.azure.com \
groq,llama-3.1-8b-instant,https://api.groq.com/openai/v1 \
hunyuan,hunyuan-large,https://api.hunyuan.cloud.tencent.com/v1 \
minimax,MiniMax-Text-01,https://api.minimax.chat/v1 \
mistral,mistral-small-latest,https://api.mistral.ai/v1 \
moonshot,moonshot-v1-8k,https://api.moonshot.cn/v1 \
openrouter,openai/gpt-4o-mini,https://openrouter.ai/api/v1 \
perplexity,llama-3.1-8b-instruct,https://api.perplexity.ai \
qianwen,qwen-turbo-latest,https://dashscope.aliyuncs.com/compatible-mode/v1 \
xai,grok-beta,https://api.x.ai/v1 \
zhipuai,glm-4-0520,https://open.bigmodel.cn/api/paas/v4 \
)
stream="true"
if [[ -n "$argc_no_stream" ]]; then
stream="false"
fi
}
_openai_compatible_models() {
api_base="${api_base:-"$argc_api_base"}"
api_key="${api_key:-"$argc_api_key"}"
url="${api_base}/models"
if [[ "$argc_provider" == "cloudflare" ]]; then
url="https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT_ID}/ai/models/search"
fi
_wrapper curl -fsSL "$url" \
-H "Authorization: Bearer $api_key" \
}
_retrieve_api_base() {
api_base="${provider_config##*,}"
if [[ -z "$api_base" ]]; then
key="$(echo $argc_provider | tr '[:lower:]' '[:upper:]')_API_BASE"
api_base="${!key}"
if [[ -z "$api_base" ]]; then
_die "error: miss api_base for $argc_provider; please set $key"
fi
fi
}
_choice_model() {
aichat --list-models
}
_choice_provider_model() {
_choice_provider
_choice_model
}
_choice_provider() {
_choice_client
_choice_openai_compatible_provider
}
_choice_client() {
printf "%s\n" gemini claude cohere azure-openai vertexai bedrock
}
_choice_openai_compatible_provider() {
for provider_config in "${OPENAI_COMPATIBLE_PROVIDERS[@]}"; do
echo "${provider_config%%,*}"
done
}
_build_body() {
kind="$1"
if [[ "$#" -eq 1 ]]; then
file="${BODY_FILE:-"tmp/body/$1.json"}"
if [[ -f "$file" ]]; then
cat "$file" | \
sed -E \
-e 's%"model": ".*"%"model": "'"$argc_model"'"%' \
-e 's%"stream": (true|false)%"stream": '$stream'%' \
fi
else
shift
case "$kind" in
openai|cohere)
echo '{
"model": "'$argc_model'",
"messages": [
{
"role": "user",
"content": "'"$*"'"
}
],
"stream": '$stream'
}'
;;
claude)
echo '{
"model": "'$argc_model'",
"messages": [
{
"role": "user",
"content": "'"$*"'"
}
],
"max_tokens": 4096,
"stream": '$stream'
}'
;;
gemini|vertexai)
echo '{
"contents": [{
"role": "user",
"parts": [
{
"text": "'"$*"'"
}
]
}],
"safetySettings":[{"category":"HARM_CATEGORY_HARASSMENT","threshold":"BLOCK_ONLY_HIGH"},{"category":"HARM_CATEGORY_HATE_SPEECH","threshold":"BLOCK_ONLY_HIGH"},{"category":"HARM_CATEGORY_SEXUALLY_EXPLICIT","threshold":"BLOCK_ONLY_HIGH"},{"category":"HARM_CATEGORY_DANGEROUS_CONTENT","threshold":"BLOCK_ONLY_HIGH"}]
}'
;;
*)
_die "error: unsupported build body for $kind"
;;
esac
fi
}
_wrapper() {
if [[ "$DRY_RUN" == "true" ]] || [[ "$DRY_RUN" == "1" ]]; then
echo "$@" >&2
else
"$@"
fi
}
_die() {
echo $*
exit 1
}
# See more details at https://github.com/sigoden/argc
eval "$(argc --argc-eval "$0" "$@")"
================================================
FILE: Cargo.toml
================================================
[package]
name = "aichat"
version = "0.30.0"
edition = "2021"
authors = ["sigoden <sigoden@gmail.com>"]
description = "All-in-one LLM CLI Tool"
license = "MIT OR Apache-2.0"
homepage = "https://github.com/sigoden/aichat"
repository = "https://github.com/sigoden/aichat"
categories = ["command-line-utilities"]
keywords = ["chatgpt", "llm", "cli", "ai", "repl"]
[dependencies]
anyhow = "1.0.69"
bytes = "1.4.0"
clap = { version = "4.4.8", features = ["derive"] }
dirs = "6.0.0"
futures-util = "0.3.29"
inquire = "0.7.0"
is-terminal = "0.4.9"
reedline = "0.40.0"
serde = { version = "1.0.152", features = ["derive"] }
serde_json = { version = "1.0.93", features = ["preserve_order"] }
serde_yaml = "0.9.17"
tokio = { version = "1.34.0", features = ["rt", "time", "macros", "signal", "rt-multi-thread"] }
tokio-graceful = "0.2.2"
tokio-stream = { version = "0.1.15", default-features = false, features = ["sync"] }
crossterm = "0.28.1"
chrono = "0.4.23"
bincode = { version = "2.0.0", features = ["serde", "std"], default-features = false }
parking_lot = "0.12.1"
fancy-regex = "0.14.0"
base64 = "0.22.0"
nu-ansi-term = "0.50.0"
async-trait = "0.1.74"
textwrap = "0.16.0"
ansi_colours = "1.2.2"
reqwest-eventsource = "0.6.0"
simplelog = "0.12.1"
log = "0.4.20"
shell-words = "1.1.0"
sha2 = "0.10.8"
unicode-width = "0.2.0"
async-recursion = "1.1.1"
http = "1.1.0"
http-body-util = "0.1"
hyper = { version = "1.0", features = ["full"] }
hyper-util = { version = "0.1", features = ["server-auto", "client-legacy"] }
time = { version = "0.3.36", features = ["macros"] }
indexmap = { version = "2.2.6", features = ["serde"] }
hmac = "0.12.1"
aws-smithy-eventstream = "0.60.4"
urlencoding = "2.1.3"
unicode-segmentation = "1.11.0"
json-patch = { version = "4.0.0", default-features = false }
bitflags = "2.5.0"
path-absolutize = "3.1.1"
hnsw_rs = "0.3.0"
rayon = "1.10.0"
uuid = { version = "1.9.1", features = ["v4"] }
scraper = { version = "0.23.1", default-features = false, features = ["deterministic"] }
sys-locale = "0.3.1"
html_to_markdown = "0.1.0"
rust-embed = "8.5.0"
os_info = { version = "3.8.2", default-features = false }
bm25 = { version = "2.0.1", features = ["parallelism"] }
which = "8.0.0"
fuzzy-matcher = "0.3.7"
terminal-colorsaurus = "0.4.8"
duct = "1.0.0"
[dependencies.reqwest]
version = "0.12.0"
features = ["json", "multipart", "socks", "rustls-tls", "rustls-tls-native-roots"]
default-features = false
[dependencies.syntect]
version = "5.0.0"
default-features = false
features = ["parsing", "regex-onig", "plist-load"]
[target.'cfg(target_os = "macos")'.dependencies]
crossterm = { version = "0.28.1", features = ["use-dev-tty"] }
[target.'cfg(target_os = "linux")'.dependencies]
arboard = { version = "3.3.0", default-features = false, features = ["wayland-data-control"] }
[target.'cfg(not(any(target_os = "linux", target_os = "android", target_os = "emscripten")))'.dependencies]
arboard = { version = "3.3.0", default-features = false }
[dev-dependencies]
pretty_assertions = "1.4.0"
rand = "0.9.0"
[profile.release]
lto = true
strip = true
opt-level = "z"
================================================
FILE: LICENSE-APACHE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: LICENSE-MIT
================================================
The MIT License (MIT)
Copyright (c) sigoden
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# AIChat: All-in-one LLM CLI Tool
[](https://github.com/sigoden/aichat/actions/workflows/ci.yaml)
[](https://crates.io/crates/aichat)
[](https://discord.gg/mr3ZZUB9hG)
AIChat is an all-in-one LLM CLI tool featuring Shell Assistant, CMD & REPL Mode, RAG, AI Tools & Agents, and More.
## Install
### Package Managers
- **Rust Developers:** `cargo install aichat`
- **Homebrew/Linuxbrew Users:** `brew install aichat`
- **Pacman Users**: `pacman -S aichat`
- **Windows Scoop Users:** `scoop install aichat`
- **Android Termux Users:** `pkg install aichat`
### Pre-built Binaries
Download pre-built binaries for macOS, Linux, and Windows from [GitHub Releases](https://github.com/sigoden/aichat/releases), extract them, and add the `aichat` binary to your `$PATH`.
## Features
### Multi-Providers
Integrate seamlessly with over 20 leading LLM providers through a unified interface. Supported providers include OpenAI, Claude, Gemini (Google AI Studio), Ollama, Groq, Azure-OpenAI, VertexAI, Bedrock, Github Models, Mistral, Deepseek, AI21, XAI Grok, Cohere, Perplexity, Cloudflare, OpenRouter, Ernie, Qianwen, Moonshot, ZhipuAI, MiniMax, Deepinfra, VoyageAI, any OpenAI-Compatible API provider.
### CMD Mode
Explore powerful command-line functionalities with AIChat's CMD mode.
### REPL Mode
Experience an interactive Chat-REPL with features like tab autocompletion, multi-line input support, history search, configurable keybindings, and custom REPL prompts.
### Shell Assistant
Elevate your command-line efficiency. Describe your tasks in natural language, and let AIChat transform them into precise shell commands. AIChat intelligently adjusts to your OS and shell environment.
### Multi-Form Input
Accept diverse input forms such as stdin, local files and directories, and remote URLs, allowing flexibility in data handling.
| Input | CMD | REPL |
| ----------------- | ------------------------------------ | -------------------------------- |
| CMD | `aichat hello` | |
| STDIN | `cat data.txt \| aichat` | |
| Last Reply | | `.file %%` |
| Local files | `aichat -f image.png -f data.txt` | `.file image.png data.txt` |
| Local directories | `aichat -f dir/` | `.file dir/` |
| Remote URLs | `aichat -f https://example.com` | `.file https://example.com` |
| External commands | ```aichat -f '`git diff`'``` | ```.file `git diff` ``` |
| Combine Inputs | `aichat -f dir/ -f data.txt explain` | `.file dir/ data.txt -- explain` |
### Role
Customize roles to tailor LLM behavior, enhancing interaction efficiency and boosting productivity.
> The role consists of a prompt and model configuration.
### Session
Maintain context-aware conversations through sessions, ensuring continuity in interactions.
> The left side uses a session, while the right side does not use a session.
### Macro
Streamline repetitive tasks by combining a series of REPL commands into a custom macro.
### RAG
Integrate external documents into your LLM conversations for more accurate and contextually relevant responses.
### Function Calling
Function calling supercharges LLMs by connecting them to external tools and data sources. This unlocks a world of possibilities, enabling LLMs to go beyond their core capabilities and tackle a wider range of tasks.
We have created a new repository [https://github.com/sigoden/llm-functions](https://github.com/sigoden/llm-functions) to help you make the most of this feature.
#### AI Tools & MCP
Integrate external tools to automate tasks, retrieve information, and perform actions directly within your workflow.
#### AI Agents (CLI version of OpenAI GPTs)
AI Agent = Instructions (Prompt) + Tools (Function Callings) + Documents (RAG).
### Local Server Capabilities
AIChat includes a lightweight built-in HTTP server for easy deployment.
```
$ aichat --serve
Chat Completions API: http://127.0.0.1:8000/v1/chat/completions
Embeddings API: http://127.0.0.1:8000/v1/embeddings
Rerank API: http://127.0.0.1:8000/v1/rerank
LLM Playground: http://127.0.0.1:8000/playground
LLM Arena: http://127.0.0.1:8000/arena?num=2
```
#### Proxy LLM APIs
The LLM Arena is a web-based platform where you can compare different LLMs side-by-side.
Test with curl:
```sh
curl -X POST -H "Content-Type: application/json" -d '{
"model":"claude:claude-3-5-sonnet-20240620",
"messages":[{"role":"user","content":"hello"}],
"stream":true
}' http://127.0.0.1:8000/v1/chat/completions
```
#### LLM Playground
A web application to interact with supported LLMs directly from your browser.
#### LLM Arena
A web platform to compare different LLMs side-by-side.
## Custom Themes
AIChat supports custom dark and light themes, which highlight response text and code blocks.
## Documentation
- [Chat-REPL Guide](https://github.com/sigoden/aichat/wiki/Chat-REPL-Guide)
- [Command-Line Guide](https://github.com/sigoden/aichat/wiki/Command-Line-Guide)
- [Role Guide](https://github.com/sigoden/aichat/wiki/Role-Guide)
- [Macro Guide](https://github.com/sigoden/aichat/wiki/Macro-Guide)
- [RAG Guide](https://github.com/sigoden/aichat/wiki/RAG-Guide)
- [Environment Variables](https://github.com/sigoden/aichat/wiki/Environment-Variables)
- [Configuration Guide](https://github.com/sigoden/aichat/wiki/Configuration-Guide)
- [Custom Theme](https://github.com/sigoden/aichat/wiki/Custom-Theme)
- [Custom REPL Prompt](https://github.com/sigoden/aichat/wiki/Custom-REPL-Prompt)
- [FAQ](https://github.com/sigoden/aichat/wiki/FAQ)
## License
Copyright (c) 2023-2025 aichat-developers.
AIChat is made available under the terms of either the MIT License or the Apache License 2.0, at your option.
See the LICENSE-APACHE and LICENSE-MIT files for license details.
================================================
FILE: assets/arena.html
================================================
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="icon" href="data:;base64,iVBORw0KGgo=">
<title>AIChat LLM Arena</title>
<link rel="stylesheet" href="//unpkg.com/katex@0.16.11/dist/katex.min.css">
<link rel="stylesheet" href="//unpkg.com/github-markdown-css@5.8.1/github-markdown.css">
<link rel="stylesheet" href="//unpkg.com/@highlightjs/cdn-assets@11.10.0/styles/github-dark.min.css"
media="screen and (prefers-color-scheme: dark)">
<link rel="stylesheet" href="//unpkg.com/@highlightjs/cdn-assets@11.10.0/styles/github.min.css"
media="screen and (prefers-color-scheme: light)">
<script src="//unpkg.com/@highlightjs/cdn-assets@11.10.0/highlight.min.js" defer></script>
<script src="//unpkg.com/marked@15.0.3/lib/marked.umd.js" defer></script>
<script src="//unpkg.com/katex@0.16.11/dist/katex.min.js" defer></script>
<script src="//unpkg.com/@sigodenjs/marked-katex-extension@1.0.0/lib/index.umd.js" defer></script>
<script src="//unpkg.com/alpinejs@3.14.6/dist/cdn.min.js" defer></script>
<style>
:root {
--fg-primary: #1652f1;
--fg-default: black;
--bg-primary: white;
--bg-default: #f9f9f9;
--bg-toast: rgba(0, 0, 0, 0.7);
--border-color: #c3c3c3;
}
[x-cloak] {
display: none !important;
}
html {
font-family: Noto Sans, SF Pro SC, SF Pro Text, SF Pro Icons, PingFang SC, Helvetica Neue, Helvetica, Arial, sans-serif
}
body,
div {
padding: 0;
margin: 0;
box-sizing: border-box;
}
textarea,
input,
select,
option {
color: var(--fg-default);
background-color: var(--bg-primary);
}
body {
font-family: Arial, sans-serif;
font-size: 1rem;
display: flex;
height: 100vh;
color: var(--fg-default);
background-color: var(--bg-default);
}
.container {
display: flex;
flex-direction: column;
background-color: var(--bg-primary);
width: 100%;
}
.chats {
display: flex;
flex-direction: row;
flex-grow: 1;
width: 100%;
}
.chat-panel {
display: flex;
flex-direction: column;
width: 100%;
}
.chat-header {
display: flex;
padding: 0.5rem;
flex-direction: row;
border-bottom: 1px solid var(--border-color);
}
.chat-header select {
width: 100%;
outline: none;
font-size: 1.25rem;
border: none;
}
.chat-body {
display: flex;
flex-direction: column;
flex-grow: 1;
overflow-x: hidden;
overflow-y: auto;
}
.chat-message {
display: flex;
padding: 0.7rem;
margin-bottom: 0.7rem;
}
.chat-avatar svg {
width: 1.25rem;
height: 1.25rem;
border-radius: 50%;
}
.chat-message-content {
position: relative;
display: flex;
flex-direction: column;
width: calc(100% - 1rem);
margin-top: -2px;
padding-left: 0.625rem;
flex-grow: 1;
}
.chat-message-content .error {
color: red;
background: none;
padding: 0;
}
.chat-message-content .message-text {
white-space: pre-wrap;
padding-top: 0.2rem;
}
.message-image-bar {
display: flex;
flex-direction: row;
overflow-x: auto;
}
.message-image {
margin: 0.25rem;
}
.message-image img {
width: 10rem;
height: 10rem;
object-fit: cover;
}
.markdown-body {
display: flex;
width: 100%;
padding: 0;
flex-direction: column;
background-color: var(--bg-primary);
}
.markdown-body:first-child {
margin-top: 0;
padding-top: 0;
}
.markdown-body pre {
overflow-x: auto;
word-wrap: break-word;
}
.code-block {
position: relative;
width: 100%;
}
.message-toolbox {
display: flex;
position: absolute;
bottom: -1.4rem;
}
.copy-message-btn,
.regenerate-message-btn,
.tts-message-btn {
top: 0.7rem;
right: 0.7rem;
cursor: pointer;
font-size: 0.9rem;
padding-right: 4px;
}
.copy-message-btn svg,
.regenerate-message-btn svg,
.tts-message-btn svg {
width: 1rem;
height: 1rem;
}
.copy-code-btn {
position: absolute;
top: 0.7rem;
right: 0.7rem;
cursor: pointer;
font-size: 0.9rem;
}
.copy-code-btn svg {
width: 1rem;
height: 1rem;
}
.scroll-to-bottom-btn {
position: absolute;
text-align: center;
cursor: pointer;
width: 1.5rem;
height: 1.5rem;
border-radius: 0.75rem;
background-color: var(--bg-primary);
}
.scroll-to-bottom-btn svg {
width: 1.5rem;
height: 1.5rem;
border-radius: 50%;
}
.input-panel {
position: relative;
border-top: 1px solid var(--border-color);
}
.input-panel-inner {
margin: 1rem;
padding: 0.5rem;
border: 1px solid var(--border-color);
border-radius: 1rem;
}
.input-panel-inner textarea {
width: 100%;
font-size: 1rem;
padding: 0.4rem;
box-sizing: border-box;
border: none;
outline: none;
resize: none;
max-height: 500px;
overflow-x: hidden;
overflow-y: auto;
}
.input-toolbox {
position: absolute;
display: flex;
right: 1.875rem;
font-size: 1rem;
bottom: 1.875rem;
cursor: pointer;
}
.input-toolbox svg {
width: 1.875rem;
height: 1.875rem;
fill: var(--fg-default);
}
.image-btn {
position: relative;
display: inline-block;
margin-right: 0.5rem;
}
.image-btn input[type="file"] {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
opacity: 0;
cursor: pointer;
}
.input-image-bar {
display: flex;
flex-direction: row;
width: 100%;
overflow-x: auto;
}
.input-image-item {
display: flex;
margin: 0.25rem;
width: 5rem;
position: relative;
}
.input-image-item img {
width: 5rem;
height: 5rem;
object-fit: cover;
}
.image-remove-btn {
font-size: 1rem;
margin-left: -0.8rem;
cursor: pointer;
}
.image-remove-btn {
width: 1rem;
height: 1rem;
}
.input-btn.disabled {
opacity: 0.3;
}
.spinner {
width: 1.1rem;
height: 1.1rem;
margin-top: 3px;
border: 2px solid var(--fg-default);
border-bottom-color: transparent;
border-radius: 50%;
display: inline-block;
animation: spinner-rotation 1s linear infinite;
}
.toast {
display: none;
position: fixed;
top: 2px;
left: 50%;
text-align: center;
transform: translate(-50%, 0);
background-color: var(--bg-toast);
color: var(--bg-primary);
padding: 0.5rem;
border-radius: 0.3rem;
z-index: 9999;
}
@keyframes spinner-rotation {
0% {
transform: rotate(0deg);
}
100% {
transform: rotate(360deg);
}
}
@media (prefers-color-scheme: dark) {
:root {
--fg-primary: #1652f1;
--fg-default: white;
--bg-primary: black;
--bg-default: #121212;
--bg-toast: rgba(255, 255, 255, 0.7);
--border-color: #3c3c3c;
}
}
@media screen and (max-width: 768px) {
body {
height: calc(100vh - 56px);
height: 100dvh;
}
.container {
padding: 3px;
}
.chat-header {
padding: 0.6rem;
}
.chat-header select {
font-size: 1rem;
}
.chat-body {
padding: 0.6rem;
}
.input-panel-inner {
margin: 0.5rem;
}
}
</style>
</head>
<body>
<div class="container" x-data="app">
<div class="chats">
<template x-for="(chat, index) in chats" :key="index">
<div class="chat-panel">
<div class="chat-header">
<select x-cloak id="model" x-model="chat.model" @change="handleModelChange">
<template x-for="model in chatModels" :key="model.id">
<option :value="model.id" :selected="model.id == chat.model" x-text="model.id"></option>
</template>
</select>
</div>
<div class="chat-body" :id="'chat-body-' + index" @scroll="(event) => handleScrollChatBody(event, index)">
<template x-for="(message, messageIndex) in chat.messages" :key="message.id">
<div class="chat-message" @mouseover="chat.hoveredMessageIndex = messageIndex"
@mouseleave="chat.messageHoveredIndex = null">
<div class="chat-avatar" :class="message.role == 'user' ? 'chat-avatar user' : 'chat-avatar assistant'">
<template x-if="message.role == 'user'">
<svg fill="currentColor" viewBox="0 0 16 16">
<path d="M11 6a3 3 0 1 1-6 0 3 3 0 0 1 6 0" />
<path fill-rule="evenodd"
d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8m8-7a7 7 0 0 0-5.468 11.37C3.242 11.226 4.805 10 8 10s4.757 1.225 5.468 2.37A7 7 0 0 0 8 1" />
</svg>
</template>
<template x-if="message.role == 'assistant'">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M6 12.5a.5.5 0 0 1 .5-.5h3a.5.5 0 0 1 0 1h-3a.5.5 0 0 1-.5-.5M3 8.062C3 6.76 4.235 5.765 5.53 5.886a26.6 26.6 0 0 0 4.94 0C11.765 5.765 13 6.76 13 8.062v1.157a.93.93 0 0 1-.765.935c-.845.147-2.34.346-4.235.346s-3.39-.2-4.235-.346A.93.93 0 0 1 3 9.219zm4.542-.827a.25.25 0 0 0-.217.068l-.92.9a25 25 0 0 1-1.871-.183.25.25 0 0 0-.068.495c.55.076 1.232.149 2.02.193a.25.25 0 0 0 .189-.071l.754-.736.847 1.71a.25.25 0 0 0 .404.062l.932-.97a25 25 0 0 0 1.922-.188.25.25 0 0 0-.068-.495c-.538.074-1.207.145-1.98.189a.25.25 0 0 0-.166.076l-.754.785-.842-1.7a.25.25 0 0 0-.182-.135" />
<path
d="M8.5 1.866a1 1 0 1 0-1 0V3h-2A4.5 4.5 0 0 0 1 7.5V8a1 1 0 0 0-1 1v2a1 1 0 0 0 1 1v1a2 2 0 0 0 2 2h10a2 2 0 0 0 2-2v-1a1 1 0 0 0 1-1V9a1 1 0 0 0-1-1v-.5A4.5 4.5 0 0 0 10.5 3h-2zM14 7.5V13a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V7.5A3.5 3.5 0 0 1 5.5 4h5A3.5 3.5 0 0 1 14 7.5" />
</svg>
</template>
</div>
<div class="chat-message-content">
<!-- message -->
<template x-if="message.role == 'assistant' && message.html">
<div class="markdown-body" x-html="message.html"></div>
</template>
<template x-if="message.role == 'assistant' && message.state == 'loading'">
<div class="spinner"></div>
</template>
<template x-if="message.role == 'user' && Array.isArray(message.content)">
<div class="message-text-images">
<template x-if="message.content[0].text">
<div class="message-text" x-text="message.content[0].text"></div>
</template>
<div class="message-image-bar">
<template x-for="part in message.content">
<template x-if="part.type == 'image_url'">
<div class="message-image">
<img :src="part.image_url.url" alt="Image Message Part">
</div>
</template>
</template>
</div>
</div>
</template>
<template
x-if="message.role == 'user' && Object.prototype.toString.call(message.content) == '[object String]'">
<div class="message-text" x-text="message.content"></div>
</template>
<!-- toolbox -->
<template x-if="messageIndex == chat.hoveredMessageIndex">
<div class="message-toolbox">
<div class="copy-message-btn" @click="handleCopyMessage(message.content)" title="Copy">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd"
d="M4 2a2 2 0 0 1 2-2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2zm2-1a1 1 0 0 0-1 1v8a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1V2a1 1 0 0 0-1-1zM2 5a1 1 0 0 0-1 1v8a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1v-1h1v1a2 2 0 0 1-2 2H2a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h1v1z" />
</svg>
</div>
<template
x-if="messageIndex == chat.messages.length - 1 && (message.state == 'succeed' || message.state == 'failed')">
<div class="regenerate-message-btn" @click="(event) => handleRegenerateMessage(index)"
title="Regenerate">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd"
d="M8 3a5 5 0 1 0 4.546 2.914.5.5 0 0 1 .908-.417A6 6 0 1 1 8 2z" />
<path
d="M8 4.466V.534a.25.25 0 0 1 .41-.192l2.36 1.966c.12.1.12.284 0 .384L8.41 4.658A.25.25 0 0 1 8 4.466" />
</svg>
</div>
</template>
<template x-if="message.state == 'succeed' && !!window.speechSynthesis">
<div class="tts-message-btn" @click="handleTTSMessage(message.content)" title="Text to speech">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M11.536 14.01A8.47 8.47 0 0 0 14.026 8a8.47 8.47 0 0 0-2.49-6.01l-.708.707A7.48 7.48 0 0 1 13.025 8c0 2.071-.84 3.946-2.197 5.303z" />
<path
d="M10.121 12.596A6.48 6.48 0 0 0 12.025 8a6.48 6.48 0 0 0-1.904-4.596l-.707.707A5.48 5.48 0 0 1 11.025 8a5.48 5.48 0 0 1-1.61 3.89z" />
<path
d="M10.025 8a4.5 4.5 0 0 1-1.318 3.182L8 10.475A3.5 3.5 0 0 0 9.025 8c0-.966-.392-1.841-1.025-2.475l.707-.707A4.5 4.5 0 0 1 10.025 8M7 4a.5.5 0 0 0-.812-.39L3.825 5.5H1.5A.5.5 0 0 0 1 6v4a.5.5 0 0 0 .5.5h2.325l2.363 1.89A.5.5 0 0 0 7 12zM4.312 6.39 6 5.04v5.92L4.312 9.61A.5.5 0 0 0 4 9.5H2v-3h2a.5.5 0 0 0 .312-.11" />
</svg>
</div>
</template>
</div>
</template>
</div>
</div>
</template>
</div>
<div class="scroll-to-bottom-btn" x-cloak x-show="chat.isShowScrollToBottomBtn"
@click="() => handleScrollToBottom(index)">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd"
d="M1 8a7 7 0 1 0 14 0A7 7 0 0 0 1 8m15 0A8 8 0 1 1 0 8a8 8 0 0 1 16 0M8.5 4.5a.5.5 0 0 0-1 0v5.793L5.354 8.146a.5.5 0 1 0-.708.708l3 3a.5.5 0 0 0 .708 0l3-3a.5.5 0 0 0-.708-.708L8.5 10.293z" />
</svg>
</div>
</div>
</template>
</div>
<div class="input-panel">
<div class="input-panel-inner">
<textarea id="chat-input" x-model="input" x-ref="input" @keydown.enter="handleEnterKeydown"
placeholder="Ask Anything" autofocus></textarea>
<div class="input-image-bar" x-show="images.length > 0">
<template x-for="(image, index) in images">
<div class="input-image-item">
<img :src="image" alt="Preview image">
<div class="image-remove-btn" @click="images.splice(index, 1);">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5m2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5m3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0z" />
<path
d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4zM2.5 3h11V2h-11z" />
</svg>
</div>
</div>
</template>
</div>
<template x-if="asking > 0">
<div class="input-toolbox">
<div class="input-btn" @click="handleCancelAsk">
<svg fill="currentColor" viewBox="0 0 16 16">
<path d="M8 15A7 7 0 1 1 8 1a7 7 0 0 1 0 14m0 1A8 8 0 1 0 8 0a8 8 0 0 0 0 16" />
<path
d="M5 6.5A1.5 1.5 0 0 1 6.5 5h3A1.5 1.5 0 0 1 11 6.5v3A1.5 1.5 0 0 1 9.5 11h-3A1.5 1.5 0 0 1 5 9.5z" />
</svg>
</div>
</div>
</template>
<template x-if="asking == 0">
<div class="input-toolbox">
<div class="image-btn" x-show="supportsVision">
<input type="file" multiple accept=".jpg,.jpeg,.png,.webp" @change="handleImageUpload">
<svg fill="currentColor" viewBox="0 0 16 16">
<path d="M6.002 5.5a1.5 1.5 0 1 1-3 0 1.5 1.5 0 0 1 3 0" />
<path
d="M2.002 1a2 2 0 0 0-2 2v10a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V3a2 2 0 0 0-2-2zm12 1a1 1 0 0 1 1 1v6.5l-3.777-1.947a.5.5 0 0 0-.577.093l-3.71 3.71-2.66-1.772a.5.5 0 0 0-.63.062L1.002 12V3a1 1 0 0 1 1-1z" />
</svg>
</div>
<div class="input-btn" :class="(input.trim() || images.length > 0) ? 'input-btn' : 'input-btn disabled'"
@click="handleAsk">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M2 16a2 2 0 0 1-2-2V2a2 2 0 0 1 2-2h12a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2zm6.5-4.5V5.707l2.146 2.147a.5.5 0 0 0 .708-.708l-3-3a.5.5 0 0 0-.708 0l-3 3a.5.5 0 1 0 .708.708L7.5 5.707V11.5a.5.5 0 0 0 1 0" />
</svg>
</div>
</div>
</template>
</div>
</div>
<div id="toast" class="toast"></div>
</div>
<script>
const QUERY = parseQueryString();
const NUM = parseInt(QUERY.num) || 2
const API_BASE = QUERY.api_base || "./v1";
const API_KEY = QUERY.api_key || "";
const CHAT_COMPLETIONS_URL = API_BASE + "/chat/completions";
const MODELS_API = API_BASE + "/models";
document.addEventListener("alpine:init", () => {
setupMarked();
setupApp();
});
function setupApp() {
let $inputPanel = document.querySelector('.input-panel');
let $chatPanels = [];
let $scrollToBottomBtns = [];
let msgIdx = 0;
Alpine.data("app", () => ({
chatModels: [],
input: "",
images: [],
asking: 0,
chats: Array.from(Array(NUM)).map(_ => ({
model: "",
messages: [],
hoveredMessageIndex: null,
askAbortController: null,
shouldScrollChatBodyToBottom: true,
isShowScrollToBottomBtn: false,
})),
async init() {
try {
const models = await fetchJSON(MODELS_API);
this.chatModels = models.filter(v => !v.type || v.type === "chat");
} catch (err) {
toast("No available model");
console.error("Failed to load models", err);
}
let models = []
if (QUERY.models) {
models = QUERY.models.split(",");
}
$chatPanels = document.querySelectorAll('.chat-panel');
$scrollToBottomBtns = document.querySelectorAll('.scroll-to-bottom-btn');
const offsets = calculateOffsets(NUM);
for (let i = 0; i < NUM; i++) {
this.chats[i].model = models[i] || "default";
$chatPanels[i].style.width = (100 / NUM) + '%';
if (i > 0) {
$chatPanels[i].style.borderLeft = '1px solid var(--border-color)';
}
$scrollToBottomBtns[i].style.left = offsets[i];
}
this.$refs.input.addEventListener("paste", (e) => this.handlePaste(e));
this.$watch("input", () => this.autosizeInput(this.$refs.input));
new ResizeObserver(() => {
this.autoHeightChatPanel();
}).observe($inputPanel)
},
get supportsVision() {
return this.chats.every(v => !!retrieveModel(this.chatModels, v.model)?.supports_vision)
},
handleAsk() {
const isEmptyInput = this.input.trim() === "";
const isEmptyImage = this.images.length === 0;
if (this.asking > 0 || (isEmptyImage && isEmptyInput)) {
return;
}
for (let index = 0; index < this.chats.length; index++) {
const chat = this.chats[index];
if (isEmptyImage) {
chat.messages.push({
id: msgIdx++,
role: "user",
content: this.input,
});
} else {
const parts = [];
if (!isEmptyInput) {
parts.push({ type: "text", text: this.input });
}
for (const image of this.images) {
parts.push({ type: "image_url", image_url: { url: image } });
}
chat.messages.push({
id: msgIdx++,
role: "user",
content: parts,
})
}
chat.messages.push({
id: msgIdx++,
role: "assistant",
content: "",
state: "loading", // streaming, succeed, failed
error: "",
html: "",
});
}
for (let index = 0; index < this.chats.length; index++) {
this.asking++;
this.ask(index);
}
this.input = "";
this.images = [];
},
handleRegenerateMessage(index) {
const chat = this.chats[index];
const lastIndex = chat.messages.length - 1;
if (lastIndex !== chat.hoveredMessageIndex) {
return
}
let lastMessage = chat.messages[lastIndex];
lastMessage.content = "";
lastMessage.state = "loading";
lastMessage.error = "";
lastMessage.html = "";
this.asking++;
this.ask(index);
},
/**
* @param {string} messageToUtter
*/
handleTTSMessage(messageToUtter) {
if (!!window.speechSynthesis) {
if (window.speechSynthesis.speaking || window.speechSynthesis.pending) {
window.speechSynthesis.cancel();
} else {
let utterance = new SpeechSynthesisUtterance(messageToUtter);
window.speechSynthesis.speak(utterance);
}
}
},
handleCancelAsk() {
for (const chat of this.chats) {
chat.askAbortController?.abort();
}
},
handleModelChange() {
this.updateUrl();
},
handleScrollChatBody(event, index) {
const chat = this.chats[index];
const $chatBody = event.target;
const { scrollTop, clientHeight, scrollHeight, _prevScrollTop = 0 } = $chatBody;
if (scrollTop + clientHeight > scrollHeight - 5) {
chat.isShowScrollToBottomBtn = false;
chat.shouldScrollChatBodyToBottom = true;
}
if (scrollHeight > clientHeight && _prevScrollTop > 1 && _prevScrollTop > scrollTop + 1) {
chat.shouldScrollChatBodyToBottom = false;
chat.isShowScrollToBottomBtn = true;
}
$chatBody._prevScrollTop = scrollTop;
},
handleScrollToBottom(index) {
const chat = this.chats[index];
const $chatBody = document.querySelector('#chat-body-' + index);
$chatBody.scrollTop = $chatBody.scrollHeight;
chat.isShowScrollToBottomBtn = false;
chat.shouldScrollChatBodyToBottom = true;
},
handleEnterKeydown(event) {
if (event.shiftKey) {
return;
}
event.preventDefault();
this.handleAsk();
},
handleCopyCode(event) {
const $btn = event.target;
const $code = $btn.closest('.code-block').querySelector("code");
if ($code) {
const range = document.createRange();
range.selectNodeContents($code);
window.getSelection().removeAllRanges();
window.getSelection().addRange(range);
document.execCommand('copy');
window.getSelection().removeAllRanges();
toast("Copied Code");
}
},
handleCopyMessage(content) {
if (Array.isArray(content)) {
content = content.map(v => v.text || "").join("");
}
const $tempTextArea = document.createElement("textarea");
$tempTextArea.value = content;
document.body.appendChild($tempTextArea);
$tempTextArea.select();
$tempTextArea.setSelectionRange(0, 99999); // For mobile devices
document.execCommand("copy");
document.body.removeChild($tempTextArea);
toast("Copied Message")
},
async handleImageUpload(event) {
const files = event.target.files;
if (!files || files.length === 0) {
return;
}
const urls = await Promise.all(Array.from(files).map(file => convertImageToDataURL(file)));
this.images.push(...urls);
event.target.value = "";
},
async handlePaste(event) {
const files = Array.from(event.clipboardData.items).filter(v => v.type.startsWith('image/')).map(v => v.getAsFile());
const urls = await Promise.all(files.map(file => convertImageToDataURL(file)));
this.images.push(...urls);
},
updateUrl() {
const newUrl = new URL(location.href);
const models = this.chats.map(v => v.model).join(",");
newUrl.searchParams.set("models", models);
history.replaceState(null, '', newUrl.toString());
},
autoHeightChatPanel() {
const height = $inputPanel.offsetHeight;
for (let i = 0; i < this.chats.length; i++) {
$chatPanels[i].style.height = (window.innerHeight - height - 5) + "px";
$scrollToBottomBtns[i].style.bottom = (height + 20) + "px";
}
},
autoScrollChatBodyToBottom(index) {
const chat = this.chats[index];
if (chat.shouldScrollChatBodyToBottom) {
const $chatBody = document.querySelector('#chat-body-' + index);
if ($chatBody) {
$chatBody.scrollTop = $chatBody.scrollHeight;
}
}
},
autosizeInput($input) {
$input.style.height = 'auto';
$input.style.height = $input.scrollHeight + 'px';
},
async ask(index) {
const chat = this.chats[index];
chat.askAbortController = new AbortController();
chat.shouldScrollChatBodyToBottom = true;
this.$nextTick(() => {
this.autoScrollChatBodyToBottom(index);
});
const lastMessage = chat.messages[chat.messages.length - 1];
const body = this.buildBody(index);
let succeed = false;
try {
const stream = await fetchChatCompletions(CHAT_COMPLETIONS_URL, body, chat.askAbortController.signal)
for await (const chunk of stream) {
lastMessage.state = "streaming";
lastMessage.content += chunk?.choices[0]?.delta?.content || "";
lastMessage.html = renderMarkdown(lastMessage.content, lastMessage.error);
this.$nextTick(() => {
this.autoScrollChatBodyToBottom(index);
});
}
lastMessage.state = "succeed";
succeed = true;
} catch (err) {
lastMessage.state = "failed";
if (this.askAbortController?.signal?.aborted) {
lastMessage.error = "";
} else {
lastMessage.error = err?.message || err;
}
lastMessage.html = renderMarkdown(lastMessage.content, lastMessage.error);
}
this.asking--;
},
buildBody(index) {
const chat = this.chats[index];
const messages = [];
for ([userMessage, assistantMessage] of chunkArray(chat.messages, 2)) {
if (assistantMessage.state === "failed") {
continue;
} else if (assistantMessage.state === "loading") {
messages.push({
role: userMessage.role,
content: userMessage.content,
});
} else {
messages.push({
role: userMessage.role,
content: userMessage.content,
});
messages.push({
role: assistantMessage.role,
content: assistantMessage.content,
});
}
}
sanitizeMessages(messages);
const body = {
model: chat.model,
messages: messages,
stream: true,
};
const { max_output_token, require_max_tokens } = retrieveModel(this.chatModels, chat.model);
if (!body["max_tokens"] && require_max_tokens) {
body["max_tokens"] = max_output_token;
};
return body;
},
}));
}
async function fetchJSON(url) {
const res = await fetch(url, { headers: getHeaders() });
const data = await res.json()
return data.data;
}
async function* fetchChatCompletions(url, body, signal) {
const stream = body.stream;
const response = await fetch(url, {
method: "POST",
signal,
headers: getHeaders(),
body: JSON.stringify(body),
});
if (!response.ok) {
const error = await response.json();
throw error?.error || error;
}
if (!stream) {
const data = await response.json();
return data;
}
const reader = response.body.getReader();
const decoder = new TextDecoder();
let done = false;
let reamingChunkValue = "";
while (!done) {
if (signal?.aborted) {
reader.cancel();
break;
}
const { value, done: doneReading } = await reader.read();
done = doneReading;
const chunkValue = decoder.decode(value);
const lines = (reamingChunkValue + chunkValue).split("\n").filter(line => line.trim().length > 0);
reamingChunkValue = "";
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
const message = line.replace(/^data: /, "");
if (message === "[DONE]") {
continue
}
try {
const parsed = JSON.parse(message);
yield parsed;
} catch {
if (i === lines.length - 1) {
reamingChunkValue += line;
break;
}
}
}
}
}
function getHeaders() {
const headers = {
"content-type": "application/json",
};
if (API_KEY) {
headers["authorization"] = `Bearer ${API_KEY}`;
}
return headers
}
function retrieveModel(models, id) {
const model = models.find(model => model.id === id);
if (!model) return {};
const max_output_token = model.max_output_tokens;
const supports_vision = !!model.supports_vision;
const require_max_tokens = !!model.require_max_tokens;
return {
id,
max_output_token,
supports_vision,
require_max_tokens,
}
}
function toast(text, duration = 2500) {
const $toast = document.getElementById("toast");
clearTimeout($toast._timer);
$toast.textContent = text;
$toast.style.display = "block";
$toast._timer = setTimeout(function () {
$toast.style.display = "none";
}, duration);
}
function convertImageToDataURL(imageFile) {
return new Promise((resolve, reject) => {
if (!imageFile) {
reject(new Error("Please select an image file."));
return;
}
const reader = new FileReader();
reader.readAsDataURL(imageFile);
reader.onload = (event) => resolve(event.target.result);
reader.onerror = (error) => reject(error);
});
}
function sanitizeMessages(messages) {
let messagesLen = messages.length;
for (let i = 0; i < messagesLen; i++) {
const message = messages[i];
if (typeof message.content === "string" && message.role === "assistant" && i !== messagesLen - 1) {
message.content = stripThinkTag(message.content);
}
}
}
function stripThinkTag(text) {
return text.replace(/^\s*<think>([\s\S]*?)<\/think>(\s*|$)/g, '')
}
function setupMarked() {
const renderer = {
code({ text, lang }) {
const validLang = !!(lang && hljs.getLanguage(lang));
const highlighted = validLang
? hljs.highlight(text, { language: lang }).value
: escapeForHTML(text);
return `<div class="code-block">
<pre><code class="hljs ${lang}">${highlighted}</code></pre>
<div class="copy-code-btn" @click="handleCopyCode" title="Copy code">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd" d="M4 2a2 2 0 0 1 2-2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2zm2-1a1 1 0 0 0-1 1v8a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1V2a1 1 0 0 0-1-1zM2 5a1 1 0 0 0-1 1v8a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1v-1h1v1a2 2 0 0 1-2 2H2a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h1v1z"/>
</svg>
</div>
</div>`;
}
};
const thinkExtension = {
name: 'think',
level: 'block',
start(src) {
const match = /^(\s*)<think>/.exec(src);
if (match) {
return match[1].length
} else {
return -1;
}
},
tokenizer(src, tokens) {
const rule = /^\s*<think>([\s\S]*?)(<\/think>|$)/;
const match = rule.exec(src);
if (match) {
return {
type: 'think',
raw: match[0],
text: match[1].trim(),
};
}
},
renderer(token) {
const text = '<p>' + token.text.trim().replace(/\n+/g, '</p><p>') + '</p>';
return `<details open class="think">
<summary>Deeply thought</summary>
<blockquote>${text}</blockquote>
</details>`;
},
};
marked.use({ renderer });
marked.use(markedKatex({ throwOnError: false, inlineTolerantNoSpace: true }));
marked.use({ extensions: [thinkExtension] })
}
function escapeForHTML(input) {
const escapeMap = {
"&": "&",
"<": "<",
">": ">",
'"': """,
"'": "'"
};
return input.replace(/([&<>'"])/g, char => escapeMap[char]);
}
function parseQueryString() {
const params = new URLSearchParams(location.search);
const queryObject = {};
params.forEach((value, key) => {
queryObject[key] = value;
});
return queryObject;
}
function chunkArray(array, chunkSize) {
const chunks = [];
for (let i = 0; i < array.length; i += chunkSize) {
chunks.push(array.slice(i, i + chunkSize));
}
return chunks;
}
function renderMarkdown(text, error = '') {
return marked.marked(text) + (error ? `<pre class="error">${error}</pre>` : '');
}
function calculateOffsets(pieces) {
const offsets = [];
for (let i = 1; i <= pieces; i++) {
const offset = ((i - 0.5) / pieces) * 100;
offsets.push(`${offset.toFixed(1)}%`);
}
return offsets;
}
</script>
</body>
</html>
================================================
FILE: assets/playground.html
================================================
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="icon" href="data:;base64,iVBORw0KGgo=">
<title>AIChat LLM Playground</title>
<link rel="stylesheet" href="//unpkg.com/katex@0.16.11/dist/katex.min.css">
<link rel="stylesheet" href="//unpkg.com/github-markdown-css@5.8.1/github-markdown.css">
<link rel="stylesheet" href="//unpkg.com/@highlightjs/cdn-assets@11.10.0/styles/github-dark.min.css"
media="screen and (prefers-color-scheme: dark)">
<link rel="stylesheet" href="//unpkg.com/@highlightjs/cdn-assets@11.10.0/styles/github.min.css"
media="screen and (prefers-color-scheme: light)">
<script src="//unpkg.com/@highlightjs/cdn-assets@11.10.0/highlight.min.js" defer></script>
<script src="//unpkg.com/marked@15.0.3/lib/marked.umd.js" defer></script>
<script src="//unpkg.com/katex@0.16.11/dist/katex.min.js" defer></script>
<script src="//unpkg.com/@sigodenjs/marked-katex-extension@1.0.0/lib/index.umd.js" defer></script>
<script src="//unpkg.com/alpinejs@3.14.6/dist/cdn.min.js" defer></script>
<style>
:root {
--fg-primary: #1652f1;
--fg-default: black;
--bg-primary: white;
--bg-default: #f9f9f9;
--bg-toast: rgba(0, 0, 0, 0.7);
--bg-cover: rgba(0, 0, 0, 0.5);
--bg-hover: #f0f0f0;
--border-color: #c3c3c3;
--shadow-color: rgba(0, 0, 0, 0.1);
}
[x-cloak] {
display: none !important;
}
html {
font-family: Noto Sans, SF Pro SC, SF Pro Text, SF Pro Icons, PingFang SC, Helvetica Neue, Helvetica, Arial, sans-serif
}
body,
div {
padding: 0;
margin: 0;
box-sizing: border-box;
}
textarea,
input,
select,
option {
color: var(--fg-default);
background-color: var(--bg-primary);
}
body {
font-family: Arial, sans-serif;
font-size: 1rem;
display: flex;
height: 100vh;
color: var(--fg-default);
background-color: var(--bg-default);
}
.container {
width: 100%;
padding: 1.25rem;
box-sizing: border-box;
display: flex;
}
.sidebar {
width: 360px;
flex-shrink: 0;
margin-right: 1.25rem;
background-color: var(--bg-primary);
box-shadow: 0 0 0.3rem var(--shadow-color);
border-radius: 0.3rem;
}
.sidebar-header {
display: flex;
align-items: center;
padding: 1.25rem;
}
.sidebar-header .title {
font-size: 1.25rem;
font-weight: bold;
}
.sidebar-header .subtitle {
font-size: 0.8rem;
padding-top: 0.3rem;
}
.sidebar-right {
display: flex;
flex-direction: row;
margin-left: auto;
gap: 6px;
}
.sidebar-btn {
cursor: pointer;
width: 1.2rem;
height: 1.2rem;
}
.hide-sidebar-btn {
display: none;
}
.settings {
padding: 1.25rem;
}
.settings label {
display: block;
margin-bottom: 0.3rem;
}
.settings select,
.settings input[type="number"] {
width: 100%;
padding: 0.5rem;
margin-bottom: 0.625rem;
border: 1px solid var(--border-color);
border-radius: 0.25rem;
box-sizing: border-box;
}
.settings textarea {
width: 100%;
height: 150px;
padding: 0.5rem;
border: 1px solid var(--border-color);
border-radius: 0.25rem;
box-sizing: border-box;
margin-bottom: 0.625rem;
}
.checkbox-group {
display: flex;
align-items: center;
}
.checkbox-group input[type="checkbox"] {
margin-left: auto;
}
.main-panel {
display: flex;
flex-direction: column;
width: calc(100vw - 360px - 2.5rem);
background-color: var(--bg-primary);
box-shadow: 0 0 0.3rem var(--shadow-color);
border-radius: 0.3rem;
}
.chat-header {
display: flex;
flex-direction: row;
padding: 1.25rem;
border-bottom: 1px solid var(--border-color);
}
.chat-header select {
width: 100%;
outline: none;
font-size: 1.25rem;
border: none;
}
.show-sidebar-btn {
display: none;
width: 1.5rem;
height: 1.5rem;
}
.chat-header .toolbar {
margin-left: auto;
}
.chat-body {
display: flex;
flex-direction: column;
padding: 0.5rem;
flex-grow: 1;
overflow-x: hidden;
overflow-y: auto;
}
.chat-message {
display: flex;
padding: 0.7rem;
margin-bottom: 0.7rem;
}
.chat-avatar svg {
width: 1.25rem;
height: 1.25rem;
border-radius: 50%;
}
.chat-message-content {
position: relative;
display: flex;
flex-direction: column;
width: calc(100% - 1rem);
margin-top: -2px;
padding-left: 0.625rem;
flex-grow: 1;
}
.chat-message-content .error {
color: red;
background: none;
padding: 0;
}
.chat-message-content .message-text {
white-space: pre-wrap;
padding-top: 0.2rem;
}
.message-image-bar {
display: flex;
flex-direction: row;
overflow-x: auto;
}
.message-image {
margin: 0.25rem;
}
.message-image img {
width: 10rem;
height: 10rem;
object-fit: cover;
}
.markdown-body {
display: flex;
width: 100%;
padding: 0;
flex-direction: column;
background-color: var(--bg-primary);
}
.markdown-body:first-child {
margin-top: 0;
padding-top: 0;
}
.markdown-body pre {
overflow-x: auto;
word-wrap: break-word;
}
.code-block {
position: relative;
width: 100%;
}
.message-toolbox {
display: flex;
position: absolute;
bottom: -1.4rem;
}
.copy-message-btn,
.regenerate-message-btn,
.tts-message-btn {
top: 0.7rem;
right: 0.7rem;
cursor: pointer;
font-size: 0.9rem;
padding-right: 4px;
}
.copy-message-btn svg,
.regenerate-message-btn svg,
.tts-message-btn svg {
width: 1rem;
height: 1rem;
}
.copy-code-btn {
position: absolute;
top: 0.7rem;
right: 0.7rem;
cursor: pointer;
font-size: 0.9rem;
}
.copy-code-btn svg {
width: 1rem;
height: 1rem;
}
.scroll-to-bottom-btn {
position: absolute;
text-align: center;
cursor: pointer;
width: 1.5rem;
height: 1.5rem;
right: calc(50vw - 180px);
bottom: 140px;
border-radius: 0.75rem;
background-color: var(--bg-primary);
}
.scroll-to-bottom-btn svg {
width: 1.5rem;
height: 1.5rem;
border-radius: 50%;
}
.input-panel {
position: relative;
border-top: 1px solid var(--border-color);
}
.input-panel-inner {
margin: 1rem;
padding: 0.5rem;
border: 1px solid var(--border-color);
border-radius: 1rem;
}
.input-panel-inner textarea {
width: 100%;
font-size: 1rem;
padding: 0.4rem;
box-sizing: border-box;
border: none;
outline: none;
resize: none;
max-height: 500px;
overflow-x: hidden;
overflow-y: auto;
}
.input-toolbox {
position: absolute;
display: flex;
right: 1.875rem;
font-size: 1rem;
bottom: 1.875rem;
cursor: pointer;
}
.input-toolbox svg {
width: 1.875rem;
height: 1.875rem;
fill: var(--fg-default);
}
.image-btn {
position: relative;
display: inline-block;
margin-right: 0.5rem;
}
.image-btn input[type="file"] {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
opacity: 0;
cursor: pointer;
}
.input-image-bar {
display: flex;
flex-direction: row;
width: 100%;
overflow-x: auto;
}
.input-image-item {
display: flex;
margin: 0.25rem;
width: 5rem;
position: relative;
}
.input-image-item img {
width: 5rem;
height: 5rem;
object-fit: cover;
}
.image-remove-btn {
font-size: 1rem;
margin-left: -0.8rem;
cursor: pointer;
}
.image-remove-btn {
width: 1rem;
height: 1rem;
}
.input-btn.disabled {
opacity: 0.3;
}
.session-list {
padding-top: 0.4rem;
max-height: 80vh;
font-size: 0.8rem;
overflow-y: auto;
overflow-x: hidden;
}
.session-item {
padding: 5px;
border-bottom: 1px solid var(--border-color);
cursor: pointer;
}
.session-item:hover {
background-color: var(--bg-hover);
}
.session-title {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.modal {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background-color: var(--bg-cover);
z-index: 1000;
display: flex;
align-items: flex-start;
justify-content: center;
padding-top: 50px;
}
.modal-content {
position: relative;
padding: 0.8rem;
border-radius: 8px;
max-width: 1000px;
width: calc(100% - 100px);
background-color: var(--bg-primary);
box-shadow: 0 4px 8px rgba(0, 0, 0, 0.2);
}
.modal-header {
display: flex;
flex-direction: row;
align-items: center;
}
.modal-header .title {
font-weight: 500;
font-size: 1.5rem;
}
.modal-header .close-btn {
margin-left: auto;
color: var(--fg-default);
background: none;
border: none;
font-size: 24px;
cursor: pointer;
}
.spinner {
width: 1.1rem;
height: 1.1rem;
margin-top: 3px;
border: 2px solid var(--fg-default);
border-bottom-color: transparent;
border-radius: 50%;
display: inline-block;
animation: spinner-rotation 1s linear infinite;
}
.toast {
display: none;
position: fixed;
bottom: 1rem;
left: 1rem;
text-align: center;
background-color: var(--bg-toast);
color: var(--bg-primary);
padding: 0.5rem;
border-radius: 0.3rem;
z-index: 9999;
}
@keyframes spinner-rotation {
0% {
transform: rotate(0deg);
}
100% {
transform: rotate(360deg);
}
}
@media (prefers-color-scheme: dark) {
:root {
--fg-primary: #1652f1;
--fg-default: white;
--bg-primary: black;
--bg-default: #121212;
--bg-toast: rgba(255, 255, 255, 0.7);
--bg-cover: rgba(255, 255, 255, 0.5);
--bg-hover: #1f1f1f;
--border-color: #3c3c3c;
--shadow-color: rgba(255, 255, 255, 0.1);
}
}
@media screen and (max-width: 768px) {
body {
height: calc(100vh - 56px);
height: 100dvh;
}
.container {
padding: 3px;
}
.sidebar {
display: none;
width: 100%;
height: 100%;
margin-right: 0;
}
.main-panel {
width: 100%;
}
.chat-header {
padding: 0.6rem;
}
.chat-header select {
font-size: 1rem;
}
.chat-body {
padding: 0.6rem;
}
.input-panel-inner {
margin: 0.5rem;
}
.scroll-to-bottom-btn {
right: 50%;
}
.hide-sidebar-btn {
display: block;
}
.show-sidebar-btn {
display: block;
}
}
</style>
</head>
<body>
<div class="container" x-data="app">
<div class="sidebar" x-ref="sidebar">
<div class="sidebar-header">
<div class="sidebar-left">
<div class="title">AIChat</div>
<div class="subtitle">All-in-one AI-Powered Chat</div>
</div>
<div class="sidebar-right">
<div class="sidebar-btn new-chat-btn" title="New Chat (Ctrl/Cmd+Shift+O)" @click="handleNewChat">
<svg fill="currentColor" viewBox="0 0 16 16">
<path d="M8 15A7 7 0 1 1 8 1a7 7 0 0 1 0 14m0 1A8 8 0 1 0 8 0a8 8 0 0 0 0 16" />
<path
d="M8 4a.5.5 0 0 1 .5.5v3h3a.5.5 0 0 1 0 1h-3v3a.5.5 0 0 1-1 0v-3h-3a.5.5 0 0 1 0-1h3v-3A.5.5 0 0 1 8 4" />
</svg>
</div>
<div class="sidebar-btn list-sessions-btn" title="List Sessions (Ctrl/Cmd+Shift+L)"
@click="showModal = 'list-sessions'">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd"
d="M2 2.5a.5.5 0 0 0-.5.5v1a.5.5 0 0 0 .5.5h1a.5.5 0 0 0 .5-.5V3a.5.5 0 0 0-.5-.5zM3 3H2v1h1z" />
<path
d="M5 3.5a.5.5 0 0 1 .5-.5h9a.5.5 0 0 1 0 1h-9a.5.5 0 0 1-.5-.5M5.5 7a.5.5 0 0 0 0 1h9a.5.5 0 0 0 0-1zm0 4a.5.5 0 0 0 0 1h9a.5.5 0 0 0 0-1z" />
<path fill-rule="evenodd"
d="M1.5 7a.5.5 0 0 1 .5-.5h1a.5.5 0 0 1 .5.5v1a.5.5 0 0 1-.5.5H2a.5.5 0 0 1-.5-.5zM2 7h1v1H2zm0 3.5a.5.5 0 0 0-.5.5v1a.5.5 0 0 0 .5.5h1a.5.5 0 0 0 .5-.5v-1a.5.5 0 0 0-.5-.5zm1 .5H2v1h1z" />
</svg>
</div>
<div class="sidebar-btn hide-sidebar-btn" @click="handleHideSidebarBtnClick">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M4.646 4.646a.5.5 0 0 1 .708 0L8 7.293l2.646-2.647a.5.5 0 0 1 .708.708L8.707 8l2.647 2.646a.5.5 0 0 1-.708.708L8 8.707l-2.646 2.647a.5.5 0 0 1-.708-.708L7.293 8 4.646 5.354a.5.5 0 0 1 0-.708" />
</svg>
</div>
</div>
</div>
<div class="settings">
<div class="control">
<label for="role">RAG</label>
<select id="role" x-model="settings.rag" :disabled="sessionMode">
<template x-for="rag in rags">
<option :value="rag" :selected="rag == settings.rag" x-text="rag"></option>
</template>
</select>
</div>
<div class="control">
<label for="role">Role</label>
<select id="role" x-model="settings.role" :disabled="sessionMode">
<template x-for="role in roles">
<option :value="role.name" :selected="role.name == settings.role" x-text="role.name"></option>
</template>
</select>
</div>
<div class="control">
<label for="prompt">System Prompt</label>
<textarea id="prompt" x-model="settings.prompt" :disabled="sessionMode"></textarea>
</div>
<div class="control">
<label for="max_output_tokens"
x-text="'Max Output Tokens' + (modelData.max_output_token ? ' [1, ' + modelData.max_output_token + ']' : '')">Max
Output Tokens</label>
<input type="number" id="max_output_tokens" x-model.number="settings.max_output_tokens">
</div>
<div class="control">
<label for="temperature">Temperature</label>
<input type="number" id="temperature" x-model.number="settings.temperature">
</div>
<div class="control">
<label for="top_p">Top P</label>
<input type="number" id="top_p" x-model.number="settings.top_p">
</div>
</div>
</div>
<div class="main-panel" x-ref="main-panel">
<div class="chat-header">
<select id="model" x-model="settings.model">
<template x-for="model in models" :key="model.id">
<option :value="model.id" :selected="model.id == settings.model" x-text="model.id"></option>
</template>
</select>
<div class="toolbar">
<div class="show-sidebar-btn" @click="handleShowSidebarBtnClick">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M3 9.5a1.5 1.5 0 1 1 0-3 1.5 1.5 0 0 1 0 3m5 0a1.5 1.5 0 1 1 0-3 1.5 1.5 0 0 1 0 3m5 0a1.5 1.5 0 1 1 0-3 1.5 1.5 0 0 1 0 3" />
</svg>
</div>
</div>
</div>
<div class="chat-body" x-ref="chat-body" @scroll="handleScrollChatBody">
<template x-for="(message, index) in messages" :key="message.id">
<div class="chat-message" @mouseover="hoveredMessageIndex = index" @mouseleave="messageHoveredIndex = null">
<div class="chat-avatar" :class="message.role == 'user' ? 'chat-avatar user' : 'chat-avatar assistant'">
<template x-if="message.role == 'user'">
<svg fill="currentColor" viewBox="0 0 16 16">
<path d="M11 6a3 3 0 1 1-6 0 3 3 0 0 1 6 0" />
<path fill-rule="evenodd"
d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8m8-7a7 7 0 0 0-5.468 11.37C3.242 11.226 4.805 10 8 10s4.757 1.225 5.468 2.37A7 7 0 0 0 8 1" />
</svg>
</template>
<template x-if="message.role == 'assistant'">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M6 12.5a.5.5 0 0 1 .5-.5h3a.5.5 0 0 1 0 1h-3a.5.5 0 0 1-.5-.5M3 8.062C3 6.76 4.235 5.765 5.53 5.886a26.6 26.6 0 0 0 4.94 0C11.765 5.765 13 6.76 13 8.062v1.157a.93.93 0 0 1-.765.935c-.845.147-2.34.346-4.235.346s-3.39-.2-4.235-.346A.93.93 0 0 1 3 9.219zm4.542-.827a.25.25 0 0 0-.217.068l-.92.9a25 25 0 0 1-1.871-.183.25.25 0 0 0-.068.495c.55.076 1.232.149 2.02.193a.25.25 0 0 0 .189-.071l.754-.736.847 1.71a.25.25 0 0 0 .404.062l.932-.97a25 25 0 0 0 1.922-.188.25.25 0 0 0-.068-.495c-.538.074-1.207.145-1.98.189a.25.25 0 0 0-.166.076l-.754.785-.842-1.7a.25.25 0 0 0-.182-.135" />
<path
d="M8.5 1.866a1 1 0 1 0-1 0V3h-2A4.5 4.5 0 0 0 1 7.5V8a1 1 0 0 0-1 1v2a1 1 0 0 0 1 1v1a2 2 0 0 0 2 2h10a2 2 0 0 0 2-2v-1a1 1 0 0 0 1-1V9a1 1 0 0 0-1-1v-.5A4.5 4.5 0 0 0 10.5 3h-2zM14 7.5V13a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V7.5A3.5 3.5 0 0 1 5.5 4h5A3.5 3.5 0 0 1 14 7.5" />
</svg>
</template>
</div>
<div class="chat-message-content">
<!-- message -->
<template x-if="message.role == 'assistant' && message.html">
<div class="markdown-body" x-html="message.html"></div>
</template>
<template x-if="message.role == 'assistant' && message.state == 'loading'">
<div class="spinner"></div>
</template>
<template x-if="message.role == 'user' && Array.isArray(message.content)">
<div class="message-text-images">
<template x-if="message.content[0].text">
<div class="message-text" x-text="message.content[0].text"></div>
</template>
<div class="message-image-bar">
<template x-for="part in message.content">
<template x-if="part.type == 'image_url'">
<div class="message-image">
<img :src="part.image_url.url" alt="Image Message Part">
</div>
</template>
</template>
</div>
</div>
</template>
<template
x-if="message.role == 'user' && Object.prototype.toString.call(message.content) == '[object String]'">
<div class="message-text" x-text="message.content"></div>
</template>
<!-- toolbox -->
<template x-if="index == hoveredMessageIndex">
<div class="message-toolbox">
<div class="copy-message-btn" @click="handleCopyMessage(message.content)" title=" Copy">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd"
d="M4 2a2 2 0 0 1 2-2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2zm2-1a1 1 0 0 0-1 1v8a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1V2a1 1 0 0 0-1-1zM2 5a1 1 0 0 0-1 1v8a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1v-1h1v1a2 2 0 0 1-2 2H2a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h1v1z" />
</svg>
</div>
<template
x-if="index == messages.length - 1 && (message.state == 'succeed' || message.state == 'failed')">
<div class="regenerate-message-btn" @click="handleRegenerateMessage" title="Regenerate">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd" d="M8 3a5 5 0 1 0 4.546 2.914.5.5 0 0 1 .908-.417A6 6 0 1 1 8 2z" />
<path
d="M8 4.466V.534a.25.25 0 0 1 .41-.192l2.36 1.966c.12.1.12.284 0 .384L8.41 4.658A.25.25 0 0 1 8 4.466" />
</svg>
</div>
</template>
<template x-if="message.state == 'succeed' && !!window.speechSynthesis">
<div class="tts-message-btn" @click="handleTTSMessage(message.content)" title="Text to speech">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M11.536 14.01A8.47 8.47 0 0 0 14.026 8a8.47 8.47 0 0 0-2.49-6.01l-.708.707A7.48 7.48 0 0 1 13.025 8c0 2.071-.84 3.946-2.197 5.303z" />
<path
d="M10.121 12.596A6.48 6.48 0 0 0 12.025 8a6.48 6.48 0 0 0-1.904-4.596l-.707.707A5.48 5.48 0 0 1 11.025 8a5.48 5.48 0 0 1-1.61 3.89z" />
<path
d="M10.025 8a4.5 4.5 0 0 1-1.318 3.182L8 10.475A3.5 3.5 0 0 0 9.025 8c0-.966-.392-1.841-1.025-2.475l.707-.707A4.5 4.5 0 0 1 10.025 8M7 4a.5.5 0 0 0-.812-.39L3.825 5.5H1.5A.5.5 0 0 0 1 6v4a.5.5 0 0 0 .5.5h2.325l2.363 1.89A.5.5 0 0 0 7 12zM4.312 6.39 6 5.04v5.92L4.312 9.61A.5.5 0 0 0 4 9.5H2v-3h2a.5.5 0 0 0 .312-.11" />
</svg>
</div>
</template>
</div>
</template>
</div>
</div>
</template>
</div>
<div class="scroll-to-bottom-btn" x-cloak x-show="isShowScrollToBottomBtn" @click="handleScrollToBottom">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd"
d="M1 8a7 7 0 1 0 14 0A7 7 0 0 0 1 8m15 0A8 8 0 1 1 0 8a8 8 0 0 1 16 0M8.5 4.5a.5.5 0 0 0-1 0v5.793L5.354 8.146a.5.5 0 1 0-.708.708l3 3a.5.5 0 0 0 .708 0l3-3a.5.5 0 0 0-.708-.708L8.5 10.293z" />
</svg>
</div>
<div class="input-panel">
<div class="input-panel-inner">
<textarea id="chat-input" x-model="input" x-ref="input" @keydown.enter="handleEnterKeyDown"
placeholder="Ask Anything" autofocus></textarea>
<div class="input-image-bar" x-show="images.length > 0">
<template x-for="(image, index) in images">
<div class="input-image-item">
<img :src="image" alt="Preview image">
<div class="image-remove-btn" @click="images.splice(index, 1);">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5m2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5m3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0z" />
<path
d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4zM2.5 3h11V2h-11z" />
</svg>
</div>
</div>
</template>
</div>
<template x-if="asking">
<div class="input-toolbox">
<div class="input-btn" @click="handleCancelAsk">
<svg fill="currentColor" viewBox="0 0 16 16">
<path d="M8 15A7 7 0 1 1 8 1a7 7 0 0 1 0 14m0 1A8 8 0 1 0 8 0a8 8 0 0 0 0 16" />
<path
d="M5 6.5A1.5 1.5 0 0 1 6.5 5h3A1.5 1.5 0 0 1 11 6.5v3A1.5 1.5 0 0 1 9.5 11h-3A1.5 1.5 0 0 1 5 9.5z" />
</svg>
</div>
</div>
</template>
<template x-if="!asking">
<div class="input-toolbox">
<div class="image-btn" x-show="modelData.supports_vision">
<input type="file" multiple accept=".jpg,.jpeg,.png,.webp" @change="handleImageUpload">
<svg fill="currentColor" viewBox="0 0 16 16">
<path d="M6.002 5.5a1.5 1.5 0 1 1-3 0 1.5 1.5 0 0 1 3 0" />
<path
d="M2.002 1a2 2 0 0 0-2 2v10a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V3a2 2 0 0 0-2-2zm12 1a1 1 0 0 1 1 1v6.5l-3.777-1.947a.5.5 0 0 0-.577.093l-3.71 3.71-2.66-1.772a.5.5 0 0 0-.63.062L1.002 12V3a1 1 0 0 1 1-1z" />
</svg>
</div>
<div class="input-btn" :class="(input.trim() || images.length > 0) ? 'input-btn' : 'input-btn disabled'"
@click="handleAsk">
<svg fill="currentColor" viewBox="0 0 16 16">
<path
d="M2 16a2 2 0 0 1-2-2V2a2 2 0 0 1 2-2h12a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2zm6.5-4.5V5.707l2.146 2.147a.5.5 0 0 0 .708-.708l-3-3a.5.5 0 0 0-.708 0l-3 3a.5.5 0 1 0 .708.708L7.5 5.707V11.5a.5.5 0 0 0 1 0" />
</svg>
</div>
</div>
</template>
</div>
</div>
</div>
<div class="modal" x-cloak x-show="showModal == 'list-sessions'"
@click="if ($event.target == $el) { showModal = ''}">
<div class="modal-content">
<div class="modal-header">
<div class="title">Sessions</div>
<button class="close-btn" @click="showModal = ''">×</button>
</div>
<div class="session-list">
<template x-for="session in sessions" :key="session.id">
<div class="session-item" @click="handleSelectSession(session.id)">
<div class="session-title" x-text="session.sessionTitle"></div>
</div>
</template>
</div>
</div>
</div>
<div id="toast" class="toast"></div>
</div>
<script>
const QUERY = parseQueryString();
const API_BASE = QUERY.api_base || "./v1";
const API_KEY = QUERY.api_key || "";
const CHAT_COMPLETIONS_URL = API_BASE + "/chat/completions";
const MODELS_API = API_BASE + "/models";
const ROLES_API = API_BASE + "/roles";
const RAGS_API = API_BASE + "/rags";
const SEARCH_RAG_API = API_BASE + "/rags/search";
document.addEventListener("alpine:init", () => {
setupMarked();
setupApp();
});
function setupApp() {
let msgIdx = 0;
let defaultSettings = {
model: QUERY.model || "default",
rag: QUERY.rag || "",
role: QUERY.role || "",
prompt: "",
max_output_tokens: parseInt(QUERY.max_output_tokens) || null,
temperature: QUERY.temperature ? parseFloat(QUERY.temperature) : null,
top_p: QUERY.top_p ? parseFloat(QUERY.top_p) : null,
};
Alpine.data("app", () => ({
models: [],
rags: [""],
roles: [{ name: "", prompt: "" }],
settings: defaultSettings,
modelData: {},
messages: [],
input: "",
images: [],
asking: false,
askAbortController: null,
hoveredMessageIndex: null,
shouldScrollChatBodyToBottom: true,
isShowScrollToBottomBtn: false,
showModal: "",
sessionMode: false,
sessionTitle: "",
selectSessionId: null,
sessions: [],
async init() {
await Promise.all([
fetchJSON(MODELS_API).then(models => {
this.models = models.filter(v => !v.type || v.type === "chat");
}).catch(err => {
toast("No model available");
console.error("Failed to load models", err);
}),
fetchJSON(RAGS_API).then(rags => {
this.rags.push(...rags);
}).catch(() => { }),
fetchJSON(ROLES_API).then(roles => {
this.roles.push(...roles.filter(v => !!v.prompt));
}).catch(() => { }),
])
this.$refs.input.addEventListener("paste", (e) => this.handlePaste(e));
this.$watch("input", () => this.autosizeInput(this.$refs.input));
this.$watch("settings", () => this.updateUrl());
this.$watch("settings.model", () => this.handleModelChange());
if (this.models.find(model => model.id === this.settings.model)) {
this.handleModelChange();
} else {
this.settings.model = "default";
}
if (!this.rags.find(rag => rag === this.settings.rag)) {
this.settings.rag = "";
}
this.$watch("settings.role", () => this.handleRoleChange())
if (this.roles.find(role => role.name === this.settings.role)) {
this.handleRoleChange();
} else {
this.settings.role = "";
}
document.addEventListener("keydown", (event) => this.handleKeyDown(event))
},
handleAsk() {
const isEmptyInput = this.input.trim() === "";
const isEmptyImage = this.images.length === 0;
if (this.asking || (isEmptyImage && isEmptyInput)) {
return;
}
if (this.messages.length === 0) {
let sessionTitle = ""
if (this.images.length > 0) {
sessionTitle = `🖼️x${this.images.length} `
}
if (this.input) {
sessionTitle += this.input.trim().replace(/\n/g, "↵").slice(0, 200);
}
this.sessionTitle = sessionTitle;
}
if (isEmptyImage) {
this.messages.push({
id: msgIdx++,
role: "user",
content: this.input,
});
} else {
const parts = [];
if (!isEmptyInput) {
parts.push({ type: "text", text: this.input });
}
for (const image of this.images) {
parts.push({ type: "image_url", image_url: { url: image } });
}
this.messages.push({
id: msgIdx++,
role: "user",
content: parts,
})
}
this.messages.push({
id: msgIdx++,
role: "assistant",
content: "",
state: "loading", // streaming, succeed, failed
error: "",
html: "",
});
this.input = "";
this.asking = true;
this.images = [];
this.ask();
},
handleRegenerateMessage() {
const lastIndex = this.messages.length - 1;
if (lastIndex !== this.hoveredMessageIndex) {
return
}
let lastMessage = this.messages[lastIndex];
lastMessage.content = "";
lastMessage.state = "loading";
lastMessage.error = "";
lastMessage.html = "";
this.asking = true;
this.ask();
},
/**
* @param {string} messageToUtter
*/
handleTTSMessage(messageToUtter) {
if (!!window.speechSynthesis) {
if (window.speechSynthesis.speaking || window.speechSynthesis.pending) {
window.speechSynthesis.cancel();
} else {
let utterance = new SpeechSynthesisUtterance(messageToUtter);
window.speechSynthesis.speak(utterance);
}
}
},
handleCancelAsk() {
this.askAbortController?.abort();
},
handleModelChange() {
this.modelData = retrieveModel(this.models, this.settings.model);
},
handleRoleChange() {
if (this.settings.prompt && !this.settings.role) {
return;
}
this.settings.prompt = this.roles.find(role => role.name === this.settings.role).prompt;
},
handleScrollChatBody(event) {
const $chatBody = event.target;
const { scrollTop, clientHeight, scrollHeight, _prevScrollTop = 0 } = $chatBody;
if (scrollTop + clientHeight > scrollHeight - 5) {
this.isShowScrollToBottomBtn = false;
this.shouldScrollChatBodyToBottom = true;
}
if (scrollHeight > clientHeight && _prevScrollTop > 1 && _prevScrollTop > scrollTop + 1) {
this.shouldScrollChatBodyToBottom = false;
this.isShowScrollToBottomBtn = true;
}
$chatBody._prevScrollTop = scrollTop;
},
handleScrollToBottom() {
const $chatBody = this.$refs["chat-body"];
$chatBody.scrollTop = $chatBody.scrollHeight;
this.isShowScrollToBottomBtn = false;
this.shouldScrollChatBodyToBottom = true;
},
handleShowSidebarBtnClick() {
this.$refs.sidebar.style.display = 'block';
this.$refs["main-panel"]._display = this.$refs["main-panel"].style.display;
this.$refs["main-panel"].style.display = "none";
},
handleHideSidebarBtnClick() {
this.$refs.sidebar.style.display = 'none';
this.$refs["main-panel"].style.display = this.$refs["main-panel"]._display;
},
handleEnterKeyDown(event) {
if (event.shiftKey) {
return;
}
event.preventDefault();
this.handleAsk();
},
handleCopyCode(event) {
const $btn = event.target;
const $code = $btn.closest('.code-block').querySelector("code");
if ($code) {
const range = document.createRange();
range.selectNodeContents($code);
window.getSelection().removeAllRanges();
window.getSelection().addRange(range);
document.execCommand('copy');
window.getSelection().removeAllRanges();
toast("Copied Code");
}
},
handleCopyMessage(content) {
if (Array.isArray(content)) {
content = content.map(v => v.text || "").join("");
}
const $tempTextArea = document.createElement("textarea");
$tempTextArea.value = content;
document.body.appendChild($tempTextArea);
$tempTextArea.select();
$tempTextArea.setSelectionRange(0, 99999); // For mobile devices
document.execCommand("copy");
document.body.removeChild($tempTextArea);
toast("Copied Message")
},
async handleImageUpload(event) {
const files = event.target.files;
if (!files || files.length === 0) {
return;
}
const urls = await Promise.all(Array.from(files).map(file => convertImageToDataURL(file)));
this.images.push(...urls);
event.target.value = "";
},
async handlePaste(event) {
const files = Array.from(event.clipboardData.items).filter(v => v.type.startsWith('image/')).map(v => v.getAsFile());
const urls = await Promise.all(files.map(file => convertImageToDataURL(file)));
this.images.push(...urls);
},
handleKeyDown(event) {
const isMac = navigator.platform.toUpperCase().indexOf('MAC') > -1;
const controlKey = isMac ? event.metaKey : event.ctrlKey;
if (controlKey && event.shiftKey && event.key.toLowerCase() === 'o') {
event.preventDefault();
this.handleNewChat();
} else if (controlKey && event.shiftKey && event.key.toLowerCase() === 'l') {
event.preventDefault();
this.showModal = 'list-sessions'
} else if (event.shiftKey && event.key === "Escape") {
event.preventDefault();
this.focusInput();
} else if (this.showModal && event.key === "Escape") {
event.preventDefault();
this.showModal = "";
}
},
handleNewChat() {
if (this.asking) {
this.askAbortController?.abort();
}
if (this.sessionTitle) {
const lastMessage = this.messages[this.messages.length - 1];
if (lastMessage.state === "loading") {
lastMessage.state = "failed";
lastMessage.error = "Error: Aborted";
lastMessage.html = renderMarkdown(lastMessage.content, lastMessage.error);
}
const sessionData = JSON.parse(JSON.stringify({
settings: this.settings,
messages: this.messages,
sessionMode: this.sessionMode,
sessionTitle: this.sessionTitle,
}));
let session = this.sessions.find(v => v.id === this.selectSessionId);
if (session) {
Object.assign(session, sessionData);
} else {
this.sessions.unshift({
id: randomUUID(),
createdAt: Date.now(),
...sessionData,
});
}
}
this.messages = [];
this.asking = false;
this.askAbortController = null;
this.hoveredMessageIndex = null;
this.shouldScrollChatBodyToBottom = true;
this.isShowScrollToBottomBtn = false;
this.showModal = "";
this.sessionMode = false;
this.sessionTitle = "";
this.selectSessionId = null;
this.focusInput();
},
handleSelectSession(id) {
const session = this.sessions.find(v => v.id === id);
if (!session || id === this.selectSessionId) {
this.showModal = "";
this.focusInput();
return;
}
this.handleNewChat();
this.settings = session.settings;
this.messages = session.messages;
this.sessionMode = session.sessionMode;
this.sessionTitle = session.sessionTitle;
this.selectSessionId = session.id;
},
updateUrl() {
const newUrl = new URL(location.href);
["model", "rag", "role", "max_output_tokens", "temperature", "top_p"].forEach(key => {
if (this.settings[key] || typeof this.settings[key] === "number") {
newUrl.searchParams.set(key, this.settings[key]);
} else {
newUrl.searchParams.delete(key);
}
});
history.replaceState(null, '', newUrl.toString());
},
autoScrollChatBodyToBottom() {
if (this.shouldScrollChatBodyToBottom) {
let $chatBody = this.$refs["chat-body"];
if (!$chatBody) {
$chatBody = document.querySelector('[x-ref="chat-body"]')
}
$chatBody.scrollTop = $chatBody.scrollHeight;
}
},
autosizeInput($input) {
$input.style.height = 'auto';
$input.style.height = $input.scrollHeight + 'px';
},
focusInput() {
this.$refs?.input?.focus();
},
async ask() {
this.askAbortController = new AbortController();
this.shouldScrollChatBodyToBottom = true;
this.$nextTick(() => {
this.autoScrollChatBodyToBottom();
});
const lastMessage = this.messages[this.messages.length - 1];
const body = this.buildBody();
let succeed = false;
try {
if (this.settings.rag) {
const message = body.messages[body.messages.length - 1];
if (message.role === "user" && typeof message.content === "string") {
message.content = await this.searchRag(this.settings.rag, message.content);
}
}
const stream = await fetchChatCompletions(CHAT_COMPLETIONS_URL, body, this.askAbortController.signal)
for await (const chunk of stream) {
lastMessage.state = "streaming";
lastMessage.content += chunk?.choices[0]?.delta?.content || "";
lastMessage.html = renderMarkdown(lastMessage.content, lastMessage.error);
this.$nextTick(() => {
this.autoScrollChatBodyToBottom();
});
}
lastMessage.state = "succeed";
succeed = true;
} catch (err) {
lastMessage.state = "failed";
if (this.askAbortController?.signal?.aborted) {
lastMessage.error = "Error: Aborted";
} else {
lastMessage.error = err?.message || err;
}
lastMessage.html = renderMarkdown(lastMessage.content, lastMessage.error);
}
if (succeed) {
this.sessionMode = true;
}
this.asking = false;
},
async searchRag(name, input) {
const res = await fetch(SEARCH_RAG_API, {
method: "POST",
headers: getHeaders(),
signal: this.askAbortController.signal,
body: JSON.stringify({
name,
input
})
});
const data = await res.json();
return data.data;
},
buildBody() {
let messages = [];
for ([userMessage, assistantMessage] of chunkArray(this.messages, 2)) {
if (assistantMessage.state === "failed") {
continue;
} else if (assistantMessage.state === "loading") {
messages.push({
role: userMessage.role,
content: userMessage.content,
});
} else {
messages.push({
role: userMessage.role,
content: userMessage.content,
});
messages.push({
role: assistantMessage.role,
content: assistantMessage.content,
});
}
}
const systemPrompt = this.settings.prompt.trim();
if (systemPrompt) {
if (messages[0]?.content?.indexOf("__INPUT__") > -1) {
messages[0].content = systemPrompt.replace("__INPUT__", messages[0].content);
} else {
const { system, cases } = parseStructurePrompt(systemPrompt);
const promptMessages = [];
if (system) {
promptMessages.push({
role: "system",
content: system,
});
}
for (const item of cases) {
promptMessages.push({
role: "user",
content: item.input,
});
promptMessages.push({
role: "assistant",
content: item.output,
});
}
messages = [...promptMessages, ...messages];
}
}
sanitizeMessages(messages);
const body = {
model: this.settings.model,
messages: messages,
stream: true,
};
[["max_output_tokens", "max_tokens"], ["temperature"], ["top_p"]].forEach(([setting_key, body_key]) => {
if (typeof this.settings[setting_key] === "number") {
body[body_key || setting_key] = this.settings[setting_key];
}
});
const { max_output_token, require_max_tokens } = this.modelData;
if (!body["max_tokens"] && require_max_tokens) {
body["max_tokens"] = max_output_token;
};
return body;
},
}));
}
async function fetchJSON(url) {
const res = await fetch(url, { headers: getHeaders() });
const data = await res.json()
return data.data;
}
async function* fetchChatCompletions(url, body, signal) {
const stream = body.stream;
const response = await fetch(url, {
method: "POST",
signal,
headers: getHeaders(),
body: JSON.stringify(body),
});
if (!response.ok) {
const error = await response.json();
throw error?.error || error;
}
if (!stream) {
const data = await response.json();
return data;
}
const reader = response.body.getReader();
const decoder = new TextDecoder();
let done = false;
let reamingChunkValue = "";
while (!done) {
if (signal?.aborted) {
reader.cancel();
break;
}
const { value, done: doneReading } = await reader.read();
done = doneReading;
const chunkValue = decoder.decode(value);
const lines = (reamingChunkValue + chunkValue).split("\n").filter(line => line.trim().length > 0);
reamingChunkValue = "";
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
const message = line.replace(/^data: /, "");
if (message === "[DONE]") {
continue
}
try {
const parsed = JSON.parse(message);
yield parsed;
} catch {
if (i === lines.length - 1) {
reamingChunkValue += line;
break;
}
}
}
}
}
function getHeaders() {
const headers = {
"content-type": "application/json",
};
if (API_KEY) {
headers["authorization"] = `Bearer ${API_KEY}`;
}
return headers
}
function retrieveModel(models, id) {
const model = models.find(model => model.id === id);
if (!model) return {};
const max_output_token = model.max_output_tokens;
const supports_vision = !!model.supports_vision;
const require_max_tokens = !!model.require_max_tokens;
return {
id,
max_output_token,
supports_vision,
require_max_tokens,
}
}
function toast(text, duration = 2500) {
const $toast = document.getElementById("toast");
clearTimeout($toast._timer);
$toast.textContent = text;
$toast.style.display = "block";
$toast._timer = setTimeout(function () {
$toast.style.display = "none";
}, duration);
}
function parseStructurePrompt(prompt) {
let text = prompt;
let searchInput = true;
let system = null;
let parts = [];
while (text) {
const search = searchInput ? "### INPUT:" : "### OUTPUT:";
const index = text.indexOf(search);
if (index !== -1) {
if (system === null) {
system = text.slice(0, index);
} else {
parts.push(text.slice(0, index));
}
searchInput = !searchInput;
text = text.slice(index + search.length);
} else {
if (text.trim()) {
if (system === null) {
system = text;
} else {
parts.push(text);
}
}
break;
}
}
const partsLength = parts.length;
if (partsLength > 0 && partsLength % 2 === 0) {
const cases = parts.reduce((acc, val, idx) => {
if (idx % 2 === 0) {
acc.push({ input: val.trim() })
} else {
acc[acc.length - 1].output = val.trim();
}
return acc;
}, []);
system = system ? system.trim() : "";
return { system, cases }
}
return { system: prompt, cases: [] }
}
function sanitizeMessages(messages) {
let messagesLen = messages.length;
for (let i = 0; i < messagesLen; i++) {
const message = messages[i];
if (typeof message.content === "string" && message.role === "assistant" && i !== messagesLen - 1) {
message.content = stripThinkTag(message.content);
}
}
}
function stripThinkTag(text) {
return text.replace(/^\s*<think>([\s\S]*?)<\/think>(\s*|$)/g, '')
}
function convertImageToDataURL(imageFile) {
return new Promise((resolve, reject) => {
if (!imageFile) {
reject(new Error("Please select an image file."));
return;
}
const reader = new FileReader();
reader.readAsDataURL(imageFile);
reader.onload = (event) => resolve(event.target.result);
reader.onerror = (error) => reject(error);
});
}
function setupMarked() {
const renderer = {
code({ text, lang }) {
const validLang = !!(lang && hljs.getLanguage(lang));
const highlighted = validLang
? hljs.highlight(text, { language: lang }).value
: escapeForHTML(text);
return `<div class="code-block">
<pre><code class="hljs ${lang}">${highlighted}</code></pre>
<div class="copy-code-btn" @click="handleCopyCode" title="Copy code">
<svg fill="currentColor" viewBox="0 0 16 16">
<path fill-rule="evenodd" d="M4 2a2 2 0 0 1 2-2h8a2 2 0 0 1 2 2v8a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2zm2-1a1 1 0 0 0-1 1v8a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1V2a1 1 0 0 0-1-1zM2 5a1 1 0 0 0-1 1v8a1 1 0 0 0 1 1h8a1 1 0 0 0 1-1v-1h1v1a2 2 0 0 1-2 2H2a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h1v1z"/>
</svg>
</div>
</div>`;
}
};
const thinkExtension = {
name: 'think',
level: 'block',
start(src) {
const match = /^(\s*)<think>/.exec(src);
if (match) {
return match[1].length
} else {
return -1;
}
},
tokenizer(src, tokens) {
const rule = /^\s*<think>([\s\S]*?)(<\/think>|$)/;
const match = rule.exec(src);
if (match) {
return {
type: 'think',
raw: match[0],
text: match[1].trim(),
};
}
},
renderer(token) {
const text = '<p>' + token.text.trim().replace(/\n+/g, '</p><p>') + '</p>';
return `<details open class="think">
<summary>Deeply thought</summary>
<blockquote>${text}</blockquote>
</details>`;
},
};
marked.use({ renderer });
marked.use(markedKatex({ throwOnError: false, inlineTolerantNoSpace: true }));
marked.use({ extensions: [thinkExtension] })
}
function escapeForHTML(input) {
const escapeMap = {
"&": "&",
"<": "<",
">": ">",
'"': """,
"'": "'"
};
return input.replace(/([&<>'"])/g, char => escapeMap[char]);
}
function parseQueryString() {
const params = new URLSearchParams(location.search);
const queryObject = {};
params.forEach((value, key) => {
queryObject[key] = value;
});
return queryObject;
}
function chunkArray(array, chunkSize) {
const chunks = [];
for (let i = 0; i < array.length; i += chunkSize) {
chunks.push(array.slice(i, i + chunkSize));
}
return chunks;
}
function randomUUID() {
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function (c) {
const r = Math.random() * 16 | 0;
const v = c === 'x' ? r : (r & 0x3 | 0x8);
return v.toString(16);
});
}
function renderMarkdown(text, error = '') {
return marked.marked(text) + (error ? `<pre class="error">${error}</pre>` : '');
}
</script>
</body>
</html>
================================================
FILE: assets/roles/%code%.md
================================================
Provide only code without comments or explanations.
### INPUT:
async sleep in js
### OUTPUT:
```javascript
async function timeout(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
```
================================================
FILE: assets/roles/%create-prompt%.md
================================================
As a professional Prompt Engineer, your role is to create effective and innovative prompts for interacting with AI models.
Your core skills include:
1. **CO-STAR Framework Application**: Utilize the CO-STAR framework to build efficient prompts, ensuring effective communication with large language models.
2. **Contextual Awareness**: Construct prompts that adapt to complex conversation contexts, ensuring relevant and coherent responses.
3. **Chain-of-Thought Prompting**: Create prompts that elicit AI models to demonstrate their reasoning process, enhancing the transparency and accuracy of answers.
4. **Zero-shot Learning**: Design prompts that enable AI models to perform specific tasks without requiring examples, reducing dependence on training data.
5. **Few-shot Learning**: Guide AI models to quickly learn and execute new tasks through a few examples.
Your output format should include:
- **Context**: Provide comprehensive background information for the task to ensure the AI understands the specific scenario and offers relevant feedback.
- **Objective**: Clearly define the task objective, guiding the AI to focus on achieving specific goals.
- **Style**: Specify writing styles according to requirements, such as imitating a particular person or industry expert.
- **Tone**: Set an appropriate emotional tone to ensure the AI's response aligns with the expected emotional context.
- **Audience**: Tailor AI responses for a specific audience, ensuring content appropriateness and ease of understanding.
- **Response**: Specify output formats for easy execution of downstream tasks, such as lists, JSON, or professional reports.
- **Workflow**: Instruct the AI on how to step-by-step complete tasks, clarifying inputs, outputs, and specific actions for each step.
- **Examples**: Show a case of input and output that fits the scenario.
Your workflow should be:
1. Extract key information from user requests to determine design objectives.
2. Based on user needs, create prompts that meet requirements, with each part being professional and detailed.
3. Must only output the newly generated and optimized prompts, without explanation, without wrapping it in markdown code block.
My first request is: __INPUT__
================================================
FILE: assets/roles/%create-title%.md
================================================
Create a concise, 3-6 word title.
**Notes**:
- Avoid quotation marks or emojis
- RESPOND ONLY WITH TITLE SLUG TEXT
**Examples**:
stock-market-trends
perfect-chocolate-chip-recipe
remote-work-productivity-tips
video-game-development-insights
================================================
FILE: assets/roles/%explain-shell%.md
================================================
Provide a terse, single sentence description of the given shell command.
Describe each argument and option of the command.
Provide short responses in about 80 words.
APPLY MARKDOWN formatting when possible.
================================================
FILE: assets/roles/%functions%.md
================================================
---
use_tools: all
---
================================================
FILE: assets/roles/%shell%.md
================================================
Provide only {{__shell__}} commands for {{__os_distro__}} without any description.
Ensure the output is a valid {{__shell__}} command.
If there is a lack of details, provide most logical solution.
If multiple steps are required, try to combine them using '&&' (For PowerShell, use ';' instead).
Output only plain text without any markdown formatting.
================================================
FILE: config.agent.example.yaml
================================================
# Agent-specific configuration
# Location `<aichat-config-dir>/agents/<agent-name>/config.yaml`
model: openai:gpt-4o # Specify the LLM to use
temperature: null # Set default temperature parameter, range (0, 1)
top_p: null # Set default top-p parameter, with a range of (0, 1) or (0, 2) depending on the model
use_tools: null # Which additional tools to use by agent. (e.g. 'fs,web_search')
agent_prelude: null # Set a session to use when starting the agent. (e.g. temp, default)
instructions: null # Override the instructions for the agent, have no effect for dynamic instructions
variables: # Custom default values for the agent variables
<key>: <value>
================================================
FILE: config.example.yaml
================================================
# ---- llm ----
model: openai:gpt-4o # Specify the LLM to use
temperature: null # Set default temperature parameter (0, 1)
top_p: null # Set default top-p parameter, with a range of (0, 1) or (0, 2) depending on the model
# ---- behavior ----
stream: true # Controls whether to use the stream-style API.
save: true # Indicates whether to persist the message
keybindings: emacs # Choose keybinding style (emacs, vi)
editor: null # Specifies the command used to edit input buffer or session. (e.g. vim, emacs, nano).
wrap: no # Controls text wrapping (no, auto, <max-width>)
wrap_code: false # Enables or disables wrapping of code blocks
# ---- function-calling ----
# Visit https://github.com/sigoden/llm-functions for setup instructions
function_calling: true # Enables or disables function calling (Globally).
mapping_tools: # Alias for a tool or toolset
fs: 'fs_cat,fs_ls,fs_mkdir,fs_rm,fs_write'
use_tools: null # Which tools to use by default. (e.g. 'fs,web_search')
# ---- prelude ----
repl_prelude: null # Set a default role or session for REPL mode (e.g. role:<name>, session:<name>, <session>:<role>)
cmd_prelude: null # Set a default role or session for CMD mode (e.g. role:<name>, session:<name>, <session>:<role>)
agent_prelude: null # Set a session to use when starting a agent (e.g. temp, default)
# ---- session ----
# Controls the persistence of the session. if true, auto save; if false, not save; if null, asking the user
save_session: null
# Compress session when token count reaches or exceeds this threshold
compress_threshold: 4000
# Text prompt used for creating a concise summary of session message
summarize_prompt: 'Summarize the discussion briefly in 200 words or less to use as a prompt for future context.'
# Text prompt used for including the summary of the entire session
summary_prompt: 'This is a summary of the chat history as a recap: '
# ---- RAG ----
# See [RAG-Guide](https://github.com/sigoden/aichat/wiki/RAG-Guide) for more details.
rag_embedding_model: null # Specifies the embedding model used for context retrieval
rag_reranker_model: null # Specifies the reranker model used for sorting retrieved documents
rag_top_k: 5 # Specifies the number of documents to retrieve for answering queries
rag_chunk_size: null # Defines the size of chunks for document processing in characters
rag_chunk_overlap: null # Defines the overlap between chunks
# Defines the query structure using variables like __CONTEXT__ and __INPUT__ to tailor searches to specific needs
rag_template: |
Answer the query based on the context while respecting the rules. (user query, some textual context and rules, all inside xml tags)
<context>
__CONTEXT__
</context>
<rules>
- If you don't know, just say so.
- If you are not sure, ask for clarification.
- Answer in the same language as the user query.
- If the context appears unreadable or of poor quality, tell the user then answer as best as you can.
- If the answer is not in the context but you think you know the answer, explain that to the user then answer with your own knowledge.
- Answer directly and without using xml tags.
</rules>
<user_query>
__INPUT__
</user_query>
# Define document loaders to control how RAG and `.file`/`--file` load files of specific formats.
document_loaders:
# You can add custom loaders using the following syntax:
# <file-extension>: <command-to-load-the-file>
# Note: Use `$1` for input file and `$2` for output file. If `$2` is omitted, use stdout as output.
pdf: 'pdftotext $1 -' # Load .pdf file, see https://poppler.freedesktop.org to set up pdftotext
docx: 'pandoc --to plain $1' # Load .docx file, see https://pandoc.org to set up pandoc
# ---- apperence ----
highlight: true # Controls syntax highlighting
light_theme: false # Activates a light color theme when true. env: AICHAT_LIGHT_THEME
# Custom REPL left/right prompts, see https://github.com/sigoden/aichat/wiki/Custom-REPL-Prompt for more details
left_prompt:
'{color.green}{?session {?agent {agent}>}{session}{?role /}}{!session {?agent {agent}>}}{role}{?rag @{rag}}{color.cyan}{?session )}{!session >}{color.reset} '
right_prompt:
'{color.purple}{?session {?consume_tokens {consume_tokens}({consume_percent}%)}{!consume_tokens {consume_tokens}}}{color.reset}'
# ---- misc ----
serve_addr: 127.0.0.1:8000 # Server listening address
user_agent: null # Set User-Agent HTTP header, use `auto` for aichat/<current-version>
save_shell_history: true # Whether to save shell execution command to the history file
# URL to sync model changes from, e.g., https://cdn.jsdelivr.net/gh/sigoden/aichat@main/models.yaml
sync_models_url: https://raw.githubusercontent.com/sigoden/aichat/refs/heads/main/models.yaml
# ---- clients ----
clients:
# All clients have the following configuration:
# - type: xxxx
# name: xxxx # Only use it to distinguish clients with the same client type. Optional
# models:
# - name: xxxx # Chat model
# max_input_tokens: 100000
# supports_vision: true
# supports_function_calling: true
# - name: xxxx # Embedding model
# type: embedding
# default_chunk_size: 1500
# max_batch_size: 100
# - name: xxxx # Reranker model
# type: reranker
# patch: # Patch api
# chat_completions: # Api type, possible values: chat_completions, embeddings, and rerank
# <regex>: # The regex to match model names, e.g. '.*' 'gpt-4o' 'gpt-4o|gpt-4-.*'
# url: '' # Patch request url
# body: # Patch request body
# <json>
# headers: # Patch request headers
# <key>: <value>
# extra:
# proxy: socks5://127.0.0.1:1080 # Set proxy
# connect_timeout: 10 # Set timeout in seconds for connect to api
# See https://platform.openai.com/docs/quickstart
- type: openai
api_base: https://api.openai.com/v1 # Optional
api_key: xxx
organization_id: org-xxx # Optional
# For any platform compatible with OpenAI's API
- type: openai-compatible
name: ollama
api_base: http://localhost:11434/v1
api_key: xxx # Optional
models:
- name: deepseek-r1
max_input_tokens: 131072
- name: llama3.1
max_input_tokens: 128000
supports_function_calling: true
- name: llama3.2-vision
max_input_tokens: 131072
supports_vision: true
- name: nomic-embed-text
type: embedding
default_chunk_size: 1000
max_batch_size: 50
# See https://ai.google.dev/docs
- type: gemini
api_base: https://generativelanguage.googleapis.com/v1beta
api_key: xxx
patch:
chat_completions:
'.*':
body:
safetySettings:
- category: HARM_CATEGORY_HARASSMENT
threshold: BLOCK_NONE
- category: HARM_CATEGORY_HATE_SPEECH
threshold: BLOCK_NONE
- category: HARM_CATEGORY_SEXUALLY_EXPLICIT
threshold: BLOCK_NONE
- category: HARM_CATEGORY_DANGEROUS_CONTENT
threshold: BLOCK_NONE
# See https://docs.anthropic.com/claude/reference/getting-started-with-the-api
- type: claude
api_base: https://api.anthropic.com/v1 # Optional
api_key: xxx
# See https://docs.mistral.ai/
- type: openai-compatible
name: mistral
api_base: https://api.mistral.ai/v1
api_key: xxx
# See https://docs.x.ai/docs
- type: openai-compatible
name: xai
api_base: https://api.x.ai/v1
api_key: xxx
# See https://docs.ai21.com/docs/quickstart
- type: openai-compatible
name: ai12
api_base: https://api.ai21.com/studio/v1
api_key: xxx
# See https://docs.cohere.com/docs/the-cohere-platform
- type: cohere
api_base: https://api.cohere.ai/v2 # Optional
api_key: xxx
# See https://docs.perplexity.ai/docs/getting-started
- type: openai-compatible
name: perplexity
api_base: https://api.perplexity.ai
api_key: xxx
# See https://console.groq.com/docs/quickstart
- type: openai-compatible
name: groq
api_base: https://api.groq.com/openai/v1
api_key: xxx
# See https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart
- type: azure-openai
api_base: https://{RESOURCE}.openai.azure.com
api_key: xxx
models:
- name: gpt-4o # Model deployment name
max_input_tokens: 128000
supports_vision: true
supports_function_calling: true
# See https://cloud.google.com/vertex-ai
- type: vertexai
project_id: xxx
location: xxx
# Specifies a application-default-credentials (adc) file
# Run `gcloud auth application-default login` to init the adc file
# see https://cloud.google.com/docs/authentication/external/set-up-adc
adc_file: <gcloud-config-dir>/application_default_credentials.json> # Optional field
patch:
chat_completions:
'gemini-.*':
body:
safetySettings:
- category: HARM_CATEGORY_HARASSMENT
threshold: BLOCK_ONLY_HIGH
- category: HARM_CATEGORY_HATE_SPEECH
threshold: BLOCK_ONLY_HIGH
- category: HARM_CATEGORY_SEXUALLY_EXPLICIT
threshold: BLOCK_ONLY_HIGH
- category: HARM_CATEGORY_DANGEROUS_CONTENT
threshold: BLOCK_ONLY_HIGH
# See https://docs.aws.amazon.com/bedrock/latest/userguide/
- type: bedrock
access_key_id: xxx
secret_access_key: xxx
region: xxx
session_token: xxx # Optional, only needed for temporary credentials
# See https://developers.cloudflare.com/workers-ai/
- type: openai-compatible
name: cloudflare
api_base: https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/v1
api_key: xxx
# See https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html
- type: openai-compatible
name: ernie
api_base: https://qianfan.baidubce.com/v2
api_key: xxx
# See https://dashscope.aliyun.com/
- type: openai-compatible
name: qianwen
api_base: https://dashscope.aliyuncs.com/compatible-mode/v1
api_key: xxx
# See https://cloud.tencent.com/product/hunyuan
- type: openai-compatible
name: hunyuan
api_base: https://api.hunyuan.cloud.tencent.com/v1
api_key: xxx
# See https://platform.moonshot.cn/docs/intro
- type: openai-compatible
name: moonshot
api_base: https://api.moonshot.cn/v1
api_key: xxx
# See https://platform.deepseek.com/api-docs/
- type: openai-compatible
name: deepseek
api_base: https://api.deepseek.com
api_key: xxx
# See https://open.bigmodel.cn/dev/howuse/introduction
- type: openai-compatible
name: zhipuai
api_base: https://open.bigmodel.cn/api/paas/v4
api_key: xxx
# See https://platform.minimaxi.com/document/Fast%20access
- type: openai-compatible
name: minimax
api_base: https://api.minimax.chat/v1
api_key: xxx
# See https://openrouter.ai/docs#quick-start
- type: openai-compatible
name: openrouter
api_base: https://openrouter.ai/api/v1
api_key: xxx
# See https://github.com/marketplace/models
- type: openai-compatible
name: github
api_base: https://models.inference.ai.azure.com
api_key: xxx
# See https://deepinfra.com/docs
- type: openai-compatible
name: deepinfra
api_base: https://api.deepinfra.com/v1/openai
api_key: xxx
# ----- RAG dedicated -----
# See https://jina.ai
- type: openai-compatible
name: jina
api_base: https://api.jina.ai/v1
api_key: xxx
# See https://docs.voyageai.com/docs/introduction
- type: openai-compatible
name: voyageai
api_base: https://api.voyageai.com/v1
api_key: xxx
================================================
FILE: models.yaml
================================================
# Links:
# - https://platform.openai.com/docs/models
# - https://platform.openai.com/docs/api-reference/chat
- provider: openai
models:
- name: gpt-5.2
max_input_tokens: 400000
max_output_tokens: 128000
input_price: 1.75
output_price: 14
supports_vision: true
supports_function_calling: true
- name: gpt-5
max_input_tokens: 400000
max_output_tokens: 128000
input_price: 1.25
output_price: 10
supports_vision: true
supports_function_calling: true
- name: gpt-5-mini
max_input_tokens: 400000
max_output_tokens: 128000
input_price: 0.25
output_price: 2
supports_vision: true
supports_function_calling: true
- name: gpt-5-nano
max_input_tokens: 400000
max_output_tokens: 128000
input_price: 0.05
output_price: 0.4
supports_vision: true
supports_function_calling: true
- name: gpt-4.1
max_input_tokens: 1047576
max_output_tokens: 32768
input_price: 2
output_price: 8
supports_vision: true
supports_function_calling: true
- name: gpt-4o
max_input_tokens: 128000
max_output_tokens: 16384
input_price: 2.5
output_price: 10
supports_vision: true
supports_function_calling: true
- name: gpt-4-turbo
max_input_tokens: 128000
max_output_tokens: 4096
input_price: 10
output_price: 30
supports_vision: true
supports_function_calling: true
- name: gpt-3.5-turbo
max_input_tokens: 16385
max_output_tokens: 4096
input_price: 0.5
output_price: 1.5
supports_function_calling: true
- name: text-embedding-3-large
type: embedding
input_price: 0.13
max_tokens_per_chunk: 8191
default_chunk_size: 2000
max_batch_size: 100
- name: text-embedding-3-small
type: embedding
input_price: 0.02
max_tokens_per_chunk: 8191
default_chunk_size: 2000
max_batch_size: 100
# Links:
# - https://ai.google.dev/models/gemini
# - https://ai.google.dev/pricing
# - https://ai.google.dev/api/rest/v1beta/models/streamGenerateContent
- provider: gemini
models:
- name: gemini-2.5-flash
max_input_tokens: 1048576
max_output_tokens: 65536
input_price: 0
output_price: 0
supports_vision: true
supports_function_calling: true
- name: gemini-2.5-pro
max_input_tokens: 1048576
max_output_tokens: 65536
input_price: 0
output_price: 0
supports_vision: true
supports_function_calling: true
- name: gemini-2.5-flash-lite
max_input_tokens: 1000000
max_output_tokens: 64000
input_price: 0
output_price: 0
supports_vision: true
supports_function_calling: true
- name: gemini-3-pro-preview
max_input_tokens: 1048576
supports_vision: true
supports_function_calling: true
- name: gemini-3-flash-preview
max_input_tokens: 1048576
supports_vision: true
supports_function_calling: true
- name: gemini-2.0-flash
max_input_tokens: 1048576
max_output_tokens: 8192
input_price: 0
output_price: 0
supports_vision: true
supports_function_calling: true
- name: gemini-2.0-flash-lite
max_input_tokens: 1048576
max_output_tokens: 8192
input_price: 0
output_price: 0
supports_vision: true
supports_function_calling: true
- name: gemma-3-27b-it
max_input_tokens: 131072
max_output_tokens: 8192
input_price: 0
output_price: 0
- name: text-embedding-004
type: embedding
input_price: 0
max_tokens_per_chunk: 2048
default_chunk_size: 1500
max_batch_size: 100
# Links:
# - https://docs.anthropic.com/en/docs/about-claude/models/all-models
# - https://docs.anthropic.com/en/api/messages
- provider: claude
models:
- name: claude-opus-4-6
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
- name: claude-opus-4-6:thinking
real_name: claude-opus-4-6
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: claude-sonnet-4-6
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
- name: claude-sonnet-4-6:thinking
real_name: claude-sonnet-4-6
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: claude-opus-4-5-20251101
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
- name: claude-opus-4-5-20251101:thinking
real_name: claude-opus-4-5-20251101
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: claude-sonnet-4-5-20250929
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
- name: claude-sonnet-4-5-20250929:thinking
real_name: claude-sonnet-4-5-20250929
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: claude-haiku-4-5-20251001
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 1
output_price: 5
supports_vision: true
supports_function_calling: true
- name: claude-haiku-4-5-20251001:thinking
real_name: claude-haiku-4-5-20251001
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 1
output_price: 5
supports_vision: true
supports_function_calling: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
# Links:
# - https://docs.mistral.ai/getting-started/models/models_overview/
# - https://mistral.ai/pricing#api-pricing
# - https://docs.mistral.ai/api/
- provider: mistral
models:
- name: mistral-large-latest
max_output_tokens: 262144
input_price: 0.5
output_price: 1.5
supports_function_calling: true
supports_vision: true
- name: mistral-medium-latest
max_input_tokens: 131072
input_price: 0.4
output_price: 2
supports_function_calling: true
supports_vision: true
- name: mistral-small-latest
max_input_tokens: 32768
input_price: 0.1
output_price: 0.3
supports_function_calling: true
supports_vision: true
- name: magistral-medium-latest
max_input_tokens: 131072
input_price: 2
output_price: 5
- name: magistral-small-latest
max_input_tokens: 131072
input_price: 0.5
output_price: 1.5
- name: devstral-medium-latest
max_input_tokens: 262144
input_price: 0.4
output_price: 2
supports_function_calling: true
- name: devstral-small-latest
max_input_tokens: 262144
input_price: 0.1
output_price: 0.3
supports_function_calling: true
- name: codestral-latest
max_input_tokens: 262144
input_price: 0.3
output_price: 0.9
supports_function_calling: true
- name: ministral-14b-latest
max_input_tokens: 262144
input_price: 0.2
output_price: 0.2
supports_function_calling: true
- name: mistral-embed
type: embedding
max_input_tokens: 8092
input_price: 0.1
max_tokens_per_chunk: 8092
default_chunk_size: 2000
# Links:
# - https://docs.ai21.com/docs/jamba-foundation-models
# - https://www.ai21.com/pricing
# - https://docs.ai21.com/reference/jamba-1-6-api-ref
- provider: ai21
models:
- name: jamba-large
max_input_tokens: 256000
input_price: 2
output_price: 8
supports_function_calling: true
- name: jamba-mini
max_input_tokens: 256000
input_price: 0.2
output_price: 0.4
supports_function_calling: true
# Links:
# - https://docs.cohere.com/docs/models
# - https://cohere.com/pricing
# - https://docs.cohere.com/reference/chat
- provider: cohere
models:
- name: command-a-03-2025
max_input_tokens: 262144
max_output_tokens: 8192
input_price: 2.5
output_price: 10
supports_function_calling: true
- name: command-a-reasoning-08-2025
max_input_tokens: 262144
max_output_tokens: 32768
input_price: 2.5
output_price: 10
- name: command-a-vision-07-2025
max_input_tokens: 131072
max_output_tokens: 8192
input_price: 2.5
output_price: 10
supports_vision: true
- name: command-r7b-12-2024
max_input_tokens: 131072
max_output_tokens: 4096
input_price: 0.0375
output_price: 0.15
- name: embed-v4.0
type: embedding
input_price: 0.12
max_tokens_per_chunk: 2048
default_chunk_size: 2000
max_batch_size: 96
- name: embed-english-v3.0
type: embedding
input_price: 0.1
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 96
- name: embed-multilingual-v3.0
type: embedding
input_price: 0.1
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 96
- name: rerank-v3.5
type: reranker
max_input_tokens: 4096
- name: rerank-english-v3.0
type: reranker
max_input_tokens: 4096
- name: rerank-multilingual-v3.0
type: reranker
max_input_tokens: 4096
# Links:
# - https://docs.x.ai/docs/models
# - https://docs.x.ai/docs/api-reference#chat-completions
- provider: xai
models:
- name: grok-4-1-fast-non-reasoning
max_input_tokens: 2000000
input_price: 0.2
output_price: 0.5
supports_function_calling: true
- name: grok-4-1-fast-reasoning
max_input_tokens: 2000000
input_price: 0.2
output_price: 0.5
supports_function_calling: true
- name: grok-code-fast-1
max_input_tokens: 256000
input_price: 0.2
output_price: 1.5
supports_function_calling: true
# Links:
# - https://docs.perplexity.ai/getting-started/models
# - https://docs.perplexity.ai/api-reference/chat-completions
- provider: perplexity
models:
- name: sonar-pro
max_input_tokens: 200000
input_price: 3
output_price: 15
- name: sonar
max_input_tokens: 128000
input_price: 1
output_price: 1
- name: sonar-reasoning-pro
max_input_tokens: 128000
input_price: 2
output_price: 8
- name: sonar-deep-research
max_input_tokens: 128000
input_price: 2
output_price: 8
# Links:
# - https://console.groq.com/docs/models
# - https://console.groq.com/docs/api-reference#chat
- provider: groq
models:
- name: openai/gpt-oss-120b
max_input_tokens: 131072
input_price: 0
output_price: 0
supports_function_calling: true
- name: openai/gpt-oss-20b
max_input_tokens: 131072
input_price: 0
output_price: 0
supports_function_calling: true
- name: meta-llama/llama-4-maverick-17b-128e-instruct
max_input_tokens: 131072
input_price: 0
output_price: 0
supports_vision: true
supports_function_calling: true
- name: meta-llama/llama-4-scout-17b-16e-instruct
max_input_tokens: 131072
input_price: 0
output_price: 0
supports_vision: true
supports_function_calling: true
- name: llama-3.3-70b-versatile
max_input_tokens: 131072
input_price: 0
output_price: 0
supports_function_calling: true
- name: moonshotai/kimi-k2-instruct-0905
max_input_tokens: 262144
input_price: 0
output_price: 0
supports_function_calling: true
- name: qwen/qwen3-32b
max_input_tokens: 131072
input_price: 0
output_price: 0
- name: groq/compound
max_input_tokens: 131072
input_price: 0
output_price: 0
- name: groq/compound-mini
max_input_tokens: 131072
input_price: 0
output_price: 0
# Links:
# - https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models
# - https://cloud.google.com/vertex-ai/generative-ai/pricing
# - https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini
- provider: vertexai
models:
- name: gemini-2.5-flash
max_input_tokens: 1048576
max_output_tokens: 65536
input_price: 0.3
output_price: 2.5
supports_vision: true
supports_function_calling: true
- name: gemini-2.5-pro
max_input_tokens: 1048576
max_output_tokens: 65536
input_price: 1.25
output_price: 10
supports_vision: true
supports_function_calling: true
- name: gemini-2.5-flash-lite
max_input_tokens: 1048576
max_output_tokens: 65536
input_price: 0.3
output_price: 0.4
supports_vision: true
supports_function_calling: true
- name: gemini-3-pro-preview
max_input_tokens: 1048576
supports_vision: true
supports_function_calling: true
- name: gemini-3-flash-preview
max_input_tokens: 1048576
supports_vision: true
supports_function_calling: true
- name: gemini-2.0-flash-001
max_input_tokens: 1048576
max_output_tokens: 8192
input_price: 0.15
output_price: 0.6
supports_vision: true
supports_function_calling: true
- name: gemini-2.0-flash-lite-001
max_input_tokens: 1048576
max_output_tokens: 8192
input_price: 0.075
output_price: 0.3
supports_vision: true
supports_function_calling: true
- name: claude-opus-4-6
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
- name: claude-opus-4-6:thinking
real_name: claude-opus-4-6
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: claude-sonnet-4-6
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
- name: claude-sonnet-4-6:thinking
real_name: claude-sonnet-4-6
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: claude-opus-4-5@20251101
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
- name: claude-opus-4-5@20251101:thinking
real_name: claude-opus-4-5@20251101
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: claude-sonnet-4-5@20250929
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
- name: claude-sonnet-4-5@20250929:thinking
real_name: claude-sonnet-4-5@20250929
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: claude-haiku-4-5@20251001
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 1
output_price: 5
supports_vision: true
supports_function_calling: true
- name: claude-haiku-4-5@20251001:thinking
real_name: claude-haiku-4-5@20251001
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 1
output_price: 5
supports_vision: true
patch:
body:
temperature: null
top_p: null
thinking:
type: enabled
budget_tokens: 16000
- name: text-embedding-005
type: embedding
max_input_tokens: 20000
input_price: 0.025
max_tokens_per_chunk: 2048
default_chunk_size: 1500
max_batch_size: 5
- name: text-multilingual-embedding-002
type: embedding
max_input_tokens: 20000
input_price: 0.2
max_tokens_per_chunk: 2048
default_chunk_size: 1500
max_batch_size: 5
# Links:
# - https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns
# - https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html
# - https://aws.amazon.com/bedrock/pricing/
# - https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-call.html
- provider: bedrock
models:
- name: us.anthropic.claude-opus-4-6-v1
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
- name: us.anthropic.claude-opus-4-6-v1:thinking
real_name: us.anthropic.claude-opus-4-6-v1
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
patch:
body:
inferenceConfig:
temperature: null
topP: null
additionalModelRequestFields:
thinking:
type: enabled
budget_tokens: 16000
- name: us.anthropic.claude-sonnet-4-6
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
- name: us.anthropic.claude-sonnet-4-6:thinking
real_name: us.anthropic.claude-sonnet-4-6
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
patch:
body:
inferenceConfig:
temperature: null
topP: null
additionalModelRequestFields:
thinking:
type: enabled
budget_tokens: 16000
- name: us.anthropic.claude-opus-4-5-20251101-v1:0
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
- name: us.anthropic.claude-opus-4-5-20251101-v1:0:thinking
real_name: us.anthropic.claude-opus-4-5-20251101-v1:0
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
patch:
body:
inferenceConfig:
temperature: null
topP: null
additionalModelRequestFields:
thinking:
type: enabled
budget_tokens: 16000
- name: us.anthropic.claude-sonnet-4-5-20250929-v1:0
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
- name: us.anthropic.claude-sonnet-4-5-20250929-v1:0:thinking
real_name: us.anthropic.claude-sonnet-4-5-20250929-v1:0
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
patch:
body:
inferenceConfig:
temperature: null
topP: null
additionalModelRequestFields:
thinking:
type: enabled
budget_tokens: 16000
- name: us.anthropic.claude-haiku-4-5-20251001-v1:0
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 1
output_price: 5
supports_vision: true
supports_function_calling: true
- name: us.anthropic.claude-haiku-4-5-20251001-v1:0:thinking
real_name: us.anthropic.claude-haiku-4-5-20251001-v1:0
max_input_tokens: 200000
max_output_tokens: 24000
require_max_tokens: true
input_price: 1
output_price: 5
supports_vision: true
patch:
body:
inferenceConfig:
temperature: null
topP: null
additionalModelRequestFields:
thinking:
type: enabled
budget_tokens: 16000
- name: us.meta.llama4-maverick-17b-instruct-v1:0
max_input_tokens: 131072
max_output_tokens: 8192
require_max_tokens: true
input_price: 0.24
output_price: 0.97
supports_function_calling: true
supports_vision: true
- name: us.meta.llama4-scout-17b-instruct-v1:0
max_input_tokens: 131072
max_output_tokens: 8192
require_max_tokens: true
input_price: 0.17
output_price: 0.66
supports_function_calling: true
supports_vision: true
- name: us.meta.llama3-3-70b-instruct-v1:0
max_input_tokens: 131072
max_output_tokens: 8192
require_max_tokens: true
input_price: 0.72
output_price: 0.72
supports_function_calling: true
- name: us.amazon.nova-premier-v1:0
max_input_tokens: 300000
max_output_tokens: 5120
input_price: 2.5
output_price: 12.5
- name: us.amazon.nova-pro-v1:0
max_input_tokens: 300000
max_output_tokens: 5120
input_price: 0.8
output_price: 3.2
supports_vision: true
- name: us.amazon.nova-lite-v1:0
max_input_tokens: 300000
max_output_tokens: 5120
input_price: 0.06
output_price: 0.24
supports_vision: true
- name: us.amazon.nova-micro-v1:0
max_input_tokens: 128000
max_output_tokens: 5120
input_price: 0.035
output_price: 0.14
- name: cohere.embed-english-v3
type: embedding
input_price: 0.1
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 96
- name: cohere.embed-multilingual-v3
type: embedding
input_price: 0.1
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 96
- name: us.deepseek.r1-v1:0
max_input_tokens: 128000
input_price: 1.35
output_price: 5.4
# Links:
# - https://developers.cloudflare.com/workers-ai/models/
# - https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/
- provider: cloudflare
models:
- name: '@cf/meta/llama-4-scout-17b-16e-instruct'
max_input_tokens: 131072
max_output_tokens: 2048
require_max_tokens: true
input_price: 0
output_price: 0
- name: '@cf/meta/llama-3.3-70b-instruct-fp8-fast'
max_input_tokens: 131072
max_output_tokens: 2048
require_max_tokens: true
input_price: 0
output_price: 0
- name: '@cf/qwen/qwen3-30b-a3b-fp8'
max_input_tokens: 131072
max_output_tokens: 2048
require_max_tokens: true
input_price: 0
output_price: 0
- name: '@cf/qwen/qwen2.5-coder-32b-instruct'
max_input_tokens: 131072
max_output_tokens: 2048
require_max_tokens: true
input_price: 0
output_price: 0
- name: '@cf/zai-org/glm-4.7-flash'
max_input_tokens: 131072
max_output_tokens: 2048
require_max_tokens: true
input_price: 0
output_price: 0
- name: '@cf/google/gemma-3-12b-it'
max_input_tokens: 131072
max_output_tokens: 2048
require_max_tokens: true
input_price: 0
output_price: 0
- name: '@cf/mistralai/mistral-small-3.1-24b-instruct'
max_input_tokens: 131072
max_output_tokens: 2048
require_max_tokens: true
input_price: 0
output_price: 0
- name: '@cf/baai/bge-large-en-v1.5'
type: embedding
input_price: 0
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 100
# Links:
# - https://cloud.baidu.com/doc/qianfan/s/rmh4stp0j
# - https://cloud.baidu.com/doc/qianfan/s/wmh4sv6ya
- provider: ernie
models:
- name: ernie-4.5-turbo-128k
max_input_tokens: 131072
input_price: 0.112
output_price: 0.448
- name: ernie-4.5-turbo-vl-32k
max_input_tokens: 32768
input_price: 0.42
output_price: 1.26
supports_vision: true
- name: ernie-5.0-thinking-preview
max_input_tokens: 131072
input_price: 1.4
output_price: 5.6
- name: ernie-x1.1-preview
max_input_tokens: 65536
input_price: 0.14
output_price: 0.56
- name: bge-large-zh
type: embedding
input_price: 0.07
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 16
- name: bge-large-en
type: embedding
input_price: 0.07
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 16
- name: bce-reranker-base
type: reranker
max_input_tokens: 1024
input_price: 0.07
# Links:
# - https://help.aliyun.com/zh/model-studio/getting-started/models
# - https://help.aliyun.com/zh/model-studio/developer-reference/use-qwen-by-calling-api
- provider: qianwen
models:
- name: qwen3.5-plus
max_input_tokens: 262144
supports_function_calling: true
patch:
body:
enable_thinking: false
- name: qwen3.5-plus:thinking
real_name: qwen3.5-plus
max_input_tokens: 262144
supports_function_calling: true
- name: qwen3-max
max_input_tokens: 262144
supports_function_calling: true
- name: qwen3-max:thinking
real_name: qwen3-max
max_input_tokens: 262144
supports_function_calling: true
patch:
body:
enable_thinking: true
- name: qwen3-vl-plus
max_input_tokens: 262144
supports_vision: true
- name: qwen3-vl-flash
max_input_tokens: 262144
supports_vision: true
- name: qwen3-coder-plus
max_input_tokens: 1000000
- name: qwen3-coder-flash
max_input_tokens: 1000000
- name: qwen3.5-397b-a17b
max_input_tokens: 262144
supports_function_calling: true
patch:
body:
enable_thinking: false
- name: qwen3.5-397b-a17b:thinking
real_name: qwen3.5-397b-a17b
max_input_tokens: 262144
supports_function_calling: true
- name: qwen3-next-80b-a3b-instruct
max_input_tokens: 131072
input_price: 0.14
output_price: 0.56
supports_function_calling: true
- name: qwen3-next-80b-a3b-thinking
max_input_tokens: 131072
input_price: 0.14
output_price: 1.4
- name: qwen3-235b-a22b-instruct-2507
max_input_tokens: 131072
input_price: 0.28
output_price: 1.12
supports_function_calling: true
- name: qwen3-235b-a22b-thinking-2507
max_input_tokens: 131072
input_price: 0.28
output_price: 2.8
- name: qwen3-30b-a3b-instruct-2507
max_input_tokens: 131072
input_price: 0.105
output_price: 0.42
supports_function_calling: true
- name: qwen3-30b-a3b-thinking-2507
max_input_tokens: 131072
input_price: 0.105
output_price: 1.05
- name: qwen3-vl-32b-instruct
max_input_tokens: 131072
input_price: 0.28
output_price: 1.12
supports_vision: true
- name: qwen3-vl-8b-instruct
max_input_tokens: 131072
input_price: 0.07
output_price: 0.28
supports_vision: true
- name: qwen3-coder-next
max_input_tokens: 262144
- name: qwen3-coder-480b-a35b-instruct
max_input_tokens: 262144
- name: qwen3-coder-30b-a3b-instruct
max_input_tokens: 262144
- name: text-embedding-v4
type: embedding
input_price: 0.1
max_tokens_per_chunk: 8192
default_chunk_size: 2000
max_batch_size: 10
- name: text-embedding-v3
type: embedding
input_price: 0.1
max_tokens_per_chunk: 8192
default_chunk_size: 2000
max_batch_size: 10
# links:
# - https://cloud.tencent.com/document/product/1729/104753
# - https://cloud.tencent.com/document/product/1729/97731
# - https://cloud.tencent.com/document/product/1729/111007
- provider: hunyuan
models:
- name: hunyuan-2.0-instruct-20251111
max_input_tokens: 131072
input_price: 0.112
output_price: 0.28
supports_function_calling: true
- name: hunyuan-2.0-thinking-20251109
max_input_tokens: 131072
input_price: 0.14
output_price: 0.56
supports_function_calling: true
- name: hunyuan-vision-1.5-instruct
max_input_tokens: 24576
input_price: 0.42
output_price: 1.26
supports_vision: true
- name: hunyuan-embedding
type: embedding
input_price: 0.01
max_tokens_per_chunk: 1024
default_chunk_size: 1000
max_batch_size: 100
# Links:
# - https://platform.moonshot.cn/docs/pricing/chat#%E8%AE%A1%E8%B4%B9%E5%9F%BA%E6%9C%AC%E6%A6%82%E5%BF%B5
# - https://platform.moonshot.cn/docs/api/chat#%E5%85%AC%E5%BC%80%E7%9A%84%E6%9C%8D%E5%8A%A1%E5%9C%B0%E5%9D%80
- provider: moonshot
models:
- name: kimi-k2.5
max_input_tokens: 262144
input_price: 0.56
output_price: 2.94
supports_vision: true
supports_function_calling: true
- name: kimi-k2-turbo-preview
max_input_tokens: 262144
input_price: 1.12
output_price: 8.12
supports_vision: true
supports_function_calling: true
- name: kimi-k2-0905-preview
max_input_tokens: 262144
input_price: 0.56
output_price: 2.24
supports_vision: true
supports_function_calling: true
- name: kimi-k2-thinking-turbo
max_input_tokens: 262144
input_price: 1.12
output_price: 8.12
supports_vision: true
- name: kimi-k2-thinking
max_input_tokens: 262144
input_price: 0.56
output_price: 2.24
supports_vision: true
# Links:
# - https://api-docs.deepseek.com/quick_start/pricing
# - https://platform.deepseek.com/api-docs/api/create-chat-completion
- provider: deepseek
models:
- name: deepseek-chat
max_input_tokens: 64000
max_output_tokens: 8192
input_price: 0.56
output_price: 1.68
supports_function_calling: true
- name: deepseek-reasoner
max_input_tokens: 64000
max_output_tokens: 32768
input_price: 0.56
output_price: 1.68
# Links:
# - https://open.bigmodel.cn/pricing
# - https://open.bigmodel.cn/dev/api#glm-4
- provider: zhipuai
models:
- name: glm-5
max_input_tokens: 202752
supports_function_calling: true
- name: glm-5:instruct
real_name: glm-5
max_input_tokens: 202752
supports_function_calling: true
patch:
body:
thinking:
type: disabled
- name: glm-4.7
max_input_tokens: 202752
supports_function_calling: true
- name: glm-4.7:instruct
real_name: glm-4.7
max_input_tokens: 202752
supports_function_calling: true
patch:
body:
thinking:
type: disabled
- name: glm-4.7-flash
max_input_tokens: 202752
input_price: 0
output_price: 0
supports_function_calling: true
- name: glm-4.6v
max_input_tokens: 65536
supports_vision: true
- name: glm-4.6v-flash
max_input_tokens: 65536
input_price: 0
output_price: 0
supports_vision: true
- name: embedding-3
type: embedding
max_input_tokens: 8192
input_price: 0.07
max_tokens_per_chunk: 8192
default_chunk_size: 2000
- name: rerank
type: reranker
max_input_tokens: 4096
input_price: 0.112
# Links:
# - https://platform.minimaxi.com/docs/guides/pricing-paygo
# - https://platform.minimaxi.com/document/ChatCompletion%20v2
- provider: minimax
models:
- name: minimax-m2.5
max_input_tokens: 204800
input_price: 0.294
output_price: 1.176
supports_function_calling: true
- name: minimax-m2.5-highspeed
max_input_tokens: 204800
input_price: 0.588
output_price: 2.352
supports_function_calling: true
- name: minimax-m2.1
max_input_tokens: 204800
input_price: 0.294
output_price: 1.176
supports_function_calling: true
- name: minimax-m2.1-highspeed
max_input_tokens: 204800
input_price: 0.588
output_price: 2.352
supports_function_calling: true
# Links:
# - https://openrouter.ai/models
# - https://openrouter.ai/docs/api-reference/chat-completion
- provider: openrouter
models:
- name: openai/gpt-5.2
max_input_tokens: 400000
max_output_tokens: 128000
input_price: 1.75
output_price: 14
supports_vision: true
supports_function_calling: true
- name: openai/gpt-5
max_input_tokens: 400000
max_output_tokens: 128000
input_price: 1.25
output_price: 10
supports_vision: true
supports_function_calling: true
- name: openai/gpt-5-mini
max_input_tokens: 400000
max_output_tokens: 128000
input_price: 0.25
output_price: 2
supports_vision: true
supports_function_calling: true
- name: openai/gpt-5-nano
max_input_tokens: 400000
max_output_tokens: 128000
input_price: 0.05
output_price: 0.4
supports_vision: true
supports_function_calling: true
- name: openai/gpt-4.1
max_input_tokens: 1047576
max_output_tokens: 32768
input_price: 2
output_price: 8
supports_vision: true
supports_function_calling: true
- name: openai/gpt-4o
max_input_tokens: 128000
input_price: 2.5
output_price: 10
supports_vision: true
supports_function_calling: true
- name: openai/gpt-oss-120b
max_input_tokens: 131072
input_price: 0.09
output_price: 0.45
supports_function_calling: true
- name: openai/gpt-oss-20b
max_input_tokens: 131072
input_price: 0.04
output_price: 0.16
supports_function_calling: true
- name: google/gemini-2.5-flash
max_input_tokens: 1048576
input_price: 0.3
output_price: 2.5
supports_vision: true
supports_function_calling: true
- name: google/gemini-2.5-pro
max_input_tokens: 1048576
input_price: 1.25
output_price: 10
supports_vision: true
supports_function_calling: true
- name: google/gemini-2.5-flash-lite
max_input_tokens: 1048576
input_price: 0.3
output_price: 0.4
supports_vision: true
- name: google/gemini-2.0-flash-001
max_input_tokens: 1000000
input_price: 0.15
output_price: 0.6
supports_vision: true
supports_function_calling: true
- name: google/gemini-2.0-flash-lite-001
max_input_tokens: 1048576
input_price: 0.075
output_price: 0.3
supports_vision: true
supports_function_calling: true
- name: google/gemma-3-27b-it
max_input_tokens: 131072
input_price: 0.1
output_price: 0.2
- name: anthropic/claude-opus-4.6
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
- name: anthropic/claude-sonnet-4.6
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
- name: anthropic/claude-opus-4.5
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 5
output_price: 25
supports_vision: true
supports_function_calling: true
- name: anthropic/claude-sonnet-4.5
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 3
output_price: 15
supports_vision: true
supports_function_calling: true
- name: anthropic/claude-haiku-4.5
max_input_tokens: 200000
max_output_tokens: 8192
require_max_tokens: true
input_price: 1
output_price: 5
supports_vision: true
supports_function_calling: true
- name: meta-llama/llama-4-maverick
max_input_tokens: 1048576
input_price: 0.18
output_price: 0.6
supports_vision: true
supports_function_calling: true
- name: meta-llama/llama-4-scout
max_input_tokens: 327680
input_price: 0.08
output_price: 0.3
supports_vision: true
supports_function_calling: true
- name: meta-llama/llama-3.3-70b-instruct
max_input_tokens: 131072
input_price: 0.12
output_price: 0.3
- name: mistralai/mistral-large-2512
max_input_tokens: 262144
input_price: 0.5
output_price: 1.5
supports_function_calling: true
- name: mistralai/mistral-medium-3.1
max_input_tokens: 131072
input_price: 0.4
output_price: 2
supports_function_calling: true
supports_vision: true
- name: mistralai/mistral-small-3.2-24b-instruct
max_input_tokens: 131072
input_price: 0.1
output_price: 0.3
supports_vision: true
- name: mistralai/devstral-2512
max_input_tokens: 262144
input_price: 0.5
output_price: 0.22
supports_function_calling: true
- name: mistralai/devstral-small
max_input_tokens: 131072
input_price: 0.07
output_price: 0.28
supports_function_calling: true
- name: mistralai/codestral-2508
max_input_tokens: 256000
input_price: 0.3
output_price: 0.9
supports_function_calling: true
- name: mistralai/ministral-14b-2512
max_input_tokens: 262144
input_price: 0.2
output_price: 0.2
supports_function_calling: true
- name: ai21/jamba-large-1.7
max_input_tokens: 256000
input_price: 2
output_price: 8
supports_function_calling: true
- name: cohere/command-a
max_input_tokens: 256000
input_price: 2.5
output_price: 10
supports_function_calling: true
- name: cohere/command-r7b-12-2024
max_input_tokens: 128000
max_output_tokens: 4096
input_price: 0.0375
output_price: 0.15
- name: deepseek/deepseek-v3.2
max_input_tokens: 163840
input_price: 0.25
output_price: 0.38
- name: qwen/qwen3-max
max_input_tokens: 262144
input_price: 1.2
output_price: 6
supports_function_calling: true
- name: qwen/qwen3-max-thinking
max_input_tokens: 262144
input_price: 1.2
output_price: 6
supports_function_calling: true
- name: qwen/qwen3.5-plus-02-15
max_input_tokens: 1000000
max_output_tokens: 8192
input_price: 0.4
output_price: 2.4
supports_function_calling: true
- name: qwen/qwen3.5-397b-a17b
max_input_tokens: 262144
max_output_tokens: 8192
input_price: 0.15
output_price: 1
supports_function_calling: true
- name: qwen/qwen3-next-80b-a3b-instruct
max_input_tokens: 262144
input_price: 0.1
output_price: 0.8
supports_function_calling: true
- name: qwen/qwen3-next-80b-a3b-thinking
max_input_tokens: 262144
input_price: 0.1
output_price: 0.8
- name: qwen/qwen3-235b-a22b-2507 # Qwen3 235B A22B Instruct 2507
max_input_tokens: 262144
input_price: 0.12
output_price: 0.59
supports_function_calling: true
- name: qwen/qwen3-235b-a22b-thinking-2507
max_input_tokens: 262144
input_price: 0.118
output_price: 0.118
- name: qwen/qwen3-30b-a3b-instruct-2507
max_input_tokens: 131072
input_price: 0.2
output_price: 0.8
- name: qwen/qwen3-30b-a3b-thinking-2507
max_input_tokens: 262144
input_price: 0.071
output_price: 0.285
- name: qwen/qwen3-vl-32b-instruct
max_input_tokens: 262144
input_price: 0.35
output_price: 1.1
supports_vision: true
- name: qwen/qwen3-vl-8b-instruct
max_input_tokens: 262144
input_price: 0.08
output_price: 0.50
supports_vision: true
- name: qwen/qwen3-coder-next
max_input_tokens: 262144
input_price: 0.12
output_price: 0.75
supports_function_calling: true
- name: qwen/qwen3-coder-plus
max_input_tokens: 128000
input_price: 1
output_price: 5
supports_function_calling: true
- name: qwen/qwen3-coder-flash
max_input_tokens: 128000
input_price: 0.3
output_price: 1.5
supports_function_calling: true
- name: qwen/qwen3-coder # Qwen3 Coder 480B A35B
max_input_tokens: 262144
input_price: 0.22
output_price: 0.95
supports_function_calling: true
- name: qwen/qwen3-coder-30b-a3b-instruct
max_input_tokens: 262144
input_price: 0.052
output_price: 0.207
supports_function_calling: true
- name: moonshotai/kimi-k2.5
max_input_tokens: 262144
input_price: 0.57
output_price: 2.85
supports_vision: true
supports_function_calling: true
- name: moonshotai/kimi-k2-0905
max_input_tokens: 262144
input_price: 0.296
output_price: 1.185
supports_vision: true
supports_function_calling: true
- name: moonshotai/kimi-k2-thinking
max_input_tokens: 262144
input_price: 0.45
output_price: 2.35
supports_function_calling: true
- name: x-ai/grok-4.1-fast
max_input_tokens: 2000000
input_price: 0.2
output_price: 0.5
supports_function_calling: true
- name: x-ai/grok-code-fast-1
max_input_tokens: 256000
input_price: 0.2
output_price: 1.5
supports_function_calling: true
- name: amazon/nova-premier-v1
max_input_tokens: 1000000
input_price: 2.5
output_price: 12.5
supports_vision: true
- name: amazon/nova-pro-v1
max_input_tokens: 300000
max_output_tokens: 5120
input_price: 0.8
output_price: 3.2
supports_vision: true
- name: amazon/nova-lite-v1
max_input_tokens: 300000
max_output_tokens: 5120
input_price: 0.06
output_price: 0.24
supports_vision: true
- name: amazon/nova-micro-v1
max_input_tokens: 128000
max_output_tokens: 5120
input_price: 0.035
output_price: 0.14
- name: perplexity/sonar-pro
max_input_tokens: 200000
input_price: 3
output_price: 15
- name: perplexity/sonar
max_input_tokens: 127072
input_price: 1
output_price: 1
- name: perplexity/sonar-reasoning-pro
max_input_tokens: 128000
input_price: 2
output_price: 8
patch:
body:
include_reasoning: true
- name: perplexity/sonar-deep-research
max_input_tokens: 200000
input_price: 2
output_price: 8
patch:
body:
include_reasoning: true
- name: minimax/minimax-m2.5
max_input_tokens: 196608
input_price: 0.3
output_price: 1.1
supports_function_calling: true
- name: minimax/minimax-m2.1
max_input_tokens: 196608
input_price: 0.12
output_price: 0.48
supports_function_calling: true
- name: z-ai/glm-5
max_input_tokens: 204800
input_price: 0.95
output_price: 2.55
supports_function_calling: true
- name: z-ai/glm-4.7
max_input_tokens: 202752
input_price: 0.16
output_price: 0.80
supports_function_calling: true
- name: z-ai/glm-4.7-flash
max_input_tokens: 202752
input_price: 0.07
output_price: 0.40
supports_function_calling: true
- name: z-ai/glm-4.6v
max_input_tokens: 131072
input_price: 0.3
output_price: 0.9
supports_vision: true
# Links:
# - https://github.com/marketplace?type=models
- provider: github
models:
- name: gpt-5
max_input_tokens: 400000
max_output_tokens: 128000
supports_vision: true
supports_function_calling: true
- name: gpt-5-mini
max_input_tokens: 400000
max_output_tokens: 128000
supports_vision: true
supports_function_calling: true
- name: gpt-5-nano
max_input_tokens: 400000
max_output_tokens: 128000
supports_vision: true
supports_function_calling: true
- name: gpt-4.1
max_input_tokens: 1047576
max_output_tokens: 32768
supports_vision: true
supports_function_calling: true
- name: gpt-4o
max_input_tokens: 128000
max_output_tokens: 16384
supports_function_calling: true
- name: text-embedding-3-large
type: embedding
max_tokens_per_chunk: 8191
default_chunk_size: 2000
max_batch_size: 100
- name: text-embedding-3-small
type: embedding
max_tokens_per_chunk: 8191
default_chunk_size: 2000
max_batch_size: 100
- name: llama-4-maverick-17b-128e-instruct-fp8
max_input_tokens: 1048576
supports_vision: true
- name: llama-4-scout-17b-16e-instruct
max_input_tokens: 327680
supports_vision: true
- name: llama-3.3-70b-instruct
max_input_tokens: 131072
- name: mistral-medium-2505
max_input_tokens: 131072
supports_function_calling: true
- name: mistral-small-2503
max_input_tokens: 131072
supports_function_calling: true
- name: codestral-2501
max_input_tokens: 256000
supports_function_calling: true
- name: cohere-embed-v3-english
type: embedding
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 96
- name: cohere-embed-v3-multilingual
type: embedding
max_tokens_per_chunk: 512
default_chunk_size: 1000
max_batch_size: 96
- name: deepseek-r1-0528
max_input_tokens: 163840
- name: deepseek-v3-0324
max_input_tokens: 163840
- name: mai-ds-r1
max_input_tokens: 163840
- name: phi-4
max_input_tokens: 16384
- name: phi-4-mini-instruct
max_input_tokens: 131072
- name: phi-4-reasoning
max_input_tokens: 33792
- name: phi-4-mini-reasoning
max_input_tokens: 131072
- name: grok-3
max_input_tokens: 131072
- name: grok-3-mini
max_input_tokens: 131072
# Links:
# - https://deepinfra.com/models
# - https://deepinfra.com/docs/openai_api
- provider: deepinfra
models:
- name: openai/gpt-oss-120b
max_input_tokens: 131072
input_price: 0.09
output_price: 0.45
supports_function_calling: true
- name: openai/gpt-oss-20b
max_input_tokens: 131072
input_price: 0.04
output_price: 0.16
supports_function_calling: true
- name: meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8
max_input_tokens: 1048576
input_price: 0.18
output_price: 0.6
supports_vision: true
- name: meta-llama/Llama-4-Scout-17B-16E-Instruct
max_input_tokens: 327680
input_price: 0.08
output_price: 0.3
supports_vision: true
- name: Qwen/Qwen3-Max
max_input_tokens: 262144
input_price: 1.2
output_price: 6
supports_function_calling: true
- name: Qwen/Qwen3-
gitextract_9_1bw_kt/
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug_report.md
│ │ └── feature_request.md
│ └── workflows/
│ ├── ci.yaml
│ └── release.yaml
├── .gitignore
├── Argcfile.sh
├── Cargo.toml
├── LICENSE-APACHE
├── LICENSE-MIT
├── README.md
├── assets/
│ ├── arena.html
│ ├── playground.html
│ └── roles/
│ ├── %code%.md
│ ├── %create-prompt%.md
│ ├── %create-title%.md
│ ├── %explain-shell%.md
│ ├── %functions%.md
│ └── %shell%.md
├── config.agent.example.yaml
├── config.example.yaml
├── models.yaml
├── scripts/
│ ├── completions/
│ │ ├── aichat.bash
│ │ ├── aichat.fish
│ │ ├── aichat.nu
│ │ ├── aichat.ps1
│ │ └── aichat.zsh
│ └── shell-integration/
│ ├── integration.bash
│ ├── integration.fish
│ ├── integration.nu
│ ├── integration.ps1
│ └── integration.zsh
└── src/
├── cli.rs
├── client/
│ ├── access_token.rs
│ ├── azure_openai.rs
│ ├── bedrock.rs
│ ├── claude.rs
│ ├── cohere.rs
│ ├── common.rs
│ ├── gemini.rs
│ ├── macros.rs
│ ├── message.rs
│ ├── mod.rs
│ ├── model.rs
│ ├── openai.rs
│ ├── openai_compatible.rs
│ ├── stream.rs
│ └── vertexai.rs
├── config/
│ ├── agent.rs
│ ├── input.rs
│ ├── mod.rs
│ ├── role.rs
│ └── session.rs
├── function.rs
├── main.rs
├── rag/
│ ├── mod.rs
│ ├── serde_vectors.rs
│ └── splitter/
│ ├── language.rs
│ └── mod.rs
├── render/
│ ├── markdown.rs
│ ├── mod.rs
│ └── stream.rs
├── repl/
│ ├── completer.rs
│ ├── highlighter.rs
│ ├── mod.rs
│ └── prompt.rs
├── serve.rs
└── utils/
├── abort_signal.rs
├── clipboard.rs
├── command.rs
├── crypto.rs
├── html_to_md.rs
├── input.rs
├── loader.rs
├── mod.rs
├── path.rs
├── render_prompt.rs
├── request.rs
├── spinner.rs
└── variables.rs
SYMBOL INDEX (930 symbols across 47 files)
FILE: src/cli.rs
type Cli (line 8) | pub struct Cli {
method text (line 90) | pub fn text(&self) -> Result<Option<String>> {
FILE: src/client/access_token.rs
function get_access_token (line 10) | pub fn get_access_token(client_name: &str) -> Result<String> {
function is_valid_access_token (line 18) | pub fn is_valid_access_token(client_name: &str) -> bool {
function set_access_token (line 27) | pub fn set_access_token(client_name: &str, token: String, expires_at: i6...
FILE: src/client/azure_openai.rs
type AzureOpenAIConfig (line 8) | pub struct AzureOpenAIConfig {
constant PROMPTS (line 22) | pub const PROMPTS: [PromptAction<'static>; 2] = [
function prepare_chat_completions (line 43) | fn prepare_chat_completions(
function prepare_embeddings (line 65) | fn prepare_embeddings(self_: &AzureOpenAIClient, data: &EmbeddingsData) ...
FILE: src/client/bedrock.rs
type BedrockConfig (line 17) | pub struct BedrockConfig {
constant PROMPTS (line 35) | pub const PROMPTS: [PromptAction<'static>; 3] = [
method chat_completions_builder (line 41) | fn chat_completions_builder(
method embeddings_builder (line 92) | fn embeddings_builder(
method chat_completions_inner (line 150) | async fn chat_completions_inner(
method chat_completions_streaming_inner (line 159) | async fn chat_completions_streaming_inner(
method embeddings_inner (line 169) | async fn embeddings_inner(
function chat_completions (line 179) | async fn chat_completions(builder: RequestBuilder) -> Result<ChatComplet...
function chat_completions_streaming (line 192) | async fn chat_completions_streaming(
function embeddings (line 302) | async fn embeddings(builder: RequestBuilder) -> Result<EmbeddingsOutput> {
type EmbeddingsResBody (line 317) | struct EmbeddingsResBody {
function build_chat_completions_body (line 321) | fn build_chat_completions_body(data: ChatCompletionsData, model: &Model)...
function extract_chat_completions (line 480) | fn extract_chat_completions(data: &Value) -> Result<ChatCompletionsOutpu...
type AwsCredentials (line 532) | struct AwsCredentials {
type AwsRequest (line 540) | struct AwsRequest {
function aws_fetch (line 550) | fn aws_fetch(
function gen_signing_key (line 638) | fn gen_signing_key(key: &str, date_stamp: &str, region: &str, service: &...
FILE: src/client/claude.rs
constant API_BASE (line 10) | const API_BASE: &str = "https://api.anthropic.com/v1";
type ClaudeConfig (line 13) | pub struct ClaudeConfig {
constant PROMPTS (line 27) | pub const PROMPTS: [PromptAction<'static>; 1] = [("api_key", "API Key", ...
function prepare_chat_completions (line 41) | fn prepare_chat_completions(
function claude_chat_completions (line 61) | pub async fn claude_chat_completions(
function claude_chat_completions_streaming (line 75) | pub async fn claude_chat_completions_streaming(
function claude_build_chat_completions_body (line 156) | pub fn claude_build_chat_completions_body(
function claude_extract_chat_completions (line 300) | pub fn claude_extract_chat_completions(data: &Value) -> Result<ChatCompl...
FILE: src/client/cohere.rs
constant API_BASE (line 10) | const API_BASE: &str = "https://api.cohere.ai/v2";
type CohereConfig (line 13) | pub struct CohereConfig {
constant PROMPTS (line 27) | pub const PROMPTS: [PromptAction<'static>; 1] = [("api_key", "API Key", ...
function prepare_chat_completions (line 41) | fn prepare_chat_completions(
function prepare_embeddings (line 65) | fn prepare_embeddings(self_: &CohereClient, data: &EmbeddingsData) -> Re...
function prepare_rerank (line 92) | fn prepare_rerank(self_: &CohereClient, data: &RerankData) -> Result<Req...
function chat_completions (line 108) | async fn chat_completions(
function chat_completions_streaming (line 123) | async fn chat_completions_streaming(
function embeddings (line 191) | async fn embeddings(builder: RequestBuilder, _model: &Model) -> Result<E...
type EmbeddingsResBody (line 204) | struct EmbeddingsResBody {
type EmbeddingsResBodyEmbeddings (line 209) | struct EmbeddingsResBodyEmbeddings {
function extract_chat_completions (line 213) | fn extract_chat_completions(data: &Value) -> Result<ChatCompletionsOutpu...
FILE: src/client/common.rs
constant MODELS_YAML (line 23) | const MODELS_YAML: &str = include_str!("../../models.yaml");
type Client (line 38) | pub trait Client: Sync + Send {
method global_config (line 39) | fn global_config(&self) -> &GlobalConfig;
method extra_config (line 41) | fn extra_config(&self) -> Option<&ExtraConfig>;
method patch_config (line 43) | fn patch_config(&self) -> Option<&RequestPatch>;
method name (line 45) | fn name(&self) -> &str;
method model (line 47) | fn model(&self) -> &Model;
method model_mut (line 49) | fn model_mut(&mut self) -> &mut Model;
method build_client (line 51) | fn build_client(&self) -> Result<ReqwestClient> {
method chat_completions (line 68) | async fn chat_completions(&self, input: Input) -> Result<ChatCompletio...
method chat_completions_streaming (line 80) | async fn chat_completions_streaming(
method embeddings (line 108) | async fn embeddings(&self, data: &EmbeddingsData) -> Result<Vec<Vec<f3...
method rerank (line 115) | async fn rerank(&self, data: &RerankData) -> Result<RerankOutput> {
method chat_completions_inner (line 122) | async fn chat_completions_inner(
method chat_completions_streaming_inner (line 128) | async fn chat_completions_streaming_inner(
method embeddings_inner (line 135) | async fn embeddings_inner(
method rerank_inner (line 143) | async fn rerank_inner(
method request_builder (line 151) | fn request_builder(
method patch_request_data (line 160) | fn patch_request_data(&self, request_data: &mut RequestData) {
method default (line 195) | fn default() -> Self {
type ExtraConfig (line 201) | pub struct ExtraConfig {
type RequestPatch (line 207) | pub struct RequestPatch {
type ApiPatch (line 213) | pub type ApiPatch = IndexMap<String, Value>;
type RequestData (line 215) | pub struct RequestData {
method new (line 222) | pub fn new<T>(url: T, body: Value) -> Self
method bearer_auth (line 233) | pub fn bearer_auth<T>(&mut self, auth: T)
method header (line 241) | pub fn header<K, V>(&mut self, key: K, value: V)
method into_builder (line 249) | pub fn into_builder(self, client: &ReqwestClient) -> RequestBuilder {
method apply_patch (line 261) | pub fn apply_patch(&mut self, patch: Value) {
type ChatCompletionsData (line 281) | pub struct ChatCompletionsData {
type ChatCompletionsOutput (line 290) | pub struct ChatCompletionsOutput {
method new (line 299) | pub fn new(text: &str) -> Self {
type EmbeddingsData (line 308) | pub struct EmbeddingsData {
method new (line 314) | pub fn new(texts: Vec<String>, query: bool) -> Self {
type EmbeddingsOutput (line 319) | pub type EmbeddingsOutput = Vec<Vec<f32>>;
type RerankData (line 322) | pub struct RerankData {
method new (line 329) | pub fn new(query: String, documents: Vec<String>, top_n: usize) -> Self {
type RerankOutput (line 338) | pub type RerankOutput = Vec<RerankResult>;
type RerankResult (line 341) | pub struct RerankResult {
type PromptAction (line 346) | pub type PromptAction<'a> = (&'a str, &'a str, Option<&'a str>);
function create_config (line 348) | pub async fn create_config(
function create_openai_compatible_client_config (line 368) | pub async fn create_openai_compatible_client_config(
function call_chat_completions (line 406) | pub async fn call_chat_completions(
function call_chat_completions_streaming (line 441) | pub async fn call_chat_completions_streaming(
function noop_prepare_embeddings (line 477) | pub fn noop_prepare_embeddings<T>(_client: &T, _data: &EmbeddingsData) -...
function noop_embeddings (line 481) | pub async fn noop_embeddings(_builder: RequestBuilder, _model: &Model) -...
function noop_prepare_rerank (line 485) | pub fn noop_prepare_rerank<T>(_client: &T, _data: &RerankData) -> Result...
function noop_rerank (line 489) | pub async fn noop_rerank(_builder: RequestBuilder, _model: &Model) -> Re...
function catch_error (line 493) | pub fn catch_error(data: &Value, status: u16) -> Result<()> {
function json_str_from_map (line 535) | pub fn json_str_from_map<'a>(
function set_client_models_config (line 542) | async fn set_client_models_config(client_config: &mut Value, client: &st...
function select_model (line 645) | fn select_model(model_names: Vec<String>) -> Result<String> {
function prompt_input_string (line 657) | fn prompt_input_string(
FILE: src/client/gemini.rs
constant API_BASE (line 9) | const API_BASE: &str = "https://generativelanguage.googleapis.com/v1beta";
type GeminiConfig (line 12) | pub struct GeminiConfig {
constant PROMPTS (line 26) | pub const PROMPTS: [PromptAction<'static>; 1] = [("api_key", "API Key", ...
function prepare_chat_completions (line 40) | fn prepare_chat_completions(
function prepare_embeddings (line 70) | fn prepare_embeddings(self_: &GeminiClient, data: &EmbeddingsData) -> Re...
function embeddings (line 111) | async fn embeddings(builder: RequestBuilder, _model: &Model) -> Result<E...
type EmbeddingsResBody (line 129) | struct EmbeddingsResBody {
type EmbeddingsResBodyEmbedding (line 134) | struct EmbeddingsResBodyEmbedding {
FILE: src/client/message.rs
type Message (line 8) | pub struct Message {
method new (line 23) | pub fn new(role: MessageRole, content: MessageContent) -> Self {
method merge_system (line 27) | pub fn merge_system(&mut self, system: MessageContent) {
method default (line 14) | fn default() -> Self {
type MessageRole (line 57) | pub enum MessageRole {
method is_system (line 66) | pub fn is_system(&self) -> bool {
method is_user (line 70) | pub fn is_user(&self) -> bool {
method is_assistant (line 74) | pub fn is_assistant(&self) -> bool {
type MessageContent (line 81) | pub enum MessageContent {
method render_input (line 89) | pub fn render_input(
method merge_prompt (line 136) | pub fn merge_prompt(&mut self, replace_fn: impl Fn(&str) -> String) {
method to_text (line 152) | pub fn to_text(&self) -> String {
type MessageContentPart (line 171) | pub enum MessageContentPart {
type ImageUrl (line 177) | pub struct ImageUrl {
type MessageContentToolCalls (line 182) | pub struct MessageContentToolCalls {
method new (line 189) | pub fn new(tool_results: Vec<ToolResult>, text: String) -> Self {
method merge (line 197) | pub fn merge(&mut self, tool_results: Vec<ToolResult>, _text: String) {
function patch_messages (line 204) | pub fn patch_messages(messages: &mut Vec<Message>, model: &Model) {
function extract_system_message (line 229) | pub fn extract_system_message(messages: &mut Vec<Message>) -> Option<Str...
FILE: src/client/mod.rs
constant OPENAI_COMPATIBLE_PROVIDERS (line 36) | pub const OPENAI_COMPATIBLE_PROVIDERS: [(&str, &str); 18] = [
FILE: src/client/model.rs
constant PER_MESSAGES_TOKENS (line 15) | const PER_MESSAGES_TOKENS: usize = 5;
constant BASIS_TOKENS (line 16) | const BASIS_TOKENS: usize = 2;
type Model (line 19) | pub struct Model {
method new (line 31) | pub fn new(client_name: &str, name: &str) -> Self {
method from_config (line 38) | pub fn from_config(client_name: &str, models: &[ModelData]) -> Vec<Sel...
method retrieve_model (line 48) | pub fn retrieve_model(config: &Config, model_id: &str, model_type: Mod...
method id (line 91) | pub fn id(&self) -> String {
method client_name (line 99) | pub fn client_name(&self) -> &str {
method name (line 103) | pub fn name(&self) -> &str {
method real_name (line 107) | pub fn real_name(&self) -> &str {
method model_type (line 111) | pub fn model_type(&self) -> ModelType {
method data (line 121) | pub fn data(&self) -> &ModelData {
method data_mut (line 125) | pub fn data_mut(&mut self) -> &mut ModelData {
method description (line 129) | pub fn description(&self) -> String {
method patch (line 177) | pub fn patch(&self) -> Option<&Value> {
method max_input_tokens (line 181) | pub fn max_input_tokens(&self) -> Option<usize> {
method max_output_tokens (line 185) | pub fn max_output_tokens(&self) -> Option<isize> {
method no_stream (line 189) | pub fn no_stream(&self) -> bool {
method no_system_message (line 193) | pub fn no_system_message(&self) -> bool {
method system_prompt_prefix (line 197) | pub fn system_prompt_prefix(&self) -> Option<&str> {
method max_tokens_per_chunk (line 201) | pub fn max_tokens_per_chunk(&self) -> Option<usize> {
method default_chunk_size (line 205) | pub fn default_chunk_size(&self) -> usize {
method max_batch_size (line 209) | pub fn max_batch_size(&self) -> Option<usize> {
method max_tokens_param (line 213) | pub fn max_tokens_param(&self) -> Option<isize> {
method set_max_tokens (line 221) | pub fn set_max_tokens(
method messages_tokens (line 234) | pub fn messages_tokens(&self, messages: &[Message]) -> usize {
method total_tokens (line 271) | pub fn total_tokens(&self, messages: &[Message]) -> usize {
method guard_max_input_tokens (line 284) | pub fn guard_max_input_tokens(&self, messages: &[Message]) -> Result<(...
method default (line 25) | fn default() -> Self {
type ModelData (line 296) | pub struct ModelData {
method new (line 337) | pub fn new(name: &str) -> Self {
type ProviderModels (line 347) | pub struct ProviderModels {
function default_model_type (line 352) | fn default_model_type() -> String {
type ModelType (line 357) | pub enum ModelType {
method can_create_from_name (line 374) | pub fn can_create_from_name(self) -> bool {
method api_name (line 382) | pub fn api_name(self) -> &'static str {
method extract_patch (line 390) | pub fn extract_patch(self, patch: &RequestPatch) -> Option<&ApiPatch> {
method fmt (line 364) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
function stringify_option_value (line 399) | fn stringify_option_value<T>(value: &Option<T>) -> String
FILE: src/client/openai.rs
constant API_BASE (line 10) | const API_BASE: &str = "https://api.openai.com/v1";
type OpenAIConfig (line 13) | pub struct OpenAIConfig {
constant PROMPTS (line 28) | pub const PROMPTS: [PromptAction<'static>; 1] = [("api_key", "API Key", ...
function prepare_chat_completions (line 42) | fn prepare_chat_completions(
function prepare_embeddings (line 65) | fn prepare_embeddings(self_: &OpenAIClient, data: &EmbeddingsData) -> Re...
function openai_chat_completions (line 85) | pub async fn openai_chat_completions(
function openai_chat_completions_streaming (line 100) | pub async fn openai_chat_completions_streaming(
function openai_embeddings (line 200) | pub async fn openai_embeddings(
type EmbeddingsResBody (line 217) | struct EmbeddingsResBody {
type EmbeddingsResBodyEmbedding (line 222) | struct EmbeddingsResBodyEmbedding {
function openai_build_chat_completions_body (line 226) | pub fn openai_build_chat_completions_body(data: ChatCompletionsData, mod...
function openai_build_embeddings_body (line 346) | pub fn openai_build_embeddings_body(data: &EmbeddingsData, model: &Model...
function openai_extract_chat_completions (line 353) | pub fn openai_extract_chat_completions(data: &Value) -> Result<ChatCompl...
function normalize_function_id (line 402) | fn normalize_function_id(value: &str) -> Option<String> {
FILE: src/client/openai_compatible.rs
type OpenAICompatibleConfig (line 10) | pub struct OpenAICompatibleConfig {
constant PROMPTS (line 24) | pub const PROMPTS: [PromptAction<'static>; 0] = [];
function prepare_chat_completions (line 38) | fn prepare_chat_completions(
function prepare_embeddings (line 58) | fn prepare_embeddings(
function prepare_rerank (line 78) | fn prepare_rerank(self_: &OpenAICompatibleClient, data: &RerankData) -> ...
function get_api_base_ext (line 99) | fn get_api_base_ext(self_: &OpenAICompatibleClient) -> Result<String> {
function generic_rerank (line 120) | pub async fn generic_rerank(builder: RequestBuilder, _model: &Model) -> ...
type GenericRerankResBody (line 140) | pub struct GenericRerankResBody {
function generic_build_rerank_body (line 144) | pub fn generic_build_rerank_body(data: &RerankData, model: &Model) -> Va...
FILE: src/client/stream.rs
type SseHandler (line 11) | pub struct SseHandler {
method new (line 19) | pub fn new(sender: UnboundedSender<SseEvent>, abort_signal: AbortSigna...
method text (line 28) | pub fn text(&mut self, text: &str) -> Result<()> {
method done (line 47) | pub fn done(&mut self) {
method tool_call (line 58) | pub fn tool_call(&mut self, call: ToolCall) -> Result<()> {
method abort (line 64) | pub fn abort(&self) -> AbortSignal {
method tool_calls (line 68) | pub fn tool_calls(&self) -> &[ToolCall] {
method take (line 72) | pub fn take(self) -> (String, Vec<ToolCall>) {
type SseEvent (line 81) | pub enum SseEvent {
type SseMmessage (line 87) | pub struct SseMmessage {
function sse_stream (line 93) | pub async fn sse_stream<F>(builder: RequestBuilder, mut handle: F) -> Re...
function json_stream (line 144) | pub async fn json_stream<S, F, E>(mut stream: S, mut handle: F) -> Resul...
type JsonStreamParser (line 175) | struct JsonStreamParser {
method process (line 185) | fn process<F>(&mut self, text: &str, handle: &mut F) -> Result<()>
function split_chunks (line 248) | fn split_chunks(text: &str) -> Vec<Vec<u8>> {
function test_json_stream_ndjson (line 278) | async fn test_json_stream_ndjson() {
function test_json_stream_array (line 286) | async fn test_json_stream_array() {
FILE: src/client/vertexai.rs
type VertexAIConfig (line 14) | pub struct VertexAIConfig {
constant PROMPTS (line 29) | pub const PROMPTS: [PromptAction<'static>; 2] = [
method chat_completions_inner (line 39) | async fn chat_completions_inner(
method chat_completions_streaming_inner (line 56) | async fn chat_completions_streaming_inner(
method embeddings_inner (line 80) | async fn embeddings_inner(
function prepare_chat_completions (line 92) | fn prepare_chat_completions(
function prepare_embeddings (line 155) | fn prepare_embeddings(self_: &VertexAIClient, data: &EmbeddingsData) -> ...
function gemini_chat_completions (line 183) | pub async fn gemini_chat_completions(
function gemini_chat_completions_streaming (line 197) | pub async fn gemini_chat_completions_streaming(
function embeddings (line 239) | async fn embeddings(builder: RequestBuilder, _model: &Model) -> Result<E...
type EmbeddingsResBody (line 257) | struct EmbeddingsResBody {
type EmbeddingsResBodyPrediction (line 262) | struct EmbeddingsResBodyPrediction {
type EmbeddingsResBodyPredictionEmbeddings (line 267) | struct EmbeddingsResBodyPredictionEmbeddings {
function gemini_extract_chat_completions_text (line 271) | fn gemini_extract_chat_completions_text(data: &Value) -> Result<ChatComp...
function gemini_build_chat_completions_body (line 309) | pub fn gemini_build_chat_completions_body(
type ModelCategory (line 428) | enum ModelCategory {
type Err (line 435) | type Err = anyhow::Error;
method from_str (line 437) | fn from_str(s: &str) -> std::result::Result<Self, Self::Err> {
function prepare_gcloud_access_token (line 450) | pub async fn prepare_gcloud_access_token(
function fetch_access_token (line 467) | async fn fetch_access_token(
function load_adc (line 491) | async fn load_adc(file: &Option<String>) -> Result<Value> {
function default_adc_file (line 516) | fn default_adc_file() -> Option<PathBuf> {
function default_adc_file (line 525) | fn default_adc_file() -> Option<PathBuf> {
function strip_model_version (line 532) | fn strip_model_version(name: &str) -> &str {
FILE: src/config/agent.rs
constant DEFAULT_AGENT_NAME (line 14) | const DEFAULT_AGENT_NAME: &str = "rag";
type AgentVariables (line 16) | pub type AgentVariables = IndexMap<String, String>;
type Agent (line 19) | pub struct Agent {
method init (line 33) | pub async fn init(
method init_agent_variables (line 121) | pub fn init_agent_variables(
method export (line 183) | pub fn export(&self) -> Result<String> {
method banner (line 210) | pub fn banner(&self) -> String {
method name (line 214) | pub fn name(&self) -> &str {
method functions (line 218) | pub fn functions(&self) -> &Functions {
method rag (line 222) | pub fn rag(&self) -> Option<Arc<Rag>> {
method conversation_staters (line 226) | pub fn conversation_staters(&self) -> &[String] {
method interpolated_instructions (line 230) | pub fn interpolated_instructions(&self) -> String {
method agent_prelude (line 244) | pub fn agent_prelude(&self) -> Option<&str> {
method variables (line 248) | pub fn variables(&self) -> &AgentVariables {
method variable_envs (line 255) | pub fn variable_envs(&self) -> HashMap<String, String> {
method config_variables (line 267) | pub fn config_variables(&self) -> &AgentVariables {
method shared_variables (line 271) | pub fn shared_variables(&self) -> &AgentVariables {
method set_shared_variables (line 275) | pub fn set_shared_variables(&mut self, shared_variables: AgentVariable...
method set_session_variables (line 279) | pub fn set_session_variables(&mut self, session_variables: AgentVariab...
method defined_variables (line 283) | pub fn defined_variables(&self) -> &[AgentVariable] {
method exit_session (line 287) | pub fn exit_session(&mut self) {
method is_dynamic_instructions (line 292) | pub fn is_dynamic_instructions(&self) -> bool {
method update_shared_dynamic_instructions (line 296) | pub fn update_shared_dynamic_instructions(&mut self, force: bool) -> R...
method update_session_dynamic_instructions (line 303) | pub fn update_session_dynamic_instructions(&mut self, value: Option<St...
method run_instructions_fn (line 314) | fn run_instructions_fn(&self) -> Result<String> {
method to_role (line 328) | fn to_role(&self) -> Role {
method model (line 335) | fn model(&self) -> &Model {
method temperature (line 339) | fn temperature(&self) -> Option<f64> {
method top_p (line 343) | fn top_p(&self) -> Option<f64> {
method use_tools (line 347) | fn use_tools(&self) -> Option<String> {
method set_model (line 351) | fn set_model(&mut self, model: Model) {
method set_temperature (line 356) | fn set_temperature(&mut self, value: Option<f64>) {
method set_top_p (line 360) | fn set_top_p(&mut self, value: Option<f64>) {
method set_use_tools (line 364) | fn set_use_tools(&mut self, value: Option<String>) {
type AgentConfig (line 370) | pub struct AgentConfig {
method new (line 388) | pub fn new(config: &Config) -> Self {
method load (line 396) | pub fn load(path: &Path) -> Result<Self> {
method load_envs (line 404) | fn load_envs(&mut self, name: &str) {
type AgentDefinition (line 434) | pub struct AgentDefinition {
method load (line 453) | pub fn load(path: &Path) -> Result<Self> {
method banner (line 461) | fn banner(&self) -> String {
method replace_tools_placeholder (line 490) | fn replace_tools_placeholder(&mut self, functions: &Functions) {
type AgentVariable (line 512) | pub struct AgentVariable {
function list_agents (line 521) | pub fn list_agents() -> Vec<String> {
function complete_agent_variables (line 540) | pub fn complete_agent_variables(agent_name: &str) -> Vec<(String, Option...
FILE: src/config/input.rs
constant IMAGE_EXTS (line 15) | const IMAGE_EXTS: [&str; 5] = ["png", "jpeg", "jpg", "webp", "gif"];
constant SUMMARY_MAX_WIDTH (line 16) | const SUMMARY_MAX_WIDTH: usize = 80;
type Input (line 19) | pub struct Input {
method from_str (line 37) | pub fn from_str(config: &GlobalConfig, text: &str, role: Option<Role>)...
method from_files (line 57) | pub async fn from_files(
method from_files_with_spinner (line 124) | pub async fn from_files_with_spinner(
method is_empty (line 139) | pub fn is_empty(&self) -> bool {
method data_urls (line 143) | pub fn data_urls(&self) -> HashMap<String, String> {
method tool_calls (line 147) | pub fn tool_calls(&self) -> &Option<MessageContentToolCalls> {
method text (line 151) | pub fn text(&self) -> String {
method clear_patch (line 158) | pub fn clear_patch(&mut self) {
method set_text (line 162) | pub fn set_text(&mut self, text: String) {
method stream (line 166) | pub fn stream(&self) -> bool {
method continue_output (line 170) | pub fn continue_output(&self) -> Option<&str> {
method set_continue_output (line 174) | pub fn set_continue_output(&mut self, output: &str) {
method regenerate (line 182) | pub fn regenerate(&self) -> bool {
method set_regenerate (line 186) | pub fn set_regenerate(&mut self) {
method use_embeddings (line 195) | pub async fn use_embeddings(&mut self, abort_signal: AbortSignal) -> R...
method rag_name (line 208) | pub fn rag_name(&self) -> Option<&str> {
method merge_tool_results (line 212) | pub fn merge_tool_results(mut self, output: String, tool_results: Vec<...
method create_client (line 222) | pub fn create_client(&self) -> Result<Box<dyn Client>> {
method fetch_chat_text (line 226) | pub async fn fetch_chat_text(&self) -> Result<String> {
method prepare_completion_data (line 233) | pub fn prepare_completion_data(
method build_messages (line 252) | pub fn build_messages(&self) -> Result<Vec<Message>> {
method echo_messages (line 267) | pub fn echo_messages(&self) -> String {
method role (line 275) | pub fn role(&self) -> &Role {
method session (line 279) | pub fn session<'a>(&self, session: &'a Option<Session>) -> Option<&'a ...
method session_mut (line 287) | pub fn session_mut<'a>(&self, session: &'a mut Option<Session>) -> Opt...
method with_agent (line 295) | pub fn with_agent(&self) -> bool {
method summary (line 299) | pub fn summary(&self) -> String {
method raw (line 323) | pub fn raw(&self) -> String {
method render (line 338) | pub fn render(&self) -> String {
method message_content (line 357) | pub fn message_content(&self) -> MessageContent {
function resolve_role (line 377) | fn resolve_role(config: &Config, role: Option<Role>) -> (Role, bool, boo...
type ResolvePathsOutput (line 388) | type ResolvePathsOutput = (
function resolve_paths (line 397) | fn resolve_paths(
function load_documents (line 441) | async fn load_documents(
function resolve_data_url (line 505) | pub fn resolve_data_url(data_urls: &HashMap<String, String>, data_url: S...
function is_image (line 517) | fn is_image(path: &str) -> bool {
function read_media_to_data_url (line 523) | fn read_media_to_data_url(image_path: &str) -> Result<String> {
FILE: src/config/mod.rs
constant TEMP_ROLE_NAME (line 44) | pub const TEMP_ROLE_NAME: &str = "%%";
constant TEMP_RAG_NAME (line 45) | pub const TEMP_RAG_NAME: &str = "temp";
constant TEMP_SESSION_NAME (line 46) | pub const TEMP_SESSION_NAME: &str = "temp";
constant DARK_THEME (line 49) | const DARK_THEME: &[u8] = include_bytes!("../../assets/monokai-extended....
constant LIGHT_THEME (line 50) | const LIGHT_THEME: &[u8] = include_bytes!("../../assets/monokai-extended...
constant CONFIG_FILE_NAME (line 52) | const CONFIG_FILE_NAME: &str = "config.yaml";
constant ROLES_DIR_NAME (line 53) | const ROLES_DIR_NAME: &str = "roles";
constant MACROS_DIR_NAME (line 54) | const MACROS_DIR_NAME: &str = "macros";
constant ENV_FILE_NAME (line 55) | const ENV_FILE_NAME: &str = ".env";
constant MESSAGES_FILE_NAME (line 56) | const MESSAGES_FILE_NAME: &str = "messages.md";
constant SESSIONS_DIR_NAME (line 57) | const SESSIONS_DIR_NAME: &str = "sessions";
constant RAGS_DIR_NAME (line 58) | const RAGS_DIR_NAME: &str = "rags";
constant FUNCTIONS_DIR_NAME (line 59) | const FUNCTIONS_DIR_NAME: &str = "functions";
constant FUNCTIONS_FILE_NAME (line 60) | const FUNCTIONS_FILE_NAME: &str = "functions.json";
constant FUNCTIONS_BIN_DIR_NAME (line 61) | const FUNCTIONS_BIN_DIR_NAME: &str = "bin";
constant AGENTS_DIR_NAME (line 62) | const AGENTS_DIR_NAME: &str = "agents";
constant CLIENTS_FIELD (line 64) | const CLIENTS_FIELD: &str = "clients";
constant SERVE_ADDR (line 66) | const SERVE_ADDR: &str = "127.0.0.1:8000";
constant SYNC_MODELS_URL (line 68) | const SYNC_MODELS_URL: &str =
constant SUMMARIZE_PROMPT (line 71) | const SUMMARIZE_PROMPT: &str =
constant SUMMARY_PROMPT (line 73) | const SUMMARY_PROMPT: &str = "This is a summary of the chat history as a...
constant RAG_TEMPLATE (line 75) | const RAG_TEMPLATE: &str = r#"Answer the query based on the context whil...
constant LEFT_PROMPT (line 94) | const LEFT_PROMPT: &str = "{color.green}{?session {?agent {agent}>}{sess...
constant RIGHT_PROMPT (line 95) | const RIGHT_PROMPT: &str = "{color.purple}{?session {?consume_tokens {co...
type Config (line 101) | pub struct Config {
method init (line 246) | pub async fn init(working_mode: WorkingMode, info_flag: bool) -> Resul...
method config_dir (line 289) | pub fn config_dir() -> PathBuf {
method local_path (line 300) | pub fn local_path(name: &str) -> PathBuf {
method config_file (line 304) | pub fn config_file() -> PathBuf {
method roles_dir (line 311) | pub fn roles_dir() -> PathBuf {
method role_file (line 318) | pub fn role_file(name: &str) -> PathBuf {
method macros_dir (line 322) | pub fn macros_dir() -> PathBuf {
method macro_file (line 329) | pub fn macro_file(name: &str) -> PathBuf {
method env_file (line 333) | pub fn env_file() -> PathBuf {
method messages_file (line 340) | pub fn messages_file(&self) -> PathBuf {
method sessions_dir (line 350) | pub fn sessions_dir(&self) -> PathBuf {
method rags_dir (line 360) | pub fn rags_dir() -> PathBuf {
method functions_dir (line 367) | pub fn functions_dir() -> PathBuf {
method functions_file (line 374) | pub fn functions_file() -> PathBuf {
method functions_bin_dir (line 378) | pub fn functions_bin_dir() -> PathBuf {
method session_file (line 382) | pub fn session_file(&self, name: &str) -> PathBuf {
method rag_file (line 389) | pub fn rag_file(&self, name: &str) -> PathBuf {
method agents_data_dir (line 396) | pub fn agents_data_dir() -> PathBuf {
method agent_data_dir (line 400) | pub fn agent_data_dir(name: &str) -> PathBuf {
method agent_config_file (line 407) | pub fn agent_config_file(name: &str) -> PathBuf {
method agent_rag_file (line 414) | pub fn agent_rag_file(agent_name: &str, rag_name: &str) -> PathBuf {
method agents_functions_dir (line 418) | pub fn agents_functions_dir() -> PathBuf {
method agent_functions_dir (line 422) | pub fn agent_functions_dir(name: &str) -> PathBuf {
method models_override_file (line 429) | pub fn models_override_file() -> PathBuf {
method state (line 433) | pub fn state(&self) -> StateFlags {
method serve_addr (line 456) | pub fn serve_addr(&self) -> String {
method log_config (line 460) | pub fn log_config(is_serve: bool) -> Result<(LevelFilter, Option<PathB...
method edit_config (line 490) | pub fn edit_config(&self) -> Result<()> {
method current_model (line 502) | pub fn current_model(&self) -> &Model {
method role_like_mut (line 514) | pub fn role_like_mut(&mut self) -> Option<&mut dyn RoleLike> {
method extract_role (line 526) | pub fn extract_role(&self) -> Role {
method info (line 545) | pub fn info(&self) -> Result<String> {
method sysinfo (line 570) | pub fn sysinfo(&self) -> Result<String> {
method update (line 629) | pub fn update(config: &GlobalConfig, data: &str) -> Result<()> {
method delete (line 697) | pub fn delete(config: &GlobalConfig, kind: &str) -> Result<()> {
method set_temperature (line 766) | pub fn set_temperature(&mut self, value: Option<f64>) {
method set_top_p (line 773) | pub fn set_top_p(&mut self, value: Option<f64>) {
method set_use_tools (line 780) | pub fn set_use_tools(&mut self, value: Option<String>) {
method set_save_session (line 787) | pub fn set_save_session(&mut self, value: Option<bool>) {
method set_compress_threshold (line 795) | pub fn set_compress_threshold(&mut self, value: Option<usize>) {
method set_rag_reranker_model (line 803) | pub fn set_rag_reranker_model(config: &GlobalConfig, value: Option<Str...
method set_rag_top_k (line 818) | pub fn set_rag_top_k(config: &GlobalConfig, value: usize) -> Result<()> {
method set_wrap (line 830) | pub fn set_wrap(&mut self, value: &str) -> Result<()> {
method set_max_output_tokens (line 844) | pub fn set_max_output_tokens(&mut self, value: Option<isize>) {
method set_model (line 857) | pub fn set_model(&mut self, model_id: &str) -> Result<()> {
method use_prompt (line 868) | pub fn use_prompt(&mut self, prompt: &str) -> Result<()> {
method use_role (line 874) | pub fn use_role(&mut self, name: &str) -> Result<()> {
method use_role_obj (line 879) | pub fn use_role_obj(&mut self, role: Role) -> Result<()> {
method role_info (line 892) | pub fn role_info(&self) -> Result<String> {
method exit_role (line 907) | pub fn exit_role(&mut self) -> Result<()> {
method retrieve_role (line 917) | pub fn retrieve_role(&self, name: &str) -> Result<Role> {
method new_role (line 949) | pub fn new_role(&mut self, name: &str) -> Result<()> {
method edit_role (line 964) | pub fn edit_role(&mut self) -> Result<()> {
method upsert_role (line 984) | pub fn upsert_role(&mut self, name: &str) -> Result<()> {
method save_role (line 995) | pub fn save_role(&mut self, name: Option<&str>) -> Result<()> {
method all_roles (line 1030) | pub fn all_roles() -> Vec<Role> {
method list_roles (line 1047) | pub fn list_roles(with_builtin: bool) -> Vec<String> {
method has_role (line 1068) | pub fn has_role(name: &str) -> bool {
method use_session (line 1073) | pub fn use_session(&mut self, session_name: Option<&str>) -> Result<()> {
method session_info (line 1129) | pub fn session_info(&self) -> Result<String> {
method exit_session (line 1148) | pub fn exit_session(&mut self) -> Result<()> {
method save_session (line 1157) | pub fn save_session(&mut self, name: Option<&str>) -> Result<()> {
method edit_session (line 1175) | pub fn edit_session(&mut self) -> Result<()> {
method empty_session (line 1194) | pub fn empty_session(&mut self) -> Result<()> {
method set_save_session_this_time (line 1207) | pub fn set_save_session_this_time(&mut self) -> Result<()> {
method list_sessions (line 1216) | pub fn list_sessions(&self) -> Vec<String> {
method list_autoname_sessions (line 1220) | pub fn list_autoname_sessions(&self) -> Vec<String> {
method maybe_compress_session (line 1224) | pub fn maybe_compress_session(config: GlobalConfig) {
method compress_session (line 1258) | pub async fn compress_session(config: &GlobalConfig) -> Result<()> {
method is_compressing_session (line 1287) | pub fn is_compressing_session(&self) -> bool {
method maybe_autoname_session (line 1294) | pub fn maybe_autoname_session(config: GlobalConfig) {
method autoname_session (line 1321) | pub async fn autoname_session(config: &GlobalConfig) -> Result<()> {
method use_rag (line 1340) | pub async fn use_rag(
method edit_rag_docs (line 1374) | pub async fn edit_rag_docs(config: &GlobalConfig, abort_signal: AbortS...
method rebuild_rag (line 1410) | pub async fn rebuild_rag(config: &GlobalConfig, abort_signal: AbortSig...
method rag_sources (line 1422) | pub fn rag_sources(config: &GlobalConfig) -> Result<String> {
method rag_info (line 1432) | pub fn rag_info(&self) -> Result<String> {
method exit_rag (line 1440) | pub fn exit_rag(&mut self) -> Result<()> {
method search_rag (line 1445) | pub async fn search_rag(
method list_rags (line 1460) | pub fn list_rags() -> Vec<String> {
method rag_template (line 1477) | pub fn rag_template(&self, embeddings: &str, text: &str) -> String {
method use_agent (line 1488) | pub async fn use_agent(
method agent_info (line 1518) | pub fn agent_info(&self) -> Result<String> {
method agent_banner (line 1526) | pub fn agent_banner(&self) -> Result<String> {
method edit_agent_config (line 1534) | pub fn edit_agent_config(&self) -> Result<()> {
method exit_agent (line 1557) | pub fn exit_agent(&mut self) -> Result<()> {
method exit_agent_session (line 1566) | pub fn exit_agent_session(&mut self) -> Result<()> {
method list_macros (line 1577) | pub fn list_macros() -> Vec<String> {
method load_macro (line 1581) | pub fn load_macro(name: &str) -> Result<Macro> {
method has_macro (line 1589) | pub fn has_macro(name: &str) -> bool {
method new_macro (line 1594) | pub fn new_macro(&mut self, name: &str) -> Result<()> {
method apply_prelude (line 1612) | pub fn apply_prelude(&mut self) -> Result<()> {
method select_functions (line 1652) | pub fn select_functions(&self, role: &Role) -> Option<Vec<FunctionDecl...
method editor (line 1721) | pub fn editor(&self) -> Result<String> {
method repl_complete (line 1738) | pub fn repl_complete(
method sync_models_url (line 1863) | pub fn sync_models_url(&self) -> String {
method sync_models (line 1869) | pub async fn sync_models(url: &str, abort_signal: AbortSignal) -> Resu...
method loal_models_override (line 1891) | pub fn loal_models_override() -> Result<Vec<ProviderModels>> {
method light_theme (line 1907) | pub fn light_theme(&self) -> bool {
method render_options (line 1911) | pub fn render_options(&self) -> Result<RenderOptions> {
method render_prompt_left (line 1943) | pub fn render_prompt_left(&self) -> String {
method render_prompt_right (line 1949) | pub fn render_prompt_right(&self) -> String {
method print_markdown (line 1955) | pub fn print_markdown(&self, text: &str) -> Result<()> {
method generate_prompt_context (line 1966) | fn generate_prompt_context(&self) -> HashMap<&str, String> {
method before_chat_completion (line 2049) | pub fn before_chat_completion(&mut self, input: &Input) -> Result<()> {
method after_chat_completion (line 2054) | pub fn after_chat_completion(
method discontinuous_last_message (line 2070) | fn discontinuous_last_message(&mut self) {
method save_message (line 2076) | fn save_message(&mut self, input: &Input, output: &str) -> Result<()> {
method init_agent_shared_variables (line 2130) | fn init_agent_shared_variables(&mut self) -> Result<()> {
method init_agent_session_variables (line 2153) | fn init_agent_session_variables(&mut self, new_session: bool) -> Resul...
method open_message_file (line 2191) | fn open_message_file(&self) -> Result<File> {
method load_from_file (line 2201) | fn load_from_file(config_path: &Path) -> Result<Self> {
method load_dynamic (line 2225) | fn load_dynamic(model_id: &str) -> Result<Self> {
method load_envs (line 2248) | fn load_envs(&mut self) {
method load_functions (line 2383) | fn load_functions(&mut self) -> Result<()> {
method setup_model (line 2388) | fn setup_model(&mut self) -> Result<()> {
method setup_document_loaders (line 2402) | fn setup_document_loaders(&mut self) {
method setup_user_agent (line 2411) | fn setup_user_agent(&mut self) {
method default (line 178) | fn default() -> Self {
type GlobalConfig (line 243) | pub type GlobalConfig = Arc<RwLock<Config>>;
function load_env_file (line 2422) | pub fn load_env_file() -> Result<()> {
type WorkingMode (line 2442) | pub enum WorkingMode {
method is_cmd (line 2449) | pub fn is_cmd(&self) -> bool {
method is_repl (line 2452) | pub fn is_repl(&self) -> bool {
method is_serve (line 2455) | pub fn is_serve(&self) -> bool {
function macro_execute (line 2461) | pub async fn macro_execute(
type Macro (line 2498) | pub struct Macro {
method resolve_variables (line 2505) | pub fn resolve_variables(&self, args: &[String]) -> Result<IndexMap<St...
method usage (line 2526) | pub fn usage(&self, name: &str) -> String {
method interpolate_command (line 2543) | pub fn interpolate_command(command: &str, variables: &IndexMap<String,...
type MacroVariable (line 2553) | pub struct MacroVariable {
type ModelsOverride (line 2561) | pub struct ModelsOverride {
type LastMessage (line 2567) | pub struct LastMessage {
method new (line 2574) | pub fn new(input: Input, output: String) -> Self {
type AssertState (line 2595) | pub enum AssertState {
method pass (line 2603) | pub fn pass() -> Self {
method bare (line 2607) | pub fn bare() -> Self {
method assert (line 2611) | pub fn assert(self, flags: StateFlags) -> bool {
function create_config_file (line 2624) | async fn create_config_file(config_path: &Path) -> Result<()> {
function ensure_parent_exists (line 2659) | pub(crate) fn ensure_parent_exists(path: &Path) -> Result<()> {
function read_env_value (line 2677) | fn read_env_value<T>(key: &str) -> Option<Option<T>>
function parse_value (line 2686) | fn parse_value<T>(value: &str) -> Result<Option<T>>
function read_env_bool (line 2702) | fn read_env_bool(key: &str) -> Option<Option<bool>> {
function complete_bool (line 2707) | fn complete_bool(value: bool) -> Vec<String> {
function complete_option_bool (line 2711) | fn complete_option_bool(value: Option<bool>) -> Vec<String> {
function map_completion_values (line 2719) | fn map_completion_values<T: ToString>(value: Vec<T>) -> Vec<(String, Opt...
function update_rag (line 2723) | fn update_rag<F>(config: &GlobalConfig, f: F) -> Result<()>
function format_option_value (line 2736) | fn format_option_value<T>(value: &Option<T>) -> String
FILE: src/config/role.rs
constant SHELL_ROLE (line 12) | pub const SHELL_ROLE: &str = "%shell%";
constant EXPLAIN_SHELL_ROLE (line 13) | pub const EXPLAIN_SHELL_ROLE: &str = "%explain-shell%";
constant CODE_ROLE (line 14) | pub const CODE_ROLE: &str = "%code%";
constant CREATE_TITLE_ROLE (line 15) | pub const CREATE_TITLE_ROLE: &str = "%create-title%";
constant INPUT_PLACEHOLDER (line 17) | pub const INPUT_PLACEHOLDER: &str = "__INPUT__";
type RolesAsset (line 21) | struct RolesAsset;
type RoleLike (line 26) | pub trait RoleLike {
method to_role (line 27) | fn to_role(&self) -> Role;
method model (line 28) | fn model(&self) -> &Model;
method temperature (line 29) | fn temperature(&self) -> Option<f64>;
method top_p (line 30) | fn top_p(&self) -> Option<f64>;
method use_tools (line 31) | fn use_tools(&self) -> Option<String>;
method set_model (line 32) | fn set_model(&mut self, model: Model);
method set_temperature (line 33) | fn set_temperature(&mut self, value: Option<f64>);
method set_top_p (line 34) | fn set_top_p(&mut self, value: Option<f64>);
method set_use_tools (line 35) | fn set_use_tools(&mut self, value: Option<String>);
method to_role (line 263) | fn to_role(&self) -> Role {
method model (line 267) | fn model(&self) -> &Model {
method temperature (line 271) | fn temperature(&self) -> Option<f64> {
method top_p (line 275) | fn top_p(&self) -> Option<f64> {
method use_tools (line 279) | fn use_tools(&self) -> Option<String> {
method set_model (line 283) | fn set_model(&mut self, model: Model) {
method set_temperature (line 290) | fn set_temperature(&mut self, value: Option<f64>) {
method set_top_p (line 294) | fn set_top_p(&mut self, value: Option<f64>) {
method set_use_tools (line 298) | fn set_use_tools(&mut self, value: Option<String>) {
type Role (line 39) | pub struct Role {
method new (line 60) | pub fn new(name: &str, content: &str) -> Self {
method builtin (line 94) | pub fn builtin(name: &str) -> Result<Self> {
method list_builtin_role_names (line 101) | pub fn list_builtin_role_names() -> Vec<String> {
method list_builtin_roles (line 107) | pub fn list_builtin_roles() -> Vec<Self> {
method has_args (line 113) | pub fn has_args(&self) -> bool {
method export (line 117) | pub fn export(&self) -> String {
method save (line 140) | pub fn save(&mut self, role_name: &str, role_path: &Path, is_repl: boo...
method sync (line 163) | pub fn sync<T: RoleLike>(&mut self, role_like: &T) {
method batch_set (line 171) | pub fn batch_set(
method is_derived (line 190) | pub fn is_derived(&self) -> bool {
method name (line 194) | pub fn name(&self) -> &str {
method model_id (line 198) | pub fn model_id(&self) -> Option<&str> {
method prompt (line 202) | pub fn prompt(&self) -> &str {
method is_empty_prompt (line 206) | pub fn is_empty_prompt(&self) -> bool {
method is_embedded_prompt (line 210) | pub fn is_embedded_prompt(&self) -> bool {
method echo_messages (line 214) | pub fn echo_messages(&self, input: &Input) -> String {
method build_messages (line 225) | pub fn build_messages(&self, input: &Input) -> Vec<Message> {
function parse_structure_prompt (line 303) | fn parse_structure_prompt(prompt: &str) -> (&str, Vec<(&str, &str)>) {
function test_parse_structure_prompt1 (line 356) | fn test_parse_structure_prompt1() {
function test_parse_structure_prompt2 (line 371) | fn test_parse_structure_prompt2() {
function test_parse_structure_prompt3 (line 385) | fn test_parse_structure_prompt3() {
FILE: src/config/session.rs
type Session (line 20) | pub struct Session {
method new (line 68) | pub fn new(config: &Config, name: &str) -> Self {
method load (line 80) | pub fn load(config: &Config, name: &str, path: &Path) -> Result<Self> {
method is_empty (line 110) | pub fn is_empty(&self) -> bool {
method name (line 114) | pub fn name(&self) -> &str {
method role_name (line 118) | pub fn role_name(&self) -> Option<&str> {
method dirty (line 122) | pub fn dirty(&self) -> bool {
method save_session (line 126) | pub fn save_session(&self) -> Option<bool> {
method tokens (line 130) | pub fn tokens(&self) -> usize {
method update_tokens (line 134) | pub fn update_tokens(&mut self) {
method has_user_messages (line 138) | pub fn has_user_messages(&self) -> bool {
method user_messages_len (line 142) | pub fn user_messages_len(&self) -> usize {
method export (line 146) | pub fn export(&self) -> Result<String> {
method render (line 178) | pub fn render(
method tokens_usage (line 258) | pub fn tokens_usage(&self) -> (usize, f32) {
method set_role (line 270) | pub fn set_role(&mut self, role: Role) {
method clear_role (line 282) | pub fn clear_role(&mut self) {
method sync_agent (line 287) | pub fn sync_agent(&mut self, agent: &Agent) {
method agent_variables (line 294) | pub fn agent_variables(&self) -> &AgentVariables {
method agent_instructions (line 298) | pub fn agent_instructions(&self) -> &str {
method set_save_session (line 302) | pub fn set_save_session(&mut self, value: Option<bool>) {
method set_save_session_this_time (line 309) | pub fn set_save_session_this_time(&mut self) {
method set_compress_threshold (line 313) | pub fn set_compress_threshold(&mut self, value: Option<usize>) {
method need_compress (line 320) | pub fn need_compress(&self, global_compress_threshold: usize) -> bool {
method compressing (line 331) | pub fn compressing(&self) -> bool {
method set_compressing (line 335) | pub fn set_compressing(&mut self, compressing: bool) {
method compress (line 339) | pub fn compress(&mut self, mut prompt: String) {
method need_autoname (line 360) | pub fn need_autoname(&self) -> bool {
method set_autonaming (line 364) | pub fn set_autonaming(&mut self, naming: bool) {
method chat_history_for_autonaming (line 370) | pub fn chat_history_for_autonaming(&self) -> Option<String> {
method autoname (line 374) | pub fn autoname(&self) -> Option<&str> {
method set_autoname (line 378) | pub fn set_autoname(&mut self, value: &str) {
method exit (line 386) | pub fn exit(&mut self, session_dir: &Path, is_repl: bool) -> Result<()> {
method save (line 434) | pub fn save(&mut self, session_name: &str, session_path: &Path, is_rep...
method guard_empty (line 462) | pub fn guard_empty(&self) -> Result<()> {
method add_message (line 469) | pub fn add_message(&mut self, input: &Input, output: &str) -> Result<(...
method clear_messages (line 511) | pub fn clear_messages(&mut self) {
method echo_messages (line 520) | pub fn echo_messages(&self, input: &Input) -> String {
method build_messages (line 525) | pub fn build_messages(&self, input: &Input) -> Vec<Message> {
method to_role (line 561) | fn to_role(&self) -> Role {
method model (line 568) | fn model(&self) -> &Model {
method temperature (line 572) | fn temperature(&self) -> Option<f64> {
method top_p (line 576) | fn top_p(&self) -> Option<f64> {
method use_tools (line 580) | fn use_tools(&self) -> Option<String> {
method set_model (line 584) | fn set_model(&mut self, model: Model) {
method set_temperature (line 593) | fn set_temperature(&mut self, value: Option<f64>) {
method set_top_p (line 600) | fn set_top_p(&mut self, value: Option<f64>) {
method set_use_tools (line 607) | fn set_use_tools(&mut self, value: Option<String>) {
type AutoName (line 616) | struct AutoName {
method new (line 623) | pub fn new(name: String) -> Self {
method new_from_chat_history (line 629) | pub fn new_from_chat_history(chat_history: String) -> Self {
method need (line 635) | pub fn need(&self) -> bool {
FILE: src/function.rs
constant PATH_SEP (line 17) | const PATH_SEP: &str = ";";
constant PATH_SEP (line 19) | const PATH_SEP: &str = ":";
function eval_tool_calls (line 21) | pub fn eval_tool_calls(config: &GlobalConfig, mut calls: Vec<ToolCall>) ...
type ToolResult (line 47) | pub struct ToolResult {
method new (line 53) | pub fn new(call: ToolCall, output: Value) -> Self {
type Functions (line 59) | pub struct Functions {
method init (line 64) | pub fn init(declarations_path: &Path) -> Result<Self> {
method find (line 81) | pub fn find(&self, name: &str) -> Option<&FunctionDeclaration> {
method contains (line 85) | pub fn contains(&self, name: &str) -> bool {
method declarations (line 89) | pub fn declarations(&self) -> &[FunctionDeclaration] {
method is_empty (line 93) | pub fn is_empty(&self) -> bool {
type FunctionDeclaration (line 99) | pub struct FunctionDeclaration {
type JsonSchema (line 108) | pub struct JsonSchema {
method is_empty_properties (line 128) | pub fn is_empty_properties(&self) -> bool {
type ToolCall (line 137) | pub struct ToolCall {
method dedup (line 146) | pub fn dedup(calls: Vec<Self>) -> Vec<Self> {
method new (line 165) | pub fn new(name: String, arguments: Value, id: Option<String>) -> Self {
method eval (line 173) | pub fn eval(&self, config: &GlobalConfig) -> Result<Value> {
method extract_call_config_from_agent (line 205) | fn extract_call_config_from_agent(
method extract_call_config_from_config (line 234) | fn extract_call_config_from_config(&self, config: &GlobalConfig) -> Re...
type CallConfig (line 143) | type CallConfig = (String, String, Vec<String>, HashMap<String, String>);
function run_llm_function (line 248) | pub fn run_llm_function(
function polyfill_cmd_name (line 296) | fn polyfill_cmd_name<T: AsRef<Path>>(cmd_name: &str, bin_dir: &[T]) -> S...
FILE: src/main.rs
function main (line 35) | async fn main() -> Result<()> {
function run (line 63) | async fn run(config: GlobalConfig, cli: Cli, text: Option<String>) -> Re...
function start_directive (line 196) | async fn start_directive(
function start_interactive (line 235) | async fn start_interactive(config: &GlobalConfig) -> Result<()> {
function shell_execute (line 241) | async fn shell_execute(
function create_input (line 328) | async fn create_input(
function setup_logger (line 352) | fn setup_logger(is_serve: bool) -> Result<()> {
FILE: src/rag/mod.rs
type Rag (line 21) | pub struct Rag {
method init (line 59) | pub async fn init(
method load (line 102) | pub fn load(config: &GlobalConfig, name: &str, path: &Path) -> Result<...
method create (line 109) | pub fn create(config: &GlobalConfig, name: &str, path: &Path, data: Ra...
method document_paths (line 127) | pub fn document_paths(&self) -> &[String] {
method refresh_document_paths (line 131) | pub async fn refresh_document_paths(
method create_config (line 152) | pub fn create_config(config: &GlobalConfig) -> Result<(Model, usize, u...
method get_config (line 198) | pub fn get_config(&self) -> (Option<String>, usize) {
method get_last_sources (line 202) | pub fn get_last_sources(&self) -> Option<String> {
method set_last_sources (line 206) | pub fn set_last_sources(&self, ids: &[DocumentId]) {
method set_reranker_model (line 231) | pub fn set_reranker_model(&mut self, reranker_model: Option<String>) -...
method set_top_k (line 237) | pub fn set_top_k(&mut self, top_k: usize) -> Result<()> {
method save (line 243) | pub fn save(&self) -> Result<bool> {
method export (line 259) | pub fn export(&self) -> Result<String> {
method name (line 287) | pub fn name(&self) -> &str {
method is_temp (line 291) | pub fn is_temp(&self) -> bool {
method search (line 295) | pub async fn search(
method sync_documents (line 313) | pub async fn sync_documents(
method hybird_search (line 509) | async fn hybird_search(
method vector_search (line 577) | async fn vector_search(
method keyword_search (line 611) | async fn keyword_search(
method create_embeddings (line 632) | async fn create_embeddings(
method fmt (line 33) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
method clone (line 44) | fn clone(&self) -> Self {
type RagData (line 693) | pub struct RagData {
method new (line 724) | pub fn new(
method get (line 746) | pub fn get(&self, id: DocumentId) -> Option<&RagDocument> {
method del (line 753) | pub fn del(&mut self, file_ids: Vec<FileId>) {
method add (line 764) | pub fn add(
method build_hnsw (line 777) | pub fn build_hnsw(&self) -> Hnsw<'static, f32, DistCosine> {
method build_bm25 (line 784) | pub fn build_bm25(&self) -> SearchEngine<DocumentId> {
method fmt (line 708) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
type RagFile (line 800) | pub struct RagFile {
type RagDocument (line 807) | pub struct RagDocument {
method new (line 813) | pub fn new<S: Into<String>>(page_content: S) -> Self {
method default (line 822) | fn default() -> Self {
type FileId (line 830) | pub type FileId = usize;
type DocumentId (line 833) | pub struct DocumentId(usize);
method new (line 843) | pub fn new(file_index: usize, document_index: usize) -> Self {
method split (line 848) | pub fn split(self) -> (usize, usize) {
method fmt (line 836) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
function select_embedding_model (line 857) | fn select_embedding_model(models: &[&Model]) -> Result<String> {
type SelectOption (line 867) | struct SelectOption {
method new (line 873) | pub fn new(value: String, description: String) -> Self {
method fmt (line 879) | fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
function set_chunk_size (line 884) | fn set_chunk_size(model: &Model) -> Result<usize> {
function set_chunk_overlay (line 906) | fn set_chunk_overlay(default_value: usize) -> Result<usize> {
function add_documents (line 920) | fn add_documents() -> Result<Vec<String>> {
function resolve_paths (line 939) | async fn resolve_paths<T: AsRef<str>>(
function progress (line 984) | fn progress(spinner: &Option<Spinner>, message: String) {
function reciprocal_rank_fusion (line 990) | fn reciprocal_rank_fusion(
FILE: src/rag/serde_vectors.rs
function serialize (line 6) | pub fn serialize<S>(
function deserialize (line 30) | pub fn deserialize<'de, D>(deserializer: D) -> Result<IndexMap<DocumentI...
FILE: src/rag/splitter/language.rs
type Language (line 2) | pub enum Language {
method separators (line 22) | pub fn separators(&self) -> Vec<&str> {
FILE: src/rag/splitter/mod.rs
constant DEFAULT_SEPARATES (line 7) | pub const DEFAULT_SEPARATES: [&str; 4] = ["\n\n", "\n", " ", ""];
function get_separators (line 9) | pub fn get_separators(extension: &str) -> Vec<&'static str> {
type RecursiveCharacterTextSplitter (line 31) | pub struct RecursiveCharacterTextSplitter {
method new (line 50) | pub fn new(chunk_size: usize, chunk_overlap: usize, separators: &[&str...
method with_chunk_size (line 57) | pub fn with_chunk_size(mut self, chunk_size: usize) -> Self {
method with_chunk_overlap (line 62) | pub fn with_chunk_overlap(mut self, chunk_overlap: usize) -> Self {
method with_separators (line 67) | pub fn with_separators(mut self, separators: &[&str]) -> Self {
method split_documents (line 72) | pub fn split_documents(
method create_documents (line 89) | pub fn create_documents(
method split_text (line 144) | pub fn split_text(&self, text: &str) -> Vec<String> {
method split_text_impl (line 152) | fn split_text_impl(
method merge_splits (line 204) | fn merge_splits(&self, splits: &[String], separator: &str) -> Vec<Stri...
method join_docs (line 241) | fn join_docs(&self, docs: &[String], separator: &str) -> Option<String> {
method default (line 39) | fn default() -> Self {
type SplitterChunkHeaderOptions (line 251) | pub struct SplitterChunkHeaderOptions {
method with_chunk_header (line 268) | pub fn with_chunk_header(mut self, header: &str) -> Self {
method with_chunk_overlap_header (line 275) | pub fn with_chunk_overlap_header(mut self, overlap_header: &str) -> Se...
method default (line 257) | fn default() -> Self {
function split_on_separator (line 281) | fn split_on_separator<'a>(text: &'a str, separator: &str, keep_separator...
function build_metadata (line 314) | fn build_metadata(source: &str) -> Value {
function test_split_text (line 318) | fn test_split_text() {
function test_create_document (line 330) | fn test_create_document() {
function test_chunk_header (line 363) | fn test_chunk_header() {
function test_markdown_splitter (line 398) | fn test_markdown_splitter() {
function test_html_splitter (line 424) | fn test_html_splitter() {
FILE: src/render/markdown.rs
constant SYNTAXES (line 14) | const SYNTAXES: &[u8] = include_bytes!("../../assets/syntaxes.bin");
type MarkdownRender (line 23) | pub struct MarkdownRender {
method init (line 34) | pub fn init(options: RenderOptions) -> Result<Self> {
method render (line 71) | pub fn render(&mut self, text: &str) -> String {
method render_line (line 78) | pub fn render_line(&self, line: &str) -> String {
method render_line_mut (line 87) | fn render_line_mut(&mut self, line: &str) -> String {
method check_line (line 99) | fn check_line(&self, line: &str) -> (LineType, Option<SyntaxReference>...
method highlight_line (line 141) | fn highlight_line(&self, line: &str, syntax: &SyntaxReference, is_code...
method highlight_code_line (line 158) | fn highlight_code_line(&self, line: &str, code_syntax: &Option<SyntaxR...
method wrap_line (line 170) | fn wrap_line(&self, line: String, is_code: bool) -> String {
method find_syntax (line 181) | fn find_syntax(&self, lang: &str) -> Option<&SyntaxReference> {
function wrap (line 192) | fn wrap(text: &str, width: usize) -> String {
type RenderOptions (line 201) | pub struct RenderOptions {
method new (line 209) | pub(crate) fn new(
type LineType (line 225) | pub enum LineType {
function as_terminal_escaped (line 232) | fn as_terminal_escaped(ranges: &[(Style, &str)], truecolor: bool) -> Str...
function convert_color (line 248) | fn convert_color(c: SyntectColor, truecolor: bool) -> Color {
function blend_fg_color (line 266) | fn blend_fg_color(fg: SyntectColor, bg: SyntectColor) -> SyntectColor {
function detect_code_block (line 282) | fn detect_code_block(line: &str) -> Option<String> {
function get_code_color (line 295) | fn get_code_color(theme: &Theme, truecolor: bool) -> Color {
constant TEXT (line 311) | const TEXT: &str = r#"
constant TEXT_NO_WRAP_CODE (line 322) | const TEXT_NO_WRAP_CODE: &str = r#"
constant TEXT_WRAP_ALL (line 335) | const TEXT_WRAP_ALL: &str = r#"
function test_render (line 350) | fn test_render() {
function no_theme (line 357) | fn no_theme() {
function no_wrap_code (line 365) | fn no_wrap_code() {
function wrap_all (line 374) | fn wrap_all() {
function test_detect_code_block (line 386) | fn test_detect_code_block() {
FILE: src/render/mod.rs
function render_stream (line 13) | pub async fn render_stream(
function render_error (line 28) | pub fn render_error(err: anyhow::Error) {
FILE: src/render/stream.rs
function markdown_stream (line 17) | pub async fn markdown_stream(
function raw_stream (line 35) | pub async fn raw_stream(
function markdown_stream_inner (line 67) | async fn markdown_stream_inner(
function gather_events (line 165) | async fn gather_events(rx: &mut UnboundedReceiver<SseEvent>) -> Vec<SseE...
function print_block (line 192) | fn print_block(writer: &mut Stdout, text: &str, columns: u16) -> Result<...
function split_line_tail (line 206) | fn split_line_tail(text: &str) -> (&str, &str) {
function need_rows (line 214) | fn need_rows(text: &str, columns: u16) -> u16 {
FILE: src/repl/completer.rs
method complete (line 9) | fn complete(&mut self, line: &str, pos: usize) -> Vec<Suggestion> {
type ReplCompleter (line 82) | pub struct ReplCompleter {
method new (line 89) | pub fn new(config: &GlobalConfig) -> Self {
function create_suggestion (line 111) | fn create_suggestion(value: &str, description: &str, span: Span) -> Sugg...
function split_line (line 127) | fn split_line(line: &str) -> Vec<(&str, usize)> {
function test_split_line (line 149) | fn test_split_line() {
FILE: src/repl/highlighter.rs
constant DEFAULT_COLOR (line 8) | const DEFAULT_COLOR: Color = Color::Default;
constant MATCH_COLOR (line 9) | const MATCH_COLOR: Color = Color::Green;
type ReplHighlighter (line 11) | pub struct ReplHighlighter;
method new (line 14) | pub fn new(_config: &GlobalConfig) -> Self {
method highlight (line 20) | fn highlight(&self, line: &str, _cursor: usize) -> StyledText {
FILE: src/repl/mod.rs
constant MENU_NAME (line 32) | const MENU_NAME: &str = "completion_menu";
type Repl (line 193) | pub struct Repl {
method init (line 201) | pub fn init(config: &GlobalConfig) -> Result<Self> {
method run (line 215) | pub async fn run(&mut self) -> Result<()> {
method create_editor (line 263) | fn create_editor(config: &GlobalConfig) -> Result<Reedline> {
method extra_keybindings (line 294) | fn extra_keybindings(keybindings: &mut Keybindings) {
method create_edit_mode (line 320) | fn create_edit_mode(config: &GlobalConfig) -> Box<dyn EditMode> {
method create_menu (line 333) | fn create_menu() -> ReedlineMenu {
type ReplCommand (line 340) | pub struct ReplCommand {
method new (line 347) | fn new(name: &'static str, desc: &'static str, state: AssertState) -> ...
method is_valid (line 355) | fn is_valid(&self, flags: StateFlags) -> bool {
type ReplValidator (line 361) | struct ReplValidator;
method validate (line 364) | fn validate(&self, line: &str) -> ValidationResult {
function run_repl_command (line 374) | pub async fn run_repl_command(
function ask (line 720) | async fn ask(
function unknown_command (line 761) | fn unknown_command() -> Result<()> {
function dump_repl_help (line 765) | fn dump_repl_help() {
function parse_command (line 780) | fn parse_command(line: &str) -> Option<(&str, Option<&str>)> {
function split_first_arg (line 792) | fn split_first_arg(args: Option<&str>) -> Option<(&str, Option<&str>)> {
function split_args_text (line 799) | pub fn split_args_text(line: &str, is_win: bool) -> (Vec<String>, &str) {
function test_process_command_line (line 876) | fn test_process_command_line() {
function test_split_args_text (line 895) | fn test_split_args_text() {
FILE: src/repl/prompt.rs
type ReplPrompt (line 7) | pub struct ReplPrompt {
method new (line 12) | pub fn new(config: &GlobalConfig) -> Self {
method render_prompt_left (line 20) | fn render_prompt_left(&self) -> Cow<'_, str> {
method render_prompt_right (line 24) | fn render_prompt_right(&self) -> Cow<'_, str> {
method render_prompt_indicator (line 28) | fn render_prompt_indicator(&self, _prompt_mode: reedline::PromptEditMode...
method render_prompt_multiline_indicator (line 32) | fn render_prompt_multiline_indicator(&self) -> Cow<'_, str> {
method render_prompt_history_search_indicator (line 36) | fn render_prompt_history_search_indicator(
FILE: src/serve.rs
constant DEFAULT_MODEL_NAME (line 35) | const DEFAULT_MODEL_NAME: &str = "default";
constant PLAYGROUND_HTML (line 36) | const PLAYGROUND_HTML: &[u8] = include_bytes!("../assets/playground.html");
constant ARENA_HTML (line 37) | const ARENA_HTML: &[u8] = include_bytes!("../assets/arena.html");
type AppResponse (line 39) | type AppResponse = Response<BoxBody<Bytes, Infallible>>;
function run (line 41) | pub async fn run(config: GlobalConfig, addr: Option<String>) -> Result<(...
type Server (line 67) | struct Server {
method new (line 75) | fn new(config: &GlobalConfig) -> Self {
method run (line 109) | async fn run(self: Arc<Self>, listener: TcpListener) -> Result<oneshot...
method handle (line 142) | async fn handle(
method playground_page (line 198) | fn playground_page(&self) -> Result<AppResponse> {
method arena_page (line 205) | fn arena_page(&self) -> Result<AppResponse> {
method list_models (line 212) | fn list_models(&self) -> Result<AppResponse> {
method list_roles (line 220) | fn list_roles(&self) -> Result<AppResponse> {
method list_rags (line 228) | fn list_rags(&self) -> Result<AppResponse> {
method search_rag (line 236) | async fn search_rag(&self, req: hyper::Request<Incoming>) -> Result<Ap...
method chat_completions (line 261) | async fn chat_completions(&self, req: hyper::Request<Incoming>) -> Res...
method embeddings (line 470) | async fn embeddings(&self, req: hyper::Request<Incoming>) -> Result<Ap...
method rerank (line 526) | async fn rerank(&self, req: hyper::Request<Incoming>) -> Result<AppRes...
type SearchRagReqBody (line 580) | struct SearchRagReqBody {
type ChatCompletionsReqBody (line 586) | struct ChatCompletionsReqBody {
type EmbeddingsReqBody (line 598) | struct EmbeddingsReqBody {
type EmbeddingsReqBodyInput (line 605) | enum EmbeddingsReqBodyInput {
type RerankReqBody (line 611) | struct RerankReqBody {
type ResEvent (line 619) | enum ResEvent {
function shutdown_signal (line 626) | async fn shutdown_signal() {
function generate_completion_id (line 632) | fn generate_completion_id() -> String {
function set_cors_header (line 637) | fn set_cors_header(res: &mut AppResponse) {
function create_text_frame (line 652) | fn create_text_frame(id: &str, model: &str, created: i64, content: &str)...
function create_tool_calls_frame (line 667) | fn create_tool_calls_frame(
function create_done_frame (line 721) | fn create_done_frame(id: &str, model: &str, created: i64, has_tool_calls...
function build_chat_completion_chunk_json (line 732) | fn build_chat_completion_chunk_json(id: &str, model: &str, created: i64,...
function ret_non_stream (line 742) | fn ret_non_stream(id: &str, model: &str, created: i64, output: &ChatComp...
function ret_err (line 803) | fn ret_err<T: std::fmt::Display>(err: T) -> AppResponse {
function parse_messages (line 816) | fn parse_messages(message: Vec<Value>) -> Result<Vec<Message>> {
function parse_tools (line 916) | fn parse_tools(tools: Option<Vec<Value>>) -> Result<Option<Vec<FunctionD...
FILE: src/utils/abort_signal.rs
type AbortSignal (line 11) | pub type AbortSignal = Arc<AbortSignalInner>;
type AbortSignalInner (line 13) | pub struct AbortSignalInner {
method new (line 23) | pub fn new() -> AbortSignal {
method aborted (line 30) | pub fn aborted(&self) -> bool {
method aborted_ctrlc (line 40) | pub fn aborted_ctrlc(&self) -> bool {
method aborted_ctrld (line 44) | pub fn aborted_ctrld(&self) -> bool {
method reset (line 48) | pub fn reset(&self) {
method set_ctrlc (line 53) | pub fn set_ctrlc(&self) {
method set_ctrld (line 57) | pub fn set_ctrld(&self) {
function create_abort_signal (line 18) | pub fn create_abort_signal() -> AbortSignal {
function wait_abort_signal (line 62) | pub async fn wait_abort_signal(abort_signal: &AbortSignal) {
function poll_abort_signal (line 71) | pub fn poll_abort_signal(abort_signal: &AbortSignal) -> Result<bool> {
FILE: src/utils/clipboard.rs
function set_text (line 12) | pub fn set_text(text: &str) -> anyhow::Result<()> {
function set_text_osc52 (line 27) | fn set_text_osc52(text: &str) -> anyhow::Result<()> {
function set_text (line 42) | pub fn set_text(_text: &str) -> anyhow::Result<()> {
function set_text (line 47) | pub fn set_text(text: &str) -> anyhow::Result<()> {
FILE: src/utils/command.rs
type Shell (line 19) | pub struct Shell {
method new (line 26) | pub fn new(name: &str, cmd: &str, arg: &str) -> Self {
function detect_shell (line 35) | pub fn detect_shell() -> Shell {
function run_command (line 81) | pub fn run_command<T: AsRef<OsStr>>(
function run_command_with_output (line 93) | pub fn run_command_with_output<T: AsRef<OsStr>>(
function run_loader_command (line 108) | pub fn run_loader_command(path: &str, extension: &str, loader_command: &...
function edit_file (line 157) | pub fn edit_file(editor: &str, path: &Path) -> Result<()> {
function append_to_shell_history (line 163) | pub fn append_to_shell_history(shell: &str, command: &str, exit_code: i3...
function get_history_file (line 183) | fn get_history_file(shell: &str) -> Option<PathBuf> {
FILE: src/utils/crypto.rs
function sha256 (line 5) | pub fn sha256(input: &str) -> String {
function hmac_sha256 (line 11) | pub fn hmac_sha256(key: &[u8], msg: &str) -> Vec<u8> {
function hex_encode (line 17) | pub fn hex_encode(bytes: &[u8]) -> String {
function encode_uri (line 23) | pub fn encode_uri(uri: &str) -> String {
function base64_encode (line 30) | pub fn base64_encode<T: AsRef<[u8]>>(input: T) -> String {
function base64_decode (line 33) | pub fn base64_decode<T: AsRef<[u8]>>(input: T) -> Result<Vec<u8>, base64...
FILE: src/utils/html_to_md.rs
function html_to_md (line 5) | pub fn html_to_md(html: &str) -> String {
FILE: src/utils/input.rs
function read_single_key (line 8) | pub fn read_single_key(valid_chars: &[char], default: char, prompt: &str...
FILE: src/utils/loader.rs
constant EXTENSION_METADATA (line 8) | pub const EXTENSION_METADATA: &str = "__extension__";
type DocumentMetadata (line 10) | pub type DocumentMetadata = IndexMap<String, String>;
type LoadedDocument (line 13) | pub struct LoadedDocument {
method new (line 21) | pub fn new(path: String, contents: String, metadata: DocumentMetadata)...
function load_recursive_url (line 30) | pub async fn load_recursive_url(
function load_file (line 57) | pub async fn load_file(loaders: &HashMap<String, String>, path: &str) ->...
function load_url (line 65) | pub async fn load_url(loaders: &HashMap<String, String>, path: &str) -> ...
function load_plain (line 72) | async fn load_plain(path: &str, extension: &str) -> Result<LoadedDocumen...
function load_with_command (line 79) | fn load_with_command(path: &str, extension: &str, loader_command: &str) ...
function is_loader_protocol (line 86) | pub fn is_loader_protocol(loaders: &HashMap<String, String>, path: &str)...
function load_protocol_path (line 93) | pub fn load_protocol_path(
FILE: src/utils/mod.rs
function now (line 49) | pub fn now() -> String {
function now_timestamp (line 53) | pub fn now_timestamp() -> i64 {
function get_env_name (line 57) | pub fn get_env_name(key: &str) -> String {
function normalize_env_name (line 61) | pub fn normalize_env_name(value: &str) -> String {
function parse_bool (line 65) | pub fn parse_bool(value: &str) -> Option<bool> {
function estimate_token_length (line 73) | pub fn estimate_token_length(text: &str) -> usize {
function strip_think_tag (line 91) | pub fn strip_think_tag(text: &str) -> Cow<'_, str> {
function extract_code_block (line 95) | pub fn extract_code_block(text: &str) -> &str {
function convert_option_string (line 103) | pub fn convert_option_string(value: &str) -> Option<String> {
function fuzzy_filter (line 111) | pub fn fuzzy_filter<T, F>(values: Vec<T>, get: F, pattern: &str) -> Vec<T>
function pretty_error (line 127) | pub fn pretty_error(err: &anyhow::Error) -> String {
function indent_text (line 145) | pub fn indent_text<T: ToString>(s: T, size: usize) -> String {
function error_text (line 154) | pub fn error_text(input: &str) -> String {
function warning_text (line 158) | pub fn warning_text(input: &str) -> String {
function color_text (line 162) | pub fn color_text(input: &str, color: nu_ansi_term::Color) -> String {
function dimmed_text (line 172) | pub fn dimmed_text(input: &str) -> String {
function multiline_text (line 179) | pub fn multiline_text(input: &str) -> String {
function temp_file (line 194) | pub fn temp_file(prefix: &str, suffix: &str) -> PathBuf {
function is_url (line 203) | pub fn is_url(path: &str) -> bool {
function set_proxy (line 207) | pub fn set_proxy(
function decode_bin (line 219) | pub fn decode_bin<T: serde::de::DeserializeOwned>(data: &[u8]) -> Result...
function test_safe_join_path (line 230) | fn test_safe_join_path() {
function test_safe_join_path (line 241) | fn test_safe_join_path() {
FILE: src/utils/path.rs
function safe_join_path (line 7) | pub fn safe_join_path<T1: AsRef<Path>, T2: AsRef<Path>>(
function expand_glob_paths (line 33) | pub async fn expand_glob_paths<T: AsRef<str>>(
function list_file_names (line 52) | pub fn list_file_names<T: AsRef<Path>>(dir: T, ext: &str) -> Vec<String> {
function get_patch_extension (line 69) | pub fn get_patch_extension(path: &str) -> Option<String> {
function to_absolute_path (line 75) | pub fn to_absolute_path(path: &str) -> Result<String> {
function resolve_home_dir (line 79) | pub fn resolve_home_dir(path: &str) -> String {
function parse_glob (line 89) | fn parse_glob(path_str: &str) -> Result<(String, Option<Vec<String>>, bo...
function list_files (line 155) | async fn list_files(
function add_file (line 187) | fn add_file(files: &mut IndexSet<String>, suffixes: Option<&Vec<String>>...
function is_valid_extension (line 196) | fn is_valid_extension(suffixes: Option<&Vec<String>>, path: &Path) -> bo...
function test_parse_glob (line 213) | fn test_parse_glob() {
FILE: src/utils/render_prompt.rs
function render_prompt (line 11) | pub fn render_prompt(template: &str, variables: &HashMap<&str, String>) ...
function parse_template (line 16) | fn parse_template(template: &str) -> Vec<Expr> {
function parse_block (line 50) | fn parse_block(current: &mut Vec<char>) -> Expr {
function eval_exprs (line 68) | fn eval_exprs(exprs: &[Expr], variables: &HashMap<&str, String>) -> Stri...
function add_text (line 105) | fn add_text(exprs: &mut Vec<Expr>, current: &mut Vec<char>) {
function truly (line 113) | fn truly(value: &str) -> bool {
type Expr (line 118) | enum Expr {
type BlockType (line 125) | enum BlockType {
function test_render (line 144) | fn test_render() {
FILE: src/utils/request.rs
constant URL_LOADER (line 20) | pub const URL_LOADER: &str = "url";
constant RECURSIVE_URL_LOADER (line 21) | pub const RECURSIVE_URL_LOADER: &str = "recursive_url";
constant MEDIA_URL_EXTENSION (line 23) | pub const MEDIA_URL_EXTENSION: &str = "media_url";
constant DEFAULT_EXTENSION (line 24) | pub const DEFAULT_EXTENSION: &str = "txt";
constant MAX_CRAWLS (line 26) | const MAX_CRAWLS: usize = 5;
constant BREAK_ON_ERROR (line 27) | const BREAK_ON_ERROR: bool = false;
constant USER_AGENT (line 28) | const USER_AGENT: &str = "curl/8.6.0";
function fetch (line 60) | pub async fn fetch(url: &str) -> Result<String> {
function fetch_with_loaders (line 70) | pub async fn fetch_with_loaders(
function fetch_models (line 169) | pub async fn fetch_models(api_base: &str, api_key: Option<&str>) -> Resu...
type CrawlOptions (line 196) | pub struct CrawlOptions {
method preset (line 203) | pub fn preset(start_url: &str) -> CrawlOptions {
function crawl_website (line 213) | pub async fn crawl_website(start_url: &str, options: CrawlOptions) -> Re...
type Page (line 300) | pub struct Page {
function crawl_gh_tree (line 305) | async fn crawl_gh_tree(start_url: &Url, exclude: &[String]) -> Result<Ve...
function crawl_page (line 371) | async fn crawl_page(
function should_exclude_link (line 427) | fn should_exclude_link(link: &str, exclude: &[String]) -> bool {
function normalize_start_url (line 446) | fn normalize_start_url(start_url: &Url) -> Url {
function match_link (line 458) | fn match_link(path: &str, link: &str) -> bool {
FILE: src/utils/spinner.rs
type SpinnerInner (line 19) | pub struct SpinnerInner {
constant DATA (line 25) | const DATA: [&'static str; 10] = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "...
method step (line 27) | fn step(&mut self) -> Result<()> {
method set_message (line 44) | fn set_message(&mut self, message: String) -> Result<()> {
method clear_message (line 52) | fn clear_message(&mut self) -> Result<()> {
type Spinner (line 70) | pub struct Spinner(mpsc::UnboundedSender<SpinnerEvent>);
method create (line 73) | pub fn create(message: &str) -> (Self, UnboundedReceiver<SpinnerEvent>) {
method set_message (line 80) | pub fn set_message(&self, message: String) -> Result<()> {
method stop (line 86) | pub fn stop(&self) {
type SpinnerEvent (line 92) | pub enum SpinnerEvent {
function spawn_spinner (line 97) | pub fn spawn_spinner(message: &str) -> Spinner {
function abortable_run_with_spinner (line 128) | pub async fn abortable_run_with_spinner<F, T>(
function abortable_run_with_spinner_rx (line 140) | pub async fn abortable_run_with_spinner_rx<F, T>(
function run_abortable_spinner (line 178) | async fn run_abortable_spinner(
FILE: src/utils/variables.rs
function interpolate_variables (line 6) | pub fn interpolate_variables(text: &mut String) {
Condensed preview — 79 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (801K chars).
[
{
"path": ".github/ISSUE_TEMPLATE/bug_report.md",
"chars": 1061,
"preview": "---\nname: Bug report\nabout: Create a report to help us improve\ntitle: ''\nlabels: ''\nassignees: ''\n\n---\n\n<!-- Your issue "
},
{
"path": ".github/ISSUE_TEMPLATE/feature_request.md",
"chars": 727,
"preview": "---\nname: Feature request\nabout: Suggest an idea for this project\ntitle: ''\nlabels: enhancement\nassignees: ''\n\n---\n\n<!--"
},
{
"path": ".github/workflows/ci.yaml",
"chars": 682,
"preview": "name: CI\n\non:\n pull_request:\n branches:\n - '*'\n push:\n branches:\n - main\n\ndefaults:\n run:\n shell: bash"
},
{
"path": ".github/workflows/release.yaml",
"chars": 4150,
"preview": "name: Release\n\non:\n push:\n tags:\n - v[0-9]+.[0-9]+.[0-9]+*\n\njobs:\n release:\n name: Publish to GitHub Release\n"
},
{
"path": ".gitignore",
"chars": 24,
"preview": "/target\n/tmp\n/.env\n*.log"
},
{
"path": "Argcfile.sh",
"chars": 12776,
"preview": "#!/usr/bin/env bash\nset -e\n\n# @meta dotenv\n# @env DRY_RUN Dry run mode\n\n# @cmd Test configuration initialization\n# @env "
},
{
"path": "Cargo.toml",
"chars": 3088,
"preview": "[package]\nname = \"aichat\"\nversion = \"0.30.0\"\nedition = \"2021\"\nauthors = [\"sigoden <sigoden@gmail.com>\"]\ndescription = \"A"
},
{
"path": "LICENSE-APACHE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "LICENSE-MIT",
"chars": 1069,
"preview": "The MIT License (MIT)\n\nCopyright (c) sigoden\n\nPermission is hereby granted, free of charge, to any person obtaining a co"
},
{
"path": "README.md",
"chars": 7482,
"preview": "# AIChat: All-in-one LLM CLI Tool\n\n[](https:"
},
{
"path": "assets/arena.html",
"chars": 36693,
"preview": "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"UTF-8\">\n <meta name=\"viewport\" content=\"width=device-width, i"
},
{
"path": "assets/playground.html",
"chars": 51658,
"preview": "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"UTF-8\">\n <meta name=\"viewport\" content=\"width=device-width, i"
},
{
"path": "assets/roles/%code%.md",
"chars": 200,
"preview": "Provide only code without comments or explanations.\n### INPUT:\nasync sleep in js\n### OUTPUT:\n```javascript\nasync functio"
},
{
"path": "assets/roles/%create-prompt%.md",
"chars": 2228,
"preview": "As a professional Prompt Engineer, your role is to create effective and innovative prompts for interacting with AI model"
},
{
"path": "assets/roles/%create-title%.md",
"chars": 243,
"preview": "Create a concise, 3-6 word title.\n\n**Notes**:\n- Avoid quotation marks or emojis\n- RESPOND ONLY WITH TITLE SLUG TEXT\n\n**E"
},
{
"path": "assets/roles/%explain-shell%.md",
"chars": 206,
"preview": "Provide a terse, single sentence description of the given shell command.\nDescribe each argument and option of the comman"
},
{
"path": "assets/roles/%functions%.md",
"chars": 23,
"preview": "---\nuse_tools: all\n---\n"
},
{
"path": "assets/roles/%shell%.md",
"chars": 351,
"preview": "Provide only {{__shell__}} commands for {{__os_distro__}} without any description.\nEnsure the output is a valid {{__shel"
},
{
"path": "config.agent.example.yaml",
"chars": 772,
"preview": "# Agent-specific configuration\n# Location `<aichat-config-dir>/agents/<agent-name>/config.yaml`\n\nmodel: openai:gpt-4o "
},
{
"path": "config.example.yaml",
"chars": 12711,
"preview": "# ---- llm ----\nmodel: openai:gpt-4o # Specify the LLM to use\ntemperature: null # Set default"
},
{
"path": "models.yaml",
"chars": 55424,
"preview": "# Links:\n# - https://platform.openai.com/docs/models\n# - https://platform.openai.com/docs/api-reference/chat\n- provide"
},
{
"path": "scripts/completions/aichat.bash",
"chars": 3281,
"preview": "_aichat() {\n local cur prev words cword i opts cmd\n COMPREPLY=()\n\n _get_comp_words_by_ref -n : cur prev words c"
},
{
"path": "scripts/completions/aichat.fish",
"chars": 1845,
"preview": "complete -c aichat -s m -l model -x -a \"(aichat --list-models)\" -d 'Select a LLM model' -r\ncomplete -c aichat -l prompt "
},
{
"path": "scripts/completions/aichat.nu",
"chars": 3071,
"preview": "module completions {\n\n def \"nu-complete aichat completions\" [] {\n [ \"bash\" \"zsh\" \"fish\" \"powershell\" \"nushell\" ]\n }"
},
{
"path": "scripts/completions/aichat.ps1",
"chars": 6709,
"preview": "using namespace System.Management.Automation\nusing namespace System.Management.Automation.Language\n\nRegister-ArgumentCom"
},
{
"path": "scripts/completions/aichat.zsh",
"chars": 2474,
"preview": "#compdef aichat\n\nautoload -U is-at-least\n\n_aichat() {\n typeset -A opt_args\n typeset -a _arguments_options\n loca"
},
{
"path": "scripts/shell-integration/integration.bash",
"chars": 187,
"preview": "_aichat_bash() {\n if [[ -n \"$READLINE_LINE\" ]]; then\n READLINE_LINE=$(aichat -e \"$READLINE_LINE\")\n READ"
},
{
"path": "scripts/shell-integration/integration.fish",
"chars": 195,
"preview": "function _aichat_fish\n set -l _old (commandline)\n if test -n $_old\n echo -n \"⌛\"\n commandline -f repa"
},
{
"path": "scripts/shell-integration/integration.nu",
"chars": 469,
"preview": "def _aichat_nushell [] {\n let _prev = (commandline)\n if ($_prev != \"\") {\n print '⌛'\n commandline edi"
},
{
"path": "scripts/shell-integration/integration.ps1",
"chars": 405,
"preview": "Set-PSReadLineKeyHandler -Chord \"alt+e\" -ScriptBlock {\n $_old = $null\n [Microsoft.PowerShell.PSConsoleReadline]::G"
},
{
"path": "scripts/shell-integration/integration.zsh",
"chars": 240,
"preview": "_aichat_zsh() {\n if [[ -n \"$BUFFER\" ]]; then\n local _old=$BUFFER\n BUFFER+=\"⌛\"\n zle -I && zle red"
},
{
"path": "src/cli.rs",
"chars": 3806,
"preview": "use anyhow::{Context, Result};\nuse clap::Parser;\nuse is_terminal::IsTerminal;\nuse std::io::{stdin, Read};\n\n#[derive(Pars"
},
{
"path": "src/client/access_token.rs",
"chars": 1021,
"preview": "use anyhow::{anyhow, Result};\nuse chrono::Utc;\nuse indexmap::IndexMap;\nuse parking_lot::RwLock;\nuse std::sync::LazyLock;"
},
{
"path": "src/client/azure_openai.rs",
"chars": 2066,
"preview": "use super::openai::*;\nuse super::*;\n\nuse anyhow::Result;\nuse serde::Deserialize;\n\n#[derive(Debug, Clone, Deserialize)]\np"
},
{
"path": "src/client/bedrock.rs",
"chars": 21995,
"preview": "use super::*;\n\nuse crate::utils::{base64_decode, encode_uri, hex_encode, hmac_sha256, sha256, strip_think_tag};\n\nuse any"
},
{
"path": "src/client/claude.rs",
"chars": 12504,
"preview": "use super::*;\n\nuse crate::utils::strip_think_tag;\n\nuse anyhow::{bail, Context, Result};\nuse reqwest::RequestBuilder;\nuse"
},
{
"path": "src/client/cohere.rs",
"chars": 8085,
"preview": "use super::openai::*;\nuse super::openai_compatible::*;\nuse super::*;\n\nuse anyhow::{bail, Context, Result};\nuse reqwest::"
},
{
"path": "src/client/common.rs",
"chars": 20165,
"preview": "use super::*;\n\nuse crate::{\n config::{Config, GlobalConfig, Input},\n function::{eval_tool_calls, FunctionDeclarati"
},
{
"path": "src/client/gemini.rs",
"chars": 3408,
"preview": "use super::vertexai::*;\nuse super::*;\n\nuse anyhow::{Context, Result};\nuse reqwest::RequestBuilder;\nuse serde::Deserializ"
},
{
"path": "src/client/macros.rs",
"chars": 9018,
"preview": "#[macro_export]\nmacro_rules! register_client {\n (\n $(($module:ident, $name:literal, $config:ident, $client:ide"
},
{
"path": "src/client/message.rs",
"chars": 7533,
"preview": "use super::Model;\n\nuse crate::{function::ToolResult, multiline_text, utils::dimmed_text};\n\nuse serde::{Deserialize, Seri"
},
{
"path": "src/client/mod.rs",
"chars": 1877,
"preview": "mod access_token;\nmod common;\nmod message;\n#[macro_use]\nmod macros;\nmod model;\nmod stream;\n\npub use crate::function::Too"
},
{
"path": "src/client/model.rs",
"chars": 12847,
"preview": "use super::{\n list_all_models, list_client_names,\n message::{Message, MessageContent, MessageContentPart},\n Api"
},
{
"path": "src/client/openai.rs",
"chars": 13882,
"preview": "use super::*;\n\nuse crate::utils::strip_think_tag;\n\nuse anyhow::{bail, Context, Result};\nuse reqwest::RequestBuilder;\nuse"
},
{
"path": "src/client/openai_compatible.rs",
"chars": 4346,
"preview": "use super::openai::*;\nuse super::*;\n\nuse anyhow::{Context, Result};\nuse reqwest::RequestBuilder;\nuse serde::Deserialize;"
},
{
"path": "src/client/stream.rs",
"chars": 8558,
"preview": "use super::{catch_error, ToolCall};\nuse crate::utils::AbortSignal;\n\nuse anyhow::{anyhow, bail, Context, Result};\nuse fut"
},
{
"path": "src/client/vertexai.rs",
"chars": 18458,
"preview": "use super::access_token::*;\nuse super::claude::*;\nuse super::openai::*;\nuse super::*;\n\nuse anyhow::{anyhow, bail, Contex"
},
{
"path": "src/config/agent.rs",
"chars": 17872,
"preview": "use super::*;\n\nuse crate::{\n client::Model,\n function::{run_llm_function, Functions},\n};\n\nuse anyhow::{Context, Re"
},
{
"path": "src/config/input.rs",
"chars": 16649,
"preview": "use super::*;\n\nuse crate::client::{\n init_client, patch_messages, ChatCompletionsData, Client, ImageUrl, Message, Mes"
},
{
"path": "src/config/mod.rs",
"chars": 94687,
"preview": "mod agent;\nmod input;\nmod role;\nmod session;\n\npub use self::agent::{complete_agent_variables, list_agents, Agent, AgentV"
},
{
"path": "src/config/role.rs",
"chars": 11295,
"preview": "use super::*;\n\nuse crate::client::{Message, MessageContent, MessageRole, Model};\n\nuse anyhow::Result;\nuse fancy_regex::R"
},
{
"path": "src/config/session.rs",
"chars": 20580,
"preview": "use super::input::*;\nuse super::*;\n\nuse crate::client::{Message, MessageContent, MessageRole};\nuse crate::render::Markdo"
},
{
"path": "src/function.rs",
"chars": 9319,
"preview": "use crate::{\n config::{Agent, Config, GlobalConfig},\n utils::*,\n};\n\nuse anyhow::{anyhow, bail, Context, Result};\nu"
},
{
"path": "src/main.rs",
"chars": 11813,
"preview": "mod cli;\nmod client;\nmod config;\nmod function;\nmod rag;\nmod render;\nmod repl;\nmod serve;\n#[macro_use]\nmod utils;\n\n#[macr"
},
{
"path": "src/rag/mod.rs",
"chars": 33825,
"preview": "use self::splitter::*;\n\nuse crate::client::*;\nuse crate::config::*;\nuse crate::utils::*;\n\nmod serde_vectors;\nmod splitte"
},
{
"path": "src/rag/serde_vectors.rs",
"chars": 2080,
"preview": "use super::*;\n\nuse base64::{engine::general_purpose::STANDARD, Engine};\nuse serde::{de, Deserializer, Serializer};\n\npub "
},
{
"path": "src/rag/splitter/language.rs",
"chars": 6306,
"preview": "#[derive(PartialEq, Eq, Hash)]\npub enum Language {\n Cpp,\n Go,\n Java,\n Js,\n Php,\n Proto,\n Python,\n "
},
{
"path": "src/rag/splitter/mod.rs",
"chars": 15639,
"preview": "mod language;\n\npub use self::language::*;\n\nuse super::{DocumentMetadata, RagDocument};\n\npub const DEFAULT_SEPARATES: [&s"
},
{
"path": "src/render/markdown.rs",
"chars": 12170,
"preview": "use crate::utils::decode_bin;\n\nuse ansi_colours::AsRGB;\nuse anyhow::{anyhow, Context, Result};\nuse crossterm::style::{Co"
},
{
"path": "src/render/mod.rs",
"chars": 952,
"preview": "mod markdown;\nmod stream;\n\npub use self::markdown::{MarkdownRender, RenderOptions};\nuse self::stream::{markdown_stream, "
},
{
"path": "src/render/stream.rs",
"chars": 6549,
"preview": "use super::{MarkdownRender, SseEvent};\n\nuse crate::utils::{poll_abort_signal, spawn_spinner, AbortSignal};\n\nuse anyhow::"
},
{
"path": "src/repl/completer.rs",
"chars": 4635,
"preview": "use super::{ReplCommand, REPL_COMMANDS};\n\nuse crate::{config::GlobalConfig, utils::fuzzy_filter};\n\nuse reedline::{Comple"
},
{
"path": "src/repl/highlighter.rs",
"chars": 1617,
"preview": "use super::REPL_COMMANDS;\n\nuse crate::{config::GlobalConfig, utils::NO_COLOR};\n\nuse nu_ansi_term::{Color, Style};\nuse re"
},
{
"path": "src/repl/mod.rs",
"chars": 32750,
"preview": "mod completer;\nmod highlighter;\nmod prompt;\n\nuse self::completer::ReplCompleter;\nuse self::highlighter::ReplHighlighter;"
},
{
"path": "src/repl/prompt.rs",
"chars": 1416,
"preview": "use crate::config::GlobalConfig;\n\nuse reedline::{Prompt, PromptHistorySearch, PromptHistorySearchStatus};\nuse std::borro"
},
{
"path": "src/serve.rs",
"chars": 32789,
"preview": "use crate::{client::*, config::*, function::*, rag::*, utils::*};\n\nuse anyhow::{anyhow, bail, Result};\nuse bytes::Bytes;"
},
{
"path": "src/utils/abort_signal.rs",
"chars": 2128,
"preview": "use anyhow::Result;\nuse crossterm::event::{self, Event, KeyCode, KeyModifiers};\nuse std::{\n sync::{\n atomic::{"
},
{
"path": "src/utils/clipboard.rs",
"chars": 1764,
"preview": "use anyhow::Context;\n\n#[cfg(not(any(target_os = \"android\", target_os = \"emscripten\")))]\nmod internal {\n use arboard::"
},
{
"path": "src/utils/command.rs",
"chars": 7027,
"preview": "use super::*;\n\nuse std::{\n collections::HashMap,\n env,\n ffi::OsStr,\n fs::OpenOptions,\n io::{self, Write},"
},
{
"path": "src/utils/crypto.rs",
"chars": 974,
"preview": "use base64::{engine::general_purpose::STANDARD, Engine};\nuse hmac::{Hmac, Mac};\nuse sha2::{Digest, Sha256};\n\npub fn sha2"
},
{
"path": "src/utils/html_to_md.rs",
"chars": 714,
"preview": "use std::{cell::RefCell, rc::Rc};\n\nuse html_to_markdown::{markdown, TagHandler};\n\npub fn html_to_md(html: &str) -> Strin"
},
{
"path": "src/utils/input.rs",
"chars": 1453,
"preview": "use anyhow::Result;\nuse crossterm::event::{self, Event, KeyCode, KeyEvent, KeyModifiers};\nuse crossterm::terminal::{disa"
},
{
"path": "src/utils/loader.rs",
"chars": 4420,
"preview": "use super::*;\n\nuse anyhow::{anyhow, Context, Result};\nuse indexmap::IndexMap;\nuse serde::{Deserialize, Serialize};\nuse s"
},
{
"path": "src/utils/mod.rs",
"chars": 6747,
"preview": "mod abort_signal;\nmod clipboard;\nmod command;\nmod crypto;\nmod html_to_md;\nmod input;\nmod loader;\nmod path;\nmod render_pr"
},
{
"path": "src/utils/path.rs",
"chars": 8115,
"preview": "use std::path::{Component, Path, PathBuf};\n\nuse anyhow::{bail, Result};\nuse indexmap::IndexSet;\nuse path_absolutize::Abs"
},
{
"path": "src/utils/render_prompt.rs",
"chars": 4809,
"preview": "use std::collections::HashMap;\n\n/// Render REPL prompt\n///\n/// The template comprises plain text and `{...}`.\n///\n/// Th"
},
{
"path": "src/utils/request.rs",
"chars": 15282,
"preview": "use super::*;\n\nuse anyhow::{anyhow, bail, Context, Result};\nuse fancy_regex::Regex;\nuse futures_util::{stream, StreamExt"
},
{
"path": "src/utils/spinner.rs",
"chars": 5906,
"preview": "use super::{poll_abort_signal, wait_abort_signal, AbortSignal, IS_STDOUT_TERMINAL};\n\nuse anyhow::{bail, Result};\nuse cro"
},
{
"path": "src/utils/variables.rs",
"chars": 1243,
"preview": "use super::*;\nuse fancy_regex::{Captures, Regex};\nuse std::sync::LazyLock;\n\npub static RE_VARIABLE: LazyLock<Regex> = La"
}
]
About this extraction
This page contains the full source code of the sigoden/aichat GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 79 files (747.0 KB), approximately 184.2k tokens, and a symbol index with 930 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.