## 🦞 Enhanced OpenClaw with MemOS Plugin

🦞 Your lobster now has a working memory system — choose **Cloud** or **Local** to get started.
### ☁️ Cloud Plugin — Hosted Memory Service
- [**72% lower token usage**](https://x.com/MemOS_dev/status/2020854044583924111) — intelligent memory retrieval instead of loading full chat history
- [**Multi-agent memory sharing**](https://x.com/MemOS_dev/status/2020538135487062094) — multi-instance agents share memory via same user_id, automatic context handoff
Get your API key: [MemOS Dashboard](https://memos-dashboard.openmem.net/cn/login/)
Full tutorial → [MemOS-Cloud-OpenClaw-Plugin](https://github.com/MemTensor/MemOS-Cloud-OpenClaw-Plugin)
### 🧠 Local Plugin — 100% On-Device Memory
- **Zero cloud dependency** — all data stays on your machine, persistent local SQLite storage
- **Hybrid search + task & skill evolution** — FTS5 + vector search, auto task summarization, reusable skills that self-upgrade
- **Multi-agent collaboration + Memory Viewer** — memory isolation, skill sharing, full web dashboard with 7 management pages
🌐 [Homepage](https://memos-claw.openmem.net) ·
📖 [Documentation](https://memos-claw.openmem.net/docs/index.html) · 📦 [NPM](https://www.npmjs.com/package/@memtensor/memos-local-openclaw-plugin)
## 📌 MemOS: Memory Operating System for AI Agents
**MemOS** is a Memory Operating System for LLMs and AI agents that unifies **store / retrieve / manage** for long-term memory, enabling **context-aware and personalized** interactions with **KB**, **multi-modal**, **tool memory**, and **enterprise-grade** optimizations built in.
### Key Features
- **Unified Memory API**: A single API to add, retrieve, edit, and delete memory—structured as a graph, inspectable and editable by design, not a black-box embedding store.
- **Multi-Modal Memory**: Natively supports text, images, tool traces, and personas, retrieved and reasoned together in one memory system.
- **Multi-Cube Knowledge Base Management**: Manage multiple knowledge bases as composable memory cubes, enabling isolation, controlled sharing, and dynamic composition across users, projects, and agents.
- **Asynchronous Ingestion via MemScheduler**: Run memory operations asynchronously with millisecond-level latency for production stability under high concurrency.
- **Memory Feedback & Correction**: Refine memory with natural-language feedback—correcting, supplementing, or replacing existing memories over time.
### News
- **2026-03-08** · 🦞 **MemOS OpenClaw Plugin — Cloud & Local**
Official OpenClaw memory plugins launched. **Cloud Plugin**: hosted memory service with 72% lower token usage and multi-agent memory sharing ([MemOS-Cloud-OpenClaw-Plugin](https://github.com/MemTensor/MemOS-Cloud-OpenClaw-Plugin)). **Local Plugin** (`v1.0.0`): 100% on-device memory with persistent SQLite, hybrid search (FTS5 + vector), task summarization & skill evolution, multi-agent collaboration, and a full Memory Viewer dashboard.
- **2025-12-24** · 🎉 **MemOS v2.0: Stardust (星尘) Release**
Comprehensive KB (doc/URL parsing + cross-project sharing), memory feedback & precise deletion, multi-modal memory (images/charts), tool memory for agent planning, Redis Streams scheduling + DB optimizations, streaming/non-streaming chat, MCP upgrade, and lightweight quick/full deployment.
✨ New Features
**Knowledge Base & Memory**
- Added knowledge base support for long-term memory from documents and URLs
**Feedback & Memory Management**
- Added natural language feedback and correction for memories
- Added memory deletion API by memory ID
- Added MCP support for memory deletion and feedback
**Conversation & Retrieval**
- Added chat API with memory-aware retrieval
- Added memory filtering with custom tags (Cloud & Open Source)
**Multimodal & Tool Memory**
- Added tool memory for tool usage history
- Added image memory support for conversations and documents
📈 Improvements
**Data & Infrastructure**
- Upgraded database for better stability and performance
**Scheduler**
- Rebuilt task scheduler with Redis Streams and queue isolation
- Added task priority, auto-recovery, and quota-based scheduling
**Deployment & Engineering**
- Added lightweight deployment with quick and full modes
🐞 Bug Fixes
**Memory Scheduling & Updates**
- Fixed legacy scheduling API to ensure correct memory isolation
- Fixed memory update logging to show new memories correctly
- **2025-08-07** · 🎉 **MemOS v1.0.0 (MemCube) Release**
First MemCube release with a word-game demo, LongMemEval evaluation, BochaAISearchRetriever integration, NebulaGraph support, improved search capabilities, and the official Playground launch.
✨ New Features
**Playground**
- Expanded Playground features and algorithm performance.
**MemCube Construction**
- Added a text game demo based on the MemCube novel.
**Extended Evaluation Set**
- Added LongMemEval evaluation results and scripts.
📈 Improvements
**Plaintext Memory**
- Integrated internet search with Bocha.
- Added support for Nebula database.
- Added contextual understanding for the tree-structured plaintext memory search interface.
🐞 Bug Fixes
**KV Cache Concatenation**
- Fixed the concat_cache method.
**Plaintext Memory**
- Fixed Nebula search-related issues.
- **2025-07-07** · 🎉 **MemOS v1.0: Stellar (星河) Preview Release**
A SOTA Memory OS for LLMs is now open-sourced.
- **2025-07-04** · 🎉 **MemOS Paper Release**
[MemOS: A Memory OS for AI System](https://arxiv.org/abs/2507.03724) is available on arXiv.
- **2024-07-04** · 🎉 **Memory3 Model Release at WAIC 2024**
The Memory3 model, featuring a memory-layered architecture, was unveiled at the 2024 World Artificial Intelligence Conference.
## 🚀 Quickstart Guide
### ☁️ 1、Cloud API (Hosted)
#### Get API Key
- Sign up on the [MemOS dashboard](https://memos-dashboard.openmem.net/cn/quickstart/?source=landing)
- Go to **API Keys** and copy your key
#### Next Steps
- [MemOS Cloud Getting Started](https://memos-docs.openmem.net/memos_cloud/quick_start/)
Connect to MemOS Cloud and enable memory in minutes.
- [MemOS Cloud Platform](https://memos.openmem.net/?from=/quickstart/)
Explore the Cloud dashboard, features, and workflows.
### 🖥️ 2、Self-Hosted (Local/Private)
1. Get the repository.
```bash
git clone https://github.com/MemTensor/MemOS.git
cd MemOS
pip install -r ./docker/requirements.txt
```
2. Configure `docker/.env.example` and copy to `MemOS/.env`
- The `OPENAI_API_KEY`,`MOS_EMBEDDER_API_KEY`,`MEMRADER_API_KEY` and others can be applied for through [`BaiLian`](https://bailian.console.aliyun.com/?spm=a2c4g.11186623.0.0.2f2165b08fRk4l&tab=api#/api).
- Fill in the corresponding configuration in the `MemOS/.env` file.
3. Start the service.
- Launch via Docker
###### Tips: Please ensure that Docker Compose is installed successfully and that you have navigated to the docker directory (via `cd docker`) before executing the following command.
```bash
# Enter docker directory
docker compose up
```
##### For detailed steps, see the[`Docker Reference`](https://docs.openmem.net/open_source/getting_started/rest_api_server/#method-1-docker-use-repository-dependency-package-imagestart-recommended-use).
- Launch via the uvicorn command line interface (CLI)
###### Tips: Please ensure that Neo4j and Qdrant are running before executing the following command.
```bash
cd src
uvicorn memos.api.server_api:app --host 0.0.0.0 --port 8001 --workers 1
```
##### For detailed integration steps, see the [`CLI Reference`](https://docs.openmem.net/open_source/getting_started/rest_api_server/#method-3client-install-with-CLI).
### Basic Usage (Self-Hosted)
- Add User Message
```python
import requests
import json
data = {
"user_id": "8736b16e-1d20-4163-980b-a5063c3facdc",
"mem_cube_id": "b32d0977-435d-4828-a86f-4f47f8b55bca",
"messages": [
{
"role": "user",
"content": "I like strawberry"
}
],
"async_mode": "sync"
}
headers = {
"Content-Type": "application/json"
}
url = "http://localhost:8000/product/add"
res = requests.post(url=url, headers=headers, data=json.dumps(data))
print(f"result: {res.json()}")
```
- Search User Memory
```python
import requests
import json
data = {
"query": "What do I like",
"user_id": "8736b16e-1d20-4163-980b-a5063c3facdc",
"mem_cube_id": "b32d0977-435d-4828-a86f-4f47f8b55bca"
}
headers = {
"Content-Type": "application/json"
}
url = "http://localhost:8000/product/search"
res = requests.post(url=url, headers=headers, data=json.dumps(data))
print(f"result: {res.json()}")
```
## 📚 Resources
- **Awesome-AI-Memory**
This is a curated repository dedicated to resources on memory and memory systems for large language models. It systematically collects relevant research papers, frameworks, tools, and practical insights. The repository aims to organize and present the rapidly evolving research landscape of LLM memory, bridging multiple research directions including natural language processing, information retrieval, agentic systems, and cognitive science.
- **Get started** 👉 [IAAR-Shanghai/Awesome-AI-Memory](https://github.com/IAAR-Shanghai/Awesome-AI-Memory)
- **MemOS Cloud OpenClaw Plugin**
Official OpenClaw lifecycle plugin for MemOS Cloud. It automatically recalls context from MemOS before the agent starts and saves the conversation back to MemOS after the agent finishes.
- **Get started** 👉 [MemTensor/MemOS-Cloud-OpenClaw-Plugin](https://github.com/MemTensor/MemOS-Cloud-OpenClaw-Plugin)
## 💬 Community & Support
Join our community to ask questions, share your projects, and connect with other developers.
- **GitHub Issues**: Report bugs or request features in our GitHub Issues.
- **GitHub Pull Requests**: Contribute code improvements via Pull Requests.
- **GitHub Discussions**: Participate in our GitHub Discussions to ask questions or share ideas.
- **Discord**: Join our Discord Server.
- **WeChat**: Scan the QR code to join our WeChat group.
## 📜 Citation
> [!NOTE]
> We publicly released the Short Version on **May 28, 2025**, making it the earliest work to propose the concept of a Memory Operating System for LLMs.
If you use MemOS in your research, we would appreciate citations to our papers.
```bibtex
@article{li2025memos_long,
title={MemOS: A Memory OS for AI System},
author={Li, Zhiyu and Song, Shichao and Xi, Chenyang and Wang, Hanyu and Tang, Chen and Niu, Simin and Chen, Ding and Yang, Jiawei and Li, Chunyu and Yu, Qingchen and Zhao, Jihao and Wang, Yezhaohui and Liu, Peng and Lin, Zehao and Wang, Pengyuan and Huo, Jiahao and Chen, Tianyi and Chen, Kai and Li, Kehang and Tao, Zhen and Ren, Junpeng and Lai, Huayi and Wu, Hao and Tang, Bo and Wang, Zhenren and Fan, Zhaoxin and Zhang, Ningyu and Zhang, Linfeng and Yan, Junchi and Yang, Mingchuan and Xu, Tong and Xu, Wei and Chen, Huajun and Wang, Haofeng and Yang, Hongkang and Zhang, Wentao and Xu, Zhi-Qin John and Chen, Siheng and Xiong, Feiyu},
journal={arXiv preprint arXiv:2507.03724},
year={2025},
url={https://arxiv.org/abs/2507.03724}
}
@article{li2025memos_short,
title={MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models},
author={Li, Zhiyu and Song, Shichao and Wang, Hanyu and Niu, Simin and Chen, Ding and Yang, Jiawei and Xi, Chenyang and Lai, Huayi and Zhao, Jihao and Wang, Yezhaohui and others},
journal={arXiv preprint arXiv:2505.22101},
year={2025},
url={https://arxiv.org/abs/2505.22101}
}
@article{yang2024memory3,
author = {Yang, Hongkang and Zehao, Lin and Wenjin, Wang and Wu, Hao and Zhiyu, Li and Tang, Bo and Wenqiang, Wei and Wang, Jinbo and Zeyun, Tang and Song, Shichao and Xi, Chenyang and Yu, Yu and Kai, Chen and Xiong, Feiyu and Tang, Linpeng and Weinan, E},
title = {Memory$^3$: Language Modeling with Explicit Memory},
journal = {Journal of Machine Learning},
year = {2024},
volume = {3},
number = {3},
pages = {300--346},
issn = {2790-2048},
doi = {https://doi.org/10.4208/jml.240708},
url = {https://global-sci.com/article/91443/memory3-language-modeling-with-explicit-memory}
}
```
## 🙌 Contributing
We welcome contributions from the community! Please read our [contribution guidelines](https://memos-docs.openmem.net/open_source/contribution/overview/) to get started.
## 📄 License
MemOS is licensed under the [Apache 2.0 License](./LICENSE).
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/.gitignore
================================================
# Dependencies
node_modules
# Environment variables
.env
.env.*
# NPM
.npmrc
# System
.DS_Store
Thumbs.db
# Logs
npm-debug.log*
yarn-debug.log*
yarn-error.log*
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/README.md
================================================
# MemOS Cloud OpenClaw Plugin (Lifecycle)
Official plugin maintained by MemTensor.
A minimal OpenClaw lifecycle plugin that **recalls** memories from MemOS Cloud before each run and **adds** new messages to MemOS Cloud after each run.
## Features
- **Recall**: `before_agent_start` → `/search/memory`
- **Add**: `agent_end` → `/add/message`
- Uses **Token** auth (`Authorization: Token `)
## Install
### Option A — NPM (Recommended)
```bash
openclaw plugins install @memtensor/memos-cloud-openclaw-plugin@latest
openclaw gateway restart
```
> **Note for Windows Users**:
> If you encounter `Error: spawn EINVAL`, this is a known issue with OpenClaw's plugin installer on Windows. Please use **Option B** (Manual Install) below.
Make sure it’s enabled in `~/.openclaw/openclaw.json`:
```json
{
"plugins": {
"entries": {
"memos-cloud-openclaw-plugin": { "enabled": true }
}
}
}
```
### Option B — Manual Install (Workaround for Windows)
1. Download the latest `.tgz` from [NPM](https://www.npmjs.com/package/@memtensor/memos-cloud-openclaw-plugin).
2. Extract it to a local folder (e.g., `C:\Users\YourName\.openclaw\extensions\memos-cloud-openclaw-plugin`).
3. Configure `~/.openclaw/openclaw.json` (or `%USERPROFILE%\.openclaw\openclaw.json`):
```json
{
"plugins": {
"entries": {
"memos-cloud-openclaw-plugin": { "enabled": true }
},
"load": {
"paths": [
"C:\\Users\\YourName\\.openclaw\\extensions\\memos-cloud-openclaw-plugin\\package"
]
}
}
}
```
*Note: The extracted folder usually contains a `package` subfolder. Point to the folder containing `package.json`.*
Restart the gateway after config changes.
## Environment Variables
The plugin tries env files in order (**openclaw → moltbot → clawdbot**). For each key, the first file with a value wins.
If none of these files exist (or the key is missing), it falls back to the process environment.
**Where to configure**
- Files (priority order):
- `~/.openclaw/.env`
- `~/.moltbot/.env`
- `~/.clawdbot/.env`
- Each line is `KEY=value`
**Quick setup (shell)**
```bash
echo 'export MEMOS_API_KEY="mpg-..."' >> ~/.zshrc
source ~/.zshrc
# or
echo 'export MEMOS_API_KEY="mpg-..."' >> ~/.bashrc
source ~/.bashrc
```
**Quick setup (Windows PowerShell)**
```powershell
[System.Environment]::SetEnvironmentVariable("MEMOS_API_KEY", "mpg-...", "User")
```
If `MEMOS_API_KEY` is missing, the plugin will warn with setup instructions and the API key URL.
**Minimal config**
```env
MEMOS_API_KEY=YOUR_TOKEN
```
**Optional config**
- `MEMOS_BASE_URL` (default: `https://memos.memtensor.cn/api/openmem/v1`)
- `MEMOS_API_KEY` (required; Token auth) — get it at https://memos-dashboard.openmem.net/cn/apikeys/
- `MEMOS_USER_ID` (optional; default: `openclaw-user`)
- `MEMOS_CONVERSATION_ID` (optional override)
- `MEMOS_RECALL_GLOBAL` (default: `true`; when true, search does **not** pass conversation_id)
- `MEMOS_MULTI_AGENT_MODE` (default: `false`; enable multi-agent data isolation)
- `MEMOS_CONVERSATION_PREFIX` / `MEMOS_CONVERSATION_SUFFIX` (optional)
- `MEMOS_CONVERSATION_SUFFIX_MODE` (`none` | `counter`, default: `none`)
- `MEMOS_CONVERSATION_RESET_ON_NEW` (default: `true`, requires hooks.internal.enabled)
- `MEMOS_RECALL_FILTER_ENABLED` (default: `false`; run model-based memory filtering before injection)
- `MEMOS_RECALL_FILTER_BASE_URL` (OpenAI-compatible base URL, e.g. `http://127.0.0.1:11434/v1`)
- `MEMOS_RECALL_FILTER_API_KEY` (optional; required if your endpoint needs auth)
- `MEMOS_RECALL_FILTER_MODEL` (model name used to filter recall candidates)
- `MEMOS_RECALL_FILTER_TIMEOUT_MS` (default: `6000`)
- `MEMOS_RECALL_FILTER_RETRIES` (default: `0`)
- `MEMOS_RECALL_FILTER_CANDIDATE_LIMIT` (default: `30` per category)
- `MEMOS_RECALL_FILTER_MAX_ITEM_CHARS` (default: `500`)
- `MEMOS_RECALL_FILTER_FAIL_OPEN` (default: `true`; fallback to unfiltered recall on failure)
## Optional Plugin Config
In `plugins.entries.memos-cloud-openclaw-plugin.config`:
```json
{
"baseUrl": "https://memos.memtensor.cn/api/openmem/v1",
"apiKey": "YOUR_API_KEY",
"userId": "memos_user_123",
"conversationId": "openclaw-main",
"queryPrefix": "important user context preferences decisions ",
"recallEnabled": true,
"recallGlobal": true,
"addEnabled": true,
"captureStrategy": "last_turn",
"maxItemChars": 8000,
"includeAssistant": true,
"conversationIdPrefix": "",
"conversationIdSuffix": "",
"conversationSuffixMode": "none",
"resetOnNew": true,
"knowledgebaseIds": [],
"memoryLimitNumber": 6,
"preferenceLimitNumber": 6,
"includePreference": true,
"includeToolMemory": false,
"toolMemoryLimitNumber": 6,
"relativity": 0.45,
"tags": ["openclaw"],
"agentId": "",
"multiAgentMode": false,
"asyncMode": true,
"recallFilterEnabled": false,
"recallFilterBaseUrl": "http://127.0.0.1:11434/v1",
"recallFilterApiKey": "",
"recallFilterModel": "qwen2.5:7b",
"recallFilterTimeoutMs": 6000,
"recallFilterRetries": 0,
"recallFilterCandidateLimit": 30,
"recallFilterMaxItemChars": 500,
"recallFilterFailOpen": true
}
```
## How it Works
- **Recall** (`before_agent_start`)
- Builds a `/search/memory` request using `user_id`, `query` (= prompt + optional prefix), and optional filters.
- Default **global recall**: when `recallGlobal=true`, it does **not** pass `conversation_id`.
- Optional second-pass filtering: if `recallFilterEnabled=true`, candidates are sent to your configured model and only returned `keep` items are injected.
- Injects a stable MemOS recall protocol via `appendSystemContext`, while the retrieved `` block remains in `prependContext`.
- **Add** (`agent_end`)
- Builds a `/add/message` request with the **last turn** by default (user + assistant).
- Sends `messages` with `user_id`, `conversation_id`, and optional `tags/info/agent_id/app_id`.
## Multi-Agent Support
The plugin provides native support for multi-agent architectures (via the `agent_id` parameter):
- **Enable Mode**: Set `"multiAgentMode": true` in config or `MEMOS_MULTI_AGENT_MODE=true` in env variables (default is `false`).
- **Dynamic Context**: When enabled, it automatically captures `ctx.agentId` during OpenClaw lifecycle hooks. (Note: the default OpenClaw agent `"main"` is ignored to preserve backwards compatibility for single-agent users).
- **Data Isolation**: The `agent_id` is automatically injected into both `/search/memory` and `/add/message` requests. This ensures completely isolated memory and message histories for different agents, even under the same user or session.
- **Static Override**: You can also force a specific agent ID by setting `"agentId": "your_agent_id"` in the plugin's `config`.
## Notes
- `conversation_id` defaults to OpenClaw `sessionKey` (unless `conversationId` is provided). **TODO**: consider binding to OpenClaw `sessionId` directly.
- Optional **prefix/suffix** via env or config; `conversationSuffixMode=counter` increments on `/new` (requires `hooks.internal.enabled`).
## Acknowledgements
- Thanks to @anatolykoptev (Contributor) — LinkedIn: https://www.linkedin.com/in/koptev?utm_source=share&utm_campaign=share_via&utm_content=profile&utm_medium=ios_app
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/README_ZH.md
================================================
# MemOS Cloud OpenClaw Plugin(Lifecycle 插件)
官方维护:MemTensor。
这是一个最小可用的 OpenClaw lifecycle 插件,功能是:
- **召回记忆**:在每轮对话前从 MemOS Cloud 检索记忆并注入上下文
- **添加记忆**:在每轮对话结束后把消息写回 MemOS Cloud
## 功能
- **Recall**:`before_agent_start` → `/search/memory`
- **Add**:`agent_end` → `/add/message`
- 使用 **Token** 认证(`Authorization: Token `)
## 安装
### 方式 A — NPM(推荐)
```bash
openclaw plugins install @memtensor/memos-cloud-openclaw-plugin@latest
openclaw gateway restart
```
> **Windows 用户注意**:
> 如果遇到 `Error: spawn EINVAL` 报错,这是 OpenClaw Windows 安装器的已知问题。请使用下方的 **方式 B**(手动安装)。
确认 `~/.openclaw/openclaw.json` 中已启用:
```json
{
"plugins": {
"entries": {
"memos-cloud-openclaw-plugin": { "enabled": true }
}
}
}
```
### 方式 B — 手动安装(Windows 解决方案)
1. 从 [NPM](https://www.npmjs.com/package/@memtensor/memos-cloud-openclaw-plugin) 下载最新的 `.tgz` 包。
2. 解压到本地目录(例如 `C:\Users\YourName\.openclaw\extensions\memos-cloud-openclaw-plugin`)。
3. 修改配置 `~/.openclaw/openclaw.json`(或 `%USERPROFILE%\.openclaw\openclaw.json`):
```json
{
"plugins": {
"entries": {
"memos-cloud-openclaw-plugin": { "enabled": true }
},
"load": {
"paths": [
"C:\\Users\\YourName\\.openclaw\\extensions\\memos-cloud-openclaw-plugin\\package"
]
}
}
}
```
*注意:解压后的文件夹通常包含一个 `package` 子文件夹,请指向包含 `package.json` 的那层目录。*
修改配置后需要重启 gateway。
## 环境变量
插件按顺序读取 env 文件(**openclaw → moltbot → clawdbot**),每个键优先使用最先匹配到的值。
若三个文件都不存在(或该键未找到),才会回退到进程环境变量。
**配置位置**
- 文件(优先级顺序):
- `~/.openclaw/.env`
- `~/.moltbot/.env`
- `~/.clawdbot/.env`
- 每行格式:`KEY=value`
**快速配置(Shell)**
```bash
echo 'export MEMOS_API_KEY="mpg-..."' >> ~/.zshrc
source ~/.zshrc
# 或者
echo 'export MEMOS_API_KEY="mpg-..."' >> ~/.bashrc
source ~/.bashrc
```
**快速配置(Windows PowerShell)**
```powershell
[System.Environment]::SetEnvironmentVariable("MEMOS_API_KEY", "mpg-...", "User")
```
若未读取到 `MEMOS_API_KEY`,插件会提示配置方式并附 API Key 获取地址。
**最小配置**
```env
MEMOS_API_KEY=YOUR_TOKEN
```
**可选配置**
- `MEMOS_BASE_URL`(默认 `https://memos.memtensor.cn/api/openmem/v1`)
- `MEMOS_API_KEY`(必填,Token 认证)—— 获取地址:https://memos-dashboard.openmem.net/cn/apikeys/
- `MEMOS_USER_ID`(可选,默认 `openclaw-user`)
- `MEMOS_CONVERSATION_ID`(可选覆盖)
- `MEMOS_RECALL_GLOBAL`(默认 `true`;为 true 时检索不传 conversation_id)
- `MEMOS_MULTI_AGENT_MODE`(默认 `false`;是否开启多 Agent 数据隔离模式)
- `MEMOS_CONVERSATION_PREFIX` / `MEMOS_CONVERSATION_SUFFIX`(可选)
- `MEMOS_CONVERSATION_SUFFIX_MODE`(`none` | `counter`,默认 `none`)
- `MEMOS_CONVERSATION_RESET_ON_NEW`(默认 `true`,需 hooks.internal.enabled)
- `MEMOS_RECALL_FILTER_ENABLED`(默认 `false`;开启后先用你指定的模型过滤召回记忆再注入)
- `MEMOS_RECALL_FILTER_BASE_URL`(OpenAI 兼容接口,例如 `http://127.0.0.1:11434/v1`)
- `MEMOS_RECALL_FILTER_API_KEY`(可选,若你的接口需要鉴权)
- `MEMOS_RECALL_FILTER_MODEL`(用于筛选记忆的模型名)
- `MEMOS_RECALL_FILTER_TIMEOUT_MS`(默认 `6000`)
- `MEMOS_RECALL_FILTER_RETRIES`(默认 `0`)
- `MEMOS_RECALL_FILTER_CANDIDATE_LIMIT`(默认每类 `30` 条)
- `MEMOS_RECALL_FILTER_MAX_ITEM_CHARS`(默认 `500`)
- `MEMOS_RECALL_FILTER_FAIL_OPEN`(默认 `true`;筛选失败时回退为“不过滤”)
## 可选插件配置
在 `plugins.entries.memos-cloud-openclaw-plugin.config` 中设置:
```json
{
"baseUrl": "https://memos.memtensor.cn/api/openmem/v1",
"apiKey": "YOUR_API_KEY",
"userId": "memos_user_123",
"conversationId": "openclaw-main",
"queryPrefix": "important user context preferences decisions ",
"recallEnabled": true,
"recallGlobal": true,
"addEnabled": true,
"captureStrategy": "last_turn",
"includeAssistant": true,
"conversationIdPrefix": "",
"conversationIdSuffix": "",
"conversationSuffixMode": "none",
"resetOnNew": true,
"memoryLimitNumber": 6,
"preferenceLimitNumber": 6,
"knowledgebaseIds": [],
"includePreference": true,
"includeToolMemory": false,
"toolMemoryLimitNumber": 6,
"tags": ["openclaw"],
"agentId": "",
"multiAgentMode": false,
"asyncMode": true,
"recallFilterEnabled": false,
"recallFilterBaseUrl": "http://127.0.0.1:11434/v1",
"recallFilterApiKey": "",
"recallFilterModel": "qwen2.5:7b",
"recallFilterTimeoutMs": 6000,
"recallFilterRetries": 0,
"recallFilterCandidateLimit": 30,
"recallFilterMaxItemChars": 500,
"recallFilterFailOpen": true
}
```
## 工作原理
### 1) 召回(before_agent_start)
- 组装 `/search/memory` 请求
- `user_id`、`query`(= prompt + 可选前缀)
- 默认**全局召回**:`recallGlobal=true` 时不传 `conversation_id`
- 可选 `filter` / `knowledgebase_ids`
- (可选)若开启 `recallFilterEnabled`,会先把 `memory/preference/tool_memory` 候选发给你配置的模型做二次筛选,只保留 `keep` 的条目
- 将稳定的 MemOS 召回协议通过 `appendSystemContext` 注入,而检索到的 `` 数据块继续通过 `prependContext` 注入
### 2) 添加(agent_end)
- 默认只写**最后一轮**(user + assistant)
- 构造 `/add/message` 请求:
- `user_id`、`conversation_id`
- `messages` 列表
- 可选 `tags / info / agent_id / app_id`
## 多Agent支持(Multi-Agent)
插件内置对多Agent模式的支持(`agent_id` 参数):
- **开启模式**:需要在配置中设置 `"multiAgentMode": true` 或在环境变量中设置 `MEMOS_MULTI_AGENT_MODE=true`(默认为 `false`)。
- **动态获取**:开启后,执行生命周期钩子时会自动读取上下文中的 `ctx.agentId`。(注:OpenClaw 的默认 Agent `"main"` 会被自动忽略,以保证老用户的单 Agent 数据兼容性)。
- **数据隔离**:在调用 `/search/memory`(检索记忆)和 `/add/message`(添加记录)时会自动附带该 `agent_id`,从而保证即使是同一用户下的不同 Agent 之间,记忆和反馈数据也是完全隔离的。
- **静态配置**:如果需要,也可在上述插件的 `config` 中显式指定 `"agentId": "your_agent_id"` 作为固定值。
## 说明
- 未显式指定 `conversation_id` 时,默认使用 OpenClaw `sessionKey`。**TODO**:后续考虑直接绑定 OpenClaw `sessionId`。
- 可配置前后缀;`conversationSuffixMode=counter` 时会在 `/new` 递增(需 `hooks.internal.enabled`)。
## 致谢
- 感谢 @anatolykoptev(Contributor)— 领英:https://www.linkedin.com/in/koptev?utm_source=share&utm_campaign=share_via&utm_content=profile&utm_medium=ios_app
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/clawdbot.plugin.json
================================================
{
"id": "memos-cloud-openclaw-plugin",
"name": "MemOS Cloud OpenClaw Plugin",
"description": "MemOS Cloud recall + add memory via lifecycle hooks",
"version": "0.1.9",
"kind": "lifecycle",
"main": "./index.js",
"configSchema": {
"type": "object",
"properties": {
"baseUrl": {
"type": "string",
"description": "MemOS Cloud base URL"
},
"apiKey": {
"type": "string",
"description": "MemOS API Key (Token auth; supports ~/.openclaw/.env, ~/.moltbot/.env, ~/.clawdbot/.env; falls back to process env)"
},
"userId": {
"type": "string",
"description": "MemOS user_id (default: openclaw-user)",
"default": "openclaw-user"
},
"conversationId": {
"type": "string",
"description": "Override conversation_id"
},
"conversationIdPrefix": {
"type": "string",
"description": "conversation_id prefix"
},
"conversationIdSuffix": {
"type": "string",
"description": "conversation_id suffix"
},
"conversationSuffixMode": {
"type": "string",
"enum": [
"none",
"counter"
],
"default": "none"
},
"resetOnNew": {
"type": "boolean",
"default": true
},
"queryPrefix": {
"type": "string",
"description": "Prefix added to search queries"
},
"maxQueryChars": {
"type": "integer",
"description": "Max chars for search query"
},
"recallEnabled": {
"type": "boolean",
"default": true
},
"recallGlobal": {
"type": "boolean",
"default": true
},
"addEnabled": {
"type": "boolean",
"default": true
},
"captureStrategy": {
"type": "string",
"enum": [
"last_turn",
"full_session"
],
"default": "last_turn"
},
"maxMessageChars": {
"type": "integer",
"description": "Max chars per message when adding",
"default": 20000
},
"maxItemChars": {
"type": "integer",
"description": "Max chars per memory item when injecting prompt",
"default": 8000
},
"includeAssistant": {
"type": "boolean",
"default": true
},
"memoryLimitNumber": {
"type": "integer",
"default": 6
},
"preferenceLimitNumber": {
"type": "integer",
"default": 6
},
"includePreference": {
"type": "boolean",
"default": true
},
"includeToolMemory": {
"type": "boolean",
"default": false
},
"toolMemoryLimitNumber": {
"type": "integer",
"default": 6
},
"filter": {
"type": "object",
"description": "MemOS search filter"
},
"knowledgebaseIds": {
"type": "array",
"items": {
"type": "string"
}
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
},
"info": {
"type": "object",
"additionalProperties": true
},
"agentId": {
"type": "string"
},
"multiAgentMode": {
"type": "boolean",
"default": false
},
"appId": {
"type": "string"
},
"allowPublic": {
"type": "boolean",
"default": false
},
"allowKnowledgebaseIds": {
"type": "array",
"items": {
"type": "string"
}
},
"asyncMode": {
"type": "boolean",
"default": true
},
"timeoutMs": {
"type": "integer",
"default": 5000
},
"retries": {
"type": "integer",
"default": 1
},
"throttleMs": {
"type": "integer",
"default": 0
}
},
"additionalProperties": false
}
}
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/index.js
================================================
#!/usr/bin/env node
import {
addMessage,
buildConfig,
extractResultData,
extractText,
formatRecallHookResult,
USER_QUERY_MARKER,
searchMemory,
} from "./lib/memos-cloud-api.js";
import { startUpdateChecker } from "./lib/check-update.js";
let lastCaptureTime = 0;
const conversationCounters = new Map();
const API_KEY_HELP_URL = "https://memos-dashboard.openmem.net/cn/apikeys/";
const ENV_FILE_SEARCH_HINTS = ["~/.openclaw/.env", "~/.moltbot/.env", "~/.clawdbot/.env"];
const MEMOS_SOURCE = "openclaw";
function warnMissingApiKey(log, context) {
const heading = "[memos-cloud] Missing MEMOS_API_KEY (Token auth)";
const header = `${heading}${context ? `; ${context} skipped` : ""}. Configure it with:`;
log.warn?.(
[
header,
"echo 'export MEMOS_API_KEY=\"mpg-...\"' >> ~/.zshrc",
"source ~/.zshrc",
"or",
"echo 'export MEMOS_API_KEY=\"mpg-...\"' >> ~/.bashrc",
"source ~/.bashrc",
"or",
"[System.Environment]::SetEnvironmentVariable(\"MEMOS_API_KEY\", \"mpg-...\", \"User\")",
`Get API key: ${API_KEY_HELP_URL}`,
].join("\n"),
);
}
function stripPrependedPrompt(content) {
if (!content) return content;
const idx = content.lastIndexOf(USER_QUERY_MARKER);
if (idx === -1) return content;
return content.slice(idx + USER_QUERY_MARKER.length).trimStart();
}
function getCounterSuffix(sessionKey) {
if (!sessionKey) return "";
const current = conversationCounters.get(sessionKey) ?? 0;
return current > 0 ? `#${current}` : "";
}
function bumpConversationCounter(sessionKey) {
if (!sessionKey) return;
const current = conversationCounters.get(sessionKey) ?? 0;
conversationCounters.set(sessionKey, current + 1);
}
function getEffectiveAgentId(cfg, ctx) {
if (!cfg.multiAgentMode) {
return cfg.agentId;
}
const agentId = ctx?.agentId || cfg.agentId;
return agentId === "main" ? undefined : agentId;
}
function resolveConversationId(cfg, ctx) {
if (cfg.conversationId) return cfg.conversationId;
// TODO: consider binding conversation_id directly to OpenClaw sessionId (prefer ctx.sessionId).
const agentId = getEffectiveAgentId(cfg, ctx);
const base = ctx?.sessionKey || ctx?.sessionId || (agentId ? `openclaw:${agentId}` : "");
const dynamicSuffix = cfg.conversationSuffixMode === "counter" ? getCounterSuffix(ctx?.sessionKey) : "";
const prefix = cfg.conversationIdPrefix || "";
const suffix = cfg.conversationIdSuffix || "";
if (base) return `${prefix}${base}${dynamicSuffix}${suffix}`;
return `${prefix}openclaw-${Date.now()}${dynamicSuffix}${suffix}`;
}
function buildSearchPayload(cfg, prompt, ctx) {
const queryRaw = `${cfg.queryPrefix || ""}${prompt}`;
const query =
Number.isFinite(cfg.maxQueryChars) && cfg.maxQueryChars > 0
? queryRaw.slice(0, cfg.maxQueryChars)
: queryRaw;
const payload = {
user_id: cfg.userId,
query,
source: MEMOS_SOURCE,
};
if (!cfg.recallGlobal) {
const conversationId = resolveConversationId(cfg, ctx);
if (conversationId) payload.conversation_id = conversationId;
}
let filterObj = cfg.filter ? JSON.parse(JSON.stringify(cfg.filter)) : null;
const agentId = getEffectiveAgentId(cfg, ctx);
if (agentId) {
if (filterObj) {
if (Array.isArray(filterObj.and)) {
filterObj.and.push({ agent_id: agentId });
} else {
filterObj = { and: [filterObj, { agent_id: agentId }] };
}
} else {
filterObj = { agent_id: agentId };
}
}
if (filterObj) payload.filter = filterObj;
if (cfg.knowledgebaseIds?.length) payload.knowledgebase_ids = cfg.knowledgebaseIds;
payload.memory_limit_number = cfg.memoryLimitNumber;
payload.include_preference = cfg.includePreference;
payload.preference_limit_number = cfg.preferenceLimitNumber;
payload.include_tool_memory = cfg.includeToolMemory;
payload.tool_memory_limit_number = cfg.toolMemoryLimitNumber;
payload.relativity = cfg.relativity;
return payload;
}
function buildAddMessagePayload(cfg, messages, ctx) {
const payload = {
user_id: cfg.userId,
conversation_id: resolveConversationId(cfg, ctx),
messages,
source: MEMOS_SOURCE,
};
const agentId = getEffectiveAgentId(cfg, ctx);
if (agentId) payload.agent_id = agentId;
if (cfg.appId) payload.app_id = cfg.appId;
if (cfg.tags?.length) payload.tags = cfg.tags;
const info = {
source: "openclaw",
sessionKey: ctx?.sessionKey,
agentId: ctx?.agentId,
...(cfg.info || {}),
};
if (Object.keys(info).length > 0) payload.info = info;
payload.allow_public = cfg.allowPublic;
if (cfg.allowKnowledgebaseIds?.length) payload.allow_knowledgebase_ids = cfg.allowKnowledgebaseIds;
payload.async_mode = cfg.asyncMode;
return payload;
}
function pickLastTurnMessages(messages, cfg) {
const lastUserIndex = messages
.map((m, idx) => ({ m, idx }))
.filter(({ m }) => m?.role === "user")
.map(({ idx }) => idx)
.pop();
if (lastUserIndex === undefined) return [];
const slice = messages.slice(lastUserIndex);
const results = [];
for (const msg of slice) {
if (!msg || !msg.role) continue;
if (msg.role === "user") {
const content = stripPrependedPrompt(extractText(msg.content));
if (content) results.push({ role: "user", content: truncate(content, cfg.maxMessageChars) });
continue;
}
if (msg.role === "assistant" && cfg.includeAssistant) {
const content = extractText(msg.content);
if (content) results.push({ role: "assistant", content: truncate(content, cfg.maxMessageChars) });
}
}
return results;
}
function pickFullSessionMessages(messages, cfg) {
const results = [];
for (const msg of messages) {
if (!msg || !msg.role) continue;
if (msg.role === "user") {
const content = stripPrependedPrompt(extractText(msg.content));
if (content) results.push({ role: "user", content: truncate(content, cfg.maxMessageChars) });
}
if (msg.role === "assistant" && cfg.includeAssistant) {
const content = extractText(msg.content);
if (content) results.push({ role: "assistant", content: truncate(content, cfg.maxMessageChars) });
}
}
return results;
}
function truncate(text, maxLen) {
if (!text) return "";
if (!maxLen) return text;
return text.length > maxLen ? `${text.slice(0, maxLen)}...` : text;
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function parseModelJson(text) {
if (!text || typeof text !== "string") return null;
const trimmed = text.trim();
if (!trimmed) return null;
try {
return JSON.parse(trimmed);
} catch {
// Some models wrap JSON in markdown code fences.
}
const fenceMatch = trimmed.match(/```(?:json)?\s*([\s\S]*?)\s*```/i);
if (fenceMatch?.[1]) {
try {
return JSON.parse(fenceMatch[1].trim());
} catch {
return null;
}
}
const first = trimmed.indexOf("{");
const last = trimmed.lastIndexOf("}");
if (first >= 0 && last > first) {
try {
return JSON.parse(trimmed.slice(first, last + 1));
} catch {
return null;
}
}
return null;
}
function normalizeIndexList(value, maxLen) {
if (!Array.isArray(value)) return [];
const seen = new Set();
const out = [];
for (const v of value) {
if (!Number.isInteger(v)) continue;
if (v < 0 || v >= maxLen) continue;
if (seen.has(v)) continue;
seen.add(v);
out.push(v);
}
return out;
}
function buildRecallCandidates(data, cfg) {
const limit = Number.isFinite(cfg.recallFilterCandidateLimit) ? Math.max(0, cfg.recallFilterCandidateLimit) : 30;
const maxChars = Number.isFinite(cfg.recallFilterMaxItemChars) ? Math.max(80, cfg.recallFilterMaxItemChars) : 500;
const memoryList = Array.isArray(data?.memory_detail_list) ? data.memory_detail_list : [];
const preferenceList = Array.isArray(data?.preference_detail_list) ? data.preference_detail_list : [];
const toolList = Array.isArray(data?.tool_memory_detail_list) ? data.tool_memory_detail_list : [];
const memoryCandidates = memoryList.slice(0, limit).map((item, idx) => ({
idx,
text: truncate(item?.memory_value || item?.memory_key || "", maxChars),
relativity: item?.relativity,
}));
const preferenceCandidates = preferenceList.slice(0, limit).map((item, idx) => ({
idx,
text: truncate(item?.preference || "", maxChars),
relativity: item?.relativity,
preference_type: item?.preference_type || "",
}));
const toolCandidates = toolList.slice(0, limit).map((item, idx) => ({
idx,
text: truncate(item?.tool_value || "", maxChars),
relativity: item?.relativity,
}));
return {
memoryList,
preferenceList,
toolList,
candidatePayload: {
memory: memoryCandidates,
preference: preferenceCandidates,
tool_memory: toolCandidates,
},
};
}
function applyRecallDecision(data, decision, lists) {
const keep = decision?.keep || {};
const memoryIdx = normalizeIndexList(keep.memory, lists.memoryList.length);
const preferenceIdx = normalizeIndexList(keep.preference, lists.preferenceList.length);
const toolIdx = normalizeIndexList(keep.tool_memory, lists.toolList.length);
return {
...data,
memory_detail_list: memoryIdx.map((idx) => lists.memoryList[idx]),
preference_detail_list: preferenceIdx.map((idx) => lists.preferenceList[idx]),
tool_memory_detail_list: toolIdx.map((idx) => lists.toolList[idx]),
};
}
async function callRecallFilterModel(cfg, userPrompt, candidatePayload) {
const headers = {
"Content-Type": "application/json",
};
if (cfg.recallFilterApiKey) {
headers.Authorization = `Bearer ${cfg.recallFilterApiKey}`;
}
const modelInput = {
user_query: userPrompt,
candidate_memories: candidatePayload,
output_schema: {
keep: {
memory: ["number index"],
preference: ["number index"],
tool_memory: ["number index"],
},
reason: "optional short string",
},
};
const body = {
model: cfg.recallFilterModel,
temperature: 0,
messages: [
{
role: "system",
content:
"You are a strict memory relevance judge. Return JSON only. Keep only items directly useful for answering current user query. If unsure, do not keep.",
},
{
role: "user",
content: JSON.stringify(modelInput),
},
],
};
let lastError;
const retries = Number.isFinite(cfg.recallFilterRetries) ? Math.max(0, cfg.recallFilterRetries) : 0;
const timeoutMs = Number.isFinite(cfg.recallFilterTimeoutMs) ? Math.max(1000, cfg.recallFilterTimeoutMs) : 6000;
for (let attempt = 0; attempt <= retries; attempt += 1) {
try {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
const res = await fetch(`${cfg.recallFilterBaseUrl}/chat/completions`, {
method: "POST",
headers,
body: JSON.stringify(body),
signal: controller.signal,
});
clearTimeout(timeoutId);
if (!res.ok) {
throw new Error(`HTTP ${res.status}`);
}
const json = await res.json();
const text = json?.choices?.[0]?.message?.content || "";
const parsed = parseModelJson(text);
if (!parsed || typeof parsed !== "object") {
throw new Error("invalid JSON output from recall filter model");
}
return parsed;
} catch (err) {
lastError = err;
if (attempt < retries) {
await sleep(120 * (attempt + 1));
}
}
}
throw lastError;
}
async function maybeFilterRecallData(cfg, data, userPrompt, log) {
if (!cfg.recallFilterEnabled) return data;
if (!cfg.recallFilterBaseUrl || !cfg.recallFilterModel) {
log.warn?.("[memos-cloud] recall filter enabled but missing recallFilterBaseUrl/recallFilterModel; skip filter");
return data;
}
const lists = buildRecallCandidates(data, cfg);
const hasCandidates =
lists.candidatePayload.memory.length > 0 ||
lists.candidatePayload.preference.length > 0 ||
lists.candidatePayload.tool_memory.length > 0;
if (!hasCandidates) return data;
try {
const decision = await callRecallFilterModel(cfg, userPrompt, lists.candidatePayload);
return applyRecallDecision(data, decision, lists);
} catch (err) {
log.warn?.(`[memos-cloud] recall filter failed: ${String(err)}`);
return cfg.recallFilterFailOpen ? data : { ...data, memory_detail_list: [], preference_detail_list: [], tool_memory_detail_list: [] };
}
}
export default {
id: "memos-cloud-openclaw-plugin",
name: "MemOS Cloud OpenClaw Plugin",
description: "MemOS Cloud recall + add memory via lifecycle hooks",
kind: "lifecycle",
register(api) {
const cfg = buildConfig(api.pluginConfig);
const log = api.logger ?? console;
// Start 12-hour background update interval
startUpdateChecker(log);
if (!cfg.envFileStatus?.found) {
const searchPaths = cfg.envFileStatus?.searchPaths?.join(", ") ?? ENV_FILE_SEARCH_HINTS.join(", ");
log.warn?.(`[memos-cloud] No .env found in ${searchPaths}; falling back to process env or plugin config.`);
}
if (cfg.conversationSuffixMode === "counter" && cfg.resetOnNew) {
if (api.config?.hooks?.internal?.enabled !== true) {
log.warn?.("[memos-cloud] command:new hook requires hooks.internal.enabled = true");
}
api.registerHook(
["command:new"],
(event) => {
if (event?.type === "command" && event?.action === "new") {
bumpConversationCounter(event.sessionKey);
}
},
{
name: "memos-cloud-conversation-new",
description: "Increment MemOS conversation suffix on /new",
},
);
}
api.on("before_agent_start", async (event, ctx) => {
if (!cfg.recallEnabled) return;
if (!event?.prompt || event.prompt.length < 3) return;
if (!cfg.apiKey) {
warnMissingApiKey(log, "recall");
return;
}
try {
const payload = buildSearchPayload(cfg, event.prompt, ctx);
const result = await searchMemory(cfg, payload);
const resultData = extractResultData(result);
if (!resultData) return;
const filteredData = await maybeFilterRecallData(cfg, resultData, event.prompt, log);
const hookResult = formatRecallHookResult({ data: filteredData }, {
wrapTagBlocks: true,
relativity: payload.relativity,
maxItemChars: cfg.maxItemChars,
});
if (!hookResult.appendSystemContext && !hookResult.prependContext) return;
return hookResult;
} catch (err) {
log.warn?.(`[memos-cloud] recall failed: ${String(err)}`);
}
});
api.on("agent_end", async (event, ctx) => {
if (!cfg.addEnabled) return;
if (!event?.success || !event?.messages?.length) return;
if (!cfg.apiKey) {
warnMissingApiKey(log, "add");
return;
}
const now = Date.now();
if (cfg.throttleMs && now - lastCaptureTime < cfg.throttleMs) {
return;
}
lastCaptureTime = now;
try {
const messages =
cfg.captureStrategy === "full_session"
? pickFullSessionMessages(event.messages, cfg)
: pickLastTurnMessages(event.messages, cfg);
if (!messages.length) return;
const payload = buildAddMessagePayload(cfg, messages, ctx);
await addMessage(cfg, payload);
} catch (err) {
log.warn?.(`[memos-cloud] add failed: ${String(err)}`);
}
});
},
};
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/lib/check-update.js
================================================
import https from "https";
import fs from "fs";
import path from "path";
import { fileURLToPath } from "url";
import { spawn, exec } from "child_process";
import os from "os";
/**
* Kill a spawned child process and its entire process tree.
*/
function killProcessTree(child) {
try {
if (process.platform === "win32") {
exec(`taskkill /pid ${child.pid} /T /F`, () => {});
} else {
// On Unix, kill the process group
process.kill(-child.pid, "SIGKILL");
}
} catch (e) {
// Fallback: try the basic kill
try { child.kill("SIGKILL"); } catch (_) {}
}
}
let isUpdating = false;
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const CHECK_INTERVAL = 12 * 60 * 60 * 1000; // 12 hours check interval
const UPDATE_TIMEOUT = 3 * 60 * 1000; // 3 minutes timeout for the CLI update command to finish
const PLUGIN_NAME = "@memtensor/memos-cloud-openclaw-plugin";
const CHECK_FILE = path.join(os.tmpdir(), "memos_openclaw_update_check.json");
const ANSI = {
RESET: "\x1b[0m",
GREEN: "\x1b[32m",
YELLOW: "\x1b[33m",
CYAN: "\x1b[36m",
RED: "\x1b[31m"
};
function getPackageVersion() {
try {
const pkgPath = path.join(__dirname, "..", "package.json");
const pkgData = fs.readFileSync(pkgPath, "utf-8");
const pkg = JSON.parse(pkgData);
return pkg.version;
} catch (err) {
return null;
}
}
function getLatestVersion(log) {
return new Promise((resolve, reject) => {
const req = https.get(
`https://registry.npmjs.org/${PLUGIN_NAME}/latest`,
{ timeout: 5000 },
(res) => {
if (res.statusCode !== 200) {
req.destroy();
return reject(new Error(`Failed to fetch version, status: ${res.statusCode}`));
}
let body = "";
res.on("data", (chunk) => {
body += chunk;
});
res.on("end", () => {
try {
const data = JSON.parse(body);
resolve(data.version);
} catch (err) {
reject(err);
}
});
}
);
req.on("error", (err) => {
reject(err);
});
req.on("timeout", () => {
req.destroy();
reject(new Error("Timeout getting latest version"));
});
});
}
function compareVersions(v1, v2) {
// Split pre-release tags (e.g. 0.1.8-beta.1 -> "0.1.8" and "beta.1")
const split1 = v1.split("-");
const split2 = v2.split("-");
const parts1 = split1[0].split(".").map(Number);
const parts2 = split2[0].split(".").map(Number);
// Compare major.minor.patch
for (let i = 0; i < Math.max(parts1.length, parts2.length); i++) {
const p1 = parts1[i] || 0;
const p2 = parts2[i] || 0;
if (p1 > p2) return 1;
if (p1 < p2) return -1;
}
// If base versions are equal, compare pre-release tags.
// A version WITH a pre-release tag is LOWER than a version WITHOUT one.
// e.g. 0.1.8-beta is less than 0.1.8. 0.1.8 is the final release.
const hasPre1 = split1.length > 1;
const hasPre2 = split2.length > 1;
if (hasPre1 && !hasPre2) return -1; // v1 is a beta, v2 is a full release
if (!hasPre1 && hasPre2) return 1; // v1 is a full release, v2 is a beta
if (!hasPre1 && !hasPre2) return 0; // both are full releases and equal
// If both are pre-releases, do a basic string compare on the tag
// "alpha" < "beta" < "rc"
if (split1[1] > split2[1]) return 1;
if (split1[1] < split2[1]) return -1;
return 0;
}
export function startUpdateChecker(log) {
// Only start the interval if we are in the gateway
const isGateway = process.argv.includes("gateway");
if (!isGateway) {
return;
}
const runCheck = async () => {
if (isUpdating) {
log.info?.(`${ANSI.YELLOW}[memos-cloud] An update sequence is currently in progress, skipping this check.${ANSI.RESET}`);
return;
}
// TRULY PREVENT LOOPS: The instant we start a check, record the time BEFORE any network or processing happens.
// This absolutely guarantees that even if the network hangs, NPM crashes, or openclaw update causes an immediate hot reload,
// the system has already advanced the 12-hour/1-min clock and will NOT re-enter this function on boot.
try {
fs.writeFileSync(CHECK_FILE, JSON.stringify({ time: Date.now() }));
} catch (e) {
log.warn?.(`${ANSI.RED}[memos-cloud] Failed to write timestamp file: ${e.message}${ANSI.RESET}`);
}
const currentVersion = getPackageVersion();
if (!currentVersion) {
log.warn?.(`${ANSI.RED}[memos-cloud] Could not read current version from package.json${ANSI.RESET}`);
return;
}
try {
const latestVersion = await getLatestVersion(log);
// Normal version check
if (compareVersions(latestVersion, currentVersion) <= 0) {
return;
}
log.info?.(`${ANSI.YELLOW}[memos-cloud] Update available: ${currentVersion} -> ${latestVersion}. Updating in background...${ANSI.RESET}`);
let dotCount = 0;
const progressInterval = setInterval(() => {
dotCount++;
const dots = ".".repeat(dotCount % 4);
log.info?.(`${ANSI.YELLOW}[memos-cloud] Update in progress for memos-cloud-openclaw-plugin${dots}${ANSI.RESET}`);
}, 30000); // Log every 30 seconds to show it's still alive without spamming
const cliName = (() => {
// Check the full path of the entry script (e.g., .../moltbot/bin/index.js) or the executable
const scriptPath = process.argv[1] ? process.argv[1].toLowerCase() : "";
const execPath = process.execPath ? process.execPath.toLowerCase() : "";
if (scriptPath.includes("moltbot") || execPath.includes("moltbot")) return "moltbot";
if (scriptPath.includes("clawdbot") || execPath.includes("clawdbot")) return "clawdbot";
return "openclaw";
})();
isUpdating = true;
const spawnOpts = { shell: true };
// On Unix, detach the process so we can kill the entire process group on timeout
if (process.platform !== "win32") {
spawnOpts.detached = true;
}
const child = spawn(cliName, ["plugins", "update", "memos-cloud-openclaw-plugin"], spawnOpts);
// Timeout mechanism: forcefully kill the update process if it hangs for more than the configured timeout
const updateTimeout = setTimeout(() => {
log.warn?.(`${ANSI.RED}[memos-cloud] Update process timed out. Please try manually running: ${cliName} plugins update memos-cloud-openclaw-plugin${ANSI.RESET}`);
killProcessTree(child);
// Fallback: if kill failed and the close event never fires, forcefully release the lock after 5 seconds
setTimeout(() => {
if (isUpdating) {
clearInterval(progressInterval);
isUpdating = false;
}
}, 5000);
}, UPDATE_TIMEOUT);
child.stdout.on("data", (data) => {
const outText = data.toString();
log.info?.(`${ANSI.CYAN}[${cliName}-cli]${ANSI.RESET}\n${outText.trim()}`);
// Auto-reply to any [y/N] prompts from the CLI
if (outText.toLowerCase().includes("[y/n]")) {
child.stdin.write("y\n");
}
});
child.stderr.on("data", (data) => {
const errText = data.toString();
log.warn?.(`${ANSI.RED}[${cliName}-cli]${ANSI.RESET}\n${errText.trim()}`);
// Some CLIs output interactive prompts to stderr instead of stdout
if (errText.toLowerCase().includes("[y/n]")) {
child.stdin.write("y\n");
}
});
child.on("close", (code) => {
clearTimeout(updateTimeout);
clearInterval(progressInterval);
isUpdating = false;
// Wait for a brief moment to let file system sync if needed
setTimeout(() => {
const postUpdateVersion = getPackageVersion();
const actuallyUpdated = (postUpdateVersion === latestVersion) && (postUpdateVersion !== currentVersion);
if (code !== 0 || !actuallyUpdated) {
log.warn?.(`${ANSI.RED}[memos-cloud] Auto-update failed or version did not change. Please refer to the CLI logs above, or run manually: ${cliName} plugins update memos-cloud-openclaw-plugin${ANSI.RESET}`);
} else {
log.info?.(`${ANSI.GREEN}[memos-cloud] Successfully updated to version ${latestVersion}. Please restart the gateway to apply changes.${ANSI.RESET}`);
}
}, 1000); // Small 1-second buffer for file systems
});
} catch (error) {
log.warn?.(`${ANSI.RED}[memos-cloud] Update check failed entirely: ${error.message}${ANSI.RESET}`);
}
};
// Check when we last ran
let lastCheckTime = 0;
try {
if (fs.existsSync(CHECK_FILE)) {
const data = JSON.parse(fs.readFileSync(CHECK_FILE, "utf-8"));
lastCheckTime = data.time || 0;
}
} catch (e) {}
const now = Date.now();
const timeSinceLastCheck = now - lastCheckTime;
// If the interval has passed, run it IMMEDIATELY without delay.
// The immediate file-write at the top of runCheck() will prevent loop scenarios.
if (timeSinceLastCheck >= CHECK_INTERVAL) {
runCheck();
setInterval(runCheck, CHECK_INTERVAL);
} else {
// If it hasn't been the full interval yet, wait the remaining time, then trigger interval
const timeUntilNextCheck = CHECK_INTERVAL - timeSinceLastCheck;
setTimeout(() => {
runCheck();
setInterval(runCheck, CHECK_INTERVAL);
}, timeUntilNextCheck);
}
}
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/lib/memos-cloud-api.js
================================================
import { readFileSync } from "node:fs";
import { join } from "node:path";
import { homedir } from "node:os";
import { setTimeout as delay } from "node:timers/promises";
const DEFAULT_BASE_URL = "https://memos.memtensor.cn/api/openmem/v1";
export const USER_QUERY_MARKER = "user\u200b原\u200b始\u200bquery\u200b:\u200b\u200b\u200b\u200b";
const ENV_SOURCES = [
{ name: "openclaw", path: join(homedir(), ".openclaw", ".env") },
{ name: "moltbot", path: join(homedir(), ".moltbot", ".env") },
{ name: "clawdbot", path: join(homedir(), ".clawdbot", ".env") },
];
let envFilesLoaded = false;
const envFileContents = new Map();
const envFileValues = new Map();
function stripQuotes(value) {
if (!value) return value;
const trimmed = value.trim();
if (
(trimmed.startsWith("\"") && trimmed.endsWith("\"")) ||
(trimmed.startsWith("'") && trimmed.endsWith("'"))
) {
return trimmed.slice(1, -1);
}
return trimmed;
}
export function extractResultData(result) {
if (!result || typeof result !== "object") return null;
return result.data ?? result.data?.data ?? result.data?.result ?? null;
}
function pad2(value) {
return String(value).padStart(2, "0");
}
function formatTime(value) {
if (value === undefined || value === null || value === "") return "";
if (typeof value === "number") {
const date = new Date(value);
if (Number.isNaN(date.getTime())) return "";
return `${date.getFullYear()}-${pad2(date.getMonth() + 1)}-${pad2(date.getDate())} ${pad2(
date.getHours(),
)}:${pad2(date.getMinutes())}`;
}
if (typeof value === "string") {
const trimmed = value.trim();
if (!trimmed) return "";
if (/^\d+$/.test(trimmed)) return formatTime(Number(trimmed));
return trimmed;
}
return "";
}
function parseEnvFile(content) {
const values = new Map();
for (const line of content.split(/\r?\n/)) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
const idx = trimmed.indexOf("=");
if (idx <= 0) continue;
const key = trimmed.slice(0, idx).trim();
const rawValue = trimmed.slice(idx + 1);
if (!key) continue;
values.set(key, stripQuotes(rawValue));
}
return values;
}
function loadEnvFiles() {
if (envFilesLoaded) return;
envFilesLoaded = true;
for (const source of ENV_SOURCES) {
try {
const content = readFileSync(source.path, "utf-8");
envFileContents.set(source.name, content);
envFileValues.set(source.name, parseEnvFile(content));
} catch {
// ignore missing files
}
}
}
function loadEnvFromFiles(name) {
for (const source of ENV_SOURCES) {
const values = envFileValues.get(source.name);
if (!values) continue;
if (values.has(name)) return values.get(name);
}
return undefined;
}
function loadEnvVar(name) {
loadEnvFiles();
const fromFiles = loadEnvFromFiles(name);
if (fromFiles !== undefined) return fromFiles;
if (envFileContents.size === 0) return process.env[name];
return undefined;
}
export function getEnvFileStatus() {
loadEnvFiles();
const sources = ENV_SOURCES.filter((source) => envFileContents.has(source.name));
return {
found: sources.length > 0,
sources: sources.map((source) => source.name),
paths: sources.map((source) => source.path),
searchPaths: ENV_SOURCES.map((source) => source.path),
};
}
function parseBool(value, fallback) {
if (value === undefined || value === null || value === "") return fallback;
if (typeof value === "boolean") return value;
const normalized = String(value).trim().toLowerCase();
if (["1", "true", "yes", "y", "on"].includes(normalized)) return true;
if (["0", "false", "no", "n", "off"].includes(normalized)) return false;
return fallback;
}
function parseNumber(value, fallback) {
if (value === undefined || value === null || value === "") return fallback;
const n = Number(value);
return Number.isFinite(n) ? n : fallback;
}
export function buildConfig(pluginConfig = {}) {
const cfg = pluginConfig ?? {};
const baseUrl = cfg.baseUrl || loadEnvVar("MEMOS_BASE_URL") || DEFAULT_BASE_URL;
const apiKey = cfg.apiKey || loadEnvVar("MEMOS_API_KEY") || "";
const userId = cfg.userId || loadEnvVar("MEMOS_USER_ID") || "openclaw-user";
const conversationId = cfg.conversationId || loadEnvVar("MEMOS_CONVERSATION_ID") || "";
const recallGlobal = parseBool(
cfg.recallGlobal,
parseBool(loadEnvVar("MEMOS_RECALL_GLOBAL"), true),
);
const conversationIdPrefix = cfg.conversationIdPrefix ?? loadEnvVar("MEMOS_CONVERSATION_PREFIX") ?? "";
const conversationIdSuffix = cfg.conversationIdSuffix ?? loadEnvVar("MEMOS_CONVERSATION_SUFFIX") ?? "";
const conversationSuffixMode =
cfg.conversationSuffixMode ?? loadEnvVar("MEMOS_CONVERSATION_SUFFIX_MODE") ?? "none";
const resetOnNew = parseBool(
cfg.resetOnNew,
parseBool(loadEnvVar("MEMOS_CONVERSATION_RESET_ON_NEW"), true),
);
const multiAgentMode = parseBool(
cfg.multiAgentMode,
parseBool(loadEnvVar("MEMOS_MULTI_AGENT_MODE"), false),
);
const recallFilterEnabled = parseBool(
cfg.recallFilterEnabled,
parseBool(loadEnvVar("MEMOS_RECALL_FILTER_ENABLED"), false),
);
const recallFilterFailOpen = parseBool(
cfg.recallFilterFailOpen,
parseBool(loadEnvVar("MEMOS_RECALL_FILTER_FAIL_OPEN"), true),
);
return {
baseUrl: baseUrl.replace(/\/+$/, ""),
apiKey,
userId,
conversationId,
conversationIdPrefix,
conversationIdSuffix,
conversationSuffixMode,
recallGlobal,
resetOnNew,
envFileStatus: getEnvFileStatus(),
queryPrefix: cfg.queryPrefix ?? "",
maxQueryChars: cfg.maxQueryChars ?? 0,
recallEnabled: cfg.recallEnabled !== false,
addEnabled: cfg.addEnabled !== false,
captureStrategy: cfg.captureStrategy ?? "last_turn",
maxMessageChars: cfg.maxMessageChars ?? 20000,
maxItemChars: cfg.maxItemChars ?? 8000,
includeAssistant: cfg.includeAssistant !== false,
memoryLimitNumber: cfg.memoryLimitNumber ?? 9,
preferenceLimitNumber: cfg.preferenceLimitNumber ?? 6,
includePreference: cfg.includePreference !== false,
includeToolMemory: cfg.includeToolMemory === true,
toolMemoryLimitNumber: cfg.toolMemoryLimitNumber ?? 6,
relativity: cfg.relativity ?? ((() => {
const v = loadEnvVar("MEMOS_RELATIVITY");
return v ? parseFloat(v) : 0.45;
})()),
filter: cfg.filter,
knowledgebaseIds: cfg.knowledgebaseIds ?? [],
tags: cfg.tags ?? ["openclaw"],
info: cfg.info ?? {},
agentId: cfg.agentId,
appId: cfg.appId,
allowPublic: cfg.allowPublic ?? false,
allowKnowledgebaseIds: cfg.allowKnowledgebaseIds ?? [],
asyncMode: cfg.asyncMode ?? true,
multiAgentMode,
recallFilterEnabled,
recallFilterBaseUrl:
(cfg.recallFilterBaseUrl ?? loadEnvVar("MEMOS_RECALL_FILTER_BASE_URL") ?? "").replace(/\/+$/, ""),
recallFilterApiKey: cfg.recallFilterApiKey ?? loadEnvVar("MEMOS_RECALL_FILTER_API_KEY") ?? "",
recallFilterModel: cfg.recallFilterModel ?? loadEnvVar("MEMOS_RECALL_FILTER_MODEL") ?? "",
recallFilterTimeoutMs: parseNumber(
cfg.recallFilterTimeoutMs ?? loadEnvVar("MEMOS_RECALL_FILTER_TIMEOUT_MS"),
6000,
),
recallFilterRetries: parseNumber(cfg.recallFilterRetries ?? loadEnvVar("MEMOS_RECALL_FILTER_RETRIES"), 0),
recallFilterCandidateLimit:
parseNumber(cfg.recallFilterCandidateLimit ?? loadEnvVar("MEMOS_RECALL_FILTER_CANDIDATE_LIMIT"), 30),
recallFilterMaxItemChars:
parseNumber(cfg.recallFilterMaxItemChars ?? loadEnvVar("MEMOS_RECALL_FILTER_MAX_ITEM_CHARS"), 500),
recallFilterFailOpen,
timeoutMs: cfg.timeoutMs ?? 5000,
retries: cfg.retries ?? 1,
throttleMs: cfg.throttleMs ?? 0,
};
}
export async function callApi({ baseUrl, apiKey, timeoutMs = 5000, retries = 1 }, path, body) {
if (!apiKey) {
throw new Error("Missing MEMOS API key (Token auth)");
}
const headers = {
"Content-Type": "application/json",
Authorization: `Token ${apiKey}`,
};
let lastError;
for (let attempt = 0; attempt <= retries; attempt += 1) {
try {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
const res = await fetch(`${baseUrl}${path}`, {
method: "POST",
headers,
body: JSON.stringify(body),
signal: controller.signal,
});
clearTimeout(timeoutId);
if (!res.ok) {
throw new Error(`HTTP ${res.status}`);
}
return await res.json();
} catch (err) {
lastError = err;
if (attempt < retries) {
await delay(100 * (attempt + 1));
}
}
}
throw lastError;
}
export async function searchMemory(cfg, payload) {
return callApi(cfg, "/search/memory", payload);
}
export async function addMessage(cfg, payload) {
return callApi(cfg, "/add/message", payload);
}
export function extractText(content) {
if (!content) return "";
if (typeof content === "string") return content;
if (Array.isArray(content)) {
return content
.filter((block) => block && typeof block === "object" && block.type === "text")
.map((block) => block.text)
.join(" ");
}
return "";
}
function normalizePreferenceType(value) {
if (!value) return "";
const normalized = String(value).trim().toLowerCase();
if (!normalized) return "";
if (normalized.includes("explicit")) return "Explicit Preference";
if (normalized.includes("implicit")) return "Implicit Preference";
return String(value)
.replace(/[_-]+/g, " ")
.replace(/\b\w/g, (ch) => ch.toUpperCase());
}
function sanitizeInlineText(text) {
if (text === undefined || text === null) return "";
return String(text).replace(/\r?\n+/g, " ").trim();
}
function formatMemoryLine(item, text, options = {}) {
const cleaned = sanitizeInlineText(text);
if (!cleaned) return "";
const maxChars = options.maxItemChars;
const truncated = truncate(cleaned, maxChars);
const time = formatTime(item?.create_time);
if (time) return ` -[${time}] ${truncated}`;
return ` - ${truncated}`;
}
function formatPreferenceLine(item, text, options = {}) {
const cleaned = sanitizeInlineText(text);
if (!cleaned) return "";
const maxChars = options.maxItemChars;
const truncated = truncate(cleaned, maxChars);
const time = formatTime(item?.create_time);
const type = normalizePreferenceType(item?.preference_type);
const typeLabel = type ? ` [${type}]` : "";
if (time) return ` -[${time}]${typeLabel} ${truncated}`;
return ` -${typeLabel} ${truncated}`;
}
function wrapCodeBlock(lines, options = {}) {
if (!options.wrapTagBlocks) return lines;
return ["```text", ...lines, "```"];
}
function buildMemorySections(data, options = {}) {
const memoryList = data?.memory_detail_list ?? [];
const preferenceList = data?.preference_detail_list ?? [];
const memoryLines = memoryList
.filter((item) => {
const score = item?.relativity ?? 1;
const threshold = options.relativity ?? 0;
return score > threshold;
})
.map((item) => {
const text = item?.memory_value || item?.memory_key || "";
return formatMemoryLine(item, text, options);
})
.filter(Boolean);
const preferenceLines = preferenceList
.filter((item) => {
const score = item?.relativity ?? 1;
const threshold = options.relativity ?? 0;
return score > threshold;
})
.map((item) => {
const text = item?.preference || "";
return formatPreferenceLine(item, text, options);
})
.filter(Boolean);
return { memoryLines, preferenceLines };
}
const STATIC_RECALL_SYSTEM_PROMPT = [
"# Role",
"",
"You are an intelligent assistant with long-term memory capabilities (MemOS Assistant). Your goal is to combine retrieved memory fragments to provide highly personalized, accurate, and logically rigorous responses.",
"",
"# System Context",
"",
"* Current Time: Use the runtime-provided current time as the baseline for freshness checks.",
"* Additional memory context for the current turn may be prepended before the original user query as a structured `` block.",
"",
"# Memory Data",
"",
'Below is the information retrieved by MemOS, categorized into "Facts" and "Preferences".',
"* **Facts**: May include user attributes, historical conversations, or third-party details.",
"* **Special Note**: Content tagged with '[assistant观点]' or '[模型总结]' represents **past AI inference**, **not** direct user statements.",
"* **Preferences**: The user's explicit or implicit requirements on response style, format, or reasoning.",
"",
"# Critical Protocol: Memory Safety",
"",
"Retrieved memories may contain **AI speculation**, **irrelevant noise**, or **wrong subject attribution**. You must strictly apply the **Four-Step Verdict**. If any step fails, **discard the memory**:",
"",
"1. **Source Verification**:",
"* **Core**: Distinguish direct user statements from AI inference.",
"* If a memory has tags like '[assistant观点]' or '[模型总结]', treat it as a **hypothesis**, not a user-grounded fact.",
"* *Counterexample*: If memory says '[assistant观点] User loves mangoes' but the user never said that, do not assume it as fact.",
"* **Principle: AI summaries are reference-only and have much lower authority than direct user statements.**",
"",
"2. **Attribution Check**:",
"* Is the subject in memory definitely the user?",
"* If the memory describes a **third party** (e.g., candidate, interviewee, fictional character, case data), never attribute it to the user.",
"",
"3. **Strong Relevance Check**:",
"* Does the memory directly help answer the current 'Original Query'?",
"* If it is only a keyword overlap with different context, ignore it.",
"",
"4. **Freshness Check**:",
"* If memory conflicts with the user's latest intent, prioritize the current 'Original Query' as the highest source of truth.",
"",
"# Instructions",
"",
"1. **Review**: Read '' first and apply the Four-Step Verdict to remove noise and unreliable AI inference.",
"2. **Execute**:",
" - Use only memories that pass filtering as context.",
" - Strictly follow style requirements from ''.",
"3. **Output**: Answer directly. Never mention internal terms such as \"memory store\", \"retrieval\", or \"AI opinions\".",
"4. **Attention**: Additional memory context may already be provided before the original user query. Do not read from or write to local `MEMORY.md` or `memory/*` files for reference, as they may be outdated or irrelevant to the current query.",
].join("\n");
function buildMemoryPrependBlock(data, options = {}) {
const { memoryLines, preferenceLines } = buildMemorySections(data, options);
const hasContent = memoryLines.length > 0 || preferenceLines.length > 0;
if (!hasContent) return "";
const memoriesBlock = [
"",
" ",
...memoryLines,
" ",
" ",
...preferenceLines,
" ",
"",
];
return [...wrapCodeBlock(memoriesBlock, options), "", USER_QUERY_MARKER].join("\n");
}
export function formatPromptBlockFromData(data, options = {}) {
if (!data || typeof data !== "object") return "";
return buildMemoryPrependBlock(data, options);
}
export function formatPromptBlock(result, options = {}) {
const data = extractResultData(result);
return formatPromptBlockFromData(data, options);
}
export function formatContextBlock(result, options = {}) {
const data = extractResultData(result);
if (!data) return "";
const memoryList = data.memory_detail_list ?? [];
const prefList = data.preference_detail_list ?? [];
const toolList = data.tool_memory_detail_list ?? [];
const preferenceNote = data.preference_note;
const lines = [];
if (memoryList.length > 0) {
lines.push("Facts:");
for (const item of memoryList) {
const text = item?.memory_value || item?.memory_key || "";
if (!text) continue;
lines.push(`- ${truncate(text, options.maxItemChars)}`);
}
}
if (prefList.length > 0) {
lines.push("Preferences:");
for (const item of prefList) {
const pref = item?.preference || "";
const type = item?.preference_type ? `(${item.preference_type}) ` : "";
if (!pref) continue;
lines.push(`- ${type}${truncate(pref, options.maxItemChars)}`);
}
}
if (toolList.length > 0) {
lines.push("Tool Memories:");
for (const item of toolList) {
const value = item?.tool_value || "";
if (!value) continue;
lines.push(`- ${truncate(value, options.maxItemChars)}`);
}
}
if (preferenceNote) {
lines.push(`Preference Note: ${truncate(preferenceNote, options.maxItemChars)}`);
}
return lines.length > 0 ? lines.join("\n") : "";
}
export function formatRecallHookResult(result, options = {}) {
const data = extractResultData(result);
if (!data) {
return {
appendSystemContext: "",
prependContext: "",
};
}
return {
// Keep this system addendum byte-stable across turns so provider-side prefix caching can hit.
appendSystemContext: STATIC_RECALL_SYSTEM_PROMPT,
prependContext: buildMemoryPrependBlock(data, options),
};
}
function truncate(text, maxLen) {
if (!text) return "";
const limit = maxLen || 10000;
return text.length > limit ? `${text.slice(0, limit)}...` : text;
}
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/moltbot.plugin.json
================================================
{
"id": "memos-cloud-openclaw-plugin",
"name": "MemOS Cloud OpenClaw Plugin",
"description": "MemOS Cloud recall + add memory via lifecycle hooks",
"version": "0.1.9",
"kind": "lifecycle",
"main": "./index.js",
"configSchema": {
"type": "object",
"properties": {
"baseUrl": {
"type": "string",
"description": "MemOS Cloud base URL"
},
"apiKey": {
"type": "string",
"description": "MemOS API Key (Token auth; supports ~/.openclaw/.env, ~/.moltbot/.env, ~/.clawdbot/.env; falls back to process env)"
},
"userId": {
"type": "string",
"description": "MemOS user_id (default: openclaw-user)",
"default": "openclaw-user"
},
"conversationId": {
"type": "string",
"description": "Override conversation_id"
},
"conversationIdPrefix": {
"type": "string",
"description": "conversation_id prefix"
},
"conversationIdSuffix": {
"type": "string",
"description": "conversation_id suffix"
},
"conversationSuffixMode": {
"type": "string",
"enum": [
"none",
"counter"
],
"default": "none"
},
"resetOnNew": {
"type": "boolean",
"default": true
},
"queryPrefix": {
"type": "string",
"description": "Prefix added to search queries"
},
"maxQueryChars": {
"type": "integer",
"description": "Max chars for search query"
},
"recallEnabled": {
"type": "boolean",
"default": true
},
"recallGlobal": {
"type": "boolean",
"default": true
},
"addEnabled": {
"type": "boolean",
"default": true
},
"captureStrategy": {
"type": "string",
"enum": [
"last_turn",
"full_session"
],
"default": "last_turn"
},
"maxMessageChars": {
"type": "integer",
"description": "Max chars per message when adding",
"default": 20000
},
"maxItemChars": {
"type": "integer",
"description": "Max chars per memory item when injecting prompt",
"default": 8000
},
"includeAssistant": {
"type": "boolean",
"default": true
},
"memoryLimitNumber": {
"type": "integer",
"default": 6
},
"preferenceLimitNumber": {
"type": "integer",
"default": 6
},
"includePreference": {
"type": "boolean",
"default": true
},
"includeToolMemory": {
"type": "boolean",
"default": false
},
"toolMemoryLimitNumber": {
"type": "integer",
"default": 6
},
"filter": {
"type": "object",
"description": "MemOS search filter"
},
"knowledgebaseIds": {
"type": "array",
"items": {
"type": "string"
}
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
},
"info": {
"type": "object",
"additionalProperties": true
},
"agentId": {
"type": "string"
},
"multiAgentMode": {
"type": "boolean",
"default": false
},
"appId": {
"type": "string"
},
"allowPublic": {
"type": "boolean",
"default": false
},
"allowKnowledgebaseIds": {
"type": "array",
"items": {
"type": "string"
}
},
"asyncMode": {
"type": "boolean",
"default": true
},
"timeoutMs": {
"type": "integer",
"default": 5000
},
"retries": {
"type": "integer",
"default": 1
},
"throttleMs": {
"type": "integer",
"default": 0
}
},
"additionalProperties": false
}
}
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/openclaw.plugin.json
================================================
{
"id": "memos-cloud-openclaw-plugin",
"name": "MemOS Cloud OpenClaw Plugin",
"description": "MemOS Cloud recall + add memory via lifecycle hooks",
"version": "0.1.9",
"kind": "lifecycle",
"main": "./index.js",
"configSchema": {
"type": "object",
"properties": {
"baseUrl": {
"type": "string",
"description": "MemOS Cloud base URL"
},
"apiKey": {
"type": "string",
"description": "MemOS API Key (Token auth; supports ~/.openclaw/.env, ~/.moltbot/.env, ~/.clawdbot/.env; falls back to process env)"
},
"userId": {
"type": "string",
"description": "MemOS user_id (default: openclaw-user)",
"default": "openclaw-user"
},
"conversationId": {
"type": "string",
"description": "Override conversation_id"
},
"conversationIdPrefix": {
"type": "string",
"description": "conversation_id prefix"
},
"conversationIdSuffix": {
"type": "string",
"description": "conversation_id suffix"
},
"conversationSuffixMode": {
"type": "string",
"enum": [
"none",
"counter"
],
"default": "none"
},
"resetOnNew": {
"type": "boolean",
"default": true
},
"queryPrefix": {
"type": "string",
"description": "Prefix added to search queries"
},
"maxQueryChars": {
"type": "integer",
"description": "Max chars for search query"
},
"recallEnabled": {
"type": "boolean",
"default": true
},
"recallGlobal": {
"type": "boolean",
"default": true
},
"addEnabled": {
"type": "boolean",
"default": true
},
"captureStrategy": {
"type": "string",
"enum": [
"last_turn",
"full_session"
],
"default": "last_turn"
},
"maxMessageChars": {
"type": "integer",
"description": "Max chars per message when adding",
"default": 20000
},
"maxItemChars": {
"type": "integer",
"description": "Max chars per memory item when injecting prompt",
"default": 8000
},
"includeAssistant": {
"type": "boolean",
"default": true
},
"memoryLimitNumber": {
"type": "integer",
"default": 6
},
"preferenceLimitNumber": {
"type": "integer",
"default": 6
},
"includePreference": {
"type": "boolean",
"default": true
},
"includeToolMemory": {
"type": "boolean",
"default": false
},
"toolMemoryLimitNumber": {
"type": "integer",
"default": 6
},
"filter": {
"type": "object",
"description": "MemOS search filter"
},
"knowledgebaseIds": {
"type": "array",
"items": {
"type": "string"
}
},
"tags": {
"type": "array",
"items": {
"type": "string"
}
},
"info": {
"type": "object",
"additionalProperties": true
},
"agentId": {
"type": "string"
},
"multiAgentMode": {
"type": "boolean",
"default": false
},
"appId": {
"type": "string"
},
"allowPublic": {
"type": "boolean",
"default": false
},
"allowKnowledgebaseIds": {
"type": "array",
"items": {
"type": "string"
}
},
"asyncMode": {
"type": "boolean",
"default": true
},
"timeoutMs": {
"type": "integer",
"default": 5000
},
"retries": {
"type": "integer",
"default": 1
},
"throttleMs": {
"type": "integer",
"default": 0
}
},
"additionalProperties": false
}
}
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/package.json
================================================
{
"name": "@memtensor/memos-cloud-openclaw-plugin",
"version": "0.1.9",
"description": "OpenClaw lifecycle plugin for MemOS Cloud (add + recall memory)",
"scripts": {
"sync-version": "node scripts/sync-version.js",
"version": "npm run sync-version && git add openclaw.plugin.json moltbot.plugin.json clawdbot.plugin.json",
"publish-beta": "npm publish --tag beta",
"publish-beta-patch": "npm version prepatch --preid=beta && npm publish --tag beta",
"publish-latest": "npm version $(node -p \"require('./package.json').version.split('-')[0]\") && npm publish",
"publish-latest-patch": "npm version patch && npm publish"
},
"keywords": [
"memos",
"memos-cloud",
"openclaw",
"plugin",
"memory"
],
"homepage": "https://github.com/MemTensor/MemOS-Cloud-OpenClaw-Plugin#readme",
"bugs": {
"url": "https://github.com/MemTensor/MemOS-Cloud-OpenClaw-Plugin/issues"
},
"repository": {
"type": "git",
"url": "git+https://github.com/MemTensor/MemOS-Cloud-OpenClaw-Plugin.git"
},
"type": "module",
"author": "MemTensor",
"license": "MIT",
"openclaw": {
"extensions": [
"./index.js"
]
},
"clawdbot": {
"extensions": [
"./index.js"
]
},
"moltbot": {
"extensions": [
"./index.js"
]
}
}
================================================
FILE: apps/MemOS-Cloud-OpenClaw-Plugin/scripts/sync-version.js
================================================
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
// Read the updated package.json to get the new version
const packageJsonPath = path.resolve(__dirname, '../package.json');
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
const newVersion = packageJson.version;
console.log(`Syncing version to ${newVersion}...`);
const filesToUpdate = [
'openclaw.plugin.json',
'moltbot.plugin.json',
'clawdbot.plugin.json'
];
filesToUpdate.forEach(fileName => {
const filePath = path.resolve(__dirname, '..', fileName);
if (fs.existsSync(filePath)) {
try {
const content = JSON.parse(fs.readFileSync(filePath, 'utf8'));
if (content.version !== newVersion) {
content.version = newVersion;
// Write back with 2 spaces indentation and a newline at the end
fs.writeFileSync(filePath, JSON.stringify(content, null, 2) + '\n', 'utf8');
console.log(`Updated ${fileName} to version ${newVersion}`);
} else {
console.log(`${fileName} is already at version ${newVersion}`);
}
} catch (error) {
console.error(`Error updating ${fileName}:`, error.message);
process.exit(1);
}
} else {
console.warn(`Warning: ${fileName} not found, skipping.`);
}
});
console.log('Version sync complete.');
================================================
FILE: apps/memos-local-openclaw/.gitignore
================================================
node_modules/
dist/
*.tsbuildinfo
.env
# OS files
.DS_Store
Thumbs.db
# IDE
.vscode/
.idea/
# Generated / non-essential
package-lock.json
.installed-version
ppt/
# Prebuilt native binaries (included in npm package via `files`, not in git)
prebuilds/
# Database files
*.sqlite
*.sqlite-journal
*.db
================================================
FILE: apps/memos-local-openclaw/README.md
================================================
# 🧠 MemOS — OpenClaw Memory Plugin
[](https://www.npmjs.com/package/@memtensor/memos-local-openclaw-plugin)
[](https://github.com/MemTensor/MemOS/blob/main/LICENSE)
[](https://nodejs.org/)
[](https://github.com/MemTensor/MemOS/tree/main/apps/memos-local-openclaw)
Persistent local conversation memory for [OpenClaw](https://github.com/nicepkg/openclaw) AI Agents. Every conversation is automatically captured, semantically indexed, and instantly recallable — with **task summarization & skill evolution**, and **multi-agent collaborative memory**.
**Full-write | Hybrid Search | Task Summarization & Skill Evolution | Multi-Agent Collaboration | Memory Viewer**
> **Homepage:** 🌐 [Homepage](https://memos-claw.openmem.net) · 📖 [Documentation](https://memos-claw.openmem.net/docs/index.html) · 📦 [NPM](https://www.npmjs.com/package/@memtensor/memos-local-openclaw-plugin)
## Why MemOS
| Problem | Solution |
|---------|----------|
| Agent forgets everything between sessions | **Persistent memory** — every conversation auto-captured to local SQLite |
| Fragmented context, repeated mistakes | **Task summarization & skill evolution** — conversations organized into structured tasks, then distilled into reusable skills that auto-upgrade |
| Multi-agent teams work in isolation | **Multi-agent collaboration** — memory isolation + public memory + skill sharing enables collective evolution |
| No visibility into what the agent remembers | **Memory Viewer** — full visualization of all memories, tasks, and skills |
| Privacy concerns with cloud storage | **100% local** — zero cloud uploads, anonymous opt-out telemetry only, password-protected |
## Features
### Memory Engine
- **Auto-capture** — Stores user, assistant, and tool messages after each agent turn via `agent_end` event (consecutive assistant messages merged into one)
- **Smart deduplication** — Exact content-hash skip; then Top-5 similar chunks (threshold 0.75) with LLM judge: DUPLICATE (skip), UPDATE (merge summary + append content), or NEW (create). Evolved chunks track merge history.
- **Semantic chunking** — Splits by code blocks, function bodies, paragraphs; never cuts mid-function
- **Hybrid retrieval** — FTS5 keyword + vector semantic dual-channel search with RRF fusion
- **MMR diversity** — Maximal Marginal Relevance reranking prevents near-duplicate results
- **Recency decay** — Configurable time-based decay (half-life: 14 days) biases recent memories
- **Multi-provider embedding** — OpenAI-compatible, Gemini, Cohere, Voyage, Mistral, or local offline (Xenova/all-MiniLM-L6-v2)
### Task Summarization & Skill Evolution
- **Auto task boundary detection** — Per-turn LLM topic judgment (warm-up: 1 user turn) + 2-hour idle timeout segments conversations into tasks. Strongly biased toward SAME to avoid over-splitting related topics
- **Structured summaries** — LLM generates Goal, Key Steps, Result, Key Details for each completed task
- **Key detail preservation** — Code, commands, URLs, file paths, error messages retained in summaries
- **Quality filtering** — Tasks with too few chunks, too few turns, or trivial content are auto-skipped
- **Task status** — `active` (in progress), `completed` (with LLM summary), `skipped` (too brief, excluded from search)
- **Task/Skill CRUD** — Edit title/summary, delete tasks and skills, retry skill generation from task cards
- **Automatic evaluation** — After task completion, rule filter + LLM evaluates if the task is worth distilling into a skill
- **Skill generation** — Multi-step LLM pipeline creates SKILL.md + scripts + references + evals from real execution records
- **Skill upgrading** — When similar tasks appear, existing skills are auto-upgraded (refine / extend / fix)
- **Quality scoring** — 0-10 quality assessment; scores below 6 marked as draft
- **Version management** — Full version history with changelog, change summary, and upgrade type tracking
- **Auto-install** — Generated skills can be auto-installed into the workspace for immediate use
- **Dedicated model** — Optional separate LLM model for skill generation (e.g., Claude 4.6 for higher quality)
- **LLM fallback chain** — `skillSummarizer` → `summarizer` → OpenClaw native model (auto-detected from `openclaw.json`). If all configured models fail, the next in chain is tried automatically
### Multi-Agent Collaboration
- **Memory isolation** — Each agent's memories are tagged with `owner`. During search, agents only see their own private memories and explicitly shared `public` memories
- **Public memory** — `memory_write_public` tool allows agents to write shared knowledge accessible to all agents (e.g., team decisions, conventions, shared configs)
- **Skill sharing** — Skills have a `visibility` toggle (`private`/`public`). Public skills are discoverable by all agents via `skill_search`
- **Skill discovery** — `skill_search` combines FTS (name + description) and vector search (description embedding) with RRF fusion, followed by LLM relevance judgment. Supports `scope` parameter: `mix` (default), `self`, or `public`
- **Publish/unpublish** — `skill_publish` / `skill_unpublish` tools toggle skill visibility. Other agents can search, preview, and install public skills
- **Agent-aware capture** — `agent_end` event extracts `agentId` to tag all captured messages with the correct owner
### Memory Migration — Reconnect 🦐
- **One-click import** — Seamlessly migrate OpenClaw's native built-in memories (SQLite + JSONL) into the MemOS intelligent memory system
- **Smart deduplication** — Vector similarity + LLM judgment prevents duplicate imports; similar content auto-merged
- **Resume anytime** — Pause and resume at any time; refreshing the page auto-restores progress; already processed items are skipped
- **Post-import processing** — Optionally generate task summaries and evolve skills from imported memories; serial processing within each agent, parallel across agents
- **Agent parallelism** — Configurable concurrency (1–8) for parallel processing across agents; sessions within each agent are processed serially
- **Source tagging** — All migrated memories are tagged with 🦐, visually distinguishing them from conversation-generated memories
- **Real-time progress** — Live progress bar, stats (stored/skipped/merged/errors), and scrolling log via SSE
### Memory Viewer
- **7 management pages** — Memories, Tasks, Skills, Analytics, **Logs**, **Import**, Settings
- **Full CRUD** — Create, edit, delete, search memories; evolution badges and merge history on memory cards
- **Task browser** — Status filters, chat-bubble chunk view, structured summaries, skill generation status; edit/delete/retry-skill buttons on cards
- **Skill browser** — Version history, quality scores, visibility toggle, one-click download as ZIP; edit/delete/publish buttons on cards
- **Analytics dashboard** — Daily read/write activity, memory breakdown charts
- **Logs** — Tool call log (memory_search, auto_recall, memory_add, etc.) with input/output and duration; filter by tool, auto-refresh
- **Online configuration** — Modify embedding, summarizer, skill evolution settings via web UI
- **Security** — Password-protected, localhost-only (127.0.0.1), session cookies
- **i18n** — Chinese / English toggle
- **Themes** — Light / Dark mode
### Privacy & Security
- **100% on-device** — All data in local SQLite, no cloud uploads
- **Anonymous telemetry** — Enabled by default, opt-out via config. Only sends tool names, latencies, and version info. Never sends memory content, queries, or personal data. See [Telemetry](#telemetry) section.
- **Viewer security** — Binds to 127.0.0.1 only, password-protected with session cookies
- **Auto-recall + Skill** — Each turn, relevant memories are injected via `before_agent_start` hook (invisible to user). When nothing is recalled (e.g. long or unclear query), the agent is prompted to call `memory_search` with a self-generated short query. The bundled skill `memos-memory-guide` documents all tools and when to use them.
## Quick Start
### 1. Install
**Step 0 — Prepare build environment (macOS / Linux):**
This plugin uses `better-sqlite3`, a native C/C++ module. On **macOS** and **Linux**, prebuilt binaries may not be available, so **install C++ build tools first** to ensure a smooth installation:
```bash
# macOS
xcode-select --install
# Linux (Ubuntu / Debian)
sudo apt install build-essential python3
```
> **Windows users:** `better-sqlite3` ships prebuilt binaries for Windows + Node.js LTS, so you can usually skip this step and go directly to Step 1. If installation still fails, install [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/) (select "C++ build tools" workload).
>
> Already have build tools? Skip to Step 1. Not sure? Run the install command above — it's safe to re-run.
>
> **Still having issues?** See the [Troubleshooting](#troubleshooting) section, the [detailed troubleshooting guide](https://memtensor.github.io/MemOS/apps/memos-local-openclaw/docs/troubleshooting.html), or the [official better-sqlite3 troubleshooting docs](https://github.com/WiseLibs/better-sqlite3/blob/master/docs/troubleshooting.md).
**Step 1 — Install the plugin:**
```bash
openclaw plugins install @memtensor/memos-local-openclaw-plugin
```
The plugin is installed under `~/.openclaw/extensions/memos-local-openclaw-plugin` and registered as `memos-local-openclaw-plugin`. Dependencies and `better-sqlite3` native module are built automatically during installation.
> **Note:** The Memory Viewer starts only when the **OpenClaw gateway** is running. After install, **configure** `openclaw.json` (step 2) and **start the gateway** (step 3); the viewer will then be available at `http://127.0.0.1:18799`.
>
> **Installation failed?** If `better-sqlite3` compilation fails during install, manually rebuild after ensuring build tools are installed:
> ```bash
> cd ~/.openclaw/extensions/memos-local-openclaw-plugin && npm rebuild better-sqlite3
> ```
**From source (development):**
```bash
git clone https://github.com/MemTensor/MemOS.git
cd MemOS/apps/memos-local-openclaw
npm install && npm run build
openclaw plugins install .
```
### 2. Configure
Add the plugin config to `~/.openclaw/openclaw.json`:
```jsonc
{
"agents": {
"defaults": {
// IMPORTANT: Disable OpenClaw's built-in memory to avoid conflicts
"memorySearch": {
"enabled": false
}
}
},
"plugins": {
"slots": {
"memory": "memos-local-openclaw-plugin"
},
"entries": {
"memos-local-openclaw-plugin": {
"enabled": true,
"config": {
"embedding": {
"provider": "openai_compatible",
"endpoint": "https://your-api-endpoint/v1",
"apiKey": "sk-••••••",
"model": "bge-m3"
},
"summarizer": {
"provider": "openai_compatible",
"endpoint": "https://your-api-endpoint/v1",
"apiKey": "sk-••••••",
"model": "gpt-4o-mini",
"temperature": 0
}
}
}
}
}
}
```
> **Critical:** You must set `agents.defaults.memorySearch.enabled` to `false`. Otherwise OpenClaw's built-in memory search runs alongside this plugin, causing duplicate retrieval and wasted tokens.
#### Embedding Provider Options
| Provider | `provider` value | Example `model` | Notes |
|---|---|---|---|
| OpenAI / compatible | `openai_compatible` | `bge-m3`, `text-embedding-3-small` | Any OpenAI-compatible API |
| Gemini | `gemini` | `text-embedding-004` | Requires `apiKey` |
| Cohere | `cohere` | `embed-english-v3.0` | Separates document/query embedding |
| Voyage | `voyage` | `voyage-2` | |
| Mistral | `mistral` | `mistral-embed` | |
| Local (offline) | `local` | — | Uses `Xenova/all-MiniLM-L6-v2`, no API needed |
> **No embedding config?** The plugin falls back to the local model automatically. You can start with zero configuration and add a cloud provider later for better quality.
#### Summarizer Provider Options
| Provider | `provider` value | Example `model` |
|---|---|---|
| OpenAI / compatible | `openai_compatible` | `gpt-4o-mini` |
| Anthropic | `anthropic` | `claude-3-haiku-20240307` |
| Gemini | `gemini` | `gemini-1.5-flash` |
| AWS Bedrock | `bedrock` | `anthropic.claude-3-haiku-20240307-v1:0` |
> **No summarizer config?** The plugin automatically falls back to the OpenClaw native model (auto-detected from `~/.openclaw/openclaw.json`). If that is also unavailable, a rule-based fallback generates summaries from the first sentence + key entities. Good enough to start.
#### Skill Evolution Configuration (Optional)
You can optionally configure a dedicated model for skill generation (for higher quality skills):
```jsonc
{
"config": {
"skillSummarizer": {
"provider": "anthropic",
"apiKey": "sk-ant-xxx",
"model": "claude-sonnet-4-20250514",
"temperature": 0
},
"skillEvolution": {
"enabled": true,
"autoEvaluate": true,
"autoInstall": false
}
}
}
```
**LLM fallback chain:** `skillSummarizer` → `summarizer` → OpenClaw native model (auto-detected from `~/.openclaw/openclaw.json`). If `skillSummarizer` is not configured, the plugin tries the regular `summarizer`, then falls back to the OpenClaw native model. Each step in the chain is tried automatically if the previous one fails.
#### Environment Variable Support
Use `${ENV_VAR}` placeholders in config to avoid hardcoding keys:
```jsonc
{
"apiKey": "${OPENAI_API_KEY}"
}
```
### 3. Start or Restart the Gateway
```bash
openclaw gateway stop # if already running
openclaw gateway install # ensure LaunchAgent is installed (macOS)
openclaw gateway start
```
Once the gateway is up, the plugin loads and starts the Memory Viewer at `http://127.0.0.1:18799`.
### 4. Verify Installation
```bash
tail -20 ~/.openclaw/logs/gateway.log
```
You should see:
```
memos-local: initialized (db: ~/.openclaw/memos-local/memos.db)
memos-local: started (embedding: openai_compatible)
╔══════════════════════════════════════════╗
║ MemOS Memory Viewer ║
║ → http://127.0.0.1:18799 ║
║ Open in browser to manage memories ║
╚══════════════════════════════════════════╝
```
### 5. Verify Memory is Working
**Step A** — Have a conversation with your OpenClaw agent about anything.
**Step B** — Open the Memory Viewer at `http://127.0.0.1:18799` and check that the conversation appears.
**Step C** — In a new conversation, ask the agent to recall what you discussed:
```
You: 你还记得我之前让你帮我处理过什么事情吗?
Agent: (calls memory_search) 是的,我们之前讨论过...
```
## How It Works
### Three Intelligent Pipelines
MemOS Lite operates through three interconnected pipelines that form a continuous learning loop:
```
Conversation → Memory Write Pipeline → Task Generation Pipeline → Skill Evolution Pipeline
↓
Smart Retrieval Pipeline ← ← ← ← ← ← ← ← ←
```
### Pipeline 1: Memory Write (auto on every agent turn)
```
Conversation → Capture (filter roles, strip system prompts)
→ Semantic chunking (code blocks, paragraphs, error stacks)
→ Content hash dedup → LLM summarize each chunk
→ Vector embedding → Store (SQLite + FTS5 + Vector)
```
- System messages are skipped; tool results from the plugin's own tools are not re-stored
- Evidence wrapper blocks (`[STORED_MEMORY]...[/STORED_MEMORY]`) are stripped to prevent feedback loops
- Content hash (SHA-256, first 16 hex chars) prevents duplicate chunk ingestion within the same session+role
### Pipeline 2: Task Generation (auto after memory write)
```
New chunks → Group into user-turns → Process one turn at a time
→ Warm-up (first user turn): assign directly
→ Each subsequent user turn: LLM topic judge (context vs new message)
→ "NEW"? → Finalize current task, create new task
→ "SAME"? → Assign to current task
→ Time gap > 2h? → Always split regardless of topic
→ Finalize: Chunks ≥ 4 & turns ≥ 2? → LLM structured summary → status = "completed"
→ Otherwise → status = "skipped" (excluded from search)
```
**Why Tasks matter:**
- Raw memory chunks are fragmented — a single conversation about "deploying Nginx" might span 20 chunks
- Task summarization organizes these fragments into a structured record: Goal → Steps → Result → Key Details
- When the agent searches memory, it can quickly locate the complete experience via `task_summary`, not just fragments
- Task summaries preserve code, commands, URLs, configs, and error messages
### Pipeline 3: Skill Evolution (auto after task completion)
```
Completed task → Rule filter (min chunks, non-trivial content)
→ Search for related existing skills
→ Related skill found (confidence ≥ 0.7)?
→ Evaluate upgrade (refine/extend/fix) → Merge new experience → Version bump
→ No related skill (or confidence < 0.3)?
→ Evaluate create → Generate SKILL.md + scripts + evals
→ Quality score (0-10) → Install if score ≥ 6
```
**Why Skills matter:**
- Without skills, agents rediscover solutions every time they encounter similar problems
- Skills crystallize successful executions into reusable guides with steps, pitfall warnings, and verification checks
- Skills auto-upgrade when new tasks bring improved approaches — getting faster, more accurate, and more token-efficient
- The evolution is automatic: task completes → evaluate → create/upgrade → install
### Pipeline 4: Smart Retrieval
**Auto-recall (every turn):** The plugin hooks `before_agent_start`, runs a memory search with the user's message, then uses an LLM to filter which candidates are relevant and whether they are sufficient to answer. The filtered memories are injected into the agent's system context (invisible to the user). If no memories are found or the query is long/unclear, the agent is prompted to call `memory_search` with a self-generated short query.
**On-demand search (`memory_search`):**
```
Query → FTS5 + Vector dual recall → RRF Fusion → MMR Rerank
→ Recency Decay → Score Filter → Top-K (e.g. 20)
→ LLM relevance filter (minimum information) → Dedup by excerpt overlap
→ Return excerpts + chunkId / task_id (no summaries)
→ sufficient=false → suggest task_summary(taskId), skill_get(taskId), memory_timeline(chunkId)
```
- **RRF (Reciprocal Rank Fusion):** Merges FTS5 and vector search rankings into a unified score
- **MMR (Maximal Marginal Relevance):** Re-ranks to balance relevance with diversity
- **Recency Decay:** Recent memories get a boost (half-life: 14 days by default)
- **LLM filter:** Only memories that are genuinely useful for the query are returned; sufficiency determines whether follow-up tool tips are appended
## Retrieval Strategy
1. **Auto-recall (hook)** — On every turn, the plugin runs a memory search using the user's message and injects LLM-filtered relevant memories into the agent's context (via `before_agent_start`). The agent sees this as system context; the user does not.
2. **When nothing is recalled** — If the user's message is long, vague, or no matches are found, the plugin injects a short hint telling the agent to call **`memory_search`** with a **self-generated short query** (e.g. key topics or a rephrased question).
3. **Bundled skill** — The plugin installs `memos-memory-guide` into `~/.openclaw/workspace/skills/memos-memory-guide/` and `~/.openclaw/skills/memos-memory-guide/`. This skill documents all memory tools, when to call them, and how to write good search queries. Add `skills.load.extraDirs: ["~/.openclaw/skills"]` in `openclaw.json` if you want the skill to appear in the OpenClaw skills dashboard.
4. **Search results** — `memory_search` returns **excerpts** (original content snippets) and IDs (`chunkId`, `task_id`), not summaries. The agent uses `memory_get(chunkId)` for full original text, `task_summary(taskId)` for structured task context, `memory_timeline(chunkId)` for surrounding conversation, and `skill_get(skillId|taskId)` for reusable experience guides.
## Agent Tools
The plugin provides **12 smart tools** (11 registered tools + auto-recall) and auto-installs the **memos-memory-guide** skill:
| Tool | Purpose | When to Use |
|------|---------|-------------|
| `auto_recall` | Automatically injects relevant memories into agent context each turn (via `before_agent_start` hook) | Runs automatically — no manual call needed |
| `memory_search` | Search memories (auto-filtered to current agent + public); returns excerpts + `chunkId` / `task_id` | When auto-recall returned nothing or you need a different query |
| `memory_get` | Get full original text of a memory chunk | When you need to verify exact details from a search hit |
| `memory_timeline` | Surrounding conversation around a chunk | When you need the exact dialogue before/after a hit |
| `memory_write_public` | Write a memory to the shared public space (owner="public") | When the agent discovers knowledge all agents should access |
| `task_summary` | Full structured summary of a completed task | When a hit has `task_id` and you need the full story (goal, steps, result) |
| `skill_get` | Get skill content by `skillId` or `taskId` | When a hit has a linked task/skill and you want the reusable experience guide |
| `skill_install` | Install a skill into the agent workspace | When the skill should be permanently available for future turns |
| `skill_search` | Search skills via FTS + vector + LLM relevance; scope: `mix` / `self` / `public` | When an agent needs to discover existing skills for a task |
| `skill_publish` | Set a skill's visibility to public | When a skill should be discoverable by other agents |
| `skill_unpublish` | Set a skill's visibility back to private | When a skill should no longer be shared |
| `memory_viewer` | Get the URL of the Memory Viewer web UI | When the user asks where to view or manage their memories |
### Search Parameters
| Parameter | Default | Range | Description |
|-----------|---------|-------|-------------|
| `query` | — | — | Natural language search query (keep it short and focused) |
| `maxResults` | 20 | 1–20 | Maximum candidates before LLM filter |
| `minScore` | 0.45 | 0.35–1.0 | Minimum relevance score |
| `role` | — | `user` / `assistant` / `tool` | Filter by message role (e.g. `user` to find what the user said) |
> **Viewer search** uses a stricter threshold (`minScore` 0.64) for vector results. When no semantic matches are found, it falls back to FTS5 keyword search and returns the top 20 keyword-based results.
## Memory Viewer
Open `http://127.0.0.1:18799` in your browser after starting the gateway.
**Pages:**
| Page | Features |
|------|----------|
| **Memories** | Timeline view, pagination, session/role/kind/date filters, CRUD, semantic search; evolution badges and merge history on cards |
| **Tasks** | Task list with status filters (active/completed/skipped), chat-bubble chunk view, structured summaries, skill generation status |
| **Skills** | Skill list with status badges, version history with changelogs, quality scores, related tasks, one-click ZIP download |
| **Analytics** | Daily write/read activity charts, memory/task/skill totals, role breakdown |
| **Logs** | Tool call log (memory_search, auto_recall, memory_add, etc.) with input/output, duration, and tool filter; auto-refresh |
| **Import** | 🦐 OpenClaw native memory migration — scan, one-click import with real-time SSE progress, smart dedup, pause/resume; post-processing for task & skill generation |
| **Settings** | Online configuration for embedding model, summarizer model, skill evolution settings, viewer port |
**Viewer won't open?**
- The viewer is started by the plugin when the **gateway** starts. It does **not** run at install time.
- Ensure the gateway is running: `openclaw gateway start`
- Ensure the plugin is enabled in `~/.openclaw/openclaw.json`
- Check the log: `tail -30 ~/.openclaw/logs/gateway.log` — look for `MemOS Memory Viewer`
**Forgot password?** Click "Forgot password?" on the login page and use the reset token:
```bash
grep "password reset token:" ~/.openclaw/logs/gateway.log 2>/dev/null | tail -1
```
Copy the 32-character hex string after `password reset token:`.
## Advanced Configuration
All optional — shown with defaults:
```jsonc
{
"config": {
"recall": {
"maxResultsDefault": 6, // Default search results
"maxResultsMax": 20, // Max search results
"minScoreDefault": 0.45, // Default min score threshold
"minScoreFloor": 0.35, // Lowest allowed min score
"rrfK": 60, // RRF fusion constant
"mmrLambda": 0.7, // MMR relevance vs diversity (0-1)
"recencyHalfLifeDays": 14, // Time decay half-life
"vectorSearchMaxChunks": 0 // 0 = search all (default). Set 200000–300000 only if search is slow on huge DBs
},
"dedup": {
"similarityThreshold": 0.75, // Cosine similarity for smart-dedup candidates (Top-5)
"enableSmartMerge": true, // LLM judge: DUPLICATE / UPDATE / NEW
"maxCandidates": 5 // Max similar chunks to send to LLM
},
"skillEvolution": {
"enabled": true, // Enable skill evolution
"autoEvaluate": true, // Auto-evaluate tasks for skill generation
"minChunksForEval": 6, // Min chunks for a task to be evaluated
"minConfidence": 0.7, // Min LLM confidence to create/upgrade skill
"autoInstall": false // Auto-install generated skills
},
"viewerPort": 18799, // Memory Viewer port
"telemetry": {
"enabled": true // Anonymous usage analytics (default: true, set false to opt-out)
}
}
}
```
## Telemetry
MemOS Lite collects **anonymous** usage analytics to help us understand how the plugin is used and improve it. Telemetry is **enabled by default** and can be disabled at any time.
### What is collected
- Plugin version, OS, Node.js version, architecture
- Tool call names and latencies (e.g. "memory_search took 120ms")
- Aggregate counts (chunks ingested, skills installed)
- Daily active ping
### What is NEVER collected
- Memory content, search queries, or conversation text
- API keys, file paths, or any personally identifiable information
- Any data stored in your local database
### How to disable
Add `telemetry` to your plugin config in `~/.openclaw/openclaw.json`:
```jsonc
{
"plugins": {
"entries": {
"memos-local-openclaw-plugin": {
"enabled": true,
"config": {
"telemetry": {
"enabled": false
}
// ... other config
}
}
}
}
}
```
Or set the environment variable:
```bash
TELEMETRY_ENABLED=false
```
### Technical details
- Uses Aliyun ARMS RUM for event collection
- Each installation gets a random anonymous UUID (stored at `~/.openclaw/memos-local/.anonymous-id`)
- Events are batched and sent in the background; failures are silently ignored
- The anonymous ID is never linked to any personal information
## Upgrade
```bash
openclaw plugins update memos-local-openclaw-plugin
```
The plugin will automatically install dependencies, clean up legacy versions, and rebuild the native SQLite module. After update, restart the gateway:
```bash
openclaw gateway stop && openclaw gateway start
```
> **Tip:** To update all plugins at once: `openclaw plugins update --all`
**If `openclaw plugins update` doesn't work** (plugin not in install registry), reinstall:
```bash
rm -rf ~/.openclaw/extensions/memos-local-openclaw-plugin
openclaw plugins install @memtensor/memos-local-openclaw-plugin
```
> **Note:** `openclaw plugins install` requires the target directory to not exist. If you see `plugin already exists`, delete the directory first. Your memory data is stored separately at `~/.openclaw/memos-local/memos.db` and will not be affected.
## Troubleshooting
> 📖 **详细排查指南 / Detailed troubleshooting guide:** [docs/troubleshooting.html](https://memtensor.github.io/MemOS/apps/memos-local-openclaw/docs/troubleshooting.html) — 包含逐步排查流程、日志查看方法、完全重装步骤等。
>
> 📦 **better-sqlite3 official troubleshooting:** [better-sqlite3 Troubleshooting](https://github.com/WiseLibs/better-sqlite3/blob/master/docs/troubleshooting.md) — the upstream guide for native module build issues.
### Common Issues
1. **Note the exact error** — e.g. `plugin not found`, `Cannot find module 'xxx'`, `Invalid config`.
2. **Check plugin status**
```bash
openclaw plugins list
```
- Status is **error** → note the error message
- Not listed → not installed or not placed in `~/.openclaw/extensions/memos-local-openclaw-plugin`
3. **Check gateway logs**
```bash
tail -50 ~/.openclaw/logs/gateway.log
```
Search for `memos-local`, `failed to load`, `Error`, `Cannot find module`.
4. **Check environment**
- Node version: `node -v` (requires **>= 18**)
- Plugin directory exists: `ls ~/.openclaw/extensions/memos-local-openclaw-plugin/package.json`
- Dependencies installed: `ls ~/.openclaw/extensions/memos-local-openclaw-plugin/node_modules/@sinclair/typebox`
If missing: `cd ~/.openclaw/extensions/memos-local-openclaw-plugin && npm install --omit=dev`
5. **Check configuration** — Open `~/.openclaw/openclaw.json` and verify:
- `agents.defaults.memorySearch.enabled` = `false` (disable built-in memory)
- `plugins.slots.memory` = `"memos-local-openclaw-plugin"`
- `plugins.entries.memos-local-openclaw-plugin.enabled` = `true`
6. **better-sqlite3 native module error** — `Could not locate the bindings file` means the native SQLite addon was not compiled for your Node.js version.
```bash
cd ~/.openclaw/extensions/memos-local-openclaw-plugin
npm rebuild better-sqlite3
```
If rebuild fails, install C++ build tools first:
- **macOS:** `xcode-select --install` (if you see `xcrun: error: invalid active developer path`, run this first)
- **Linux:** `sudo apt install build-essential python3`
- **Windows:** Usually not needed — `better-sqlite3` provides prebuilt binaries for Windows + Node.js LTS. If it still fails, install [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/) (select "C++ build tools" workload)
Then retry `npm rebuild better-sqlite3` and restart the gateway.
> **Still failing?** Check the official [better-sqlite3 troubleshooting guide](https://github.com/WiseLibs/better-sqlite3/blob/master/docs/troubleshooting.md) for platform-specific solutions. For non-LTS Node.js versions (e.g., v25.x), prebuilt binaries may not be available and compilation from source is required.
7. **Memory conflict with built-in search** — If the agent calls both the built-in memory search and the plugin's `memory_search`, it means `agents.defaults.memorySearch.enabled` is not set to `false`.
8. **Skills not generating** — Check:
- `skillEvolution.enabled` is `true`
- Tasks have enough content (default requires >= 6 chunks)
- LLM model is accessible (check gateway log for `judgeNewTopic failed` or `SkillEvolver` errors)
- The LLM fallback chain will try: `skillSummarizer` → `summarizer` → OpenClaw native model. If all fail, skill generation is skipped
- Look for `SkillEvolver` output in the gateway log
9. **LLM calls failing** — All LLM-dependent features (summarization, topic detection, skill generation) use a fallback chain. If the configured model returns an error, the next model in the chain is tried automatically. Check the gateway log for messages like `failed (model), trying next`. If all models fail, the operation falls back to rule-based logic or is skipped.
## Data Location
| File | Path |
|---|---|
| Database | `~/.openclaw/memos-local/memos.db` |
| Viewer auth | `~/.openclaw/memos-local/viewer-auth.json` |
| Gateway log | `~/.openclaw/logs/gateway.log` |
| Plugin code | `~/.openclaw/extensions/memos-local-openclaw-plugin/` |
| Memory-guide skill | `~/.openclaw/workspace/skills/memos-memory-guide/SKILL.md` (and `~/.openclaw/skills/memos-memory-guide/`) |
| Generated skills | `~/.openclaw/memos-local/skills-store//` |
| Installed skills | `~/.openclaw/workspace/skills//` |
## Development Guide
This section is for contributors who want to develop, test, or modify the plugin from source.
### Prerequisites
- **Node.js >= 18** (`node -v`)
- **npm >= 9** (`npm -v`)
- **C++ build tools** (for `better-sqlite3` native module):
- macOS: `xcode-select --install`
- Linux: `sudo apt install build-essential python3`
- Windows: usually not needed (prebuilt binaries available for LTS Node.js); if build fails, install [Visual Studio Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
- **OpenClaw CLI** installed and available in PATH (`openclaw --version`)
> **`better-sqlite3` build issues?** This is the most common installation problem on macOS and Linux. If `npm install` fails, first install the C++ build tools above, then run `npm rebuild better-sqlite3`. For detailed platform-specific solutions, see the [official better-sqlite3 troubleshooting guide](https://github.com/WiseLibs/better-sqlite3/blob/master/docs/troubleshooting.md) and our [installation troubleshooting page](https://memtensor.github.io/MemOS/apps/memos-local-openclaw/docs/troubleshooting.html).
### Clone & Setup
```bash
git clone https://github.com/MemTensor/MemOS.git
cd MemOS/apps/memos-local-openclaw
npm install
```
> `npm install` triggers the `postinstall` script which automatically rebuilds `better-sqlite3` for your Node.js version.
### Project Structure
```
apps/memos-local-openclaw/
├── index.ts # Plugin entry — hooks, tool registration, lifecycle
├── plugin-impl.ts # OpenClaw plugin SDK implementation
├── src/
│ ├── index.ts # Module re-exports
│ ├── config.ts # Configuration schema & defaults
│ ├── types.ts # TypeScript type definitions
│ ├── capture/index.ts # Message capture & filtering logic
│ ├── embedding/ # Embedding providers (OpenAI, Gemini, Cohere, etc.)
│ ├── ingest/
│ │ ├── chunker.ts # Semantic chunking (code blocks, paragraphs)
│ │ ├── dedup.ts # Content-hash + vector deduplication
│ │ ├── worker.ts # Async ingestion pipeline
│ │ ├── task-processor.ts # Task boundary detection & summarization
│ │ └── providers/ # LLM providers for summarization
│ ├── recall/
│ │ ├── engine.ts # Hybrid retrieval engine (FTS5 + Vector)
│ │ ├── rrf.ts # Reciprocal Rank Fusion
│ │ ├── mmr.ts # Maximal Marginal Relevance
│ │ └── recency.ts # Time-decay scoring
│ ├── shared/
│ │ └── llm-call.ts # LLM fallback chain utility (callLLMWithFallback, buildSkillConfigChain)
│ ├── skill/ # Skill evolution pipeline (evaluator, generator, upgrader)
│ ├── storage/
│ │ ├── sqlite.ts # SQLite database layer (chunks, tasks, skills, FTS5)
│ │ └── vector.ts # Vector similarity search
│ ├── tools/ # Tool implementations (memory-search, memory-get, etc.)
│ ├── viewer/ # Memory Viewer web server & HTML templates
│ └── telemetry.ts # Anonymous usage analytics
├── tests/ # Test suite (vitest)
├── scripts/ # Utility scripts (seed data, smoke test, viewer)
├── skill/ # Bundled skill definitions (SKILL.md files)
├── openclaw.plugin.json # Plugin metadata for OpenClaw registry
├── package.json # Dependencies & scripts
├── tsconfig.json # TypeScript configuration
└── vitest.config.ts # Test runner configuration
```
**Files NOT in the repository** (generated locally, excluded via `.gitignore`):
| Directory / File | Purpose | How to generate |
|---|---|---|
| `node_modules/` | npm dependencies | `npm install` |
| `dist/` | Compiled JavaScript output | `npm run build` |
| `package-lock.json` | Dependency lock file | `npm install` (auto-generated) |
| `www/` | Memory Viewer static site (local preview) | Started automatically by the plugin |
| `docs/` | Documentation HTML pages | Built from source or viewed at the hosted URL |
| `ppt/` | Presentation files (internal use) | Not needed for development |
| `.env` | Local environment variables | Copy from `.env.example` |
### Build
```bash
npm run build # Compile TypeScript → dist/
npm run dev # Watch mode — auto-recompile on save
```
The build output goes to `dist/` (CommonJS modules with declarations and source maps).
### Configure for Local Development
1. **Copy the environment template:**
```bash
cp .env.example .env
```
2. **Edit `.env`** with your API keys (or leave blank for local-only mode):
```bash
# Embedding — leave blank to use local offline model
EMBEDDING_PROVIDER=openai_compatible
EMBEDDING_API_KEY=your-key
EMBEDDING_ENDPOINT=https://your-api.com/v1
EMBEDDING_MODEL=bge-m3
# Summarizer — leave blank for rule-based fallback
SUMMARIZER_PROVIDER=openai_compatible
SUMMARIZER_API_KEY=your-key
SUMMARIZER_ENDPOINT=https://api.openai.com/v1
SUMMARIZER_MODEL=gpt-4o-mini
```
3. **Install the plugin locally into OpenClaw:**
```bash
npm run build
openclaw plugins install .
```
4. **Configure OpenClaw** — Add the plugin to `~/.openclaw/openclaw.json` (see [Configure](#2-configure) section above).
5. **Start the gateway:**
```bash
openclaw gateway stop # stop existing
openclaw gateway start # start with new plugin
```
### Testing
Run the full test suite:
```bash
npm test # Run all tests once
npm run test:watch # Watch mode — re-run on file changes
```
Test coverage includes:
| Test File | Coverage |
|---|---|
| `tests/policy.test.ts` | Retrieval strategy, search filtering, evidence extraction, instruction stripping |
| `tests/recall.test.ts` | RRF fusion, recency decay correctness |
| `tests/capture.test.ts` | Message filtering, evidence block stripping, self-tool exclusion |
| `tests/storage.test.ts` | SQLite CRUD, FTS5, vector storage, content hash dedup |
| `tests/chunker.test.ts` | Semantic chunking for code blocks, paragraphs, function bodies |
| `tests/task-processor.test.ts` | Task boundary detection, skip logic, summary generation |
| `tests/multi-agent.test.ts` | Multi-agent memory isolation, owner filtering, public sharing |
| `tests/integration.test.ts` | End-to-end ingestion and retrieval pipeline |
> Tests use an **in-memory SQLite database** — no external services or API keys required.
### Development Workflow
1. **Make changes** to files in `src/` or `index.ts`
2. **Run tests** to verify: `npm test`
3. **Build** to check TypeScript compilation: `npm run build`
4. **Test with OpenClaw** locally:
```bash
openclaw plugins install . # re-install from local source
openclaw gateway stop && openclaw gateway start
tail -f ~/.openclaw/logs/gateway.log # watch logs
```
5. **Open Memory Viewer** at `http://127.0.0.1:18799` to verify UI changes
### Publishing to npm
```bash
npm run build # Compile TypeScript
npm publish --access public # Publish to npm registry
```
After publishing, users can install with:
```bash
openclaw plugins install @memtensor/memos-local-openclaw-plugin
```
### Utility Scripts
| Script | Command | Purpose |
|---|---|---|
| Seed test data | `npx tsx scripts/seed-test-data.ts` | Populate local DB with sample memories, tasks, and skills |
| Smoke test | `npx tsx scripts/smoke-test.ts` | Quick end-to-end verification of plugin functionality |
| Start viewer | `npx tsx scripts/start-viewer.ts` | Start Memory Viewer standalone (without gateway) |
| Refresh skills | `npx tsx scripts/refresh-skill.ts` | Re-evaluate and regenerate skills from existing tasks |
| Refresh summaries | `npx tsx scripts/refresh-summaries.ts` | Re-generate task summaries for completed tasks |
| Mock skills | `npx tsx scripts/mock-skills.ts` | Generate mock skill data for testing |
## License
MIT — See [LICENSE](../../LICENSE) for details.
================================================
FILE: apps/memos-local-openclaw/index.ts
================================================
/**
* OpenClaw Plugin Entry — memos-local
*
* Full-write local memory with hybrid retrieval (RRF + MMR + recency).
* Provides: memory_search, memory_get, memory_timeline, task_summary, skill_get, skill_install, memory_viewer
*/
import type { OpenClawPluginApi } from "openclaw/plugin-sdk";
import { Type } from "@sinclair/typebox";
import * as fs from "fs";
import * as path from "path";
import { fileURLToPath } from "url";
import { buildContext } from "./src/config";
import { ensureSqliteBinding } from "./src/storage/ensure-binding";
import { SqliteStore } from "./src/storage/sqlite";
import { Embedder } from "./src/embedding";
import { IngestWorker } from "./src/ingest/worker";
import { RecallEngine } from "./src/recall/engine";
import { captureMessages, stripInboundMetadata } from "./src/capture";
import { DEFAULTS } from "./src/types";
import { ViewerServer } from "./src/viewer/server";
import { SkillEvolver } from "./src/skill/evolver";
import { SkillInstaller } from "./src/skill/installer";
import { Summarizer } from "./src/ingest/providers";
import { MEMORY_GUIDE_SKILL_MD } from "./src/skill/bundled-memory-guide";
import { Telemetry } from "./src/telemetry";
/** Remove near-duplicate hits based on summary word overlap (>70%). Keeps first (highest-scored) hit. */
function deduplicateHits(hits: T[]): T[] {
const kept: T[] = [];
for (const hit of hits) {
const dominated = kept.some((k) => {
const a = k.summary.toLowerCase();
const b = hit.summary.toLowerCase();
if (a === b) return true;
const wordsA = new Set(a.split(/\s+/).filter(w => w.length > 1));
const wordsB = new Set(b.split(/\s+/).filter(w => w.length > 1));
if (wordsA.size === 0 || wordsB.size === 0) return false;
let overlap = 0;
for (const w of wordsB) { if (wordsA.has(w)) overlap++; }
return overlap / Math.min(wordsA.size, wordsB.size) > 0.7;
});
if (!dominated) kept.push(hit);
}
return kept;
}
const pluginConfigSchema = {
type: "object" as const,
additionalProperties: true,
properties: {
viewerPort: {
type: "number" as const,
description: "Memory Viewer HTTP port (default 18799)",
},
telemetry: {
type: "object" as const,
description: "Anonymous usage analytics (opt-out). No memory content or personal data is ever sent.",
properties: {
enabled: {
type: "boolean" as const,
description: "Enable anonymous telemetry (default: true). Set to false to opt-out.",
},
},
},
},
};
const memosLocalPlugin = {
id: "memos-local-openclaw-plugin",
name: "MemOS Local Memory",
description:
"Full-write local conversation memory with hybrid search (RRF + MMR + recency). " +
"Provides memory_search, memory_get, task_summary, memory_timeline, memory_viewer for layered retrieval.",
kind: "memory" as const,
configSchema: pluginConfigSchema,
register(api: OpenClawPluginApi) {
// ─── Ensure better-sqlite3 native module is available ───
const pluginDir = path.dirname(fileURLToPath(import.meta.url));
function normalizeFsPath(p: string): string {
return path.resolve(p).replace(/\\/g, "/").toLowerCase();
}
let sqliteReady = false;
function trySqliteLoad(): boolean {
try {
const resolved = require.resolve("better-sqlite3", { paths: [pluginDir] });
const resolvedNorm = normalizeFsPath(resolved);
const pluginNorm = normalizeFsPath(pluginDir);
if (!resolvedNorm.startsWith(pluginNorm + "/") && resolvedNorm !== pluginNorm) {
api.logger.warn(`memos-local: better-sqlite3 resolved outside plugin dir: ${resolved}`);
return false;
}
require(resolved);
return true;
} catch {
return false;
}
}
sqliteReady = trySqliteLoad();
if (!sqliteReady) {
api.logger.warn(`memos-local: better-sqlite3 not found in ${pluginDir}, attempting auto-rebuild ...`);
try {
const { spawnSync } = require("child_process");
const rebuildResult = spawnSync("npm", ["rebuild", "better-sqlite3"], {
cwd: pluginDir,
stdio: "pipe",
shell: true,
timeout: 120_000,
});
const stdout = rebuildResult.stdout?.toString() || "";
const stderr = rebuildResult.stderr?.toString() || "";
if (stdout) api.logger.info(`memos-local: rebuild stdout: ${stdout.slice(0, 500)}`);
if (stderr) api.logger.warn(`memos-local: rebuild stderr: ${stderr.slice(0, 500)}`);
if (rebuildResult.status === 0) {
Object.keys(require.cache)
.filter(k => k.includes("better-sqlite3") || k.includes("better_sqlite3"))
.forEach(k => delete require.cache[k]);
sqliteReady = trySqliteLoad();
if (sqliteReady) {
api.logger.info("memos-local: better-sqlite3 auto-rebuild succeeded!");
} else {
api.logger.warn("memos-local: rebuild exited 0 but module still not loadable from plugin dir");
}
} else {
api.logger.warn(`memos-local: rebuild exited with code ${rebuildResult.status}`);
}
} catch (rebuildErr) {
api.logger.warn(`memos-local: auto-rebuild error: ${rebuildErr}`);
}
if (!sqliteReady) {
const nodeVer = process.version;
const nodeMajor = parseInt(process.versions?.node?.split(".")[0] ?? "0", 10);
const isNode25Plus = nodeMajor >= 25;
const lines = [
"",
"╔══════════════════════════════════════════════════════════════╗",
"║ MemOS Local Memory — better-sqlite3 native module missing ║",
"╠══════════════════════════════════════════════════════════════╣",
"║ ║",
"║ Auto-rebuild failed (Node " + nodeVer + "). Run manually: ║",
"║ ║",
`║ cd ${pluginDir}`,
"║ npm rebuild better-sqlite3 ║",
"║ openclaw gateway stop && openclaw gateway start ║",
"║ ║",
"║ If rebuild fails, install build tools first: ║",
"║ macOS: xcode-select --install ║",
"║ Linux: sudo apt install build-essential python3 ║",
];
if (isNode25Plus) {
lines.push("║ ║");
lines.push("║ Node 25+ has no prebuild: build tools required, or use ║");
lines.push("║ Node LTS (20/22): nvm install 22 && nvm use 22 ║");
}
lines.push("║ ║");
lines.push("╚══════════════════════════════════════════════════════════════╝");
lines.push("");
api.logger.warn(lines.join("\n"));
throw new Error(
`better-sqlite3 native module not found (Node ${nodeVer}). Auto-rebuild failed. Fix: install build tools, then cd ${pluginDir} && npm rebuild better-sqlite3. Or use Node LTS (20/22).`
);
}
}
const pluginCfg = (api.pluginConfig ?? {}) as Record;
const stateDir = api.resolvePath("~/.openclaw");
const ctx = buildContext(stateDir, process.cwd(), pluginCfg as any, {
debug: (msg: string) => api.logger.info(`[debug] ${msg}`),
info: (msg: string) => api.logger.info(msg),
warn: (msg: string) => api.logger.warn(msg),
error: (msg: string) => api.logger.warn(`[error] ${msg}`),
});
ensureSqliteBinding(ctx.log);
const store = new SqliteStore(ctx.config.storage!.dbPath!, ctx.log);
const embedder = new Embedder(ctx.config.embedding, ctx.log);
const worker = new IngestWorker(store, embedder, ctx);
const engine = new RecallEngine(store, embedder, ctx);
const evidenceTag = ctx.config.capture?.evidenceWrapperTag ?? DEFAULTS.evidenceWrapperTag;
const workspaceDir = api.resolvePath("~/.openclaw/workspace");
const skillCtx = { ...ctx, workspaceDir };
const skillEvolver = new SkillEvolver(store, engine, skillCtx);
skillEvolver.onSkillEvolved = (name, type) => telemetry.trackSkillEvolved(name, type);
const skillInstaller = new SkillInstaller(store, skillCtx);
let pluginVersion = "0.0.0";
try {
const pkg = JSON.parse(fs.readFileSync(path.join(__dirname, "package.json"), "utf-8"));
pluginVersion = pkg.version ?? pluginVersion;
} catch {}
const telemetry = new Telemetry(ctx.config.telemetry ?? {}, stateDir, pluginVersion, ctx.log);
// Install bundled memory-guide skill so OpenClaw loads it (write from embedded content so it works regardless of deploy layout)
const workspaceSkillsDir = path.join(workspaceDir, "skills");
const memosGuideDest = path.join(workspaceSkillsDir, "memos-memory-guide");
fs.mkdirSync(memosGuideDest, { recursive: true });
fs.writeFileSync(path.join(memosGuideDest, "SKILL.md"), MEMORY_GUIDE_SKILL_MD, "utf-8");
ctx.log.info(`memos-local: installed bundled skill memos-memory-guide → ${memosGuideDest}`);
// Also ensure managed skills dir has it so dashboard/other loaders can see it
const managedSkillsDir = path.join(stateDir, "skills");
const managedMemosGuide = path.join(managedSkillsDir, "memos-memory-guide");
try {
fs.mkdirSync(managedMemosGuide, { recursive: true });
fs.writeFileSync(path.join(managedMemosGuide, "SKILL.md"), MEMORY_GUIDE_SKILL_MD, "utf-8");
ctx.log.info(`memos-local: installed bundled skill memos-memory-guide → ${managedMemosGuide} (managed)`);
} catch (e) {
ctx.log.warn(`memos-local: could not write to managed skills dir: ${e}`);
}
// Ensure plugin tools are enabled in openclaw.json tools.allow
try {
const openclawJsonPath = path.join(stateDir, "openclaw.json");
if (fs.existsSync(openclawJsonPath)) {
const raw = fs.readFileSync(openclawJsonPath, "utf-8");
const cfg = JSON.parse(raw);
const allow: string[] | undefined = cfg?.tools?.allow;
if (Array.isArray(allow) && allow.length > 0 && !allow.includes("group:plugins")) {
const lastEntry = JSON.stringify(allow[allow.length - 1]);
const patched = raw.replace(
new RegExp(`(${lastEntry})(\\s*\\])`),
`$1,\n "group:plugins"$2`,
);
if (patched !== raw && patched.includes("group:plugins")) {
fs.writeFileSync(openclawJsonPath, patched, "utf-8");
ctx.log.info("memos-local: added 'group:plugins' to tools.allow in openclaw.json");
}
}
}
} catch (e) {
ctx.log.warn(`memos-local: could not patch tools.allow: ${e}`);
}
worker.getTaskProcessor().onTaskCompleted((task) => {
skillEvolver.onTaskCompleted(task).catch((err) => {
ctx.log.warn(`SkillEvolver async error: ${err}`);
});
});
const summarizer = new Summarizer(ctx.config.summarizer, ctx.log);
api.logger.info(`memos-local: initialized (db: ${ctx.config.storage!.dbPath})`);
// Current agent ID — updated by hooks, read by tools for owner isolation.
// Falls back to "main" when no hook has fired yet (single-agent setups).
let currentAgentId = "main";
// ─── Check allowPromptInjection policy ───
// When allowPromptInjection=false, the prompt mutation fields (such as prependContext) in the hook return value
// will be stripped by the framework. Skip auto-recall to avoid unnecessary LLM/embedding calls.
const pluginEntry = (api.config as any)?.plugins?.entries?.[api.id];
const allowPromptInjection = pluginEntry?.hooks?.allowPromptInjection !== false;
if (!allowPromptInjection) {
api.logger.info("memos-local: allowPromptInjection=false, auto-recall disabled");
}
else {
api.logger.info("memos-local: allowPromptInjection=true, auto-recall enabled");
}
const trackTool = (toolName: string, fn: (...args: any[]) => Promise) =>
async (...args: any[]) => {
const t0 = performance.now();
let ok = true;
let result: any;
const inputParams = args.length > 1 ? args[1] : args[0];
try {
result = await fn(...args);
return result;
} catch (e) {
ok = false;
telemetry.trackError(toolName, (e as Error)?.name ?? "unknown");
throw e;
} finally {
const dur = performance.now() - t0;
store.recordToolCall(toolName, dur, ok);
telemetry.trackToolCalled(toolName, dur, ok);
try {
let outputText: string;
const det = result?.details;
if (det && Array.isArray(det.candidates)) {
outputText = JSON.stringify({
candidates: det.candidates,
filtered: det.hits ?? det.filtered ?? [],
});
} else {
outputText = result?.content?.[0]?.text ?? JSON.stringify(result ?? "");
}
store.recordApiLog(toolName, { ...inputParams, type: "tool_call" }, outputText, dur, ok);
} catch (_) { /* best-effort */ }
}
};
// ─── Tool: memory_search ───
api.registerTool(
{
name: "memory_search",
label: "Memory Search",
description:
"Search long-term conversation memory for past conversations, user preferences, decisions, and experiences. " +
"Relevant memories are automatically injected at the start of each turn, but call this tool when you need " +
"to search with a different query or the auto-recalled context is insufficient. " +
"Pass only a short natural-language query (2-5 key words).",
parameters: Type.Object({
query: Type.String({ description: "Short natural language search query (2-5 key words)" }),
}),
execute: trackTool("memory_search", async (_toolCallId: any, params: any) => {
const { query } = params as { query: string };
const role = undefined;
const minScore = undefined;
const agentId = currentAgentId;
const ownerFilter = [`agent:${agentId}`, "public"];
const effectiveMaxResults = 10;
ctx.log.debug(`memory_search query="${query}" maxResults=${effectiveMaxResults} minScore=${minScore ?? 0.45} role=${role ?? "all"} owner=agent:${agentId}`);
const result = await engine.search({ query, maxResults: effectiveMaxResults, minScore, role, ownerFilter });
ctx.log.debug(`memory_search raw candidates: ${result.hits.length}`);
const rawCandidates = result.hits.map((h) => ({
chunkId: h.ref.chunkId,
role: h.source.role,
score: h.score,
summary: h.summary,
original_excerpt: (h.original_excerpt ?? "").slice(0, 200),
}));
if (result.hits.length === 0) {
return {
content: [{ type: "text", text: result.meta.note ?? "No relevant memories found." }],
details: { candidates: [], meta: result.meta },
};
}
// LLM relevance + sufficiency filtering
let filteredHits = result.hits;
let sufficient = false;
const candidates = result.hits.map((h, i) => ({
index: i + 1,
role: h.source.role,
content: (h.original_excerpt ?? "").slice(0, 300),
time: h.source.ts ? new Date(h.source.ts).toISOString().slice(0, 16) : "",
}));
const filterResult = await summarizer.filterRelevant(query, candidates);
if (filterResult !== null) {
sufficient = filterResult.sufficient;
if (filterResult.relevant.length > 0) {
const indexSet = new Set(filterResult.relevant);
filteredHits = result.hits.filter((_, i) => indexSet.has(i + 1));
ctx.log.debug(`memory_search LLM filter: ${result.hits.length} → ${filteredHits.length} hits, sufficient=${sufficient}`);
} else {
return {
content: [{ type: "text", text: "No relevant memories found for this query." }],
details: { candidates: rawCandidates, filtered: [], meta: result.meta },
};
}
}
if (filteredHits.length === 0) {
return {
content: [{ type: "text", text: "No relevant memories found for this query." }],
details: { candidates: rawCandidates, filtered: [], meta: result.meta },
};
}
const beforeDedup = filteredHits.length;
filteredHits = deduplicateHits(filteredHits);
ctx.log.debug(`memory_search dedup: ${beforeDedup} → ${filteredHits.length}`);
const lines = filteredHits.map((h, i) => {
const excerpt = h.original_excerpt;
const parts = [`${i + 1}. [${h.source.role}]`];
if (excerpt) parts.push(` ${excerpt}`);
parts.push(` chunkId="${h.ref.chunkId}"`);
if (h.taskId) {
const task = store.getTask(h.taskId);
if (task && task.status !== "skipped") {
parts.push(` task_id="${h.taskId}"`);
}
}
return parts.join("\n");
});
let tipsText = "";
if (!sufficient) {
const hasTask = filteredHits.some((h) => {
if (!h.taskId) return false;
const t = store.getTask(h.taskId);
return t && t.status !== "skipped";
});
const tips: string[] = [];
if (hasTask) {
tips.push("→ call task_summary(taskId) for full task context");
tips.push("→ call skill_get(taskId=...) if the task has a proven experience guide");
}
tips.push("→ call memory_timeline(chunkId) to expand surrounding conversation");
if (tips.length > 0) {
tipsText = "\n\nThese memories may not be enough. You can fetch more context:\n" + tips.join("\n");
}
}
return {
content: [
{
type: "text",
text: `Found ${filteredHits.length} relevant memories:\n\n${lines.join("\n\n")}${tipsText}`,
},
],
details: {
candidates: rawCandidates,
hits: filteredHits.map((h) => {
let effectiveTaskId = h.taskId;
if (effectiveTaskId) {
const t = store.getTask(effectiveTaskId);
if (t && t.status === "skipped") effectiveTaskId = null;
}
return {
chunkId: h.ref.chunkId,
taskId: effectiveTaskId,
skillId: h.skillId,
role: h.source.role,
score: h.score,
summary: h.summary,
original_excerpt: (h.original_excerpt ?? "").slice(0, 200),
};
}),
meta: result.meta,
},
};
}),
},
{ name: "memory_search" },
);
// ─── Tool: memory_timeline ───
api.registerTool(
{
name: "memory_timeline",
label: "Memory Timeline",
description:
"Expand context around a memory search hit. Pass the chunkId from a search result " +
"to read the surrounding conversation messages.",
parameters: Type.Object({
chunkId: Type.String({ description: "The chunkId from a memory_search hit" }),
window: Type.Optional(Type.Number({ description: "Context window ±N (default 2)" })),
}),
execute: trackTool("memory_timeline", async (_toolCallId: any, params: any) => {
ctx.log.debug(`memory_timeline called (agent=${currentAgentId})`);
const { chunkId, window: win } = params as {
chunkId: string;
window?: number;
};
const ownerFilter = [`agent:${currentAgentId}`, "public"];
const anchorChunk = store.getChunkForOwners(chunkId, ownerFilter);
if (!anchorChunk) {
return {
content: [{ type: "text", text: `Chunk not found: ${chunkId}` }],
details: { error: "not_found" },
};
}
const w = win ?? DEFAULTS.timelineWindowDefault;
const neighbors = store.getNeighborChunks(anchorChunk.sessionKey, anchorChunk.turnId, anchorChunk.seq, w, ownerFilter);
const anchorTs = anchorChunk?.createdAt ?? 0;
const entries = neighbors.map((chunk) => {
let relation: "before" | "current" | "after" = "before";
if (chunk.id === chunkId) relation = "current";
else if (chunk.createdAt > anchorTs) relation = "after";
return {
relation,
role: chunk.role,
excerpt: chunk.content,
ts: chunk.createdAt,
};
});
const rl = (r: string) => r === "user" ? "USER" : r === "assistant" ? "ASSISTANT" : r.toUpperCase();
const text = entries
.map((e) => `[${e.relation}] ${rl(e.role)}: ${e.excerpt}`)
.join("\n");
return {
content: [{ type: "text", text: `Timeline (${entries.length} entries):\n\n${text}` }],
details: { entries, anchorRef: { sessionKey: anchorChunk.sessionKey, chunkId, turnId: anchorChunk.turnId, seq: anchorChunk.seq } },
};
}),
},
{ name: "memory_timeline" },
);
// ─── Tool: memory_get ───
api.registerTool(
{
name: "memory_get",
label: "Memory Get",
description:
"Get the full original text of a memory chunk. Use to verify exact details from a search hit.",
parameters: Type.Object({
chunkId: Type.String({ description: "From search hit ref.chunkId" }),
maxChars: Type.Optional(
Type.Number({ description: `Max chars (default ${DEFAULTS.getMaxCharsDefault}, max ${DEFAULTS.getMaxCharsMax})` }),
),
}),
execute: trackTool("memory_get", async (_toolCallId: any, params: any) => {
const { chunkId, maxChars } = params as { chunkId: string; maxChars?: number };
const limit = Math.min(maxChars ?? DEFAULTS.getMaxCharsDefault, DEFAULTS.getMaxCharsMax);
const ownerFilter = [`agent:${currentAgentId}`, "public"];
const chunk = store.getChunkForOwners(chunkId, ownerFilter);
if (!chunk) {
return {
content: [{ type: "text", text: `Chunk not found: ${chunkId}` }],
details: { error: "not_found" },
};
}
const content = chunk.content;
const who = chunk.role === "user" ? "USER said" : chunk.role === "assistant" ? "ASSISTANT replied" : chunk.role === "tool" ? "TOOL returned" : chunk.role.toUpperCase();
return {
content: [{ type: "text", text: `[${who}] (session: ${chunk.sessionKey})\n\n${content}` }],
details: {
ref: { sessionKey: chunk.sessionKey, chunkId: chunk.id, turnId: chunk.turnId, seq: chunk.seq },
source: { ts: chunk.createdAt, role: chunk.role, sessionKey: chunk.sessionKey },
},
};
}),
},
{ name: "memory_get" },
);
// ─── Tool: task_summary ───
api.registerTool(
{
name: "task_summary",
label: "Task Summary",
description:
"Get the detailed summary of a complete task. Use this when memory_search returns a hit " +
"with a task_id and you need the full context of that task. The summary preserves all " +
"critical information: URLs, file paths, commands, error codes, step-by-step instructions.",
parameters: Type.Object({
taskId: Type.String({ description: "The task_id from a memory_search hit" }),
}),
execute: trackTool("task_summary", async (_toolCallId: any, params: any) => {
const { taskId } = params as { taskId: string };
ctx.log.debug(`task_summary called for task=${taskId}`);
const task = store.getTask(taskId);
if (!task) {
return {
content: [{ type: "text", text: `Task not found: ${taskId}` }],
details: { error: "not_found" },
};
}
if (task.status === "skipped") {
return {
content: [{ type: "text", text: `Task "${task.title}" was too brief to generate a summary. Reason: ${task.summary || "conversation too short"}. Use memory_get to read individual chunks instead.` }],
details: { taskId, status: task.status },
};
}
if (!task.summary) {
const chunks = store.getChunksByTask(taskId);
if (chunks.length === 0) {
return {
content: [{ type: "text", text: `Task ${taskId} has no content yet.` }],
details: { taskId, status: task.status },
};
}
return {
content: [{
type: "text",
text: `Task "${task.title}" is still active (summary not yet generated). ` +
`It contains ${chunks.length} memory chunks. Use memory_get to read individual chunks.`,
}],
details: { taskId, status: task.status, chunkCount: chunks.length },
};
}
const relatedSkills = store.getSkillsByTask(taskId);
let skillSection = "";
if (relatedSkills.length > 0) {
const skillLines = relatedSkills.map(rs =>
`- 🔧 ${rs.skill.name} (${rs.relation}, v${rs.versionAt}) — call skill_get(skillId="${rs.skill.id}") or skill_get(taskId="${taskId}") to get the full guide`
);
skillSection = `\n\n### Related Skills\n${skillLines.join("\n")}`;
}
return {
content: [{
type: "text",
text: `## Task: ${task.title}\n\nStatus: ${task.status}\nChunks: ${store.getChunksByTask(taskId).length}\n\n${task.summary}${skillSection}`,
}],
details: {
taskId: task.id,
title: task.title,
status: task.status,
startedAt: task.startedAt,
endedAt: task.endedAt,
relatedSkills: relatedSkills.map(rs => ({ skillId: rs.skill.id, name: rs.skill.name, relation: rs.relation })),
},
};
}),
},
{ name: "task_summary" },
);
// ─── Tool: skill_get ───
api.registerTool(
{
name: "skill_get",
label: "Get Skill",
description:
"Retrieve a proven skill (experience guide) by skillId or taskId. " +
"Pass either one — if you have a task_id from memory_search, pass taskId and the system " +
"will find the associated skill automatically.",
parameters: Type.Object({
skillId: Type.Optional(Type.String({ description: "Direct skill ID" })),
taskId: Type.Optional(Type.String({ description: "Task ID — will look up the skill linked to this task" })),
}),
execute: trackTool("skill_get", async (_toolCallId: any, params: any) => {
const { skillId: directSkillId, taskId } = params as { skillId?: string; taskId?: string };
let resolvedSkillId = directSkillId;
if (!resolvedSkillId && taskId) {
const linked = store.getSkillsByTask(taskId);
if (linked.length > 0) {
resolvedSkillId = linked[0].skill.id;
} else {
return {
content: [{ type: "text", text: `No skill associated with task ${taskId}.` }],
details: { error: "no_skill_for_task", taskId },
};
}
}
if (!resolvedSkillId) {
return {
content: [{ type: "text", text: "Provide either skillId or taskId." }],
details: { error: "missing_params" },
};
}
ctx.log.debug(`skill_get resolved skill=${resolvedSkillId} (from ${directSkillId ? "skillId" : "taskId=" + taskId})`);
const skill = store.getSkill(resolvedSkillId);
if (!skill) {
return {
content: [{ type: "text", text: `Skill not found: ${resolvedSkillId}` }],
details: { error: "not_found" },
};
}
const sv = store.getLatestSkillVersion(resolvedSkillId);
if (!sv) {
return {
content: [{ type: "text", text: `Skill "${skill.name}" has no content versions.` }],
details: { skillId: resolvedSkillId, name: skill.name, error: "no_version" },
};
}
return {
content: [{
type: "text",
text: `## Skill: ${skill.name} (v${skill.version})\n\n${sv.content}\n\n---\nTo install this skill for persistent use: call skill_install(skillId="${resolvedSkillId}")`,
}],
details: {
skillId: skill.id,
name: skill.name,
version: skill.version,
status: skill.status,
installed: skill.installed,
},
};
}),
},
{ name: "skill_get" },
);
// ─── Tool: skill_install ───
api.registerTool(
{
name: "skill_install",
label: "Install Skill",
description:
"Install a learned skill into the agent workspace so it becomes permanently available. " +
"After installation, the skill will be loaded automatically in future sessions.",
parameters: Type.Object({
skillId: Type.String({ description: "The skill_id to install" }),
}),
execute: trackTool("skill_install", async (_toolCallId: any, params: any) => {
const { skillId } = params as { skillId: string };
ctx.log.debug(`skill_install called for skill=${skillId}`);
const result = skillInstaller.install(skillId);
const skill = store.getSkill(skillId);
if (skill) telemetry.trackSkillInstalled(skill.name);
return {
content: [{ type: "text", text: result.message }],
details: result,
};
}),
},
{ name: "skill_install" },
);
// ─── Tool: memory_viewer ───
const viewerPort = (pluginCfg as any).viewerPort ?? 18799;
api.registerTool(
{
name: "memory_viewer",
label: "Open Memory Viewer",
description:
"Show the MemOS Memory Viewer URL. Call this when the user asks how to view, browse, manage, " +
"or access their stored memories, or asks where the memory dashboard is. " +
"Returns the URL the user can open in their browser.",
parameters: Type.Object({}),
execute: trackTool("memory_viewer", async () => {
ctx.log.debug(`memory_viewer called`);
telemetry.trackViewerOpened();
const url = `http://127.0.0.1:${viewerPort}`;
return {
content: [
{
type: "text",
text: [
`MemOS Memory Viewer: ${url}`,
"",
"Open this URL in your browser to:",
"- Browse all stored memories with a clean timeline view",
"- Semantic search (powered by your embedding model)",
"- Create, edit, and delete memories",
"- Filter by session, role, and time range",
"",
"First visit requires setting a password to protect your data.",
].join("\n"),
},
],
details: { viewerUrl: url },
};
}),
},
{ name: "memory_viewer" },
);
// ─── Tool: memory_write_public ───
api.registerTool(
{
name: "memory_write_public",
label: "Write Public Memory",
description:
"Write a piece of information to public memory. Public memories are visible to all agents during memory_search. " +
"Use this for shared knowledge, team decisions, or cross-agent coordination information.",
parameters: Type.Object({
content: Type.String({ description: "The content to write to public memory" }),
summary: Type.Optional(Type.String({ description: "Optional short summary of the content" })),
}),
execute: trackTool("memory_write_public", async (_toolCallId: any, params: any) => {
const { content: writeContent, summary: writeSummary } = params as { content: string; summary?: string };
if (!writeContent || !writeContent.trim()) {
return { content: [{ type: "text", text: "Content cannot be empty." }] };
}
const { v4: uuidv4 } = require("uuid");
const now = Date.now();
const chunkId = uuidv4();
const chunkSummary = writeSummary ?? writeContent;
store.insertChunk({
id: chunkId,
sessionKey: "public",
turnId: `public-${now}`,
seq: 0,
role: "assistant",
content: writeContent.trim(),
kind: "paragraph",
summary: chunkSummary,
embedding: null,
taskId: null,
skillId: null,
owner: "public",
dedupStatus: "active",
dedupTarget: null,
dedupReason: null,
mergeCount: 0,
lastHitAt: null,
mergeHistory: "[]",
createdAt: now,
updatedAt: now,
});
try {
const [emb] = await embedder.embed([chunkSummary]);
if (emb) store.upsertEmbedding(chunkId, emb);
} catch (err) {
api.logger.warn(`memos-local: public memory embedding failed: ${err}`);
}
return {
content: [{ type: "text", text: `Public memory written successfully (id: ${chunkId}).` }],
details: { chunkId, owner: "public" },
};
}),
},
{ name: "memory_write_public" },
);
// ─── Tool: skill_search ───
api.registerTool(
{
name: "skill_search",
label: "Skill Search",
description:
"Search available skills by natural language. Searches your own skills, public skills, or both. " +
"Use when you need a capability or guide and don't have a matching skill at hand.",
parameters: Type.Object({
query: Type.String({ description: "Natural language description of the needed skill" }),
scope: Type.Optional(Type.String({ description: "Search scope: 'mix' (default, self + public), 'self' (own only), 'public' (public only)" })),
}),
execute: trackTool("skill_search", async (_toolCallId: any, params: any) => {
const { query: skillQuery, scope: rawScope } = params as { query: string; scope?: string };
const scope = (rawScope === "self" || rawScope === "public") ? rawScope : "mix";
const currentOwner = `agent:${currentAgentId}`;
const hits = await engine.searchSkills(skillQuery, scope as any, currentOwner);
if (hits.length === 0) {
return {
content: [{ type: "text", text: `No relevant skills found for: "${skillQuery}" (scope: ${scope})` }],
details: { query: skillQuery, scope, hits: [] },
};
}
const text = hits.map((h, i) =>
`${i + 1}. [${h.name}] ${h.description}${h.visibility === "public" ? " (public)" : ""}`,
).join("\n");
return {
content: [{ type: "text", text: `Found ${hits.length} skills:\n\n${text}` }],
details: { query: skillQuery, scope, hits },
};
}),
},
{ name: "skill_search" },
);
// ─── Tool: skill_publish ───
api.registerTool(
{
name: "skill_publish",
label: "Publish Skill",
description: "Make a skill public so other agents can discover and install it via skill_search.",
parameters: Type.Object({
skillId: Type.String({ description: "The skill ID to publish" }),
}),
execute: trackTool("skill_publish", async (_toolCallId: any, params: any) => {
const { skillId: pubSkillId } = params as { skillId: string };
const skill = store.getSkill(pubSkillId);
if (!skill) {
return { content: [{ type: "text", text: `Skill not found: ${pubSkillId}` }] };
}
store.setSkillVisibility(pubSkillId, "public");
return {
content: [{ type: "text", text: `Skill "${skill.name}" is now public.` }],
details: { skillId: pubSkillId, name: skill.name, visibility: "public" },
};
}),
},
{ name: "skill_publish" },
);
// ─── Tool: skill_unpublish ───
api.registerTool(
{
name: "skill_unpublish",
label: "Unpublish Skill",
description: "Make a skill private. Other agents will no longer be able to discover it.",
parameters: Type.Object({
skillId: Type.String({ description: "The skill ID to unpublish" }),
}),
execute: trackTool("skill_unpublish", async (_toolCallId: any, params: any) => {
const { skillId: unpubSkillId } = params as { skillId: string };
const skill = store.getSkill(unpubSkillId);
if (!skill) {
return { content: [{ type: "text", text: `Skill not found: ${unpubSkillId}` }] };
}
store.setSkillVisibility(unpubSkillId, "private");
return {
content: [{ type: "text", text: `Skill "${skill.name}" is now private.` }],
details: { skillId: unpubSkillId, name: skill.name, visibility: "private" },
};
}),
},
{ name: "skill_unpublish" },
);
// ─── Auto-recall: inject relevant memories before agent starts ───
api.on("before_agent_start", async (event: { prompt?: string; messages?: unknown[] }, hookCtx?: { agentId?: string; sessionKey?: string }) => {
if (!allowPromptInjection) return {};
if (!event.prompt || event.prompt.length < 3) return;
const recallAgentId = hookCtx?.agentId ?? "main";
currentAgentId = recallAgentId;
const recallOwnerFilter = [`agent:${recallAgentId}`, "public"];
ctx.log.info(`auto-recall: agentId=${recallAgentId} (from hookCtx)`);
const recallT0 = performance.now();
let recallQuery = "";
try {
const rawPrompt = event.prompt;
ctx.log.debug(`auto-recall: rawPrompt="${rawPrompt.slice(0, 300)}"`);
let query = rawPrompt;
const senderTag = "Sender (untrusted metadata):";
const senderPos = rawPrompt.indexOf(senderTag);
if (senderPos !== -1) {
const afterSender = rawPrompt.slice(senderPos);
const fenceStart = afterSender.indexOf("```json");
const fenceEnd = fenceStart >= 0 ? afterSender.indexOf("```\n", fenceStart + 7) : -1;
if (fenceEnd > 0) {
query = afterSender.slice(fenceEnd + 4).replace(/^\s*\n/, "").trim();
} else {
const firstDblNl = afterSender.indexOf("\n\n");
if (firstDblNl > 0) {
query = afterSender.slice(firstDblNl + 2).trim();
}
}
}
query = stripInboundMetadata(query);
query = query.replace(/<[^>]+>/g, "").trim();
recallQuery = query;
if (query.length < 2) {
ctx.log.debug("auto-recall: extracted query too short, skipping");
return;
}
ctx.log.debug(`auto-recall: query="${query.slice(0, 80)}"`);
const result = await engine.search({ query, maxResults: 10, minScore: 0.45, ownerFilter: recallOwnerFilter });
if (result.hits.length === 0) {
ctx.log.debug("auto-recall: no candidates found");
const dur = performance.now() - recallT0;
store.recordToolCall("memory_search", dur, true);
store.recordApiLog("memory_search", { type: "auto_recall", query }, JSON.stringify({ candidates: [], filtered: [] }), dur, true);
if (query.length > 50) {
const noRecallHint =
"## Memory system — ACTION REQUIRED\n\n" +
"Auto-recall found no results for a long query. " +
"You MUST call `memory_search` now with a shortened query (2-5 key words) before answering. " +
"Do NOT skip this step. Do NOT answer without searching first.";
return { prependContext: noRecallHint };
}
return;
}
const candidates = result.hits.map((h, i) => ({
index: i + 1,
role: h.source.role,
content: (h.original_excerpt ?? "").slice(0, 300),
time: h.source.ts ? new Date(h.source.ts).toISOString().slice(0, 16) : "",
}));
let filteredHits = result.hits;
let sufficient = false;
const filterResult = await summarizer.filterRelevant(query, candidates);
if (filterResult !== null) {
sufficient = filterResult.sufficient;
if (filterResult.relevant.length > 0) {
const indexSet = new Set(filterResult.relevant);
filteredHits = result.hits.filter((_, i) => indexSet.has(i + 1));
} else {
ctx.log.debug("auto-recall: LLM filter returned no relevant hits");
const dur = performance.now() - recallT0;
store.recordToolCall("memory_search", dur, true);
store.recordApiLog("memory_search", { type: "auto_recall", query }, JSON.stringify({
candidates: result.hits.map(h => ({ score: h.score, role: h.source.role, summary: h.summary, content: h.original_excerpt })),
filtered: []
}), dur, true);
if (query.length > 50) {
const noRecallHint =
"## Memory system — ACTION REQUIRED\n\n" +
"Auto-recall found no relevant results for a long query. " +
"You MUST call `memory_search` now with a shortened query (2-5 key words) before answering. " +
"Do NOT skip this step. Do NOT answer without searching first.";
return { prependContext: noRecallHint };
}
return;
}
}
const beforeDedup = filteredHits.length;
filteredHits = deduplicateHits(filteredHits);
ctx.log.debug(`auto-recall: ${result.hits.length} → ${beforeDedup} relevant → ${filteredHits.length} after dedup, sufficient=${sufficient}`);
const lines = filteredHits.map((h, i) => {
const excerpt = h.original_excerpt;
const parts: string[] = [`${i + 1}. [${h.source.role}]`];
if (excerpt) parts.push(` ${excerpt}`);
parts.push(` chunkId="${h.ref.chunkId}"`);
if (h.taskId) {
const task = store.getTask(h.taskId);
if (task && task.status !== "skipped") {
parts.push(` task_id="${h.taskId}"`);
}
}
return parts.join("\n");
});
const hasTask = filteredHits.some((h) => {
if (!h.taskId) return false;
const t = store.getTask(h.taskId);
return t && t.status !== "skipped";
});
const tips: string[] = [];
if (hasTask) {
tips.push("- A hit has `task_id` → call `task_summary(taskId=\"...\")` to get the full task context (steps, code, results)");
tips.push("- A task may have a reusable guide → call `skill_get(taskId=\"...\")` to retrieve the experience/skill");
}
tips.push("- Need more surrounding dialogue → call `memory_timeline(chunkId=\"...\")` to expand context around a hit");
const tipsText = "\n\nAvailable follow-up tools:\n" + tips.join("\n");
const contextParts = [
"## User's conversation history (from memory system)",
"",
"IMPORTANT: The following are facts from previous conversations with this user.",
"You MUST treat these as established knowledge and use them directly when answering.",
"Do NOT say you don't know or don't have information if the answer is in these memories.",
"",
lines.join("\n\n"),
];
if (tipsText) contextParts.push(tipsText);
const context = contextParts.join("\n");
const recallDur = performance.now() - recallT0;
store.recordToolCall("memory_search", recallDur, true);
store.recordApiLog("memory_search", { type: "auto_recall", query }, JSON.stringify({
candidates: result.hits.map(h => ({ score: h.score, role: h.source.role, summary: h.summary, content: h.original_excerpt })),
filtered: filteredHits.map(h => ({ score: h.score, role: h.source.role, summary: h.summary, content: h.original_excerpt }))
}), recallDur, true);
telemetry.trackAutoRecall(filteredHits.length, recallDur);
ctx.log.info(`auto-recall: returning prependContext (${context.length} chars), sufficient=${sufficient}`);
if (!sufficient) {
const searchHint =
"\n\nIf these memories don't fully answer the question, " +
"call `memory_search` with a shorter or rephrased query to find more.";
return { prependContext: context + searchHint };
}
return {
prependContext: context,
};
} catch (err) {
const dur = performance.now() - recallT0;
store.recordToolCall("memory_search", dur, false);
try { store.recordApiLog("memory_search", { type: "auto_recall", query: recallQuery }, `error: ${String(err)}`, dur, false); } catch (_) { /* best-effort */ }
ctx.log.warn(`auto-recall failed: ${String(err)}`);
}
});
// ─── Auto-capture: write conversation to memory after each agent turn ───
// Track how many messages we've already processed per session to avoid
// re-processing the entire conversation history on every agent_end.
// On first encounter after restart, skip all existing messages (they were
// already processed before the restart) and only capture future increments.
const sessionMsgCursor = new Map();
api.on("agent_end", async (event: any, hookCtx?: { agentId?: string; sessionKey?: string; sessionId?: string }) => {
if (!event.success || !event.messages || event.messages.length === 0) return;
try {
const captureAgentId = hookCtx?.agentId ?? "main";
currentAgentId = captureAgentId;
const captureOwner = `agent:${captureAgentId}`;
const sessionKey = hookCtx?.sessionKey ?? "default";
ctx.log.info(`agent_end: agentId=${captureAgentId} sessionKey=${sessionKey} (from hookCtx)`);
const cursorKey = `${sessionKey}::${captureAgentId}`;
const allMessages = event.messages;
if (!sessionMsgCursor.has(cursorKey)) {
// First time seeing this session after (re)start — find the last
// user message and capture from there (current turn only).
let lastUserIdx = -1;
for (let i = allMessages.length - 1; i >= 0; i--) {
const m = allMessages[i] as Record;
if (m && m.role === "user") { lastUserIdx = i; break; }
}
const initCursor = lastUserIdx >= 0 ? lastUserIdx : allMessages.length;
sessionMsgCursor.set(cursorKey, initCursor);
ctx.log.debug(`agent_end: first encounter session=${sessionKey} agent=${captureAgentId}, initialized cursor=${initCursor} (total=${allMessages.length})`);
}
let cursor = sessionMsgCursor.get(cursorKey)!;
// Session was reset — cursor exceeds current message count
if (cursor > allMessages.length) cursor = 0;
if (cursor >= allMessages.length) return;
const newMessages = allMessages.slice(cursor);
sessionMsgCursor.set(cursorKey, allMessages.length);
ctx.log.debug(`agent_end: session=${sessionKey} total=${allMessages.length} cursor=${cursor} new=${newMessages.length}`);
const raw: Array<{ role: string; content: string; toolName?: string }> = [];
for (const msg of newMessages) {
if (!msg || typeof msg !== "object") continue;
const m = msg as Record;
const role = m.role as string;
if (role !== "user" && role !== "assistant" && role !== "tool") continue;
let text = "";
if (typeof m.content === "string") {
text = m.content;
} else if (Array.isArray(m.content)) {
for (const block of m.content) {
if (!block || typeof block !== "object") continue;
const b = block as Record;
if (b.type === "text" && typeof b.text === "string") {
text += b.text + "\n";
} else if (typeof b.content === "string") {
text += b.content + "\n";
} else if (typeof b.text === "string") {
text += b.text + "\n";
}
}
}
text = text.trim();
if (!text) continue;
// Strip injected prefix and OpenClaw metadata wrapper
// to store only the user's actual input
if (role === "user") {
const mcTag = "";
const mcEnd = "";
const mcIdx = text.indexOf(mcTag);
if (mcIdx !== -1) {
const endIdx = text.indexOf(mcEnd);
if (endIdx !== -1) {
text = text.slice(endIdx + mcEnd.length).trim();
}
}
// Strip OpenClaw metadata envelope:
// "Sender (untrusted metadata):\n```json\n{...}\n```\n\n[timestamp] actual message"
const senderIdx = text.indexOf("Sender (untrusted metadata):");
if (senderIdx !== -1) {
const afterSender = text.slice(senderIdx);
const fenceEnd = afterSender.indexOf("```\n", afterSender.indexOf("```json"));
if (fenceEnd > 0) {
const afterFence = afterSender.slice(fenceEnd + 4).replace(/^\s*\n/, "");
if (afterFence.trim().length >= 2) text = afterFence.trim();
} else {
const firstDblNl = afterSender.indexOf("\n\n");
if (firstDblNl > 0) {
const tail = afterSender.slice(firstDblNl + 2).trim();
if (tail.length >= 2) text = tail;
}
}
}
// Strip timestamp prefix like "[Thu 2026-03-05 15:23 GMT+8] "
text = text.replace(/^\[.*?\]\s*/, "").trim();
if (!text) continue;
}
const toolName = role === "tool"
? (m.name as string) ?? (m.toolName as string) ?? (m.tool_call_id ? "unknown" : undefined)
: undefined;
raw.push({ role, content: text, toolName });
}
// Merge consecutive assistant messages into one (OpenClaw may send reply in multiple chunks)
const msgs: Array<{ role: string; content: string; toolName?: string }> = [];
for (let i = 0; i < raw.length; i++) {
const curr = raw[i];
if (curr.role !== "assistant") {
msgs.push(curr);
continue;
}
let merged = curr.content;
while (i + 1 < raw.length && raw[i + 1].role === "assistant") {
i++;
merged = merged + "\n\n" + raw[i].content;
}
msgs.push({ role: "assistant", content: merged.trim() });
}
if (msgs.length === 0) return;
const turnId = `${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
const captured = captureMessages(msgs, sessionKey, turnId, evidenceTag, ctx.log, captureOwner);
if (captured.length > 0) {
worker.enqueue(captured);
telemetry.trackMemoryIngested(captured.length);
}
} catch (err) {
api.logger.warn(`memos-local: capture failed: ${String(err)}`);
}
});
// ─── Memory Viewer (web UI) ───
const viewer = new ViewerServer({
store,
embedder,
port: viewerPort,
log: ctx.log,
dataDir: stateDir,
ctx,
});
// ─── Service lifecycle ───
api.registerService({
id: "memos-local-openclaw-plugin",
start: async () => {
try {
const viewerUrl = await viewer.start();
api.logger.info(`memos-local: started (embedding: ${embedder.provider})`);
api.logger.info(`╔══════════════════════════════════════════╗`);
api.logger.info(`║ MemOS Memory Viewer ║`);
api.logger.info(`║ → ${viewerUrl.padEnd(37)}║`);
api.logger.info(`║ Open in browser to manage memories ║`);
api.logger.info(`╚══════════════════════════════════════════╝`);
api.logger.info(`memos-local: password reset token: ${viewer.getResetToken()}`);
api.logger.info(`memos-local: forgot password? Use the reset token on the login page.`);
skillEvolver.recoverOrphanedTasks().then((count) => {
if (count > 0) api.logger.info(`memos-local: recovered ${count} orphaned skill tasks`);
}).catch((err) => {
api.logger.warn(`memos-local: skill recovery failed: ${err}`);
});
} catch (err) {
api.logger.warn(`memos-local: viewer failed to start: ${err}`);
api.logger.info(`memos-local: started (embedding: ${embedder.provider})`);
}
telemetry.trackPluginStarted(
ctx.config.embedding?.provider ?? "local",
ctx.config.summarizer?.provider ?? "none",
);
},
stop: async () => {
await telemetry.shutdown();
viewer.stop();
store.close();
api.logger.info("memos-local: stopped");
},
});
},
};
export default memosLocalPlugin;
================================================
FILE: apps/memos-local-openclaw/openclaw.plugin.json
================================================
{
"id": "memos-local-openclaw-plugin",
"name": "MemOS Local Memory",
"description": "Full-write local conversation memory with hybrid search (RRF + MMR + recency). Provides memory_search, memory_get, task_summary, memory_timeline, memory_viewer for layered retrieval.",
"kind": "memory",
"version": "0.1.11",
"skills": [
"skill/memos-memory-guide"
],
"homepage": "https://github.com/MemTensor/MemOS/tree/main/apps/memos-local-openclaw",
"configSchema": {
"type": "object",
"additionalProperties": true,
"description": "Configuration for MemOS Local Memory. Use Raw mode to edit embedding/summarizer settings.",
"properties": {
"viewerPort": {
"type": "number",
"description": "Memory Viewer HTTP port (default 18799)"
}
}
},
"requirements": {
"node": ">=18.0.0",
"openclaw": ">=2026.2.0"
},
"setup": {
"postInstall": "node scripts/postinstall.cjs",
"notes": [
"After install, add to ~/.openclaw/openclaw.json: plugins.slots.memory = \"memos-local-openclaw-plugin\"",
"Set agents.defaults.memorySearch.enabled = false to disable OpenClaw's built-in memory",
"Restart the gateway: openclaw gateway stop && openclaw gateway start",
"Memory Viewer will be available at http://127.0.0.1:18799",
"If better-sqlite3 fails to build, ensure you have C++ build tools: xcode-select --install (macOS) or build-essential (Linux)"
]
}
}
================================================
FILE: apps/memos-local-openclaw/package.json
================================================
{
"name": "@memtensor/memos-local-openclaw-plugin",
"version": "1.0.3",
"description": "MemOS Local memory plugin for OpenClaw \u2014 full-write, hybrid-recall, progressive retrieval",
"type": "module",
"main": "index.ts",
"types": "dist/index.d.ts",
"files": [
"index.ts",
"src",
"dist",
"skill",
"prebuilds",
"scripts/postinstall.cjs",
"openclaw.plugin.json",
"README.md",
".env.example"
],
"openclaw": {
"id": "memos-local-openclaw-plugin",
"extensions": [
"./index.ts"
],
"skills": [
"skill/memos-memory-guide"
],
"installDependencies": true
},
"scripts": {
"build": "tsc",
"dev": "tsc --watch",
"lint": "eslint src --ext .ts",
"test": "vitest run",
"test:watch": "vitest",
"test:accuracy": "tsx scripts/run-accuracy-test.ts",
"postinstall": "node scripts/postinstall.cjs",
"prepublishOnly": "npm run build"
},
"keywords": [
"openclaw",
"plugin",
"memory",
"memos",
"rag"
],
"license": "MIT",
"engines": {
"node": ">=18.0.0"
},
"dependencies": {
"@huggingface/transformers": "^3.8.0",
"@sinclair/typebox": "^0.34.48",
"better-sqlite3": "^12.6.2",
"puppeteer": "^24.38.0",
"semver": "^7.7.4",
"uuid": "^10.0.0"
},
"devDependencies": {
"@types/better-sqlite3": "^7.6.12",
"@types/node": "^22.10.0",
"@types/semver": "^7.7.1",
"@types/uuid": "^10.0.0",
"tsx": "^4.21.0",
"typescript": "^5.7.0",
"vitest": "^2.1.0"
}
}
================================================
FILE: apps/memos-local-openclaw/plugin-impl.ts
================================================
/**
* MemOS Local Plugin Implementation — loaded by index.ts after ensuring deps.
*/
import type { OpenClawPluginApi } from "openclaw/plugin-sdk";
import { Type } from "@sinclair/typebox";
import { buildContext } from "./src/config";
import { SqliteStore } from "./src/storage/sqlite";
import { Embedder } from "./src/embedding";
import { IngestWorker } from "./src/ingest/worker";
import { RecallEngine } from "./src/recall/engine";
import { captureMessages } from "./src/capture";
import { DEFAULTS } from "./src/types";
import { ViewerServer } from "./src/viewer/server";
import { SkillEvolver } from "./src/skill/evolver";
function ownerFilterFor(agentId: string | undefined): string[] {
const resolvedAgentId = agentId && agentId.trim().length > 0 ? agentId : "main";
return [`agent:${resolvedAgentId}`, "public"];
}
const pluginConfigSchema = {
type: "object" as const,
additionalProperties: true,
properties: {
embedding: {
type: "object" as const,
properties: {
provider: { type: "string" as const },
endpoint: { type: "string" as const },
apiKey: { type: "string" as const },
model: { type: "string" as const },
},
},
summarizer: {
type: "object" as const,
properties: {
provider: { type: "string" as const },
endpoint: { type: "string" as const },
apiKey: { type: "string" as const },
model: { type: "string" as const },
temperature: { type: "number" as const },
},
},
viewerPort: { type: "number" as const },
telemetry: {
type: "object" as const,
description: "Anonymous usage analytics (opt-out). No memory content or personal data is ever sent.",
properties: {
enabled: {
type: "boolean" as const,
description: "Enable anonymous telemetry (default: true). Set to false to opt-out.",
},
},
},
},
};
const memosLocalPlugin = {
id: "memos-local-openclaw-plugin",
name: "MemOS Local Memory",
description:
"Full-write local conversation memory with hybrid search (RRF + MMR + recency). " +
"Provides memory_search, memory_timeline, memory_get for progressive recall.",
kind: "memory" as const,
configSchema: pluginConfigSchema,
register(api: OpenClawPluginApi) {
const pluginCfg = (api.pluginConfig ?? {}) as Record;
const stateDir = api.resolvePath("~/.openclaw");
const ctx = buildContext(stateDir, process.cwd(), pluginCfg as any, {
debug: (msg: string) => api.logger.info(`[debug] ${msg}`),
info: (msg: string) => api.logger.info(msg),
warn: (msg: string) => api.logger.warn(msg),
error: (msg: string) => api.logger.warn(`[error] ${msg}`),
});
const store = new SqliteStore(ctx.config.storage!.dbPath!, ctx.log);
const embedder = new Embedder(ctx.config.embedding, ctx.log);
const worker = new IngestWorker(store, embedder, ctx);
const engine = new RecallEngine(store, embedder, ctx);
const evidenceTag = ctx.config.capture?.evidenceWrapperTag ?? DEFAULTS.evidenceWrapperTag;
api.logger.info(`memos-local: initialized (db: ${ctx.config.storage!.dbPath})`);
// ─── Tool: memory_search ───
api.registerTool(
{
name: "memory_search",
label: "Memory Search",
description:
"Search stored conversation memories. Returns summary, original_excerpt (evidence), score, and ref. " +
"Default: top 6, minScore 0.45. Increase maxResults to 12/20 or lower minScore to 0.35 if needed.",
parameters: Type.Object({
query: Type.String({ description: "Natural language search query" }),
maxResults: Type.Optional(Type.Number({ description: "Max results (default 6, max 20)" })),
minScore: Type.Optional(Type.Number({ description: "Min score 0-1 (default 0.45, floor 0.35)" })),
}),
async execute(_toolCallId, params, context) {
const { query, maxResults, minScore } = params as {
query: string;
maxResults?: number;
minScore?: number;
};
const agentId = (context as any)?.agentId ?? "main";
const ownerFilter = ownerFilterFor(agentId);
const result = await engine.search({ query, maxResults, minScore, ownerFilter });
if (result.hits.length === 0) {
return {
content: [{ type: "text", text: result.meta.note ?? "No relevant memories found." }],
details: { meta: result.meta },
};
}
const roleLabel = (r: string) => r === "user" ? "[USER said]" : r === "assistant" ? "[ASSISTANT replied]" : r === "tool" ? "[TOOL returned]" : `[${r.toUpperCase()}]`;
const text = result.hits
.map(
(h, i) =>
`${i + 1}. ${roleLabel(h.source.role)} [score=${h.score}] ${h.summary}\n Evidence: ${h.original_excerpt.slice(0, 200)}`,
)
.join("\n\n");
return {
content: [
{
type: "text",
text: `Found ${result.hits.length} memories (minScore=${result.meta.usedMinScore}):\n\n${text}`,
},
],
details: {
hits: result.hits.map((h) => ({
role: h.source.role,
summary: h.summary,
original_excerpt: h.original_excerpt,
ref: h.ref,
score: h.score,
source: h.source,
})),
meta: result.meta,
},
};
},
},
{ name: "memory_search" },
);
// ─── Tool: memory_timeline ───
api.registerTool(
{
name: "memory_timeline",
label: "Memory Timeline",
description:
"Get neighboring context around a memory ref. Use after memory_search to expand context.",
parameters: Type.Object({
sessionKey: Type.String({ description: "From search hit ref.sessionKey" }),
chunkId: Type.String({ description: "From search hit ref.chunkId" }),
turnId: Type.String({ description: "From search hit ref.turnId" }),
seq: Type.Number({ description: "From search hit ref.seq" }),
window: Type.Optional(Type.Number({ description: "Context window ±N (default 2)" })),
}),
async execute(_toolCallId, params, context) {
const { sessionKey, chunkId, turnId, seq, window: win } = params as {
sessionKey: string;
chunkId: string;
turnId: string;
seq: number;
window?: number;
};
const agentId = (context as any)?.agentId ?? "main";
const ownerFilter = ownerFilterFor(agentId);
const w = win ?? DEFAULTS.timelineWindowDefault;
const anchorChunk = store.getChunkForOwners(chunkId, ownerFilter);
if (!anchorChunk) {
return {
content: [{ type: "text", text: "Timeline (0 entries):\n\n" }],
details: { entries: [], anchorRef: { sessionKey, chunkId, turnId, seq } },
};
}
const neighbors = store.getNeighborChunks(sessionKey, turnId, seq, w, ownerFilter);
const anchorTs = anchorChunk?.createdAt ?? 0;
const entries = neighbors.map((chunk) => {
let relation: "before" | "current" | "after" = "before";
if (chunk.id === chunkId) relation = "current";
else if (chunk.createdAt > anchorTs) relation = "after";
return {
relation,
role: chunk.role,
excerpt: chunk.content.slice(0, DEFAULTS.excerptMaxChars),
ts: chunk.createdAt,
};
});
const rl = (r: string) => r === "user" ? "USER" : r === "assistant" ? "ASSISTANT" : r.toUpperCase();
const text = entries
.map((e) => `[${e.relation}] ${rl(e.role)}: ${e.excerpt.slice(0, 150)}`)
.join("\n");
return {
content: [{ type: "text", text: `Timeline (${entries.length} entries):\n\n${text}` }],
details: { entries, anchorRef: { sessionKey, chunkId, turnId, seq } },
};
},
},
{ name: "memory_timeline" },
);
// ─── Tool: memory_get ───
api.registerTool(
{
name: "memory_get",
label: "Memory Get",
description:
"Get full original text of a memory chunk. Use to verify exact details from a search hit.",
parameters: Type.Object({
chunkId: Type.String({ description: "From search hit ref.chunkId" }),
maxChars: Type.Optional(
Type.Number({ description: `Max chars (default ${DEFAULTS.getMaxCharsDefault}, max ${DEFAULTS.getMaxCharsMax})` }),
),
}),
async execute(_toolCallId, params, context) {
const { chunkId, maxChars } = params as { chunkId: string; maxChars?: number };
const limit = Math.min(maxChars ?? DEFAULTS.getMaxCharsDefault, DEFAULTS.getMaxCharsMax);
const agentId = (context as any)?.agentId ?? "main";
const chunk = store.getChunkForOwners(chunkId, ownerFilterFor(agentId));
if (!chunk) {
return {
content: [{ type: "text", text: `Chunk not found: ${chunkId}` }],
details: { error: "not_found" },
};
}
const content = chunk.content.length > limit
? chunk.content.slice(0, limit) + "…"
: chunk.content;
const who = chunk.role === "user" ? "USER said" : chunk.role === "assistant" ? "ASSISTANT replied" : chunk.role === "tool" ? "TOOL returned" : chunk.role.toUpperCase();
return {
content: [{ type: "text", text: `[${who}] (session: ${chunk.sessionKey})\n\n${content}` }],
details: {
ref: { sessionKey: chunk.sessionKey, chunkId: chunk.id, turnId: chunk.turnId, seq: chunk.seq },
source: { ts: chunk.createdAt, role: chunk.role, sessionKey: chunk.sessionKey },
},
};
},
},
{ name: "memory_get" },
);
// ─── Tool: memory_viewer ───
const viewerPort = (pluginCfg as any).viewerPort ?? 18799;
api.registerTool(
{
name: "memory_viewer",
label: "Open Memory Viewer",
description:
"Open the MemOS Memory Viewer web dashboard. Returns the URL the user can open in their browser to visually browse, search, and manage all stored memories.",
parameters: Type.Object({}),
async execute() {
const url = `http://127.0.0.1:${viewerPort}`;
return {
content: [
{
type: "text",
text: [
`MemOS Memory Viewer: ${url}`,
"",
"Open this URL in your browser to:",
"- Browse all stored memories with a clean timeline view",
"- Semantic search (powered by your embedding model)",
"- Create, edit, and delete memories",
"- Filter by session, role, and time range",
"",
"First visit requires setting a password to protect your data.",
].join("\n"),
},
],
details: { viewerUrl: url },
};
},
},
{ name: "memory_viewer" },
);
// ─── Tool: memory_write_public ───
api.registerTool(
{
name: "memory_write_public",
label: "Write Public Memory",
description:
"Write a piece of information to public memory. Public memories are visible to all agents during memory_search. " +
"Use this for shared knowledge, team decisions, or cross-agent coordination information.",
parameters: Type.Object({
content: Type.String({ description: "The content to write to public memory" }),
summary: Type.Optional(Type.String({ description: "Optional short summary of the content" })),
}),
async execute(_toolCallId, params) {
const { content, summary } = params as { content: string; summary?: string };
if (!content || !content.trim()) {
return { content: [{ type: "text", text: "Content cannot be empty." }] };
}
const { v4: uuidv4 } = await import("uuid");
const now = Date.now();
const chunkId = uuidv4();
const chunkSummary = summary ?? content.slice(0, 200);
store.insertChunk({
id: chunkId,
sessionKey: "public",
turnId: `public-${now}`,
seq: 0,
role: "assistant",
content: content.trim(),
kind: "paragraph",
summary: chunkSummary,
embedding: null,
taskId: null,
skillId: null,
owner: "public",
dedupStatus: "active",
dedupTarget: null,
dedupReason: null,
mergeCount: 0,
lastHitAt: null,
mergeHistory: "[]",
createdAt: now,
updatedAt: now,
});
try {
const [emb] = await embedder.embed([chunkSummary]);
if (emb) store.upsertEmbedding(chunkId, emb);
} catch (err) {
api.logger.warn(`memos-local: public memory embedding failed: ${err}`);
}
return {
content: [{ type: "text", text: `Public memory written successfully (id: ${chunkId}).` }],
details: { chunkId, owner: "public" },
};
},
},
{ name: "memory_write_public" },
);
// ─── Tool: skill_search ───
api.registerTool(
{
name: "skill_search",
label: "Skill Search",
description:
"Search available skills by natural language. Searches your own skills, public skills, or both. " +
"Use when you need a capability or guide and don't have a matching skill at hand.",
parameters: Type.Object({
query: Type.String({ description: "Natural language description of the needed skill" }),
scope: Type.Optional(Type.String({ description: "Search scope: 'mix' (default, self + public), 'self' (own only), 'public' (public only)" })),
}),
async execute(_toolCallId, params, context) {
const { query, scope: rawScope } = params as { query: string; scope?: string };
const scope = (rawScope === "self" || rawScope === "public") ? rawScope : "mix";
const agentId = (context as any)?.agentId ?? "main";
const currentOwner = `agent:${agentId}`;
const hits = await engine.searchSkills(query, scope as any, currentOwner);
if (hits.length === 0) {
return {
content: [{ type: "text", text: `No relevant skills found for: "${query}" (scope: ${scope})` }],
details: { query, scope, hits: [] },
};
}
const text = hits.map((h, i) =>
`${i + 1}. [${h.name}] ${h.description.slice(0, 150)}${h.visibility === "public" ? " (public)" : ""}`,
).join("\n");
return {
content: [{ type: "text", text: `Found ${hits.length} skills:\n\n${text}` }],
details: { query, scope, hits },
};
},
},
{ name: "skill_search" },
);
// ─── Tool: skill_publish ───
api.registerTool(
{
name: "skill_publish",
label: "Publish Skill",
description: "Make a skill public so other agents can discover and install it via skill_search.",
parameters: Type.Object({
skillId: Type.String({ description: "The skill ID to publish" }),
}),
async execute(_toolCallId, params) {
const { skillId } = params as { skillId: string };
const skill = store.getSkill(skillId);
if (!skill) {
return { content: [{ type: "text", text: `Skill not found: ${skillId}` }] };
}
store.setSkillVisibility(skillId, "public");
return {
content: [{ type: "text", text: `Skill "${skill.name}" is now public.` }],
details: { skillId, name: skill.name, visibility: "public" },
};
},
},
{ name: "skill_publish" },
);
// ─── Tool: skill_unpublish ───
api.registerTool(
{
name: "skill_unpublish",
label: "Unpublish Skill",
description: "Make a skill private. Other agents will no longer be able to discover it.",
parameters: Type.Object({
skillId: Type.String({ description: "The skill ID to unpublish" }),
}),
async execute(_toolCallId, params) {
const { skillId } = params as { skillId: string };
const skill = store.getSkill(skillId);
if (!skill) {
return { content: [{ type: "text", text: `Skill not found: ${skillId}` }] };
}
store.setSkillVisibility(skillId, "private");
return {
content: [{ type: "text", text: `Skill "${skill.name}" is now private.` }],
details: { skillId, name: skill.name, visibility: "private" },
};
},
},
{ name: "skill_unpublish" },
);
// ─── Auto-capture: write conversation to memory after each agent turn ───
api.on("agent_end", async (event) => {
if (!event.success || !event.messages || event.messages.length === 0) return;
try {
const agentId = (event as any).agentId ?? "main";
const owner = `agent:${agentId}`;
const msgs: Array<{ role: string; content: string; toolName?: string }> = [];
for (const msg of event.messages) {
if (!msg || typeof msg !== "object") continue;
const m = msg as Record;
const role = m.role as string;
if (role !== "user" && role !== "assistant" && role !== "tool") continue;
let text = "";
if (typeof m.content === "string") {
text = m.content;
} else if (Array.isArray(m.content)) {
for (const block of m.content) {
if (block && typeof block === "object" && (block as any).type === "text") {
text += (block as any).text + "\n";
}
}
}
if (!text.trim()) continue;
const toolName = role === "tool"
? (m.name as string) ?? (m.toolName as string) ?? (m.tool_call_id ? "unknown" : undefined)
: undefined;
msgs.push({ role, content: text.trim(), toolName });
}
if (msgs.length === 0) return;
const sessionKey = (event as any).sessionKey ?? "default";
const turnId = `${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
const captured = captureMessages(msgs, sessionKey, turnId, evidenceTag, ctx.log, owner);
if (captured.length > 0) {
worker.enqueue(captured);
}
} catch (err) {
api.logger.warn(`memos-local: capture failed: ${String(err)}`);
}
});
// ─── Memory Viewer (web UI) ───
const viewer = new ViewerServer({
store,
embedder,
port: viewerPort,
log: ctx.log,
dataDir: stateDir,
ctx,
});
// ─── Service lifecycle ───
api.registerService({
id: "memos-local-openclaw-plugin",
start: async () => {
try {
const viewerUrl = await viewer.start();
api.logger.info(`memos-local: started (embedding: ${embedder.provider})`);
api.logger.info(`╔══════════════════════════════════════════╗`);
api.logger.info(`║ MemOS Memory Viewer ║`);
api.logger.info(`║ → ${viewerUrl.padEnd(37)}║`);
api.logger.info(`║ Open in browser to manage memories ║`);
api.logger.info(`╚══════════════════════════════════════════╝`);
api.logger.info(`memos-local: password reset token: ${viewer.getResetToken()}`);
api.logger.info(`memos-local: forgot password? Use the reset token on the login page.`);
const skillEnabled = ctx.config.skillEvolution?.enabled ?? DEFAULTS.skillEvolutionEnabled;
if (skillEnabled) {
const recallEngine = new RecallEngine(store, embedder, ctx);
const evolver = new SkillEvolver(store, recallEngine, ctx, embedder);
evolver.recoverOrphanedTasks().then((count) => {
if (count > 0) api.logger.info(`memos-local: recovered ${count} orphaned skill tasks`);
}).catch((err) => {
api.logger.warn(`memos-local: skill recovery failed: ${err}`);
});
}
} catch (err) {
api.logger.warn(`memos-local: viewer failed to start: ${err}`);
api.logger.info(`memos-local: started (embedding: ${embedder.provider})`);
}
},
stop: async () => {
viewer.stop();
await worker.flush();
store.close();
api.logger.info("memos-local: stopped");
},
});
},
};
export default memosLocalPlugin;
================================================
FILE: apps/memos-local-openclaw/scripts/mock-skills.ts
================================================
/**
* Mock skill data for testing the Skills viewer page.
* Run: npx tsx scripts/mock-skills.ts
*/
import Database from "better-sqlite3";
import { v4 as uuid } from "uuid";
import * as path from "path";
import * as fs from "fs";
import * as os from "os";
const dbPath = path.join(os.homedir(), ".openclaw", "memos-local", "memos.db");
console.log(`Opening DB: ${dbPath}`);
const db = new Database(dbPath);
db.exec(`
CREATE TABLE IF NOT EXISTS skills (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
description TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 1,
status TEXT NOT NULL DEFAULT 'active',
tags TEXT NOT NULL DEFAULT '[]',
source_type TEXT NOT NULL DEFAULT 'task',
dir_path TEXT NOT NULL DEFAULT '',
installed INTEGER NOT NULL DEFAULT 0,
quality_score REAL,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_skills_status ON skills(status);
CREATE INDEX IF NOT EXISTS idx_skills_name ON skills(name);
CREATE TABLE IF NOT EXISTS skill_versions (
id TEXT PRIMARY KEY,
skill_id TEXT NOT NULL REFERENCES skills(id),
version INTEGER NOT NULL,
content TEXT NOT NULL,
changelog TEXT NOT NULL DEFAULT '',
upgrade_type TEXT NOT NULL DEFAULT 'create',
source_task_id TEXT,
metrics TEXT NOT NULL DEFAULT '{}',
quality_score REAL,
created_at INTEGER NOT NULL,
UNIQUE(skill_id, version)
);
CREATE INDEX IF NOT EXISTS idx_skill_versions_skill ON skill_versions(skill_id);
CREATE TABLE IF NOT EXISTS task_skills (
task_id TEXT NOT NULL,
skill_id TEXT NOT NULL REFERENCES skills(id),
relation TEXT NOT NULL DEFAULT 'generated_from',
version_at INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL,
PRIMARY KEY (task_id, skill_id)
);
`);
console.log("Ensured skill tables exist");
// Migrate quality_score columns if missing
try {
const skillCols = db.prepare("PRAGMA table_info(skills)").all() as Array<{ name: string }>;
if (!skillCols.some(c => c.name === "quality_score")) {
db.exec("ALTER TABLE skills ADD COLUMN quality_score REAL");
console.log("Migrated: added quality_score to skills");
}
const vCols = db.prepare("PRAGMA table_info(skill_versions)").all() as Array<{ name: string }>;
if (!vCols.some(c => c.name === "quality_score")) {
db.exec("ALTER TABLE skill_versions ADD COLUMN quality_score REAL");
console.log("Migrated: added quality_score to skill_versions");
}
if (!vCols.some(c => c.name === "change_summary")) {
db.exec("ALTER TABLE skill_versions ADD COLUMN change_summary TEXT NOT NULL DEFAULT ''");
console.log("Migrated: added change_summary to skill_versions");
}
} catch (e) { console.log("Migration check:", e); }
const now = Date.now();
const skills = [
{
id: uuid(),
name: "docker-node-deploy",
description: "如何将 Node.js 应用部署到 Docker 容器中。当用户需要容器化部署、Dockerfile 编写、镜像构建、端口映射、多阶段构建,或任何将 Node 应用打包为 Docker 容器的场景时,使用此技能。",
version: 2,
status: "active",
tags: JSON.stringify(["docker", "node.js", "deployment", "devops"]),
sourceType: "task",
dirPath: path.join(os.homedir(), ".openclaw", "skills-store", "docker-node-deploy"),
installed: 1,
createdAt: now - 7 * 86400000,
updatedAt: now - 2 * 86400000,
content_v1: `---
name: "docker-node-deploy"
description: "如何将 Node.js 应用部署到 Docker 容器中。当用户需要容器化部署、Dockerfile 编写、镜像构建、端口映射、多阶段构建,或任何将 Node 应用打包为 Docker 容器的场景时,使用此技能。"
metadata: { "openclaw": { "emoji": "🐳" } }
---
# Docker Node.js 部署指南
将 Node.js 应用安全、高效地打包为 Docker 容器并运行。
## 适用场景
- 需要将 Node.js 后端服务容器化
- 需要编写优化的 Dockerfile(多阶段构建)
- 需要处理端口映射、环境变量注入
- 需要在 CI/CD 中构建 Docker 镜像
## 步骤
### 1. 创建 Dockerfile(多阶段构建)
\`\`\`dockerfile
# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
RUN npm run build
# Production stage
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package.json ./
EXPOSE 3000
CMD ["node", "dist/index.js"]
\`\`\`
为什么用多阶段:减少最终镜像大小约 60%,不包含开发依赖和源码。
### 2. 创建 .dockerignore
\`\`\`
node_modules
.git
*.md
.env
\`\`\`
### 3. 构建和运行
\`\`\`bash
docker build -t my-app:latest .
docker run -d -p 3000:3000 --name my-app --env-file .env.production my-app:latest
\`\`\`
## 踩坑指南
**错误方式**:直接 \`COPY . .\` 不用 .dockerignore → 镜像巨大,包含 node_modules 和 .git
**正确方式**:分层 COPY,先 package.json 再源码,利用 Docker 缓存层
**错误方式**:用 \`npm install\` 而不是 \`npm ci\` → 可能安装不一致的依赖
**正确方式**:生产构建必须用 \`npm ci\`
## 关键配置
- Alpine 镜像比 Debian 小约 100MB
- 多阶段构建减少 60% 镜像体积
- \`--omit=dev\` 不安装开发依赖
## 注意事项
- Node.js >= 18 推荐使用 node:20-alpine
- 确保 .env 文件不被打包进镜像
- 健康检查建议添加 HEALTHCHECK 指令
`,
content_v2: `---
name: "docker-node-deploy"
description: "如何将 Node.js 应用部署到 Docker 容器中。当用户需要容器化部署、Dockerfile 编写、镜像构建、端口映射、多阶段构建,或任何将 Node 应用打包为 Docker 容器的场景时,使用此技能。"
metadata: { "openclaw": { "emoji": "🐳" } }
---
# Docker Node.js 部署指南
将 Node.js 应用安全、高效地打包为 Docker 容器并运行。
## 适用场景
- 需要将 Node.js 后端服务容器化
- 需要编写优化的 Dockerfile(多阶段构建)
- 需要处理端口映射、环境变量注入
- 需要在 CI/CD 中构建 Docker 镜像
- 需要配置健康检查和优雅停机
## 步骤
### 1. 创建 Dockerfile(多阶段构建 + 健康检查)
\`\`\`dockerfile
# Build stage
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
RUN npm run build
# Production stage
FROM node:20-alpine
RUN apk add --no-cache curl
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY package.json ./
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s CMD curl -f http://localhost:3000/health || exit 1
CMD ["node", "dist/index.js"]
\`\`\`
为什么用多阶段:减少最终镜像大小约 60%,不包含开发依赖和源码。
v2 新增:HEALTHCHECK 指令,确保容器健康监测。
### 2. 创建 .dockerignore
\`\`\`
node_modules
.git
*.md
.env
dist
\`\`\`
### 3. 构建和运行
\`\`\`bash
docker build -t my-app:latest .
docker run -d -p 3000:3000 --name my-app --env-file .env.production --restart unless-stopped my-app:latest
\`\`\`
v2 新增:\`--restart unless-stopped\` 确保容器异常退出后自动重启。
## 踩坑指南
**错误方式**:直接 \`COPY . .\` 不用 .dockerignore → 镜像巨大
**正确方式**:分层 COPY + .dockerignore
**错误方式**:用 \`npm install\` 而不是 \`npm ci\`
**正确方式**:生产构建必须用 \`npm ci\`
**错误方式**:不加 --restart 策略 → 容器挂了不自动恢复
**正确方式**:添加 \`--restart unless-stopped\`
## 注意事项
- Node.js >= 18 推荐使用 node:20-alpine
- 确保 .env 文件不被打包进镜像
- 添加 /health 端点用于容器健康检查
`,
},
{
id: uuid(),
name: "sqlite-migration-pattern",
description: "SQLite 数据库 schema 迁移的最佳实践。当需要给 SQLite 数据库添加新列、新表、修改索引,或处理向后兼容的 schema 变更时使用此技能。适用于任何使用 better-sqlite3 或类似驱动的 Node.js 项目。",
version: 1,
status: "active",
tags: JSON.stringify(["sqlite", "migration", "database", "schema"]),
sourceType: "task",
dirPath: path.join(os.homedir(), ".openclaw", "skills-store", "sqlite-migration-pattern"),
installed: 0,
createdAt: now - 3 * 86400000,
updatedAt: now - 3 * 86400000,
content_v1: `---
name: "sqlite-migration-pattern"
description: "SQLite 数据库 schema 迁移的最佳实践。当需要给 SQLite 数据库添加新列、新表、修改索引,或处理向后兼容的 schema 变更时使用此技能。"
metadata: { "openclaw": { "emoji": "🗄️" } }
---
# SQLite Migration 最佳实践
在 Node.js + better-sqlite3 项目中安全地进行 schema 迁移。
## 适用场景
- 需要给现有表添加新列
- 需要创建新的关联表
- 需要保持向后兼容(旧数据库能自动迁移)
## 步骤
### 1. 添加新列的安全方式
\`\`\`typescript
private migrateNewColumn(): void {
const cols = this.db.prepare("PRAGMA table_info(my_table)").all() as Array<{ name: string }>;
if (!cols.some(c => c.name === "new_column")) {
this.db.exec("ALTER TABLE my_table ADD COLUMN new_column TEXT");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_my_table_new ON my_table(new_column)");
this.log.info("Migrated: added new_column to my_table");
}
}
\`\`\`
为什么要先检查:ALTER TABLE ADD COLUMN 如果列已存在会报错,PRAGMA table_info 是安全的幂等检查。
### 2. 创建新表(幂等)
\`\`\`sql
CREATE TABLE IF NOT EXISTS new_table (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
created_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_new_table_name ON new_table(name);
\`\`\`
### 3. 在 migrate() 中按顺序调用
\`\`\`typescript
migrate(): void {
this.createCoreTables();
this.migrateV2Columns();
this.migrateV3Tables();
}
\`\`\`
## 踩坑指南
**错误方式**:直接执行 ALTER TABLE 不检查 → 第二次启动会报错
**正确方式**:用 PRAGMA table_info 检查列是否存在
**错误方式**:在 CREATE TABLE 后忘记加 IF NOT EXISTS
**正确方式**:始终使用 IF NOT EXISTS
## 注意事项
- SQLite 不支持 DROP COLUMN(3.35.0+ 才支持)
- SQLite 不支持 ALTER COLUMN,只能 ADD COLUMN
- 迁移顺序很重要:先建表、再加列、再加索引
`,
},
{
id: uuid(),
name: "typescript-strict-config",
description: "TypeScript 严格模式配置与常见类型错误修复指南。当遇到 TS 编译错误、需要配置 tsconfig.json 严格选项、处理类型推断问题、或从 JS 迁移到 TS 时使用此技能。",
version: 1,
status: "draft",
tags: JSON.stringify(["typescript", "config", "strict-mode", "type-safety"]),
sourceType: "task",
dirPath: path.join(os.homedir(), ".openclaw", "skills-store", "typescript-strict-config"),
installed: 1,
createdAt: now - 5 * 86400000,
updatedAt: now - 5 * 86400000,
content_v1: `---
name: "typescript-strict-config"
description: "TypeScript 严格模式配置与常见类型错误修复指南。当遇到 TS 编译错误、需要配置 tsconfig.json 严格选项、处理类型推断问题时使用此技能。"
metadata: { "openclaw": { "emoji": "📘" } }
---
# TypeScript 严格模式指南
配置 TypeScript 严格模式,修复常见类型错误。
## 适用场景
- 新项目需要配置 tsconfig.json
- 启用 strict 模式后出现大量报错
- 处理 null/undefined 类型安全
## 推荐配置
\`\`\`json
{
"compilerOptions": {
"target": "ES2022",
"module": "node16",
"moduleResolution": "node16",
"strict": true,
"noUncheckedIndexedAccess": true,
"exactOptionalPropertyTypes": false,
"outDir": "./dist",
"rootDir": "./src",
"declaration": true,
"sourceMap": true,
"esModuleInterop": true,
"skipLibCheck": true
}
}
\`\`\`
## 常见错误和修复
### Object is possibly undefined
\`\`\`typescript
// 错误
const val = obj.prop.nested;
// 正确
const val = obj.prop?.nested;
// 或断言(确定有值时)
const val = obj.prop!.nested;
\`\`\`
### Type X is not assignable to type Y
\`\`\`typescript
// 错误:直接用 as any
const row = db.prepare("...").get() as any;
// 正确:定义接口
interface MyRow { id: string; name: string }
const row = db.prepare("...").get() as MyRow | undefined;
\`\`\`
## 注意事项
- \`strict: true\` 等于同时启用 7 个 strict 子选项
- \`skipLibCheck: true\` 可以大幅加快编译速度
- 从 JS 迁移时建议先用 \`strict: false\`,逐步启用
`,
},
];
// Get some existing task IDs for linking
const existingTasks = db.prepare("SELECT id, title FROM tasks WHERE status = 'completed' ORDER BY started_at DESC LIMIT 5").all() as Array<{ id: string; title: string }>;
console.log(`Found ${existingTasks.length} existing tasks for linking`);
for (const skill of skills) {
// Create skill-store directory
fs.mkdirSync(skill.dirPath, { recursive: true });
fs.writeFileSync(path.join(skill.dirPath, "SKILL.md"), (skill as any).content_v2 || skill.content_v1, "utf-8");
// Create sample scripts/references for docker skill
if (skill.name === "docker-node-deploy") {
const scriptsDir = path.join(skill.dirPath, "scripts");
fs.mkdirSync(scriptsDir, { recursive: true });
fs.writeFileSync(path.join(scriptsDir, "build.sh"), "#!/bin/bash\ndocker build -t my-app:latest .\n", "utf-8");
fs.writeFileSync(path.join(scriptsDir, "run.sh"), "#!/bin/bash\ndocker run -d -p 3000:3000 --name my-app --restart unless-stopped my-app:latest\n", "utf-8");
const refsDir = path.join(skill.dirPath, "references");
fs.mkdirSync(refsDir, { recursive: true });
fs.writeFileSync(path.join(refsDir, "docker-best-practices.md"), "# Docker Best Practices\n\n- Use multi-stage builds\n- Use .dockerignore\n- Use HEALTHCHECK\n", "utf-8");
const evalsDir = path.join(skill.dirPath, "evals");
fs.mkdirSync(evalsDir, { recursive: true });
fs.writeFileSync(path.join(evalsDir, "evals.json"), JSON.stringify({
skill_name: "docker-node-deploy",
evals: [
{ id: 1, prompt: "帮我把 Node.js 项目打包成 Docker 镜像", expectations: ["使用多阶段构建", "包含 .dockerignore"] },
{ id: 2, prompt: "我的 Docker 容器经常崩溃,怎么自动重启", expectations: ["使用 --restart 策略", "添加 HEALTHCHECK"] },
],
}, null, 2), "utf-8");
}
// Insert skill
const qualityScore = skill.name === 'docker-node-deploy' ? 8.5 : skill.name === 'sqlite-migration-pattern' ? 7.2 : 5.0;
db.prepare(`INSERT OR REPLACE INTO skills (id, name, description, version, status, tags, source_type, dir_path, installed, quality_score, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(
skill.id, skill.name, skill.description, skill.version, skill.status,
skill.tags, skill.sourceType, skill.dirPath, skill.installed, qualityScore,
skill.createdAt, skill.updatedAt,
);
// Insert version 1
const v1Summary = skill.name === 'docker-node-deploy'
? '首次从 Docker 部署 Node.js 的实际执行记录中提炼生成。涵盖多阶段构建 Dockerfile 编写、.dockerignore 配置、镜像构建与运行命令。记录了生产环境常见的错误方式(如直接 COPY 不用 .dockerignore、npm install 替代 npm ci)及其正确做法。包含 2 个辅助脚本(build.sh、run.sh)和 2 个测试用例。'
: skill.name === 'sqlite-migration-pattern'
? '从实际项目中 SQLite schema 迁移的执行经验提炼而成。覆盖了添加新列的安全检查方式(PRAGMA table_info)、CREATE TABLE IF NOT EXISTS 的幂等性保证、以及按顺序组织 migrate 函数的最佳实践。避免了常见的"第二次启动报错"和"忘加 IF NOT EXISTS"问题。'
: '从 TypeScript 严格模式配置的实践中提炼。包含推荐的 tsconfig.json 配置项、Object is possibly undefined 和 Type X is not assignable to Y 的典型修复方案。适合从 JS 迁移到 TS 或首次启用 strict 模式的项目。质量评分偏低,标记为 draft 待改进。';
db.prepare(`INSERT OR REPLACE INTO skill_versions (id, skill_id, version, content, changelog, change_summary, upgrade_type, source_task_id, metrics, quality_score, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(
uuid(), skill.id, 1, skill.content_v1,
`Initial generation`,
v1Summary,
"create", existingTasks[0]?.id ?? null, "{}", qualityScore,
skill.createdAt,
);
// Insert version 2 if exists
if ((skill as any).content_v2 && skill.version >= 2) {
const v2Summary = '新增容器健康检查(HEALTHCHECK)和自动重启策略(--restart unless-stopped),解决了容器异常退出后无法自动恢复的问题。同时优化了 .dockerignore,增加了 dist 目录排除。这些改进来自一次实际的生产环境排障——容器频繁 crash 但无人察觉,加入 HEALTHCHECK 后运维平台可以自动检测并重启不健康的容器。';
db.prepare(`INSERT OR REPLACE INTO skill_versions (id, skill_id, version, content, changelog, change_summary, upgrade_type, source_task_id, metrics, quality_score, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(
uuid(), skill.id, 2, (skill as any).content_v2,
`Added HEALTHCHECK, --restart policy, optimized .dockerignore`,
v2Summary,
"extend", existingTasks[1]?.id ?? null,
JSON.stringify({ dimensions: ["more_robust", "new_scenario"], confidence: 0.85 }), qualityScore,
skill.updatedAt,
);
}
// Link to existing tasks
if (existingTasks.length > 0) {
const taskIdx = skills.indexOf(skill) % existingTasks.length;
db.prepare(`INSERT OR REPLACE INTO task_skills (task_id, skill_id, relation, version_at, created_at)
VALUES (?, ?, ?, ?, ?)`).run(
existingTasks[taskIdx].id, skill.id, "generated_from", 1, skill.createdAt,
);
}
console.log(` ✓ Skill "${skill.name}" v${skill.version} (installed=${skill.installed})`);
}
db.close();
console.log(`\nDone! Inserted ${skills.length} mock skills.`);
================================================
FILE: apps/memos-local-openclaw/scripts/postinstall.cjs
================================================
#!/usr/bin/env node
"use strict";
const { spawnSync } = require("child_process");
const path = require("path");
const fs = require("fs");
const RESET = "\x1b[0m";
const GREEN = "\x1b[32m";
const YELLOW = "\x1b[33m";
const RED = "\x1b[31m";
const CYAN = "\x1b[36m";
const BOLD = "\x1b[1m";
const DIM = "\x1b[2m";
function log(msg) { console.log(` ${CYAN}[memos-local]${RESET} ${msg}`); }
function warn(msg) { console.log(` ${YELLOW}⚠ [memos-local]${RESET} ${msg}`); }
function ok(msg) { console.log(` ${GREEN}✔ [memos-local]${RESET} ${msg}`); }
function fail(msg) { console.log(` ${RED}✖ [memos-local]${RESET} ${msg}`); }
function phase(n, title) {
console.log(`\n${CYAN}${BOLD} ─── Phase ${n}: ${title} ───${RESET}\n`);
}
const pluginDir = path.resolve(__dirname, "..");
console.log(`
${CYAN}${BOLD}┌──────────────────────────────────────────────────┐
│ MemOS Local Memory — postinstall setup │
└──────────────────────────────────────────────────┘${RESET}
`);
log(`Plugin dir: ${DIM}${pluginDir}${RESET}`);
log(`Node: ${process.version} Platform: ${process.platform}-${process.arch}`);
/* ═══════════════════════════════════════════════════════════
* Pre-phase: Clean stale build artifacts on upgrade
* When openclaw re-installs a new version over an existing
* extensions dir, old dist/node_modules can conflict.
* We nuke them so npm install gets a clean slate, but
* preserve user data (.env, data/).
* ═══════════════════════════════════════════════════════════ */
function cleanStaleArtifacts() {
const isExtensionsDir = pluginDir.includes(path.join(".openclaw", "extensions"));
if (!isExtensionsDir) return;
const pkgPath = path.join(pluginDir, "package.json");
if (!fs.existsSync(pkgPath)) return;
let installedVer = "unknown";
try {
const pkg = JSON.parse(fs.readFileSync(pkgPath, "utf-8"));
installedVer = pkg.version || "unknown";
} catch { /* ignore */ }
const markerPath = path.join(pluginDir, ".installed-version");
let prevVer = "";
try { prevVer = fs.readFileSync(markerPath, "utf-8").trim(); } catch { /* first install */ }
if (prevVer === installedVer) {
log(`Version unchanged (${installedVer}), skipping artifact cleanup.`);
return;
}
if (prevVer) {
log(`Upgrade detected: ${DIM}${prevVer}${RESET} → ${GREEN}${installedVer}${RESET}`);
} else {
log(`Fresh install: ${GREEN}${installedVer}${RESET}`);
}
const dirsToClean = ["dist", "node_modules"];
let cleaned = 0;
for (const dir of dirsToClean) {
const full = path.join(pluginDir, dir);
if (fs.existsSync(full)) {
try {
fs.rmSync(full, { recursive: true, force: true });
ok(`Cleaned stale ${dir}/`);
cleaned++;
} catch (e) {
warn(`Could not remove ${dir}/: ${e.message}`);
}
}
}
const filesToClean = ["package-lock.json"];
for (const f of filesToClean) {
const full = path.join(pluginDir, f);
if (fs.existsSync(full)) {
try { fs.unlinkSync(full); ok(`Removed stale ${f}`); cleaned++; } catch { /* ignore */ }
}
}
try { fs.writeFileSync(markerPath, installedVer + "\n", "utf-8"); } catch { /* ignore */ }
if (cleaned > 0) {
ok(`Cleaned ${cleaned} stale artifact(s). Fresh install will follow.`);
}
}
try {
cleanStaleArtifacts();
} catch (e) {
warn(`Artifact cleanup error: ${e.message}`);
}
/* ═══════════════════════════════════════════════════════════
* Phase 0: Ensure all dependencies are installed
* ═══════════════════════════════════════════════════════════ */
function ensureDependencies() {
phase(0, "检测核心依赖 / Check core dependencies");
const coreDeps = ["@sinclair/typebox", "uuid", "@huggingface/transformers"];
const missing = [];
for (const dep of coreDeps) {
try {
require.resolve(dep, { paths: [pluginDir] });
log(` ${dep} ${GREEN}✔${RESET}`);
} catch {
missing.push(dep);
log(` ${dep} ${RED}✖ missing${RESET}`);
}
}
if (missing.length === 0) {
ok("All core dependencies present.");
return;
}
warn(`Missing ${missing.length} dependencies: ${BOLD}${missing.join(", ")}${RESET}`);
log("Running: npm install --omit=dev ...");
const startMs = Date.now();
const result = spawnSync("npm", ["install", "--omit=dev"], {
cwd: pluginDir,
stdio: "pipe",
shell: true,
timeout: 120_000,
});
const elapsed = ((Date.now() - startMs) / 1000).toFixed(1);
const stderr = (result.stderr || "").toString().trim();
if (result.status === 0) {
ok(`Dependencies installed successfully (${elapsed}s).`);
} else {
fail(`npm install exited with code ${result.status} (${elapsed}s).`);
if (stderr) warn(`stderr: ${stderr.slice(0, 300)}`);
warn("Some features may not work. Try running manually:");
warn(` cd ${pluginDir} && npm install --omit=dev`);
}
}
try {
ensureDependencies();
} catch (e) {
warn(`Dependency check error: ${e.message}`);
}
/* ═══════════════════════════════════════════════════════════
* Phase 1: Clean up legacy plugin versions
* ═══════════════════════════════════════════════════════════ */
function cleanupLegacy() {
phase(1, "清理旧版本插件 / Clean up legacy plugins");
const home = process.env.HOME || process.env.USERPROFILE || "";
if (!home) { log("Cannot determine HOME directory, skipping."); return; }
const ocHome = path.join(home, ".openclaw");
if (!fs.existsSync(ocHome)) { log("No ~/.openclaw directory found, skipping."); return; }
const extDir = path.join(ocHome, "extensions");
if (!fs.existsSync(extDir)) { log("No extensions directory found, skipping."); return; }
const legacyDirs = [
path.join(extDir, "memos-local"),
path.join(extDir, "memos-lite"),
path.join(extDir, "memos-lite-openclaw-plugin"),
path.join(extDir, "node_modules", "@memtensor", "memos-lite-openclaw-plugin"),
];
let cleaned = 0;
for (const dir of legacyDirs) {
if (fs.existsSync(dir)) {
try {
fs.rmSync(dir, { recursive: true, force: true });
ok(`Removed legacy dir: ${DIM}${dir}${RESET}`);
cleaned++;
} catch (e) {
warn(`Could not remove ${dir}: ${e.message}`);
}
}
}
const cfgPath = path.join(ocHome, "openclaw.json");
if (fs.existsSync(cfgPath)) {
try {
const raw = fs.readFileSync(cfgPath, "utf-8");
const cfg = JSON.parse(raw);
const entries = cfg?.plugins?.entries;
if (entries) {
const oldKeys = ["memos-local", "memos-lite", "memos-lite-openclaw-plugin"];
let cfgChanged = false;
for (const oldKey of oldKeys) {
if (entries[oldKey]) {
const oldEntry = entries[oldKey];
if (!entries["memos-local-openclaw-plugin"]) {
entries["memos-local-openclaw-plugin"] = oldEntry;
log(`Migrated config: ${DIM}${oldKey}${RESET} → ${GREEN}memos-local-openclaw-plugin${RESET}`);
}
delete entries[oldKey];
cfgChanged = true;
ok(`Removed legacy config key: ${DIM}${oldKey}${RESET}`);
}
}
const newEntry = entries["memos-local-openclaw-plugin"];
if (newEntry && typeof newEntry.source === "string") {
const oldSource = newEntry.source;
if (oldSource.includes("memos-lite") || (oldSource.includes("memos-local") && !oldSource.includes("memos-local-openclaw-plugin"))) {
newEntry.source = oldSource
.replace(/memos-lite-openclaw-plugin/g, "memos-local-openclaw-plugin")
.replace(/memos-lite/g, "memos-local-openclaw-plugin")
.replace(/\/memos-local\//g, "/memos-local-openclaw-plugin/")
.replace(/\/memos-local$/g, "/memos-local-openclaw-plugin");
if (newEntry.source !== oldSource) {
log(`Updated source path: ${DIM}${oldSource}${RESET} → ${GREEN}${newEntry.source}${RESET}`);
cfgChanged = true;
}
}
}
const slots = cfg?.plugins?.slots;
if (slots && typeof slots.memory === "string") {
const oldSlotNames = ["memos-local", "memos-lite", "memos-lite-openclaw-plugin"];
if (oldSlotNames.includes(slots.memory)) {
log(`Migrated plugins.slots.memory: ${DIM}${slots.memory}${RESET} → ${GREEN}memos-local-openclaw-plugin${RESET}`);
slots.memory = "memos-local-openclaw-plugin";
cfgChanged = true;
}
}
if (cfgChanged) {
const backup = cfgPath + ".bak-" + Date.now();
fs.copyFileSync(cfgPath, backup);
fs.writeFileSync(cfgPath, JSON.stringify(cfg, null, 2) + "\n", "utf-8");
ok(`Config updated. Backup: ${DIM}${backup}${RESET}`);
} else {
log("No legacy config entries found.");
}
}
} catch (e) {
warn(`Could not update openclaw.json: ${e.message}`);
}
}
if (cleaned > 0) {
ok(`Legacy cleanup done: ${cleaned} old dir(s) removed.`);
} else {
ok("No legacy plugin directories found. Clean.");
}
}
try {
cleanupLegacy();
} catch (e) {
warn(`Legacy cleanup error: ${e.message}`);
}
/* ═══════════════════════════════════════════════════════════
* Phase 2: Install bundled skill (memos-memory-guide)
* ═══════════════════════════════════════════════════════════ */
function installBundledSkill() {
phase(2, "安装记忆技能 / Install memory skill");
const home = process.env.HOME || process.env.USERPROFILE || "";
if (!home) { warn("Cannot determine HOME directory, skipping skill install."); return; }
const skillSrc = path.join(pluginDir, "skill", "memos-memory-guide", "SKILL.md");
if (!fs.existsSync(skillSrc)) {
warn("Bundled SKILL.md not found, skipping skill install.");
return;
}
let pluginVersion = "0.0.0";
try {
const pkg = JSON.parse(fs.readFileSync(path.join(pluginDir, "package.json"), "utf-8"));
pluginVersion = pkg.version || pluginVersion;
} catch { /* ignore */ }
const skillContent = fs.readFileSync(skillSrc, "utf-8");
const targets = [
path.join(home, ".openclaw", "workspace", "skills", "memos-memory-guide"),
path.join(home, ".openclaw", "skills", "memos-memory-guide"),
];
const meta = JSON.stringify({ ownerId: "memos-local-openclaw-plugin", slug: "memos-memory-guide", version: pluginVersion, publishedAt: Date.now() });
const origin = JSON.stringify({ version: 1, registry: "memos-local-openclaw-plugin", slug: "memos-memory-guide", installedVersion: pluginVersion, installedAt: Date.now() });
for (const dest of targets) {
try {
fs.mkdirSync(dest, { recursive: true });
fs.writeFileSync(path.join(dest, "SKILL.md"), skillContent, "utf-8");
fs.writeFileSync(path.join(dest, "_meta.json"), meta, "utf-8");
const clawHubDir = path.join(dest, ".clawhub");
fs.mkdirSync(clawHubDir, { recursive: true });
fs.writeFileSync(path.join(clawHubDir, "origin.json"), origin, "utf-8");
ok(`Skill installed → ${DIM}${dest}${RESET}`);
} catch (e) {
warn(`Could not install skill to ${dest}: ${e.message}`);
}
}
// Register in skills-lock.json so OpenClaw Dashboard can discover it
const lockPath = path.join(home, ".openclaw", "workspace", "skills-lock.json");
try {
let lockData = { version: 1, skills: {} };
if (fs.existsSync(lockPath)) {
lockData = JSON.parse(fs.readFileSync(lockPath, "utf-8"));
}
if (!lockData.skills) lockData.skills = {};
lockData.skills["memos-memory-guide"] = { source: "memos-local-openclaw-plugin", sourceType: "plugin", computedHash: "" };
fs.writeFileSync(lockPath, JSON.stringify(lockData, null, 2) + "\n", "utf-8");
ok("Registered in skills-lock.json");
} catch (e) {
warn(`Could not update skills-lock.json: ${e.message}`);
}
}
try {
installBundledSkill();
} catch (e) {
warn(`Skill install error: ${e.message}`);
}
/* ═══════════════════════════════════════════════════════════
* Phase 3: Verify better-sqlite3 native module
* ═══════════════════════════════════════════════════════════ */
phase(3, "检查 better-sqlite3 原生模块 / Check native module");
const sqliteModulePath = path.join(pluginDir, "node_modules", "better-sqlite3");
function findSqliteBinding() {
const candidates = [
path.join(sqliteModulePath, "build", "Release", "better_sqlite3.node"),
path.join(sqliteModulePath, "build", "better_sqlite3.node"),
path.join(sqliteModulePath, "build", "Debug", "better_sqlite3.node"),
];
const prebuildDir = path.join(sqliteModulePath, "prebuilds");
if (fs.existsSync(prebuildDir)) {
try {
const platformDir = `${process.platform}-${process.arch}`;
const pbDir = path.join(prebuildDir, platformDir);
if (fs.existsSync(pbDir)) {
const files = fs.readdirSync(pbDir).filter(f => f.endsWith(".node"));
for (const f of files) candidates.push(path.join(pbDir, f));
}
} catch { /* ignore */ }
}
for (const c of candidates) {
if (fs.existsSync(c)) return c;
}
return null;
}
function sqliteBindingsExist() {
const found = findSqliteBinding();
if (found) {
log(`Native binding found: ${DIM}${found}${RESET}`);
return true;
}
return false;
}
if (sqliteBindingsExist()) {
ok("better-sqlite3 is ready.");
console.log(`
${GREEN}${BOLD} ┌──────────────────────────────────────────────────┐
│ ✔ Setup complete! │
│ │
│ Restart gateway: │
│ ${CYAN}openclaw gateway stop && openclaw gateway start${GREEN} │
└──────────────────────────────────────────────────┘${RESET}
`);
process.exit(0);
} else {
warn("better-sqlite3 native bindings not found in plugin dir.");
log(`Searched in: ${DIM}${sqliteModulePath}/build/${RESET}`);
log("Running: npm rebuild better-sqlite3 (may take 30-60s)...");
}
const startMs = Date.now();
const result = spawnSync("npm", ["rebuild", "better-sqlite3"], {
cwd: pluginDir,
stdio: "pipe",
shell: true,
timeout: 180_000,
});
const elapsed = ((Date.now() - startMs) / 1000).toFixed(1);
const stdout = (result.stdout || "").toString().trim();
const stderr = (result.stderr || "").toString().trim();
if (stdout) log(`rebuild output: ${DIM}${stdout.slice(0, 500)}${RESET}`);
if (stderr) warn(`rebuild stderr: ${DIM}${stderr.slice(0, 500)}${RESET}`);
if (result.status === 0) {
if (sqliteBindingsExist()) {
ok(`better-sqlite3 rebuilt successfully (${elapsed}s).`);
console.log(`
${GREEN}${BOLD} ┌──────────────────────────────────────────────────┐
│ ✔ Setup complete! │
│ │
│ Restart gateway: │
│ ${CYAN}openclaw gateway stop && openclaw gateway start${GREEN} │
└──────────────────────────────────────────────────┘${RESET}
`);
process.exit(0);
} else {
fail(`Rebuild completed but bindings still missing (${elapsed}s).`);
fail(`Looked in: ${sqliteModulePath}/build/`);
}
} else {
fail(`Rebuild failed with exit code ${result.status} (${elapsed}s).`);
}
console.log(`
${YELLOW}${BOLD} ╔══════════════════════════════════════════════════════════════╗
║ ✖ better-sqlite3 native module build failed ║
╠══════════════════════════════════════════════════════════════╣${RESET}
${YELLOW} ║${RESET} ${YELLOW}║${RESET}
${YELLOW} ║${RESET} This plugin requires C/C++ build tools to compile ${YELLOW}║${RESET}
${YELLOW} ║${RESET} the SQLite native module on first install. ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${BOLD}Install build tools:${RESET} ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${CYAN}macOS:${RESET} xcode-select --install ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${CYAN}Ubuntu:${RESET} sudo apt install build-essential python3 ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${CYAN}Windows:${RESET} npm install -g windows-build-tools ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${BOLD}Then retry:${RESET} ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${GREEN}cd ${pluginDir}${RESET}
${YELLOW} ║${RESET} ${GREEN}npm rebuild better-sqlite3${RESET} ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${GREEN}openclaw gateway stop && openclaw gateway start${RESET} ${YELLOW}║${RESET}
${YELLOW} ║${RESET} ${YELLOW}║${RESET}
${YELLOW}${BOLD} ╚══════════════════════════════════════════════════════════════╝${RESET}
`);
process.exit(0);
================================================
FILE: apps/memos-local-openclaw/scripts/refresh-skill.ts
================================================
#!/usr/bin/env npx tsx
/**
* Regenerate a skill's SKILL.md from its source task, using updated prompts.
* Usage: npx tsx scripts/refresh-skill.ts
*/
import { buildContext } from "../src/config";
import { SqliteStore } from "../src/storage/sqlite";
import { Embedder } from "../src/embedding";
import { RecallEngine } from "../src/recall/engine";
import { SkillGenerator } from "../src/skill/generator";
const skillId = process.argv[2];
if (!skillId) {
console.error("Usage: npx tsx scripts/refresh-skill.ts ");
process.exit(1);
}
import * as fs from "fs";
const home = process.env.HOME ?? "/tmp";
const stateDir = `${home}/.openclaw`;
const workspaceDir = `${home}/.openclaw/workspace`;
// Read plugin config from openclaw.json
let pluginConfig: Record | undefined;
try {
const oc = JSON.parse(fs.readFileSync(`${stateDir}/openclaw.json`, "utf-8"));
pluginConfig = oc?.plugins?.entries?.["memos-local"]?.config;
} catch {}
const ctx = buildContext(stateDir, workspaceDir, pluginConfig, {
info: (m: string) => console.log(`[INFO] ${m}`),
debug: (m: string) => console.log(`[DEBUG] ${m}`),
warn: (m: string) => console.warn(`[WARN] ${m}`),
error: (m: string) => console.error(`[ERROR] ${m}`),
});
const store = new SqliteStore(ctx.config.storage!.dbPath, ctx.log);
const embedder = new Embedder(ctx.config.embedding!, ctx.log);
const engine = new RecallEngine(store, embedder, ctx);
const generator = new SkillGenerator(store, engine, ctx);
const skill = store.getSkill(skillId);
if (!skill) {
console.error(`Skill not found: ${skillId}`);
process.exit(1);
}
// Find source task
const db = (store as any).db;
const versionRow = db.prepare(
"SELECT source_task_id FROM skill_versions WHERE skill_id = ? ORDER BY version DESC LIMIT 1"
).get(skillId) as { source_task_id: string } | undefined;
if (!versionRow?.source_task_id) {
console.error("No source task found for this skill");
process.exit(1);
}
const task = store.getTask(versionRow.source_task_id);
if (!task) {
console.error(`Task not found: ${versionRow.source_task_id}`);
process.exit(1);
}
const chunks = store.getChunksByTask(task.id);
console.log(`Regenerating skill "${skill.name}" from task "${task.title}" (${chunks.length} chunks)...`);
const evalResult = {
shouldGenerate: true,
reason: "refresh",
suggestedName: skill.name,
suggestedTags: JSON.parse(skill.tags || "[]"),
confidence: 0.9,
};
generator.generate(task, chunks, evalResult).then((newSkill) => {
console.log(`\nDone! Skill regenerated:`);
console.log(` Name: ${newSkill.name}`);
console.log(` Status: ${newSkill.status}`);
console.log(` Quality: ${newSkill.qualityScore}`);
console.log(` Dir: ${newSkill.dirPath}`);
store.close();
}).catch((err) => {
console.error("Failed:", err);
store.close();
process.exit(1);
});
================================================
FILE: apps/memos-local-openclaw/scripts/refresh-summaries.ts
================================================
import Database from "better-sqlite3";
import * as path from "path";
import * as os from "os";
import * as fs from "fs";
const TASK_SUMMARY_PROMPT = `You create a DETAILED task summary from a multi-turn conversation. This summary will be the ONLY record of this conversation, so it must preserve ALL important information.
CRITICAL LANGUAGE RULE: You MUST write in the SAME language as the user's messages. Chinese input → Chinese output. English input → English output. NEVER mix languages.
Output EXACTLY this structure:
📌 Title
A short, descriptive title (10-30 characters). Like a chat group name.
🎯 Goal
One sentence: what the user wanted to accomplish.
📋 Key Steps
- Describe each meaningful step in detail
- Include the ACTUAL content produced: code snippets, commands, config blocks, formulas, key paragraphs
- For code: include the function signature and core logic (up to ~30 lines per block), use fenced code blocks
- For configs: include the actual config values and structure
- For lists/instructions: include the actual items, not just "provided a list"
- Merge only truly trivial back-and-forth (like "ok" / "sure")
- Do NOT over-summarize: "provided a function" is BAD; show the actual function
✅ Result
What was the final outcome? Include the final version of any code/config/content produced.
💡 Key Details
- Decisions made, trade-offs discussed, caveats noted, alternative approaches mentioned
- Specific values: numbers, versions, thresholds, URLs, file paths, model names
- Omit this section only if there truly are no noteworthy details
RULES:
- This summary is a KNOWLEDGE BASE ENTRY, not a brief note. Be thorough.
- PRESERVE verbatim: code, commands, URLs, file paths, error messages, config values, version numbers, names, amounts
- DISCARD only: greetings, filler, the assistant explaining what it will do before doing it
- Replace secrets (API keys, tokens, passwords) with [REDACTED]
- Target length: 30-50% of the original conversation length. Longer conversations need longer summaries.
- Output summary only, no preamble.`;
function parseTitleFromSummary(summary: string): { title: string; body: string } {
const titleMatch = summary.match(/📌\s*(?:Title|标题)\s*\n(.+)/);
if (titleMatch) {
const title = titleMatch[1].trim().slice(0, 80);
const body = summary.replace(/📌\s*(?:Title|标题)\s*\n.+\n?/, "").trim();
return { title, body };
}
return { title: "", body: summary };
}
async function main() {
const configPath = path.join(os.homedir(), ".openclaw", "openclaw.json");
const config = JSON.parse(fs.readFileSync(configPath, "utf-8"));
const memosConfig = config.plugins?.entries?.["memos-local"]?.config
?? config.plugins?.configs?.["memos-local"]?.config;
const cfg = memosConfig?.summarizer;
if (!cfg) {
console.error("No summarizer config found");
process.exit(1);
}
const isAnthropic = cfg.provider === "anthropic"
|| cfg.endpoint?.toLowerCase().includes("anthropic");
console.log(`Summarizer: ${cfg.provider} / ${cfg.model}`);
let endpoint = cfg.endpoint.replace(/\/+$/, "");
if (isAnthropic) {
if (!endpoint.endsWith("/v1/messages") && !endpoint.endsWith("/messages")) {
endpoint += "/v1/messages";
}
} else {
if (!endpoint.endsWith("/chat/completions")) endpoint += "/chat/completions";
}
async function callLLM(text: string): Promise {
const headers: Record = isAnthropic
? {
"Content-Type": "application/json",
"x-api-key": cfg.apiKey,
"anthropic-version": "2023-06-01",
}
: {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
};
const body = isAnthropic
? JSON.stringify({
model: cfg.model,
temperature: 0.1,
max_tokens: 4096,
system: TASK_SUMMARY_PROMPT,
messages: [{ role: "user", content: text }],
})
: JSON.stringify({
model: cfg.model,
temperature: 0.1,
max_tokens: 4096,
messages: [
{ role: "system", content: TASK_SUMMARY_PROMPT },
{ role: "user", content: text },
],
});
const resp = await fetch(endpoint, {
method: "POST",
headers,
body,
signal: AbortSignal.timeout(60_000),
});
if (!resp.ok) {
const respBody = await resp.text();
throw new Error(`API ${resp.status}: ${respBody.slice(0, 200)}`);
}
const json = (await resp.json()) as any;
if (isAnthropic) {
return json.content?.find((c: any) => c.type === "text")?.text?.trim() ?? "";
}
return json.choices[0]?.message?.content?.trim() ?? "";
}
const db = new Database(
path.join(os.homedir(), ".openclaw", "memos-local", "memos.db"),
);
const tasks = db
.prepare("SELECT * FROM tasks WHERE status = 'completed' ORDER BY started_at DESC")
.all() as any[];
console.log(`\nRefreshing ${tasks.length} completed tasks...\n`);
for (const task of tasks) {
const chunks = db
.prepare("SELECT role, content FROM chunks WHERE task_id = ? ORDER BY created_at, seq")
.all(task.id) as any[];
if (chunks.length === 0) {
console.log(` SKIP (no chunks): ${task.title.slice(0, 40)}`);
continue;
}
const conv = chunks
.map((c: any) => `[${c.role === "user" ? "User" : c.role === "assistant" ? "Assistant" : c.role}]: ${c.content}`)
.join("\n\n");
const truncated =
conv.length > 15000
? conv.slice(0, 15000) + "\n\n[... truncated ...]"
: conv;
console.log(
` Processing: "${task.title.slice(0, 40)}..." (${chunks.length} chunks)`,
);
try {
const raw = await callLLM(truncated);
const { title, body } = parseTitleFromSummary(raw);
const finalTitle = title || task.title;
db.prepare(
"UPDATE tasks SET title = ?, summary = ?, updated_at = ? WHERE id = ?",
).run(finalTitle, body, Date.now(), task.id);
console.log(` ✅ title="${finalTitle}"`);
console.log(` ${body.slice(0, 80).replace(/\n/g, " ")}...`);
console.log("");
} catch (err) {
console.error(` ❌ Failed: ${err}`);
}
await new Promise((r) => setTimeout(r, 1000));
}
console.log("Done!");
db.close();
}
main().catch(console.error);
================================================
FILE: apps/memos-local-openclaw/scripts/run-accuracy-test.ts
================================================
#!/usr/bin/env npx tsx
/**
* MemOS Accuracy Test — sends data through OpenClaw Gateway (real pipeline).
*
* Ingest uses `openclaw agent` CLI so data flows through the full gateway,
* is processed by the memos plugin, and is visible in the Viewer UI.
* Search verification uses direct DB access via initPlugin.
*
* Usage:
* npx tsx scripts/run-accuracy-test.ts # quick mode (5 ingest, verify only)
* npx tsx scripts/run-accuracy-test.ts --full # full 50+ test cases
* npx tsx scripts/run-accuracy-test.ts --workers 3 # concurrent sessions (full mode)
* npx tsx scripts/run-accuracy-test.ts --skip-ingest # only run search checks (assumes data exists)
*
* Add to package.json:
* "test:accuracy": "tsx scripts/run-accuracy-test.ts"
*/
import { execSync } from "child_process";
import * as fs from "fs";
import * as os from "os";
import * as path from "path";
import { initPlugin, type MemosLocalPlugin } from "../src/index";
// ─── CLI args ───
const args = process.argv.slice(2);
const FULL_MODE = args.includes("--full");
const SKIP_INGEST = args.includes("--skip-ingest");
const WORKERS = Number(args.find((_, i, a) => a[i - 1] === "--workers") ?? 2);
const INGEST_DELAY_MS = 3000;
// ─── Config ───
function loadConfig() {
const home = process.env.HOME ?? process.env.USERPROFILE ?? "/tmp";
const cfgPath = path.join(home, ".openclaw", "openclaw.json");
if (!fs.existsSync(cfgPath)) {
throw new Error(`OpenClaw config not found: ${cfgPath}`);
}
const raw = JSON.parse(fs.readFileSync(cfgPath, "utf-8"));
return raw?.plugins?.entries?.["memos-local-openclaw-plugin"]?.config ?? {};
}
// ─── Test framework ───
interface TestResult {
category: string;
name: string;
pass: boolean;
detail: string;
durationMs: number;
}
const results: TestResult[] = [];
const RUN_ID = Date.now();
const SESSION_PREFIX = `acc-${RUN_ID}`;
let sessionSeq = 0;
function mkSession(label: string) {
return `${SESSION_PREFIX}-${label}-${++sessionSeq}`;
}
function log(msg: string) {
const t = new Date().toLocaleTimeString("zh-CN", { hour12: false });
console.log(`[${t}] ${msg}`);
}
// ─── Progress tracker ───
class ProgressTracker {
private total: number;
private done = 0;
private startMs = Date.now();
private phaseName: string;
constructor(phaseName: string, total: number) {
this.phaseName = phaseName;
this.total = total;
}
tick(label: string) {
this.done++;
const elapsed = Date.now() - this.startMs;
const pct = Math.round((this.done / this.total) * 100);
const remaining = this.total - this.done;
const avgMs = elapsed / this.done;
const eta = Math.round(remaining * avgMs);
const barLen = 30;
const filled = Math.round(barLen * this.done / this.total);
const bar = "█".repeat(filled) + "░".repeat(barLen - filled);
log(
` [${bar}] ${this.done}/${this.total} (${pct}%)` +
` elapsed: ${fmtDur(elapsed)} ETA: ${remaining > 0 ? fmtDur(eta) : "done"}` +
` — ${label}`,
);
}
summary(): string {
const elapsed = Date.now() - this.startMs;
return `${this.phaseName}: ${this.done}/${this.total} in ${fmtDur(elapsed)}`;
}
}
function fmtDur(ms: number): string {
const s = Math.floor(ms / 1000);
if (s < 60) return `${s}s`;
const m = Math.floor(s / 60);
const sec = s % 60;
return `${m}m${sec}s`;
}
function hitContains(hits: any[], keyword: string): boolean {
return hits.some(
(h: any) =>
h.original_excerpt?.toLowerCase().includes(keyword.toLowerCase()) ||
h.summary?.toLowerCase().includes(keyword.toLowerCase()),
);
}
// ─── Send message through OpenClaw Gateway ───
function sendViaGateway(sessionId: string, message: string): boolean {
const tmpFile = path.join(os.tmpdir(), `memos-test-msg-${Date.now()}.txt`);
try {
fs.writeFileSync(tmpFile, message, "utf-8");
execSync(
`openclaw agent --session-id "${sessionId}" --message "$(cat '${tmpFile}')" --json`,
{ timeout: 120_000, stdio: "pipe" },
);
return true;
} catch (e: any) {
log(` [WARN] gateway send failed: ${e.message?.slice(0, 200)}`);
return false;
} finally {
try { fs.unlinkSync(tmpFile); } catch {}
}
}
// ─── Test data: realistic, multi-turn, long-form conversations ───
interface ConversationCase {
id: string;
label: string;
sessionId: string;
messages: string[];
group: "dedup" | "topic" | "search" | "summary" | "cross-lang";
}
function buildTestCases(): ConversationCase[] {
const cases: ConversationCase[] = [];
// ═══════════════════════════════════════════
// Group 1: Dedup — exact / semantic / merge
// ═══════════════════════════════════════════
const dedupSession1 = mkSession("dedup-exact");
cases.push({
id: "dedup-exact-1",
label: "Dedup: exact duplicate (msg 1/3)",
sessionId: dedupSession1,
group: "dedup",
messages: [
`我们的线上 Redis 集群配置如下:Redis 版本 6.2.14,部署在 3 台 AWS ElastiCache r6g.xlarge 节点上,组成 3 主 3 从的 Cluster 模式。maxmemory 设置为 12GB,淘汰策略用 allkeys-lru,连接池大小 50,超时时间 3 秒。所有缓存 key 统一加 "prod:" 前缀,TTL 默认 1 小时,热点数据(如用户 session、商品详情)TTL 设为 24 小时。`,
],
});
cases.push({
id: "dedup-exact-2",
label: "Dedup: exact duplicate (msg 2/3, same content)",
sessionId: dedupSession1,
group: "dedup",
messages: [
`我们的线上 Redis 集群配置如下:Redis 版本 6.2.14,部署在 3 台 AWS ElastiCache r6g.xlarge 节点上,组成 3 主 3 从的 Cluster 模式。maxmemory 设置为 12GB,淘汰策略用 allkeys-lru,连接池大小 50,超时时间 3 秒。所有缓存 key 统一加 "prod:" 前缀,TTL 默认 1 小时,热点数据(如用户 session、商品详情)TTL 设为 24 小时。`,
],
});
cases.push({
id: "dedup-exact-3",
label: "Dedup: exact duplicate (msg 3/3, same content again)",
sessionId: dedupSession1,
group: "dedup",
messages: [
`我们的线上 Redis 集群配置如下:Redis 版本 6.2.14,部署在 3 台 AWS ElastiCache r6g.xlarge 节点上,组成 3 主 3 从的 Cluster 模式。maxmemory 设置为 12GB,淘汰策略用 allkeys-lru,连接池大小 50,超时时间 3 秒。所有缓存 key 统一加 "prod:" 前缀,TTL 默认 1 小时,热点数据(如用户 session、商品详情)TTL 设为 24 小时。`,
],
});
const dedupSession2 = mkSession("dedup-semantic");
cases.push({
id: "dedup-sem-1",
label: "Dedup: semantic dup (PostgreSQL v1)",
sessionId: dedupSession2,
group: "dedup",
messages: [
`主数据库使用 PostgreSQL 16,部署在 AWS RDS 的 db.r6g.2xlarge 实例上。已开启读写分离,1 个 writer 实例 + 2 个 reader 副本做负载均衡。连接池用 PgBouncer,transaction pooling 模式,max_client_conn 设为 200,default_pool_size 设为 25。WAL 日志异步复制,backup 策略是每日自动快照 + 开启 Point-in-Time Recovery(PITR),保留 7 天。`,
],
});
cases.push({
id: "dedup-sem-2",
label: "Dedup: semantic dup (PostgreSQL v2 — reworded)",
sessionId: dedupSession2,
group: "dedup",
messages: [
`生产环境的核心关系型数据库是 PG 16,跑在 Amazon RDS 上面,机型选的是 db.r6g.2xlarge。数据库做了读写分离——一个主库负责写入,两个只读副本分担查询流量。中间层用 PgBouncer 做连接池管理,采用事务级池化,最大客户端连接数 200,默认池大小 25。日志走 WAL 异步复制,每天自动创建快照备份,还启用了时间点恢复(PITR),保留窗口 7 天。`,
],
});
const dedupSession3 = mkSession("dedup-merge");
cases.push({
id: "dedup-merge-1",
label: "Dedup: merge — old state (React 18 + Vite)",
sessionId: dedupSession3,
group: "dedup",
messages: [
`前端项目用 React 18.2 搭配 Vite 5.0 构建,TypeScript 5.3 严格模式。状态管理用 Zustand + React Query v5,UI 组件库用 Ant Design 5.x。打包产物部署到 CloudFront CDN,Gzip + Brotli 双压缩,首屏 LCP 控制在 1.8 秒以内。`,
],
});
cases.push({
id: "dedup-merge-2",
label: "Dedup: merge — new state (migrated to Next.js 14)",
sessionId: dedupSession3,
group: "dedup",
messages: [
`前端已经从 React 18 + Vite 迁移到了 Next.js 14 App Router,改用 Vercel 部署。状态管理保持 Zustand + React Query 不变,但 UI 组件库换成了 Shadcn/ui + Tailwind CSS。SSR + ISR 混合渲染,Core Web Vitals 全绿,LCP 降到 1.2 秒。`,
],
});
// ═══════════════════════════════════════════
// Group 2: Topic boundary detection
// ═══════════════════════════════════════════
const topicSameSession = mkSession("topic-same");
cases.push({
id: "topic-same-1",
label: "Topic: same topic (Nginx config, part 1)",
sessionId: topicSameSession,
group: "topic",
messages: [
`帮我配置生产环境的 Nginx 反向代理。需求:监听 443 端口,SSL/TLS 证书放在 /etc/nginx/ssl/ 目录下,upstream 后端是 localhost:3000 的 Node.js 应用。需要配置 worker_processes auto,worker_connections 4096,以及 proxy_set_header 把真实 IP 传到后端。`,
],
});
cases.push({
id: "topic-same-2",
label: "Topic: same topic (Nginx config, part 2 — add gzip + cache)",
sessionId: topicSameSession,
group: "topic",
messages: [
`Nginx 配置再加几个优化:开启 gzip 压缩(gzip on; gzip_types text/plain text/css application/json application/javascript; gzip_min_length 1024;),静态资源加浏览器缓存头(location ~* \\.(js|css|png|jpg|svg|woff2)$ { expires 30d; add_header Cache-Control "public, immutable"; }),还要加上 HTTP/2 和 HSTS(add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";)。`,
],
});
const topicSwitchSession = mkSession("topic-switch");
cases.push({
id: "topic-switch-1",
label: "Topic: switch — Docker (tech)",
sessionId: topicSwitchSession,
group: "topic",
messages: [
`帮我写一个多阶段 Dockerfile,用于构建 Node.js 20 的生产镜像。第一阶段用 node:20-alpine 作为 builder,安装 pnpm,复制 package.json 和 pnpm-lock.yaml,然后 pnpm install --frozen-lockfile --prod=false,再 pnpm run build。第二阶段用干净的 node:20-alpine,只复制 dist/ 和 node_modules/,暴露 3000 端口,CMD ["node", "dist/server.js"]。同时生成一个 .dockerignore 排除 node_modules、.git、.env、coverage、*.md。`,
],
});
cases.push({
id: "topic-switch-2",
label: "Topic: switch — cooking (completely different domain)",
sessionId: topicSwitchSession,
group: "topic",
messages: [
`今天想试试做正宗的红烧肉。食材清单:五花肉 500g(切 3cm 方块)、冰糖 30g、生抽 3 勺、老抽 1 勺、料酒 2 勺、八角 2 颗、桂皮 1 小段、香叶 2 片、干辣椒 2 个、生姜 4 片、葱白 3 段。步骤:五花肉冷水下锅焯水 5 分钟,捞出洗净。锅里放少量油,中小火炒冰糖至焦糖色,下五花肉翻炒上色。加料酒、生抽、老抽,放八角桂皮香叶,加没过肉的热水,大火煮开后转小火炖 50 分钟。最后大火收汁,撒葱花出锅。`,
],
});
// ═══════════════════════════════════════════
// Group 3: Search precision + recall data
// ═══════════════════════════════════════════
const searchSession = mkSession("search-data");
cases.push({
id: "search-mysql",
label: "Search: MySQL InnoDB MVCC",
sessionId: searchSession,
group: "search",
messages: [
`线上 MySQL 8.0 数据库要点总结:存储引擎统一用 InnoDB,默认行级锁,支持 MVCC 多版本并发控制。事务隔离级别设为 REPEATABLE READ(MySQL 默认),innodb_buffer_pool_size 设为物理内存的 70%(当前 28GB / 40GB),innodb_flush_log_at_trx_commit=1 保证事务持久性。慢查询日志开启,long_query_time=2 秒,定期用 pt-query-digest 分析 Top 20 慢查询。索引策略:核心业务表必须有聚簇索引,联合索引遵循最左前缀原则,覆盖索引优先避免回表。`,
],
});
cases.push({
id: "search-k8s",
label: "Search: Kubernetes cluster",
sessionId: searchSession,
group: "search",
messages: [
`Kubernetes 生产集群规模和配置:3 个 master 节点(etcd 高可用集群)+ 8 个 worker 节点,全部部署在阿里云 ECS ecs.c7.2xlarge(8c16g)上。容器运行时用 containerd 1.7,网络插件 Calico VXLAN 模式。部署方式:核心服务 Deployment + HPA(CPU 60% 触发扩容,最小 2 副本最大 10 副本),有状态服务(MySQL、Redis)用 StatefulSet + PVC。日志用 Fluent Bit DaemonSet 采集到 ES,监控用 Prometheus Operator + kube-state-metrics。`,
],
});
cases.push({
id: "search-review",
label: "Search: Code Review process",
sessionId: searchSession,
group: "search",
messages: [
`团队 Code Review 流程规范:每周三下午 2-4 点集中做 Code Review Session,其他时间异步 review。GitLab MR 模板包含:变更描述、影响范围、测试情况、截图/录屏。Review 规则:至少 2 人 approve 才能合并,其中 1 人必须是 Tech Lead 或 Senior。自动化检查:CI 跑 lint(ESLint + Prettier)、单元测试(覆盖率门禁 80%)、类型检查、依赖安全扫描(Snyk)。Code Review 重点关注:逻辑正确性 > 性能 > 可读性 > 编码风格。`,
],
});
cases.push({
id: "search-elk",
label: "Search: ELK logging stack",
sessionId: searchSession,
group: "search",
messages: [
`日志系统架构:ELK 栈。Elasticsearch 7.17 集群(3 节点,每节点 64GB 内存 + 2TB SSD),Logstash 作为日志处理管道(grok 解析 + 字段映射 + 时间戳标准化),Kibana 做可视化和告警。日志分级:应用日志走 Fluent Bit → Kafka(缓冲) → Logstash → ES,系统日志直接 Filebeat → ES。索引策略:按天滚动创建索引(logs-app-YYYY.MM.DD),ILM 策略 hot/warm/cold 三层,hot 7 天 SSD,warm 30 天 HDD,cold 90 天归档到 S3 Glacier。`,
],
});
cases.push({
id: "search-monitoring",
label: "Search: Prometheus Grafana monitoring",
sessionId: searchSession,
group: "search",
messages: [
`监控告警体系:Prometheus 2.45 + Grafana 10.x + AlertManager。Prometheus 抓取间隔 15 秒,数据保留 30 天。主要 exporter:node_exporter(主机指标)、cadvisor(容器指标)、mysqld_exporter、redis_exporter、blackbox_exporter(HTTP 探测)。Grafana 仪表盘:系统概览、应用 QPS/延迟/错误率、数据库连接池、Redis 命中率。告警规则:CPU > 80% 持续 5 分钟 → 企业微信通知,5xx 错误率 > 1% → 电话告警(PagerDuty),磁盘使用率 > 85% → 邮件通知。`,
],
});
// Recall data — DevOps tools
const recallSession = mkSession("recall-devops");
cases.push({
id: "search-jenkins",
label: "Search: Jenkins CI pipeline",
sessionId: recallSession,
group: "search",
messages: [
`CI/CD Pipeline 用 Jenkins 2.x,Jenkinsfile 放在项目根目录,采用 declarative pipeline 语法。流水线分 5 个 stage:Checkout → Lint & Type Check → Unit Test(Jest,覆盖率报告上传 SonarQube)→ Build(Docker 多阶段构建)→ Deploy(kubectl apply 到对应环境)。分支策略:feature/* 只跑 lint + test,develop 跑全量 + 部署 staging,main 跑全量 + 部署 production(需要人工审批)。Jenkins 节点用 Kubernetes Pod 作为 agent,按需弹性伸缩。`,
],
});
cases.push({
id: "search-terraform",
label: "Search: Terraform IaC",
sessionId: recallSession,
group: "search",
messages: [
`基础设施即代码用 Terraform 1.6,state 存在 S3 bucket + DynamoDB 做状态锁,防止并发修改。模块化组织:modules/networking(VPC、子网、安全组)、modules/compute(ECS 实例、Auto Scaling Group)、modules/database(RDS、ElastiCache)、modules/monitoring(CloudWatch、SNS)。环境用 workspace 隔离:dev / staging / production。变量通过 terraform.tfvars 和 CI 环境变量注入。每次变更走 PR,CI 自动执行 terraform plan,输出 diff 到 PR 评论,merge 后自动 terraform apply。`,
],
});
// ═══════════════════════════════════════════
// Group 4: Summary quality — long text
// ═══════════════════════════════════════════
const summarySession = mkSession("summary");
cases.push({
id: "summary-microservices",
label: "Summary: complex microservices architecture",
sessionId: summarySession,
group: "summary",
messages: [
`微服务架构详细设计方案如下。服务拆分:user-service 负责用户注册登录、OAuth2.0 第三方授权、RBAC 权限管理、用户画像标签;order-service 处理订单创建/取消/退款全生命周期,支持分库分表(按 user_id 取模 16 库 64 表);payment-service 对接支付宝当面付、微信 JSAPI 支付、银联快捷支付,所有支付回调统一走消息队列异步处理;inventory-service 管理商品库存,用 Redis 预扣 + MySQL 最终一致性方案防超卖;notification-service 负责短信(阿里云 SMS)、邮件(SES)、App Push(极光推送)、站内信。所有服务 Kubernetes 部署,Istio 服务网格做流量管理和灰度发布,Jaeger 全链路追踪,SkyWalking 做 APM 性能监控。服务间通信:同步走 gRPC(protobuf 序列化),异步走 RocketMQ 5.0。API Gateway 用 Kong,统一鉴权、限流、日志。`,
],
});
cases.push({
id: "summary-migration",
label: "Summary: DB migration plan",
sessionId: summarySession,
group: "summary",
messages: [
`数据库迁移三阶段实施方案。Q1(1-3 月):用户表从 MySQL 迁移到 PostgreSQL。第一步搭建 PG 目标库,用 pgloader 做初始全量同步;第二步开启 Maxwell → Kafka → PG 的实时 CDC 增量同步;第三步应用层改为双写模式(先写 MySQL 再写 PG),持续一个月做数据一致性校验(每天凌晨全表 count + 随机抽样 1000 条 hash 比对);第四步灰度切读到 PG(先 10% → 50% → 100%),确认无误后停止双写。Q2(4-6 月):订单表和支付表迁移,用 Debezium CDC 替代 Maxwell(支持 exactly-once delivery),同样双写 + 校验 + 灰度流程。Q3(7-9 月):剩余表迁移完成,停掉旧 MySQL 集群。每个阶段迁移完成后保留旧库只读权限 90 天,作为回滚保险。`,
],
});
// ═══════════════════════════════════════════
// Group 5: Cross-language
// ═══════════════════════════════════════════
const crossLangSession = mkSession("cross-lang");
cases.push({
id: "cross-lang-en",
label: "Cross-lang: Docker Compose (English)",
sessionId: crossLangSession,
group: "cross-lang",
messages: [
`Our local development setup uses Docker Compose with four services: "api" runs the Node.js backend on port 3000 with hot-reload via nodemon, "web" runs the Next.js frontend on port 3001 with Fast Refresh, "postgres" uses the official PostgreSQL 16 image with a named volume for data persistence, and "redis" uses Redis 7 Alpine for caching. We also have a "mailhog" service for testing email delivery locally. All services share a custom bridge network called "dev-net". Environment variables are injected via a .env file referenced in docker-compose.yml.`,
],
});
cases.push({
id: "cross-lang-zh",
label: "Cross-lang: Docker Compose (Chinese, same meaning)",
sessionId: crossLangSession,
group: "cross-lang",
messages: [
`本地开发环境用 Docker Compose 编排四个核心服务:api 容器跑 Node.js 后端(端口 3000,nodemon 热更新),web 容器跑 Next.js 前端(端口 3001,Fast Refresh),postgres 容器用官方 PostgreSQL 16 镜像(命名卷持久化数据),redis 容器用 Redis 7 Alpine 做缓存。另外还有一个 mailhog 容器用来本地测试邮件发送。所有容器通过自定义桥接网络 dev-net 互通。环境变量通过 .env 文件注入。`,
],
});
// ═══════════════════════════════════════════
// Full mode: additional cases for scale
// ═══════════════════════════════════════════
if (FULL_MODE) {
const fullSession = mkSession("full-extra");
cases.push({
id: "full-api-doc",
label: "Full: API documentation (Swagger/OpenAPI)",
sessionId: fullSession,
group: "search",
messages: [
`API 文档自动化方案:使用 Swagger/OpenAPI 3.0 规范,结合 swagger-jsdoc 从代码注释自动生成 API 文档。每个接口必须标注:summary、description、parameters(含类型和校验规则)、requestBody schema、responses(200/400/401/403/404/500 各场景)。CI 流水线中自动生成 openapi.json,部署到 Swagger UI(内网 /api-docs 路径)。SDK 生成:用 openapi-generator 给前端自动生成 TypeScript axios client,给移动端生成 Swift/Kotlin client。文档变更必须随代码 PR 一起提交,CI 校验 schema 兼容性(不允许破坏性变更,用 oasdiff 检测)。`,
],
});
cases.push({
id: "full-backup",
label: "Full: Database backup strategy",
sessionId: fullSession,
group: "search",
messages: [
`数据库备份策略。MySQL:每日凌晨 2 点 mysqldump 全量备份(--single-transaction --routines --triggers),每小时 binlog 增量备份,所有备份加密后上传到 S3 Standard-IA,保留 30 天。PostgreSQL:每日 pg_basebackup 全量 + 持续 WAL 归档(archive_command 到 S3),支持 PITR。恢复演练:每月第一个周六做一次恢复演练,从 S3 拉取备份恢复到演练环境,验证数据完整性(行数对比 + 业务关键数据校验)。恢复 RTO 目标 < 1 小时,RPO 目标 < 1 小时。监控:备份任务状态接入 Prometheus,失败立即 PagerDuty 告警。`,
],
});
cases.push({
id: "full-perf",
label: "Full: React performance optimization",
sessionId: fullSession,
group: "search",
messages: [
`React 前端性能优化记录。代码层面:用 React.lazy + Suspense 做路由级代码分割,首屏 JS 从 1.2MB 降到 380KB;React.memo + useMemo 避免不必要的重渲染,列表组件用 react-window 虚拟化(1 万条数据渲染从 3.2 秒降到 60ms);图片全部用 next/image 自动 WebP 转换 + 懒加载。构建层面:Vite 5 tree-shaking + dynamic import,第三方库用 CDN 外置(React/ReactDOM/Lodash)。Lighthouse 指标:Performance 从 45 提升到 92,FCP 1.1s,LCP 1.8s,CLS 0.02。监控:接入 web-vitals 库实时上报 Core Web Vitals 到 ClickHouse,Grafana 展示 P75/P90/P99 趋势。`,
],
});
const fullSession2 = mkSession("full-devops");
cases.push({
id: "full-sonarqube",
label: "Full: SonarQube quality gate",
sessionId: fullSession2,
group: "search",
messages: [
`代码质量门禁用 SonarQube 9.x。Quality Gate 规则:新代码覆盖率 > 80%,整体覆盖率 > 65%,代码重复率 < 3%,无新增 Blocker/Critical 级别的 Bug 和漏洞,Maintainability Rating 必须 A 级。CI 集成:Jenkins pipeline 中在 test stage 之后执行 sonar-scanner,扫描结果推送到 SonarQube Server,Quality Gate 不通过则 pipeline 失败。自定义规则:在默认 Sonar way profile 基础上,新增了 SQL 注入检测、硬编码密钥检测、日志敏感信息检测等自定义规则。每周一生成代码质量周报,邮件发送给团队 Tech Lead。`,
],
});
cases.push({
id: "full-ansible",
label: "Full: Ansible server management",
sessionId: fullSession2,
group: "search",
messages: [
`服务器配置管理用 Ansible 2.15。Inventory 文件按环境分组:[dev]、[staging]、[production],每个环境有独立的 group_vars。核心 Playbook:server-init.yml(系统初始化:时区/NTP/防火墙/用户/SSH 加固),deploy-app.yml(应用部署:拉取镜像/更新 compose 文件/滚动重启),monitor-setup.yml(安装 node_exporter + fluent-bit)。Ansible Vault 加密所有密钥和密码。执行策略:变更先在 staging 跑一遍(--check 模式预演),确认无误后在 production 执行(每次最多 2 台,serial: 2)。所有 playbook 执行日志记录到 ELK。`,
],
});
const fullSession3 = mkSession("full-unrelated");
cases.push({
id: "full-company-event",
label: "Full: unrelated (company annual party)",
sessionId: fullSession3,
group: "dedup",
messages: [
`公司年会安排确定了。时间:12 月 20 日(周六)下午 2 点到晚上 9 点。地点:杭州西湖国宾馆 3 号楼宴会厅,可容纳 300 人。议程:2:00-3:00 CEO 年度总结和明年规划,3:00-4:30 各部门优秀项目展示(每组 10 分钟),4:30-5:00 茶歇,5:00-6:30 年度颁奖(最佳团队、最佳个人、最佳新人、创新奖),6:30-9:00 晚宴 + 文艺表演 + 抽奖。每个部门需要准备至少一个节目,节目清单 12 月 10 日前提交给 HR 小王。预算:人均 500 元。`,
],
});
cases.push({
id: "full-training",
label: "Full: unrelated (new employee training)",
sessionId: fullSession3,
group: "dedup",
messages: [
`新员工入职培训计划(为期两周)。第一周:Day 1 公司文化和价值观介绍、HR 制度讲解、IT 账号开通;Day 2-3 技术栈总览(架构图、代码仓库结构、本地开发环境搭建);Day 4 编码规范培训(TypeScript 规范、ESLint 规则、命名约定、文件组织);Day 5 Git 工作流培训(Git Flow、分支命名、Commit Message 规范、MR 流程)。第二周:Day 6-7 跟随导师做一个入门任务(小 feature 开发);Day 8-9 Code Review 流程实践(参加 Review Session、自己提交 MR 被 review);Day 10 入职考核(代码 quiz + 流程问答 + 导师评价)。`,
],
});
}
return cases;
}
// ─── Search cases ───
interface SearchCase {
query: string;
expectKeyword: string;
category: "keyword" | "semantic" | "negative" | "recall";
topK: number;
minScore?: number;
shouldFind: boolean;
}
function buildSearchCases(): SearchCase[] {
const cases: SearchCase[] = [
{ query: "MySQL InnoDB MVCC 行锁 innodb_buffer_pool_size", expectKeyword: "InnoDB", category: "keyword", topK: 5, shouldFind: true },
{ query: "Kubernetes ECS 阿里云 容器集群 Calico", expectKeyword: "Kubernetes", category: "keyword", topK: 5, shouldFind: true },
{ query: "Prometheus Grafana AlertManager 监控告警", expectKeyword: "Prometheus", category: "keyword", topK: 5, shouldFind: true },
{ query: "ELK Elasticsearch Logstash Kibana 日志", expectKeyword: "Elasticsearch", category: "keyword", topK: 5, shouldFind: true },
{ query: "数据库事务隔离级别和并发控制机制", expectKeyword: "MVCC", category: "semantic", topK: 5, shouldFind: true },
{ query: "容器编排平台和自动扩容策略", expectKeyword: "Kubernetes", category: "semantic", topK: 5, shouldFind: true },
{ query: "代码质量审查团队协作流程", expectKeyword: "Review", category: "semantic", topK: 5, shouldFind: true },
{ query: "应用日志集中采集存储和检索", expectKeyword: "ELK", category: "semantic", topK: 5, shouldFind: true },
{ query: "深度学习 PyTorch GPU 训练模型 CUDA 显存", expectKeyword: "MySQL", category: "negative", topK: 5, minScore: 0.65, shouldFind: false },
{ query: "量化交易策略回测 Alpha 因子挖掘", expectKeyword: "Kubernetes", category: "negative", topK: 5, minScore: 0.65, shouldFind: false },
{ query: "CI/CD 流水线 自动化部署 发布流程", expectKeyword: "Jenkins", category: "recall", topK: 10, shouldFind: true },
{ query: "基础设施即代码 IaC 云资源管理", expectKeyword: "Terraform", category: "recall", topK: 10, shouldFind: true },
{ query: "Docker Compose 本地开发环境 容器编排", expectKeyword: "Docker", category: "recall", topK: 5, shouldFind: true },
];
if (FULL_MODE) {
cases.push(
{ query: "API 接口文档自动生成 Swagger OpenAPI", expectKeyword: "Swagger", category: "keyword", topK: 5, shouldFind: true },
{ query: "数据库定时备份恢复策略 mysqldump", expectKeyword: "备份", category: "keyword", topK: 5, shouldFind: true },
{ query: "React 性能优化 Lighthouse 代码分割", expectKeyword: "React", category: "keyword", topK: 5, shouldFind: true },
{ query: "代码质量门禁覆盖率重复率检测", expectKeyword: "SonarQube", category: "recall", topK: 10, shouldFind: true },
{ query: "服务器批量配置管理自动化运维 Playbook", expectKeyword: "Ansible", category: "recall", topK: 10, shouldFind: true },
);
}
return cases;
}
// ─── Register sessions into OpenClaw sessions.json so they appear in UI dropdown ───
function registerSessionsInStore(cases: ConversationCase[]) {
const home = process.env.HOME ?? process.env.USERPROFILE ?? "/tmp";
const storePath = path.join(home, ".openclaw", "agents", "main", "sessions", "sessions.json");
if (!fs.existsSync(storePath)) {
log("[WARN] sessions.json not found, skipping UI registration");
return;
}
const store = JSON.parse(fs.readFileSync(storePath, "utf-8"));
const sessionsDir = path.dirname(storePath);
const seen = new Set();
let added = 0;
for (const c of cases) {
if (seen.has(c.sessionId)) continue;
seen.add(c.sessionId);
const storeKey = `agent:main:${c.sessionId}`;
if (store[storeKey]) continue;
const sessionFile = path.join(sessionsDir, `${c.sessionId}.jsonl`);
if (!fs.existsSync(sessionFile)) continue;
// acc-1773286763918-dedup-exact-1 -> dedup-exact
const shortName = c.sessionId
.replace(/^acc-\d+-/, "")
.replace(/-\d+$/, "");
store[storeKey] = {
sessionId: c.sessionId,
updatedAt: Date.now(),
systemSent: true,
abortedLastRun: false,
chatType: "direct",
label: `[test] ${shortName}`,
displayName: `Test: ${shortName}`,
origin: {
provider: "cli",
surface: "cli",
chatType: "direct",
label: `accuracy-test:${shortName}`,
},
sessionFile,
};
added++;
}
fs.writeFileSync(storePath, JSON.stringify(store, null, 2), "utf-8");
log(`Registered ${added} test sessions in sessions.json (UI dropdown)`);
}
// ─── Ingest via Gateway ───
async function ingestPhase(cases: ConversationCase[]) {
const totalMsgs = cases.reduce((a, c) => a + c.messages.length, 0);
log(`Sending ${cases.length} conversations (${totalMsgs} messages) through OpenClaw Gateway...`);
log(`(Each message goes through full gateway → plugin pipeline, visible in Viewer)\n`);
const tracker = new ProgressTracker("Ingest", totalMsgs);
const buckets: ConversationCase[][] = Array.from({ length: WORKERS }, () => []);
cases.forEach((c, i) => buckets[i % WORKERS].push(c));
let successCount = 0;
let failCount = 0;
const workerFn = async (workerId: number, bucket: ConversationCase[]) => {
for (const c of bucket) {
for (const msg of c.messages) {
const ok = sendViaGateway(c.sessionId, msg);
if (ok) {
successCount++;
} else {
failCount++;
}
tracker.tick(`${ok ? "OK" : "FAIL"} ${c.label}`);
await new Promise((r) => setTimeout(r, INGEST_DELAY_MS));
}
}
};
const t0 = performance.now();
await Promise.all(
buckets.map((b, i) => (b.length > 0 ? workerFn(i + 1, b) : Promise.resolve())),
);
const dur = Math.round(performance.now() - t0);
log(`\nIngest complete: ${successCount} sent, ${failCount} failed (${(dur / 1000).toFixed(1)}s)\n`);
log("Waiting 10s for ingest pipeline to process all messages...");
await new Promise((r) => setTimeout(r, 10_000));
registerSessionsInStore(cases);
return { successCount, failCount };
}
// ─── Verify phase ───
async function runSearchTests(plugin: MemosLocalPlugin, cases: SearchCase[], tracker: ProgressTracker) {
const searchTool = plugin.tools.find((t) => t.name === "memory_search")!;
for (const c of cases) {
const t0 = performance.now();
const result = (await searchTool.handler({
query: c.query,
maxResults: c.topK,
minScore: c.minScore,
})) as any;
const dur = Math.round(performance.now() - t0);
const hits = result.hits ?? [];
const found = hitContains(hits, c.expectKeyword);
if (c.category === "negative") {
const pass = !found;
results.push({
category: "Precision",
name: `negative: "${c.query.slice(0, 25)}..."`,
pass,
detail: `should NOT contain "${c.expectKeyword}": ${pass ? "OK" : "FAIL"} (${hits.length} hits)`,
durationMs: dur,
});
} else if (c.category === "keyword") {
results.push({
category: "Precision",
name: `keyword: ${c.expectKeyword}`,
pass: found,
detail: `top${c.topK} contains "${c.expectKeyword}": ${found}`,
durationMs: dur,
});
} else if (c.category === "semantic") {
results.push({
category: "Precision",
name: `semantic: ${c.expectKeyword}`,
pass: found,
detail: `top${c.topK} contains "${c.expectKeyword}": ${found}`,
durationMs: dur,
});
} else if (c.category === "recall") {
results.push({
category: "Recall",
name: `recall: ${c.expectKeyword}`,
pass: found,
detail: found ? "found" : "missed",
durationMs: dur,
});
}
tracker.tick(`${c.category}: ${c.expectKeyword}`);
}
}
async function runDedupChecks(plugin: MemosLocalPlugin, tracker: ProgressTracker) {
const searchTool = plugin.tools.find((t) => t.name === "memory_search")!;
const t0 = performance.now();
const r1 = (await searchTool.handler({ query: "Redis ElastiCache 集群 maxmemory allkeys-lru 连接池", maxResults: 10 })) as any;
const redisHits = (r1.hits ?? []).filter((h: any) => hitContains([h], "Redis") || hitContains([h], "ElastiCache"));
const exactPass = redisHits.length >= 1 && redisHits.length <= 2;
results.push({ category: "Dedup", name: "exact dup (Redis x3 → 1-2)", pass: exactPass, detail: `${redisHits.length} active hits (expect 1-2)`, durationMs: Math.round(performance.now() - t0) });
tracker.tick("dedup: exact dup (Redis)");
const t1 = performance.now();
const r2 = (await searchTool.handler({ query: "PostgreSQL RDS PgBouncer 读写分离 WAL", maxResults: 10 })) as any;
const pgHits = (r2.hits ?? []).filter((h: any) => hitContains([h], "PostgreSQL") || hitContains([h], "PG ") || hitContains([h], "PgBouncer"));
const semPass = pgHits.length >= 1 && pgHits.length <= 2;
results.push({ category: "Dedup", name: "semantic dup (PG x2 → 1-2)", pass: semPass, detail: `${pgHits.length} active hits (expect 1-2)`, durationMs: Math.round(performance.now() - t1) });
tracker.tick("dedup: semantic dup (PG)");
const t2 = performance.now();
const r3 = (await searchTool.handler({ query: "前端技术栈 Next.js Shadcn Tailwind Vercel", maxResults: 10 })) as any;
const hasLatest = hitContains(r3.hits ?? [], "Next.js") || hitContains(r3.hits ?? [], "Shadcn");
results.push({ category: "Dedup", name: "merge (React/Vite → Next.js/Vercel)", pass: hasLatest, detail: `latest state present: ${hasLatest}`, durationMs: Math.round(performance.now() - t2) });
tracker.tick("dedup: merge (Next.js)");
}
async function runSummaryChecks(plugin: MemosLocalPlugin, tracker: ProgressTracker) {
const searchTool = plugin.tools.find((t) => t.name === "memory_search")!;
const queries = [
{ query: "微服务架构 user-service payment-service Istio gRPC", label: "microservices arch" },
{ query: "数据库迁移 MySQL PostgreSQL Debezium CDC 双写", label: "DB migration plan" },
];
for (const q of queries) {
const t0 = performance.now();
const r = (await searchTool.handler({ query: q.query, maxResults: 3 })) as any;
const dur = Math.round(performance.now() - t0);
if (r.hits?.length > 0) {
const h = r.hits[0];
const sl = h.summary?.length ?? 0;
const cl = h.original_excerpt?.length ?? 999;
const pass = sl > 0 && sl < cl;
results.push({ category: "Summary", name: q.label, pass, detail: `summary=${sl}chars, content=${cl}chars, shorter=${sl < cl}`, durationMs: dur });
} else {
results.push({ category: "Summary", name: q.label, pass: false, detail: "no hits found", durationMs: dur });
}
tracker.tick(`summary: ${q.label}`);
}
}
async function runTopicChecks(plugin: MemosLocalPlugin, tracker: ProgressTracker) {
const searchTool = plugin.tools.find((t) => t.name === "memory_search")!;
const t0 = performance.now();
const nginxR = (await searchTool.handler({ query: "Nginx 反向代理 SSL gzip HTTP/2 HSTS", maxResults: 10 })) as any;
const nginxHits = (nginxR.hits ?? []).filter((h: any) => hitContains([h], "Nginx") || hitContains([h], "gzip") || hitContains([h], "SSL"));
results.push({
category: "Topic",
name: "same topic merge (Nginx parts → 1 chunk)",
pass: nginxHits.length >= 1 && nginxHits.length <= 2,
detail: `${nginxHits.length} chunks (expect 1-2 merged)`,
durationMs: Math.round(performance.now() - t0),
});
tracker.tick("topic: same (Nginx)");
const t1 = performance.now();
const dockerR = (await searchTool.handler({ query: "Dockerfile 多阶段构建 pnpm node:20-alpine", maxResults: 5 })) as any;
const cookR = (await searchTool.handler({ query: "红烧肉 五花肉 冰糖 八角 桂皮", maxResults: 5 })) as any;
const dockerFound = hitContains(dockerR.hits ?? [], "Dockerfile") || hitContains(dockerR.hits ?? [], "node");
const cookFound = hitContains(cookR.hits ?? [], "五花肉") || hitContains(cookR.hits ?? [], "红烧肉");
const switchPass = dockerFound && cookFound;
results.push({
category: "Topic",
name: "topic switch (Docker → cooking)",
pass: switchPass,
detail: `Docker found=${dockerFound}, cooking found=${cookFound}`,
durationMs: Math.round(performance.now() - t1),
});
tracker.tick("topic: switch (Docker→cooking)");
}
// ─── Report ───
function printReport(totalMs: number, ingestStats?: { successCount: number; failCount: number }) {
console.log("\n");
console.log("=".repeat(70));
console.log(` MemOS Accuracy Test Report`);
console.log(` Mode: ${FULL_MODE ? "FULL" : "QUICK"} | Workers: ${WORKERS} | Duration: ${(totalMs / 1000).toFixed(1)}s`);
if (ingestStats) {
console.log(` Ingest: ${ingestStats.successCount} sent via Gateway, ${ingestStats.failCount} failed`);
}
console.log("=".repeat(70));
const categories = [...new Set(results.map((r) => r.category))];
let totalPass = 0;
let totalCount = 0;
for (const cat of categories) {
const cr = results.filter((r) => r.category === cat);
const passed = cr.filter((r) => r.pass).length;
totalPass += passed;
totalCount += cr.length;
const pct = ((passed / cr.length) * 100).toFixed(1);
console.log(`\n ${cat.padEnd(20)} ${passed}/${cr.length} (${pct}%)`);
for (const r of cr) {
const icon = r.pass ? "PASS" : "FAIL";
console.log(` [${icon}] ${r.name}: ${r.detail} (${r.durationMs}ms)`);
}
}
console.log("\n" + "-".repeat(70));
const overallPct = totalCount > 0 ? ((totalPass / totalCount) * 100).toFixed(1) : "0";
console.log(` OVERALL: ${totalPass}/${totalCount} (${overallPct}%)`);
console.log("=".repeat(70));
return totalPass === totalCount ? 0 : 1;
}
// ─── Main ───
async function main() {
const t0 = performance.now();
log("MemOS Accuracy Test starting...");
log(`Mode: ${FULL_MODE ? "FULL (50+ cases)" : "QUICK (15 cases — pass --full for all)"}`);
log("Loading OpenClaw config...");
const config = loadConfig();
const stateDir = path.join(process.env.HOME ?? "/tmp", ".openclaw");
let ingestStats: { successCount: number; failCount: number } | undefined;
if (!SKIP_INGEST) {
const testCases = buildTestCases();
const totalMsgs = testCases.reduce((a, c) => a + c.messages.length, 0);
log(`Prepared ${testCases.length} conversations (${totalMsgs} messages total)`);
ingestStats = await ingestPhase(testCases);
} else {
log("Skipping ingest (--skip-ingest), running search checks only...");
}
log("Initializing plugin for search verification (direct DB access)...");
const plugin = initPlugin({ stateDir, config });
const searchCases = buildSearchCases();
const verifyTotal = 3 + 2 + searchCases.length + 2; // dedup(3) + topic(2) + search + summary(2)
const verifyTracker = new ProgressTracker("Verify", verifyTotal);
log("Running dedup checks...");
await runDedupChecks(plugin, verifyTracker);
log("Running topic boundary checks...");
await runTopicChecks(plugin, verifyTracker);
log("Running search precision & recall tests...");
await runSearchTests(plugin, searchCases, verifyTracker);
log("Running summary quality checks...");
await runSummaryChecks(plugin, verifyTracker);
const totalMs = Math.round(performance.now() - t0);
const exitCode = printReport(totalMs, ingestStats);
await plugin.shutdown();
process.exit(exitCode);
}
main().catch((err) => {
console.error("Fatal error:", err);
process.exit(2);
});
================================================
FILE: apps/memos-local-openclaw/scripts/seed-test-data.ts
================================================
/**
* Seed script: inserts test data for Task-related features into the live database.
*
* Usage: npx tsx scripts/seed-test-data.ts
*
* After running, open Memory Viewer (http://127.0.0.1:18799/) and check:
* - Tasks page: 4 tasks (3 completed, 1 active)
* - Memories page: chunks associated with each task
* - Analytics: totals reflect new data
*/
import Database from "better-sqlite3";
import { v4 as uuid } from "uuid";
import * as path from "path";
import * as os from "os";
const DB_PATH = path.join(os.homedir(), ".openclaw", "memos-local", "memos.db");
const db = new Database(DB_PATH);
db.pragma("journal_mode = WAL");
db.pragma("foreign_keys = ON");
const now = Date.now();
const HOUR = 3600_000;
const MIN = 60_000;
// ─── Task 1: TypeScript 防抖节流(3小时前完成) ───
const task1Id = uuid();
const session1 = "test-session-debounce";
const task1Start = now - 3 * HOUR;
const task1Chunks = [
{ role: "user", content: "用 TypeScript 写一个防抖函数 debounce,支持 leading 和 trailing 选项。", summary: "用户请求:用 TS 写防抖函数 debounce,支持 leading/trailing", offset: 0 },
{ role: "assistant", content: `好的,下面是一个支持 leading 和 trailing 选项的 TypeScript 防抖函数:
\`\`\`typescript
type DebounceOptions = {
leading?: boolean;
trailing?: boolean;
};
function debounce any>(
fn: T,
delay: number,
options: DebounceOptions = { leading: false, trailing: true }
): (...args: Parameters) => void {
let timer: ReturnType | null = null;
let lastArgs: Parameters | null = null;
return function (this: any, ...args: Parameters) {
const isFirstCall = timer === null;
lastArgs = args;
if (options.leading && isFirstCall) {
fn.apply(this, args);
}
if (timer) clearTimeout(timer);
timer = setTimeout(() => {
if (options.trailing && lastArgs) {
fn.apply(this, lastArgs);
}
timer = null;
lastArgs = null;
}, delay);
};
}
\`\`\`
这个实现支持 \`leading\`(首次立即执行)和 \`trailing\`(延迟后执行)两种模式。默认只开启 trailing。`, summary: "AI 提供了完整的 TypeScript debounce 实现,支持 leading/trailing", offset: 1 },
{ role: "user", content: "再写一个节流 throttle,和 debounce 一起导出。", summary: "用户请求:写 throttle 函数,和 debounce 一起导出", offset: 3 },
{ role: "assistant", content: `好的,下面是 throttle 实现,和 debounce 一起导出:
\`\`\`typescript
function throttle any>(
fn: T,
interval: number
): (...args: Parameters) => void {
let lastTime = 0;
let timer: ReturnType | null = null;
return function (this: any, ...args: Parameters) {
const now = Date.now();
const remaining = interval - (now - lastTime);
if (remaining <= 0) {
if (timer) { clearTimeout(timer); timer = null; }
lastTime = now;
fn.apply(this, args);
} else if (!timer) {
timer = setTimeout(() => {
lastTime = Date.now();
timer = null;
fn.apply(this, args);
}, remaining);
}
};
}
export { debounce, throttle };
\`\`\``, summary: "AI 提供了 throttle 实现,并将 debounce/throttle 一起导出", offset: 4 },
{ role: "user", content: "给这两个函数各写一个单元测试。", summary: "用户请求:为 debounce 和 throttle 写单元测试", offset: 6 },
{ role: "assistant", content: `以下是使用 vitest 的测试:
\`\`\`typescript
import { describe, it, expect, vi } from 'vitest';
import { debounce, throttle } from './utils';
describe('debounce', () => {
it('should delay execution', async () => {
const fn = vi.fn();
const debounced = debounce(fn, 100);
debounced(); debounced(); debounced();
expect(fn).not.toHaveBeenCalled();
await new Promise(r => setTimeout(r, 150));
expect(fn).toHaveBeenCalledTimes(1);
});
});
describe('throttle', () => {
it('should limit call frequency', async () => {
const fn = vi.fn();
const throttled = throttle(fn, 100);
throttled(); throttled(); throttled();
expect(fn).toHaveBeenCalledTimes(1);
await new Promise(r => setTimeout(r, 150));
expect(fn).toHaveBeenCalledTimes(2);
});
});
\`\`\``, summary: "AI 提供了 debounce 和 throttle 的 vitest 单元测试", offset: 7 },
];
// ─── Task 2: 红烧肉菜谱(2小时前完成,同一 session,换话题) ───
const task2Id = uuid();
const task2Start = now - 2 * HOUR;
const task2Chunks = [
{ role: "user", content: "我想学做红烧肉,需要准备哪些食材和步骤?", summary: "用户请求:红烧肉的食材和做法步骤", offset: 0 },
{ role: "assistant", content: `红烧肉的做法如下:
**食材**
- 五花肉 500g(选三层肉最佳)
- 冰糖 30g
- 生抽 2勺、老抽 1勺
- 料酒 2勺
- 葱段、姜片、八角 2个、桂皮 1小段
**步骤**
1. 五花肉切 3cm 方块,冷水下锅焯水,撇去浮沫,捞出洗净。
2. 锅中放少量油,小火放入冰糖,炒至枣红色糖色。
3. 放入五花肉翻炒上色,加入葱姜、八角、桂皮。
4. 加入料酒、生抽、老抽翻炒均匀。
5. 加入没过肉的热水,大火烧开后转小火炖 60 分钟。
6. 最后大火收汁至浓稠即可。
小贴士:全程不要加盐,生抽和老抽的咸度已经足够。`, summary: "AI 提供了红烧肉完整做法:食材清单 + 6 步骤 + 小贴士", offset: 0 },
{ role: "user", content: "做的时候火候要注意什么?炖多久最好?", summary: "用户追问:红烧肉的火候和炖煮时长", offset: 2 },
{ role: "assistant", content: `火候是红烧肉好吃的关键:
1. **炒糖色**:全程小火,冰糖变枣红色冒小泡时立刻下肉,大火会焦苦。
2. **炖煮**:大火烧开后务必转**小火**(微沸、表面冒小泡),炖 **60~90 分钟**。小火慢炖才能让肉软烂入味又不柴。
3. **收汁**:最后 5 分钟转大火收浓,一边翻动防止粘底。
时长参考:普通锅 60~90 分钟;压力锅 25~30 分钟。肉能用筷子轻松戳透就是炖好了。`, summary: "AI 讲解红烧肉火候要点:炒糖色小火、炖煮小火60-90分钟、最后大火收汁", offset: 3 },
];
// ─── Task 3: React 性能优化(1小时前完成,新 session) ───
const task3Id = uuid();
const session2 = "test-session-react-perf";
const task3Start = now - 1 * HOUR;
const task3Chunks = [
{ role: "user", content: "我的 React 应用渲染很慢,列表有几千条数据,怎么优化?", summary: "用户请求:React 列表几千条数据渲染慢,求优化方案", offset: 0 },
{ role: "assistant", content: `几千条数据的列表优化,按优先级推荐:
**1. 虚拟滚动(最有效)**
使用 \`react-window\` 或 \`react-virtuoso\`,只渲染可视区域内的 DOM 节点。
\`\`\`tsx
import { FixedSizeList } from 'react-window';
{({ index, style }) =>
{items[index].name}
}
\`\`\`
**2. React.memo + useMemo**
避免父组件重渲导致所有列表项重新渲染。
**3. key 优化**
确保 key 是稳定唯一值(如 id),不要用 index。
**4. 分页 / 无限滚动**
如果数据来自 API,不要一次拉全量,用分页或 cursor-based pagination。
**5. 状态下沉**
把频繁变化的状态(如 hover/selected)放到子组件,不要提升到列表层。`, summary: "AI 提供 React 列表性能优化方案:虚拟滚动、React.memo、key 优化、分页、状态下沉", offset: 0 },
{ role: "user", content: "react-window 和 react-virtuoso 哪个更好用?有什么区别?", summary: "用户追问:react-window vs react-virtuoso 的区别和选择", offset: 2 },
{ role: "assistant", content: `两者对比:
| | react-window | react-virtuoso |
|---|---|---|
| 包大小 | ~6KB | ~30KB |
| 动态行高 | 需用 VariableSizeList + 手动测量 | 内建支持,自动测量 |
| 分组/表头 | 不支持 | 支持 GroupedVirtuoso |
| 无限滚动 | 需额外组合 react-window-infinite-loader | 内建 endReached |
| TypeScript | 需装 @types | 原生 TS |
| API 简洁度 | 较底层 | 较高层 |
**选择建议**:
- 如果行高固定且追求极致性能 → **react-window**
- 如果行高不固定、需要分组、或想快速集成 → **react-virtuoso**
大多数业务场景推荐 **react-virtuoso**,开箱即用更省心。`, summary: "AI 对比 react-window 和 react-virtuoso:大小、动态行高、分组、无限滚动等,推荐大多数场景用 react-virtuoso", offset: 3 },
];
// ─── Task 4: 正在进行的任务 — Git 工作流(10分钟前开始,active) ───
const task4Id = uuid();
const session3 = "test-session-git-workflow";
const task4Start = now - 10 * MIN;
const task4Chunks = [
{ role: "user", content: "帮我梳理一下 Git 常用工作流,特别是 feature branch 和 trunk-based 的区别。", summary: "用户请求:梳理 Git 工作流,feature branch vs trunk-based", offset: 0 },
{ role: "assistant", content: `两种主流 Git 工作流对比:
## Feature Branch 工作流
- 每个功能/修复从 main 拉出独立分支
- 开发完成后通过 PR/MR 合并回 main
- **优点**:隔离性强、Code Review 自然、适合大团队
- **缺点**:长期分支容易 merge conflict、集成延迟
\`\`\`
main ──────●──────●──────●──────
\\ \\ /
feature-A ●──●──●──────○
\\ /
feature-B ●──●──○
\`\`\`
## Trunk-Based 工作流
- 所有人直接向 main(trunk)提交,或用极短生命周期分支(<1天)
- 依赖 Feature Flag 控制未完成功能的可见性
- **优点**:持续集成、冲突少、部署频率高
- **缺点**:需要完善的 CI/CD 和 Feature Flag 基础设施
\`\`\`
main ──●──●──●──●──●──●──●──
\`\`\`
**选择建议**:
- 小团队(<5人)、持续部署 → Trunk-Based
- 大团队、发布周期长、需要严格审查 → Feature Branch`, summary: "AI 对比了 Feature Branch 和 Trunk-Based 两种 Git 工作流,含优缺点和选择建议", offset: 0 },
];
// ─── Insert ───
const insertTask = db.prepare(`
INSERT OR REPLACE INTO tasks (id, session_key, title, summary, status, started_at, ended_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
`);
const insertChunk = db.prepare(`
INSERT OR REPLACE INTO chunks (id, session_key, turn_id, seq, role, content, kind, summary, task_id, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
function seedTask(
taskId: string,
sessionKey: string,
title: string,
summary: string,
status: "active" | "completed",
startedAt: number,
endedAt: number | null,
chunks: Array<{ role: string; content: string; summary: string; offset: number }>,
) {
insertTask.run(taskId, sessionKey, title, summary, status, startedAt, endedAt, now);
for (const c of chunks) {
const chunkId = uuid();
const turnId = `turn-${c.offset}-${Math.random().toString(36).slice(2, 6)}`;
const chunkTs = startedAt + c.offset * MIN;
insertChunk.run(
chunkId,
sessionKey,
turnId,
c.offset,
c.role,
c.content,
c.content.includes("```") ? "code_block" : "paragraph",
c.summary,
taskId,
chunkTs,
chunkTs,
);
}
}
const insertAll = db.transaction(() => {
seedTask(
task1Id, session1,
"TypeScript 防抖 debounce 与节流 throttle 实现",
`🎯 Goal
用 TypeScript 实现防抖 debounce 和节流 throttle 函数,并编写单元测试。
📋 Key Steps
- 实现 debounce 函数:支持 leading(首次立即执行)和 trailing(延迟后执行)两种模式,通过 DebounceOptions 配置
- 实现 throttle 函数:通过时间戳间隔限制调用频率,支持尾调用
- 两个函数通过 export { debounce, throttle } 一起导出
- 使用 vitest 编写单元测试:测试 debounce 的延迟执行、测试 throttle 的频率限制
✅ Result
两个函数均已实现并通过测试,支持泛型类型推断,可直接导入使用。`,
"completed", task1Start, task1Start + 30 * MIN,
task1Chunks,
);
seedTask(
task2Id, session1,
"红烧肉做法与火候技巧",
`🎯 Goal
学做红烧肉,了解食材、步骤和火候要点。
📋 Key Steps
- 食材准备:五花肉 500g、冰糖 30g、生抽 2 勺、老抽 1 勺、料酒 2 勺、葱姜八角桂皮
- 制作流程:冷水焯水 → 小火炒冰糖至枣红色 → 五花肉翻炒上色 → 加调料和热水 → 小火炖 60-90 分钟 → 大火收汁
- 火候要点:炒糖色全程小火(大火会焦苦);炖煮保持小火微沸;最后 5 分钟大火收汁翻动防粘底
✅ Result
掌握了完整红烧肉做法。全程不加盐(生抽老抽已够)。压力锅可缩短至 25-30 分钟。`,
"completed", task2Start, task2Start + 15 * MIN,
task2Chunks,
);
seedTask(
task3Id, session2,
"React 长列表性能优化方案",
`🎯 Goal
优化 React 应用中几千条数据的列表渲染性能。
📋 Key Steps
- 方案 1(最有效):虚拟滚动,使用 react-window 或 react-virtuoso,只渲染可视区域 DOM
- 方案 2:React.memo + useMemo 避免父组件重渲导致列表项全部重新渲染
- 方案 3:key 使用稳定唯一值(如 id),不用 index
- 方案 4:分页或 cursor-based pagination,不一次拉全量数据
- 方案 5:状态下沉,把 hover/selected 等频繁变化的状态放到子组件
- 对比 react-window(6KB、底层、适合固定行高)vs react-virtuoso(30KB、高层、支持动态行高和分组)
✅ Result
推荐大多数业务场景使用 react-virtuoso(开箱即用),追求极致性能且行高固定时用 react-window。`,
"completed", task3Start, task3Start + 20 * MIN,
task3Chunks,
);
seedTask(
task4Id, session3,
"Git 工作流:Feature Branch vs Trunk-Based",
"",
"active", task4Start, null,
task4Chunks,
);
});
insertAll();
const taskCount = (db.prepare("SELECT COUNT(*) as c FROM tasks WHERE id IN (?,?,?,?)").get(task1Id, task2Id, task3Id, task4Id) as { c: number }).c;
const chunkCount = (db.prepare("SELECT COUNT(*) as c FROM chunks WHERE task_id IN (?,?,?,?)").get(task1Id, task2Id, task3Id, task4Id) as { c: number }).c;
console.log(`✅ 插入完成!`);
console.log(` Tasks: ${taskCount} 个(3 completed + 1 active)`);
console.log(` Chunks: ${chunkCount} 条记忆`);
console.log(``);
console.log(`📋 测试数据概览:`);
console.log(` Task 1: "TypeScript 防抖 debounce 与节流 throttle 实现" — completed, session=${session1}`);
console.log(` Task 2: "红烧肉做法与火候技巧" — completed, session=${session1}(同 session 换话题)`);
console.log(` Task 3: "React 长列表性能优化方案" — completed, session=${session2}(新 session)`);
console.log(` Task 4: "Git 工作流:Feature Branch vs Trunk-Based" — active, session=${session3}(进行中)`);
console.log(``);
console.log(`🌐 打开 Memory Viewer 查看: http://127.0.0.1:18799/`);
db.close();
================================================
FILE: apps/memos-local-openclaw/scripts/smoke-test.ts
================================================
/**
* Smoke Test — 用真实 API 跑通完整链路
*
* 用法:
* npx tsx scripts/smoke-test.ts
*
* 需要先在 .env 中配置好 EMBEDDING / SUMMARIZER 的 key 和 endpoint
*/
import * as fs from "fs";
import * as path from "path";
import * as os from "os";
import { initPlugin } from "../src/index";
// ─── 加载 .env ───
const envPath = path.join(__dirname, "..", ".env");
if (fs.existsSync(envPath)) {
for (const line of fs.readFileSync(envPath, "utf-8").split("\n")) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
const eq = trimmed.indexOf("=");
if (eq > 0) {
process.env[trimmed.slice(0, eq)] = trimmed.slice(eq + 1);
}
}
}
// ─── 配色输出 ───
const GREEN = "\x1b[32m";
const RED = "\x1b[31m";
const CYAN = "\x1b[36m";
const YELLOW = "\x1b[33m";
const RESET = "\x1b[0m";
const BOLD = "\x1b[1m";
function ok(msg: string) { console.log(`${GREEN} ✓ ${msg}${RESET}`); }
function fail(msg: string) { console.log(`${RED} ✗ ${msg}${RESET}`); }
function section(msg: string) { console.log(`\n${BOLD}${CYAN}━━━ ${msg} ━━━${RESET}`); }
function info(msg: string) { console.log(`${YELLOW} ℹ ${msg}${RESET}`); }
async function main() {
console.log(`\n${BOLD}🧪 MemOS Local for OpenClaw — Smoke Test${RESET}`);
console.log(` Embedding: ${process.env.EMBEDDING_ENDPOINT ?? "local"}`);
console.log(` Summarizer: ${process.env.SUMMARIZER_ENDPOINT ?? "rule-based fallback"}`);
// ─── 1. 初始化插件 ───
section("1. 初始化插件");
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "memos-smoke-"));
info(`临时数据库目录: ${tmpDir}`);
const plugin = initPlugin({
stateDir: tmpDir,
config: {
embedding: {
provider: "openai_compatible",
endpoint: process.env.EMBEDDING_ENDPOINT,
apiKey: process.env.EMBEDDING_API_KEY,
model: process.env.EMBEDDING_MODEL ?? "bge-m3",
},
summarizer: {
provider: "openai_compatible",
endpoint: process.env.SUMMARIZER_ENDPOINT,
apiKey: process.env.SUMMARIZER_API_KEY,
model: process.env.SUMMARIZER_MODEL ?? "gpt-4o-mini",
temperature: 0,
},
},
});
ok("插件初始化成功");
// ─── 2. 写入测试对话 ───
section("2. 写入测试对话");
plugin.onConversationTurn([
{
role: "user",
content:
"我正在把 API 服务部署到 port 8443,用的命令是 `docker compose -f docker-compose.prod.yml up -d`。" +
"Postgres 密码配在 POSTGRES_PASSWORD 环境变量里。另外 Nginx 反代配置在 /etc/nginx/conf.d/api.conf。",
},
{
role: "assistant",
content:
"好的,我帮你确认部署。确保防火墙放行 8443 端口,POSTGRES_PASSWORD 要在 .env 里设置。" +
"docker-compose.prod.yml 里建议配置 health check,Nginx 反代记得设 proxy_set_header。",
},
], "session-deploy");
info("第 1 轮: 部署相关对话已入队");
plugin.onConversationTurn([
{
role: "user",
content:
"现在来讨论前端。我们用的 Next.js 14 + App Router,入口页是 app/page.tsx," +
"数据从 /api/dashboard 接口拉取。样式用的 Tailwind CSS v3.4。",
},
{
role: "assistant",
content:
"Next.js 14 App Router 默认用 Server Components,app/page.tsx 可以直接 async fetch。" +
"/api/dashboard 对应 app/api/dashboard/route.ts。Tailwind 3.4 记得在 tailwind.config.ts 里配 content 路径。",
},
], "session-frontend");
info("第 2 轮: 前端相关对话已入队");
plugin.onConversationTurn([
{
role: "user",
content: `构建出错了:
Error: Module not found: Can't resolve '@/components/Chart'
at ModuleNotFoundError (webpack/lib/ModuleNotFoundError.js:28:12)
at factorize (webpack/lib/Compilation.js:2045:24)
at resolve (webpack/lib/NormalModuleFactory.js:439:20)
应该是 tsconfig.json 的 path alias 配错了。`,
},
{
role: "assistant",
content:
'这是 @/components/Chart 的 path alias 找不到。检查 tsconfig.json 的 paths 配置:' +
'"@/*": ["./src/*"],同时确认 next.config.js 没有覆盖 webpack resolve。',
},
], "session-frontend");
info("第 3 轮: 报错相关对话已入队");
// 写入一条带 [STORED_MEMORY] wrapper 的消息,验证防回写
plugin.onConversationTurn([
{
role: "assistant",
content: "根据记忆 [STORED_MEMORY]旧数据: port 3000[/STORED_MEMORY] 实际端口是 8443。",
},
], "session-deploy");
info("第 4 轮: 带防回写标记的消息已入队");
// ─── 等待异步 ingest 完成 ───
info("等待所有异步写入完成...");
await plugin.flush();
ok("所有对话已完成写入(chunking → summary → embedding → 持久化)");
// ─── 3. 测试 memory_search ───
section("3. memory_search — 检索部署细节");
const searchTool = plugin.tools.find((t) => t.name === "memory_search")!;
const r1 = (await searchTool.handler({ query: "docker 部署 端口 8443" })) as any;
console.log(` 命中 ${r1.hits.length} 条 (minScore=${r1.meta.usedMinScore}, maxResults=${r1.meta.usedMaxResults})`);
if (r1.hits.length > 0) {
ok(`Top hit score=${r1.hits[0].score}`);
info(`Summary: ${r1.hits[0].summary.slice(0, 120)}...`);
info(`Excerpt: ${r1.hits[0].original_excerpt.slice(0, 120)}...`);
info(`Ref: session=${r1.hits[0].ref.sessionKey}, chunk=${r1.hits[0].ref.chunkId.slice(0, 8)}...`);
} else {
fail("未命中任何结果!检查 embedding API 是否正常");
}
section("3b. memory_search — 检索前端细节");
const r2 = (await searchTool.handler({ query: "Next.js App Router page.tsx" })) as any;
console.log(` 命中 ${r2.hits.length} 条`);
if (r2.hits.length > 0) {
ok(`Top hit score=${r2.hits[0].score}`);
info(`Excerpt: ${r2.hits[0].original_excerpt.slice(0, 120)}...`);
} else {
fail("未命中前端相关结果");
}
section("3c. memory_search — 检索报错信息");
const r3 = (await searchTool.handler({ query: "Module not found Chart component 报错" })) as any;
console.log(` 命中 ${r3.hits.length} 条`);
if (r3.hits.length > 0) {
ok(`Top hit score=${r3.hits[0].score}`);
info(`Excerpt: ${r3.hits[0].original_excerpt.slice(0, 120)}...`);
} else {
fail("未命中报错相关结果");
}
section("3d. memory_search — 重复查询检测");
const r4 = (await searchTool.handler({ query: "docker 部署 端口 8443" })) as any;
if (r4.meta.note && r4.meta.note.includes("already")) {
ok(`重复查询检测生效: "${r4.meta.note.slice(0, 80)}..."`);
} else {
info("重复查询检测未触发(可能参数不完全相同)");
}
// ─── 4. 测试 memory_timeline ───
section("4. memory_timeline — 拉邻近上下文");
if (r1.hits.length > 0) {
const timelineTool = plugin.tools.find((t) => t.name === "memory_timeline")!;
const tl = (await timelineTool.handler({ ref: r1.hits[0].ref, window: 2 })) as any;
console.log(` 拉到 ${tl.entries.length} 条相邻上下文`);
for (const entry of tl.entries) {
const tag = entry.relation === "current" ? "→" : " ";
info(`${tag} [${entry.relation}] ${entry.role}: ${entry.excerpt.slice(0, 80)}...`);
}
ok("Timeline 返回正常");
} else {
info("跳过(无 search hit 可用)");
}
// ─── 5. 测试 memory_get ───
section("5. memory_get — 获取完整原文");
if (r1.hits.length > 0) {
const getTool = plugin.tools.find((t) => t.name === "memory_get")!;
const g = (await getTool.handler({ ref: r1.hits[0].ref, maxChars: 500 })) as any;
ok(`获取到 ${g.content.length} 字符原文`);
info(`原文: ${g.content.slice(0, 150)}...`);
info(`Source: ts=${new Date(g.source.ts).toISOString()}, role=${g.source.role}`);
} else {
info("跳过(无 search hit 可用)");
}
// ─── 6. 验证防回写 ───
section("6. 防回写验证");
const r5 = (await searchTool.handler({ query: "旧数据 port 3000" })) as any;
let antiWritebackOk = true;
for (const hit of r5.hits) {
if (hit.original_excerpt.includes("[STORED_MEMORY]") || hit.original_excerpt.includes("旧数据: port 3000")) {
fail(`检测到回写内容泄漏: ${hit.original_excerpt.slice(0, 80)}`);
antiWritebackOk = false;
}
}
if (antiWritebackOk) {
ok("防回写验证通过 — [STORED_MEMORY] 包裹的内容未入库");
}
// ─── 清理 ───
section("🏁 测试结束");
plugin.shutdown();
const passed = [r1.hits.length > 0, r2.hits.length > 0, r3.hits.length > 0, antiWritebackOk];
const total = passed.length;
const passCount = passed.filter(Boolean).length;
console.log(`\n${BOLD} 结果: ${passCount}/${total} 核心场景通过${RESET}`);
if (passCount === total) {
console.log(`${GREEN}${BOLD} 🎉 全部通过!插件可以正式接入 OpenClaw 使用了。${RESET}\n`);
} else {
console.log(`${YELLOW}${BOLD} ⚠ 部分场景未通过,请检查上方输出。${RESET}\n`);
}
fs.rmSync(tmpDir, { recursive: true, force: true });
process.exit(passCount === total ? 0 : 1);
}
main().catch((err) => {
console.error(`${RED}Fatal error: ${err}${RESET}`);
process.exit(1);
});
================================================
FILE: apps/memos-local-openclaw/scripts/start-viewer.ts
================================================
/**
* Standalone Viewer launcher — starts the Memory Viewer web UI
* without needing the full OpenClaw plugin lifecycle.
*
* Usage:
* npx tsx scripts/start-viewer.ts
*/
import * as fs from "fs";
import * as path from "path";
import * as os from "os";
import { fileURLToPath } from "url";
import { SqliteStore } from "../src/storage/sqlite";
import { Embedder } from "../src/embedding";
import { ViewerServer } from "../src/viewer/server";
import { buildContext } from "../src/config";
import type { Logger } from "../src/types";
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const envPath = path.join(__dirname, "..", ".env");
if (fs.existsSync(envPath)) {
for (const line of fs.readFileSync(envPath, "utf-8").split("\n")) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
const eq = trimmed.indexOf("=");
if (eq > 0) {
process.env[trimmed.slice(0, eq)] = trimmed.slice(eq + 1);
}
}
}
const log: Logger = {
info: (msg: string) => console.log(`\x1b[36m ℹ ${msg}\x1b[0m`),
warn: (msg: string) => console.log(`\x1b[33m ⚠ ${msg}\x1b[0m`),
error: (msg: string) => console.log(`\x1b[31m ✗ ${msg}\x1b[0m`),
debug: (msg: string) => console.log(`\x1b[90m · ${msg}\x1b[0m`),
};
async function main() {
const dataDir = path.join(os.homedir(), ".memos-local");
fs.mkdirSync(dataDir, { recursive: true });
const dbPath = path.join(dataDir, "memos.db");
log.info(`Database: ${dbPath}`);
const store = new SqliteStore(dbPath, log);
const embedder = new Embedder(
{
provider: "openai_compatible" as any,
endpoint: process.env.EMBEDDING_ENDPOINT,
apiKey: process.env.EMBEDDING_API_KEY,
model: process.env.EMBEDDING_MODEL ?? "bge-m3",
},
log,
);
const port = parseInt(process.env.VIEWER_PORT ?? "18799", 10);
const ctx = buildContext(dataDir, process.cwd(), undefined, log);
const viewer = new ViewerServer({ store, embedder, port, log, dataDir, ctx });
const url = await viewer.start();
console.log();
console.log(`\x1b[1m╔══════════════════════════════════════════╗\x1b[0m`);
console.log(`\x1b[1m║ 🧠 MemOS Memory Viewer ║\x1b[0m`);
console.log(`\x1b[1m║ → \x1b[36m${url.padEnd(37)}\x1b[0m\x1b[1m║\x1b[0m`);
console.log(`\x1b[1m║ Open in browser to manage memories ║\x1b[0m`);
console.log(`\x1b[1m╚══════════════════════════════════════════╝\x1b[0m`);
console.log();
console.log(`\x1b[90m Reset token: ${viewer.getResetToken()}\x1b[0m`);
console.log(`\x1b[90m Press Ctrl+C to stop\x1b[0m`);
process.on("SIGINT", () => {
viewer.stop();
store.close();
process.exit(0);
});
}
main().catch((err) => {
console.error("Failed to start viewer:", err);
process.exit(1);
});
================================================
FILE: apps/memos-local-openclaw/scripts/test-agent-isolation.ts
================================================
#!/usr/bin/env npx tsx
/**
* Multi-agent data isolation test.
*
* Writes data with different owner tags via initPlugin, then creates
* a separate RecallEngine to verify search isolation with ownerFilter.
*
* Usage:
* npx tsx scripts/test-agent-isolation.ts
*/
import * as fs from "fs";
import * as path from "path";
import { initPlugin } from "../src/index";
import { SqliteStore } from "../src/storage/sqlite";
import { Embedder } from "../src/embedding";
import { RecallEngine } from "../src/recall/engine";
import { buildContext } from "../src/config";
const RUN_ID = Date.now();
const AGENT_A = "iso-test-alpha";
const AGENT_B = "iso-test-beta";
const UNIQUE_A = `AlphaUniqueKey${RUN_ID}`;
const UNIQUE_B = `BetaUniqueKey${RUN_ID}`;
const MSG_A1 = `我正在用 ${UNIQUE_A} 部署一个私有 Redis 缓存集群,配置主从复制和哨兵模式,端口 6379。`;
const MSG_A2 = `${UNIQUE_A} 的 Redis 集群已经部署完成,延迟从 50ms 降到了 3ms,命中率 95%。`;
const MSG_B1 = `帮我设置 ${UNIQUE_B} 的 PostgreSQL 数据库迁移方案,从 v14 升级到 v16,数据量约 500GB。`;
const MSG_B2 = `${UNIQUE_B} 的 PostgreSQL 迁移完成了,用了 pg_upgrade --link 模式,停机只有 2 分钟。`;
let passed = 0;
let failed = 0;
function log(msg: string) {
const t = new Date().toLocaleTimeString("zh-CN", { hour12: false });
console.log(`[${t}] ${msg}`);
}
function assert(name: string, condition: boolean, detail: string) {
if (condition) {
passed++;
log(` ✅ ${name}`);
} else {
failed++;
log(` ❌ ${name}: ${detail}`);
}
}
const silentLog = { debug: () => {}, info: () => {}, warn: () => {}, error: () => {} };
async function main() {
log("═══════════════════════════════════════════════════════");
log(" Multi-Agent Data Isolation Test");
log("═══════════════════════════════════════════════════════");
log(` Agent A: ${AGENT_A} (keyword: ${UNIQUE_A})`);
log(` Agent B: ${AGENT_B} (keyword: ${UNIQUE_B})`);
log("");
const home = process.env.HOME ?? process.env.USERPROFILE ?? "/tmp";
const stateDir = path.join(home, ".openclaw");
const cfgPath = path.join(stateDir, "openclaw.json");
const raw = JSON.parse(fs.readFileSync(cfgPath, "utf-8"));
const pluginCfg = raw?.plugins?.entries?.["memos-local-openclaw-plugin"]?.config ?? {};
// ── Step 1: Ingest data with different owners ──
log("── Step 1: Ingesting data with different agent owners ──");
const plugin = initPlugin({ stateDir, config: pluginCfg, log: silentLog });
const sessionA = `iso-session-a-${RUN_ID}`;
const sessionB = `iso-session-b-${RUN_ID}`;
plugin.onConversationTurn(
[{ role: "user", content: MSG_A1 }, { role: "assistant", content: MSG_A2 }],
sessionA,
`agent:${AGENT_A}`,
);
log(` Enqueued 2 messages for agent:${AGENT_A}`);
plugin.onConversationTurn(
[{ role: "user", content: MSG_B1 }, { role: "assistant", content: MSG_B2 }],
sessionB,
`agent:${AGENT_B}`,
);
log(` Enqueued 2 messages for agent:${AGENT_B}`);
log(" Flushing ingest pipeline...");
await plugin.flush();
log(" Waiting 3s for embedding completion...");
await new Promise((r) => setTimeout(r, 3000));
await plugin.flush();
log(" Done.");
await plugin.shutdown();
// ── Step 2: Open a read-only store + engine for verification ──
log("\n── Step 2: Verify owner tags in raw DB ──");
const ctx = buildContext(stateDir, process.cwd(), pluginCfg, silentLog);
const store = new SqliteStore(ctx.config.storage!.dbPath!, silentLog);
const embedder = new Embedder(ctx.config.embedding, silentLog);
const engine = new RecallEngine(store, embedder, ctx);
const db = (store as any).db;
const chunksA = db.prepare(
`SELECT id, owner, session_key, role, substr(content, 1, 80) as preview
FROM chunks WHERE content LIKE ? AND dedup_status = 'active'`
).all(`%${UNIQUE_A}%`) as any[];
const chunksB = db.prepare(
`SELECT id, owner, session_key, role, substr(content, 1, 80) as preview
FROM chunks WHERE content LIKE ? AND dedup_status = 'active'`
).all(`%${UNIQUE_B}%`) as any[];
log(` Chunks with keyword-A: ${chunksA.length}`);
for (const c of chunksA) {
log(` owner=${c.owner} role=${c.role} preview=${c.preview.slice(0, 50)}...`);
}
log(` Chunks with keyword-B: ${chunksB.length}`);
for (const c of chunksB) {
log(` owner=${c.owner} role=${c.role} preview=${c.preview.slice(0, 50)}...`);
}
assert("Keyword-A chunks exist", chunksA.length > 0, "No chunks — ingest failed");
assert("Keyword-B chunks exist", chunksB.length > 0, "No chunks — ingest failed");
if (chunksA.length > 0) {
const ownersA = new Set(chunksA.map((c: any) => c.owner));
assert(
"Keyword-A owner = agent:" + AGENT_A,
ownersA.size === 1 && ownersA.has(`agent:${AGENT_A}`),
`Got: ${[...ownersA].join(", ")}`,
);
}
if (chunksB.length > 0) {
const ownersB = new Set(chunksB.map((c: any) => c.owner));
assert(
"Keyword-B owner = agent:" + AGENT_B,
ownersB.size === 1 && ownersB.has(`agent:${AGENT_B}`),
`Got: ${[...ownersB].join(", ")}`,
);
}
// ── Step 3: Search isolation via RecallEngine ──
log("\n── Step 3: Search isolation (RecallEngine) ──");
const search = async (query: string, owner: string) =>
engine.search({ query, maxResults: 10, ownerFilter: [`agent:${owner}`, "public"] });
const allowedOwners = (owner: string) => new Set([`agent:${owner}`, "public"]);
const checkHitOwners = (hits: any[], allowed: Set): string[] => {
const violations: string[] = [];
for (const h of hits) {
const chunk = store.getChunk(h.ref.chunkId);
if (chunk && !allowed.has(chunk.owner)) {
violations.push(`chunkId=${h.ref.chunkId} owner=${chunk.owner}`);
}
}
return violations;
};
// 3a. Agent-A searches own keyword — should find own data
const resAA = await search(UNIQUE_A, AGENT_A);
assert("Agent-A finds own keyword-A", resAA.hits.length > 0, `Got ${resAA.hits.length} hits`);
// 3b. Agent-A searches keyword-B — results must only contain Agent-A or public data
const resAB = await search(UNIQUE_B, AGENT_A);
const violationsAB = checkHitOwners(resAB.hits, allowedOwners(AGENT_A));
assert(
"Agent-A results for keyword-B contain NO agent-B data ← ISOLATION",
violationsAB.length === 0,
`Found ${violationsAB.length} leaks: ${violationsAB.join("; ")}`,
);
log(` (Agent-A got ${resAB.hits.length} hits for keyword-B, all from own/public — OK)`);
// 3c. Agent-B searches own keyword — should find own data
const resBB = await search(UNIQUE_B, AGENT_B);
assert("Agent-B finds own keyword-B", resBB.hits.length > 0, `Got ${resBB.hits.length} hits`);
// 3d. Agent-B searches keyword-A — results must only contain Agent-B or public data
const resBA = await search(UNIQUE_A, AGENT_B);
const violationsBA = checkHitOwners(resBA.hits, allowedOwners(AGENT_B));
assert(
"Agent-B results for keyword-A contain NO agent-A data ← ISOLATION",
violationsBA.length === 0,
`Found ${violationsBA.length} leaks: ${violationsBA.join("; ")}`,
);
log(` (Agent-B got ${resBA.hits.length} hits for keyword-A, all from own/public — OK)`);
// 3e. agent:main results should not contain iso-test agents' data
const resMainA = await search(UNIQUE_A, "main");
const violationsMainA = checkHitOwners(resMainA.hits, allowedOwners("main"));
assert(
"agent:main results contain no iso-test-alpha data",
violationsMainA.length === 0,
`Found ${violationsMainA.length} leaks: ${violationsMainA.join("; ")}`,
);
const resMainB = await search(UNIQUE_B, "main");
const violationsMainB = checkHitOwners(resMainB.hits, allowedOwners("main"));
assert(
"agent:main results contain no iso-test-beta data",
violationsMainB.length === 0,
`Found ${violationsMainB.length} leaks: ${violationsMainB.join("; ")}`,
);
// ── Step 4: FTS isolation ──
log("\n── Step 4: FTS isolation ──");
const ftsAA = store.ftsSearch(UNIQUE_A, 10, [`agent:${AGENT_A}`, "public"]);
assert("FTS: Agent-A finds keyword-A", ftsAA.length > 0, `Got ${ftsAA.length}`);
const ftsAB = store.ftsSearch(UNIQUE_B, 10, [`agent:${AGENT_A}`, "public"]);
assert("FTS: Agent-A cannot find keyword-B", ftsAB.length === 0, `Got ${ftsAB.length} — BROKEN!`);
const ftsBB = store.ftsSearch(UNIQUE_B, 10, [`agent:${AGENT_B}`, "public"]);
assert("FTS: Agent-B finds keyword-B", ftsBB.length > 0, `Got ${ftsBB.length}`);
const ftsBA = store.ftsSearch(UNIQUE_A, 10, [`agent:${AGENT_B}`, "public"]);
assert("FTS: Agent-B cannot find keyword-A", ftsBA.length === 0, `Got ${ftsBA.length} — BROKEN!`);
// ── Summary ──
log("\n═══════════════════════════════════════════════════════");
log(` Results: ${passed} passed, ${failed} failed`);
if (failed === 0) {
log(" 🎉 All isolation tests passed!");
} else {
log(" ⚠ Some isolation tests FAILED");
}
log("═══════════════════════════════════════════════════════");
store.close();
process.exit(failed > 0 ? 1 : 0);
}
main().catch((err) => {
console.error("Fatal error:", err);
process.exit(1);
});
================================================
FILE: apps/memos-local-openclaw/skill/browserwing-admin/SKILL.md
================================================
---
name: browserwing-admin
description: Manage and operate BrowserWing — an intelligent browser automation platform. Install dependencies, configure LLM, create/manage/execute automation scripts, use AI-driven exploration to generate scripts, browse the script marketplace, and troubleshoot issues.
---
# BrowserWing Admin Skill
## Overview
BrowserWing is an intelligent browser automation platform that allows you to:
- Record, create, and replay browser automation scripts
- Use AI to autonomously explore websites and generate replayable scripts
- Execute scripts via HTTP API or MCP protocol
- Manage LLM configurations for AI-powered features
**API Base URL:** `http://localhost:8080/api/v1`
**Authentication:** Use `X-BrowserWing-Key: ` header or `Authorization: Bearer `
---
## 1. Installing Google Chrome (Prerequisite)
BrowserWing requires Google Chrome to be installed on the host machine.
### Linux (Debian/Ubuntu)
```bash
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" | sudo tee /etc/apt/sources.list.d/google-chrome.list
sudo apt-get update
sudo apt-get install -y google-chrome-stable
```
### macOS
```bash
brew install --cask google-chrome
```
### Windows
Download and install from: https://www.google.com/chrome/
### Verify Installation
```bash
google-chrome --version
# or on macOS:
# /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --version
```
### Using Remote Chrome (Alternative)
If Chrome is running on a remote machine with debugging enabled:
```bash
google-chrome --remote-debugging-port=9222 --remote-debugging-address=0.0.0.0 --no-sandbox
```
Then configure BrowserWing's `config.toml`:
```toml
[browser]
control_url = 'http://:9222'
```
---
## 2. LLM Configuration
AI features (AI Explorer, Agent chat, smart extraction) require an LLM configuration.
### List LLM Configs
```bash
curl -X GET 'http://localhost:8080/api/v1/llm-configs'
```
### Add LLM Config
```bash
curl -X POST 'http://localhost:8080/api/v1/llm-configs' \
-H 'Content-Type: application/json' \
-d '{
"name": "my-openai",
"provider": "openai",
"api_key": "sk-xxx",
"model": "gpt-4o",
"base_url": "https://api.openai.com/v1",
"is_active": true,
"is_default": true
}'
```
**Supported providers:** `openai`, `anthropic`, `deepseek`, or any OpenAI-compatible endpoint.
### Test LLM Config
```bash
curl -X POST 'http://localhost:8080/api/v1/llm-configs/test' \
-H 'Content-Type: application/json' \
-d '{"name": "my-openai"}'
```
### Update LLM Config
```bash
curl -X PUT 'http://localhost:8080/api/v1/llm-configs/' \
-H 'Content-Type: application/json' \
-d '{"api_key": "sk-new-key", "model": "gpt-4o-mini"}'
```
### Delete LLM Config
```bash
curl -X DELETE 'http://localhost:8080/api/v1/llm-configs/'
```
---
## 3. AI Autonomous Exploration (Generate Scripts Automatically)
Use AI to browse a website, perform a task, and automatically generate a replayable script.
### Start Exploration
```bash
curl -X POST 'http://localhost:8080/api/v1/ai-explore/start' \
-H 'Content-Type: application/json' \
-d '{
"task_desc": "Go to bilibili.com, search for 'AI', and get the first page of video results",
"start_url": "https://www.bilibili.com",
"llm_config_id": "my-openai"
}'
```
**Response:** Returns a session `id` for tracking.
### Stream Exploration Events (SSE)
```bash
curl -N 'http://localhost:8080/api/v1/ai-explore//stream'
```
Returns real-time Server-Sent Events: `thinking`, `tool_call`, `progress`, `error`, `script_ready`, `done`.
### Stop Exploration
```bash
curl -X POST 'http://localhost:8080/api/v1/ai-explore//stop'
```
### Get Generated Script
```bash
curl -X GET 'http://localhost:8080/api/v1/ai-explore//script'
```
### Save Generated Script
```bash
curl -X POST 'http://localhost:8080/api/v1/ai-explore//save'
```
Saves the generated script to the local script library for future replay.
---
## 4. Script Management
### List All Scripts
```bash
curl -X GET 'http://localhost:8080/api/v1/scripts'
```
Returns all local scripts with their `id`, `name`, `description`, `actions`, `tags`, `group`, etc.
### Get Script Details
```bash
curl -X GET 'http://localhost:8080/api/v1/scripts/'
```
### Get Script Schema / Summary
```bash
curl -X GET 'http://localhost:8080/api/v1/scripts/summary'
```
Returns a concise summary of all scripts, including names, descriptions, input parameters (variables), and action counts. Useful for programmatic discovery.
### Create a New Script
```bash
curl -X POST 'http://localhost:8080/api/v1/scripts' \
-H 'Content-Type: application/json' \
-d '{
"name": "Search Bilibili",
"description": "Search for a keyword on Bilibili",
"url": "https://www.bilibili.com",
"actions": [
{"type": "navigate", "url": "https://www.bilibili.com"},
{"type": "click", "identifier": ".nav-search-input"},
{"type": "type", "identifier": ".nav-search-input", "value": "${keyword}"},
{"type": "press_key", "key": "Enter"},
{"type": "wait", "timeout": 3}
]
}'
```
**Variables:** Use `${variable_name}` syntax in action values. These become input parameters when the script is executed.
### Update a Script
```bash
curl -X PUT 'http://localhost:8080/api/v1/scripts/' \
-H 'Content-Type: application/json' \
-d '{"name": "Updated Name", "description": "Updated description"}'
```
### Delete a Script
```bash
curl -X DELETE 'http://localhost:8080/api/v1/scripts/'
```
### Export Scripts as Skill (Convert to SKILL.md)
Convert one or more scripts into a SKILL.md file that can be imported by AI agents (e.g., Claude, Cursor). This allows other AI agents to discover and execute your BrowserWing scripts.
#### Export Selected Scripts
```bash
curl -X POST 'http://localhost:8080/api/v1/scripts/export/skill' \
-H 'Content-Type: application/json' \
-d '{
"script_ids": ["script-id-1", "script-id-2", "script-id-3"]
}'
```
Merges multiple scripts into a single SKILL.md with all their actions, variables, and descriptions.
#### Export All Scripts
```bash
curl -X POST 'http://localhost:8080/api/v1/scripts/export/skill' \
-H 'Content-Type: application/json' \
-d '{"script_ids": []}'
```
Pass an empty `script_ids` array to export **all** scripts into one SKILL.md.
#### Export Executor Skill (Browser Control API)
```bash
curl -X GET 'http://localhost:8080/api/v1/executor/export/skill'
```
Exports the low-level browser automation API as a skill, allowing an AI agent to directly control the browser (navigate, click, type, extract, etc.).
**Workflow: Script → Skill → AI Agent**
```
1. Create scripts (manually, by recording, or via AI exploration)
2. Export them as SKILL.md: POST /scripts/export/skill
3. Place the SKILL.md in your AI agent's skill directory
4. The AI agent can now discover and call your scripts via POST /scripts//play
```
---
## 5. Execute Scripts
### Run a Script by ID
```bash
curl -X POST 'http://localhost:8080/api/v1/scripts//play' \
-H 'Content-Type: application/json' \
-d '{
"variables": {
"keyword": "deepseek"
}
}'
```
**Variables:** Pass values for `${variable_name}` placeholders defined in the script actions.
### Get Play Result (Extracted Data)
```bash
curl -X GET 'http://localhost:8080/api/v1/scripts/play/result'
```
Returns data extracted during the last script execution (e.g., scraped content from `execute_js` actions).
### List Script Execution History
```bash
curl -X GET 'http://localhost:8080/api/v1/script-executions?page=1&page_size=20'
```
---
## 6. Script Marketplace (Remote Scripts)
*Note: The remote script marketplace feature is under development. The following APIs may not be available yet.*
### Browse Marketplace
```bash
# TODO: curl -X GET 'http://localhost:8080/api/v1/marketplace/scripts?category=search&page=1'
```
### Install Script from Marketplace
```bash
# TODO: curl -X POST 'http://localhost:8080/api/v1/marketplace/scripts//install'
```
---
## 7. MCP (Model Context Protocol) Integration
BrowserWing exposes an MCP-compatible endpoint for AI agent integrations.
### MCP SSE Endpoint
```
SSE: http://localhost:8080/api/v1/mcp/sse
Message: http://localhost:8080/api/v1/mcp/sse_message
```
### Check MCP Status
```bash
curl -X GET 'http://localhost:8080/api/v1/mcp/status'
```
### List MCP Commands
```bash
curl -X GET 'http://localhost:8080/api/v1/mcp/commands'
```
Shows all registered MCP tools (browser tools + script-based custom commands).
---
## 8. Prompt Management
System prompts control AI behavior. Users can customize them.
### List All Prompts
```bash
curl -X GET 'http://localhost:8080/api/v1/prompts'
```
### Get a Specific Prompt
```bash
curl -X GET 'http://localhost:8080/api/v1/prompts/'
```
**System prompt IDs:** `system-extractor`, `system-formfiller`, `system-aiagent`, `system-get-mcp-info`, `system-ai-explorer`
### Update a Prompt
```bash
curl -X PUT 'http://localhost:8080/api/v1/prompts/' \
-H 'Content-Type: application/json' \
-d '{"content": "Your custom prompt content here..."}'
```
---
## 9. Browser Instance Management
### List Browser Instances
```bash
curl -X GET 'http://localhost:8080/api/v1/browser/instances'
```
### Start a Browser Instance
```bash
curl -X POST 'http://localhost:8080/api/v1/browser/instances//start'
```
### Stop a Browser Instance
```bash
curl -X POST 'http://localhost:8080/api/v1/browser/instances//stop'
```
---
## 10. Cookie Management
Manage browser cookies — view saved cookies, import cookies (e.g., for authenticated sessions), and delete cookies.
### View Saved Cookies
```bash
curl -X GET 'http://localhost:8080/api/v1/cookies/browser'
```
Returns all cookies saved under the `browser` store ID (the default store). Replace `browser` with a custom store ID if needed.
### Save Current Browser Cookies
```bash
curl -X POST 'http://localhost:8080/api/v1/browser/cookies/save'
```
Saves all cookies from the current browser session to the database. Requires the browser to be running.
### Import Cookies
```bash
curl -X POST 'http://localhost:8080/api/v1/browser/cookies/import' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://example.com",
"cookies": [
{
"name": "session_id",
"value": "abc123",
"domain": ".example.com",
"path": "/",
"secure": true,
"httpOnly": true,
"sameSite": "Lax",
"expires": 1735689600
}
]
}'
```
**Fields:** `name` and `value` are required. `domain`, `path`, `secure`, `httpOnly`, `sameSite`, `expires` are optional (`path` defaults to `/`).
### Delete a Single Cookie
```bash
curl -X POST 'http://localhost:8080/api/v1/browser/cookies/delete' \
-H 'Content-Type: application/json' \
-d '{
"id": "browser",
"name": "session_id",
"domain": ".example.com",
"path": "/"
}'
```
Deletes a specific cookie identified by `name` + `domain` + `path` from the given cookie store.
### Batch Delete Cookies
```bash
curl -X POST 'http://localhost:8080/api/v1/browser/cookies/batch/delete' \
-H 'Content-Type: application/json' \
-d '{
"id": "browser",
"cookies": [
{"name": "session_id", "domain": ".example.com", "path": "/"},
{"name": "tracking", "domain": ".example.com", "path": "/"}
]
}'
```
Deletes multiple cookies at once. Each cookie is identified by `name` + `domain` + `path`.
---
## 11. Troubleshooting
When something goes wrong, follow these steps to diagnose issues.
### Check Service Health
```bash
curl -X GET 'http://localhost:8080/health'
```
### View Logs
BrowserWing logs are stored in the path configured in `config.toml` under `[log] file`.
Default location: `./log/browserwing.log`
```bash
# View last 100 lines of logs
tail -n 100 ./log/browserwing.log
# Follow logs in real-time
tail -f ./log/browserwing.log
# Search for errors
grep -i 'error\|fail\|panic' ./log/browserwing.log | tail -20
```
### Common Issues
**1. Browser won't start**
- Check if Google Chrome is installed: `google-chrome --version`
- On Linux, ensure `--no-sandbox` flag or run as non-root
- Check for lingering Chrome lock files in user data dir (SingletonLock, lockfile)
- If using remote Chrome, verify the `control_url` in `config.toml`
- Try killing existing Chrome processes: `pkill -f chrome`
**2. AI features not working**
- Ensure LLM config is set up and active: `GET /api/v1/llm-configs`
- Test the LLM connection: `POST /api/v1/llm-configs/test`
- Check API key validity and model availability
- Check logs for LLM-related errors
**3. Script execution fails**
- Verify the script exists: `GET /api/v1/scripts/`
- Check if the browser is running: `GET /api/v1/browser/instances`
- Review execution history: `GET /api/v1/script-executions`
- Ensure all required `${variables}` are provided in the play request
- Target website may have changed — try re-recording or updating the script
**4. Page elements not found**
- Use `GET /api/v1/executor/snapshot` to see current page elements
- Elements may have dynamic selectors — prefer RefIDs from snapshot
- Page may not have finished loading — use wait actions
**5. Port conflicts**
- BrowserWing default port: 8080 (configurable in `config.toml` under `[server] port`)
- Chrome debugging port: 9222 (or as configured in `control_url`)
- Check for port usage: `lsof -i :` or `netstat -tlnp | grep `
---
## Quick Start Workflow
Here's how to get up and running:
```
1. Install Chrome (see Section 1)
2. Start BrowserWing: ./browserwing --port 8080
3. Add an LLM config (see Section 2)
4. Choose your approach:
a) AI Exploration: POST /ai-explore/start with a task description
b) Manual Creation: POST /scripts with actions array
c) Web UI: Open http://:8080 in browser to use the visual editor
5. Execute scripts: POST /scripts//play
6. View results: GET /scripts/play/result
```
## API Quick Reference
| Category | Method | Endpoint | Description |
|----------|--------|----------|-------------|
| Health | GET | `/health` | Check service status |
| LLM | GET | `/api/v1/llm-configs` | List LLM configurations |
| LLM | POST | `/api/v1/llm-configs` | Add LLM configuration |
| LLM | POST | `/api/v1/llm-configs/test` | Test LLM connection |
| Explore | POST | `/api/v1/ai-explore/start` | Start AI exploration |
| Explore | GET | `/api/v1/ai-explore/:id/stream` | Stream exploration events |
| Explore | POST | `/api/v1/ai-explore/:id/stop` | Stop exploration |
| Explore | POST | `/api/v1/ai-explore/:id/save` | Save generated script |
| Scripts | GET | `/api/v1/scripts` | List all scripts |
| Scripts | GET | `/api/v1/scripts/:id` | Get script details |
| Scripts | POST | `/api/v1/scripts` | Create new script |
| Scripts | PUT | `/api/v1/scripts/:id` | Update script |
| Scripts | DELETE | `/api/v1/scripts/:id` | Delete script |
| Scripts | GET | `/api/v1/scripts/summary` | Get scripts schema/summary |
| Scripts | POST | `/api/v1/scripts/export/skill` | Export scripts as SKILL.md |
| Execute | POST | `/api/v1/scripts/:id/play` | Execute a script |
| Execute | GET | `/api/v1/scripts/play/result` | Get execution result data |
| Execute | GET | `/api/v1/script-executions` | List execution history |
| Prompts | GET | `/api/v1/prompts` | List all prompts |
| Prompts | PUT | `/api/v1/prompts/:id` | Update prompt |
| Browser | GET | `/api/v1/browser/instances` | List browser instances |
| Cookies | GET | `/api/v1/cookies/:id` | View saved cookies |
| Cookies | POST | `/api/v1/browser/cookies/save` | Save current browser cookies |
| Cookies | POST | `/api/v1/browser/cookies/import` | Import cookies |
| Cookies | POST | `/api/v1/browser/cookies/delete` | Delete a single cookie |
| Cookies | POST | `/api/v1/browser/cookies/batch/delete` | Batch delete cookies |
| MCP | GET | `/api/v1/mcp/status` | MCP server status |
| MCP | GET | `/api/v1/mcp/commands` | List MCP commands |
| Executor | GET | `/api/v1/executor/help` | Executor API help |
| Executor | GET | `/api/v1/executor/snapshot` | Page accessibility snapshot |
| Skill | GET | `/api/v1/executor/export/skill` | Export Executor skill |
| Skill | GET | `/api/v1/admin/export/skill` | Export this Admin skill |
================================================
FILE: apps/memos-local-openclaw/skill/browserwing-executor/SKILL.md
================================================
---
name: browserwing-executor
description: Control browser automation through HTTP API. Supports page navigation, element interaction (click, type, select), data extraction, accessibility snapshot analysis, screenshot, JavaScript execution, and batch operations.
---
# BrowserWing Executor API
## Overview
BrowserWing Executor provides comprehensive browser automation capabilities through HTTP APIs. You can control browser navigation, interact with page elements, extract data, and analyze page structure.
**API Base URL:** `http://localhost:8080/api/v1/executor`
**Authentication:** Use `X-BrowserWing-Key: ` header or `Authorization: Bearer `
## Core Capabilities
- **Page Navigation:** Navigate to URLs, go back/forward, reload
- **Element Interaction:** Click, type, select, hover on page elements
- **Data Extraction:** Extract text, attributes, values from elements
- **Accessibility Analysis:** Get accessibility snapshot to understand page structure
- **Advanced Operations:** Screenshot, JavaScript execution, keyboard input
- **Batch Processing:** Execute multiple operations in sequence
## API Endpoints
### 1. Discover Available Commands
**IMPORTANT:** Always call this endpoint first to see all available commands and their parameters.
```bash
curl -X GET 'http://localhost:8080/api/v1/executor/help'
```
**Response:** Returns complete list of all commands with parameters, examples, and usage guidelines.
**Query specific command:**
```bash
curl -X GET 'http://localhost:8080/api/v1/executor/help?command=extract'
```
### 2. Get Accessibility Snapshot
**CRITICAL:** Always call this after navigation to understand page structure and get element RefIDs.
```bash
curl -X GET 'http://localhost:8080/api/v1/executor/snapshot'
```
**Response Example:**
```json
{
"success": true,
"snapshot_text": "Clickable Elements:\n @e1 Login (role: button)\n @e2 Sign Up (role: link)\n\nInput Elements:\n @e3 Email (role: textbox) [placeholder: your@email.com]\n @e4 Password (role: textbox)"
}
```
**Use Cases:**
- Understand what interactive elements are on the page
- Get element RefIDs (@e1, @e2, etc.) for precise identification
- See element labels, roles, and attributes
- The accessibility tree is cleaner than raw DOM and better for LLMs
- RefIDs are stable references that work reliably across page changes
### 3. Common Operations
#### Navigate to URL
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/navigate' \
-H 'Content-Type: application/json' \
-d '{"url": "https://example.com"}'
```
#### Click Element
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/click' \
-H 'Content-Type: application/json' \
-d '{"identifier": "@e1"}'
```
**Identifier formats:**
- **RefID (Recommended):** `@e1`, `@e2` (from snapshot)
- **CSS Selector:** `#button-id`, `.class-name`
- **XPath:** `//button[@type='submit']`
- **Text:** `Login` (text content)
#### Type Text
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/type' \
-H 'Content-Type: application/json' \
-d '{"identifier": "@e3", "text": "user@example.com"}'
```
#### Extract Data
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/extract' \
-H 'Content-Type: application/json' \
-d '{
"selector": ".product-item",
"fields": ["text", "href"],
"multiple": true
}'
```
#### Wait for Element
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/wait' \
-H 'Content-Type: application/json' \
-d '{"identifier": ".loading", "state": "hidden", "timeout": 10}'
```
#### Batch Operations
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/batch' \
-H 'Content-Type: application/json' \
-d '{
"operations": [
{"type": "navigate", "params": {"url": "https://example.com"}, "stop_on_error": true},
{"type": "click", "params": {"identifier": "@e1"}, "stop_on_error": true},
{"type": "type", "params": {"identifier": "@e3", "text": "query"}, "stop_on_error": true}
]
}'
```
## Instructions
**Step-by-step workflow:**
1. **Discover commands:** Call `GET /help` to see all available operations and their parameters (do this first if unsure).
2. **Navigate:** Use `POST /navigate` to open the target webpage.
3. **Analyze page:** Call `GET /snapshot` to understand page structure and get element RefIDs.
4. **Interact:** Use element RefIDs (like `@e1`, `@e2`) or CSS selectors to:
- Click elements: `POST /click`
- Input text: `POST /type`
- Select options: `POST /select`
- Wait for elements: `POST /wait`
5. **Extract data:** Use `POST /extract` to get information from the page.
6. **Present results:** Format and show extracted data to the user.
## Complete Example
**User Request:** "Search for 'laptop' on example.com and get the first 5 results"
**Your Actions:**
1. Navigate to search page:
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/navigate' \
-H 'Content-Type: application/json' \
-d '{"url": "https://example.com/search"}'
```
2. Get page structure to find search input:
```bash
curl -X GET 'http://localhost:8080/api/v1/executor/snapshot'
```
Response shows: `@e3 Search (role: textbox) [placeholder: Search...]`
3. Type search query:
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/type' \
-H 'Content-Type: application/json' \
-d '{"identifier": "@e3", "text": "laptop"}'
```
4. Press Enter to submit:
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/press-key' \
-H 'Content-Type: application/json' \
-d '{"key": "Enter"}'
```
5. Wait for results to load:
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/wait' \
-H 'Content-Type: application/json' \
-d '{"identifier": ".search-results", "state": "visible", "timeout": 10}'
```
6. Extract search results:
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/extract' \
-H 'Content-Type: application/json' \
-d '{
"selector": ".result-item",
"fields": ["text", "href"],
"multiple": true
}'
```
7. Present the extracted data:
```
Found 15 results for 'laptop':
1. Gaming Laptop - $1299 (https://...)
2. Business Laptop - $899 (https://...)
...
```
## Key Commands Reference
### Navigation
- `POST /navigate` - Navigate to URL
- `POST /go-back` - Go back in history
- `POST /go-forward` - Go forward in history
- `POST /reload` - Reload current page
### Element Interaction
- `POST /click` - Click element (supports: RefID `@e1`, CSS selector, XPath, text content)
- `POST /type` - Type text into input (supports: RefID `@e3`, CSS selector, XPath)
- `POST /select` - Select dropdown option
- `POST /hover` - Hover over element
- `POST /wait` - Wait for element state (visible, hidden, enabled)
- `POST /press-key` - Press keyboard key (Enter, Tab, Ctrl+S, etc.)
### Data Extraction
- `POST /extract` - Extract data from elements (supports multiple elements, custom fields)
- `POST /get-text` - Get element text content
- `POST /get-value` - Get input element value
- `GET /page-info` - Get page URL and title
- `GET /page-text` - Get all page text
- `GET /page-content` - Get full HTML
### Page Analysis
- `GET /snapshot` - Get accessibility snapshot (⭐ **ALWAYS call after navigation**)
- `GET /clickable-elements` - Get all clickable elements
- `GET /input-elements` - Get all input elements
### Advanced
- `POST /screenshot` - Take page screenshot (base64 encoded)
- `POST /evaluate` - Execute JavaScript code
- `POST /batch` - Execute multiple operations in sequence
- `POST /scroll-to-bottom` - Scroll to page bottom
- `POST /resize` - Resize browser window
- `POST /tabs` - Manage browser tabs (list, new, switch, close)
- `POST /fill-form` - Intelligently fill multiple form fields at once
### Debug & Monitoring
- `GET /console-messages` - Get browser console messages (logs, warnings, errors)
- `GET /network-requests` - Get network requests made by the page
- `POST /handle-dialog` - Configure JavaScript dialog (alert, confirm, prompt) handling
- `POST /file-upload` - Upload files to input elements
- `POST /drag` - Drag and drop elements
- `POST /close-page` - Close the current page/tab
## Element Identification
You can identify elements using:
1. **RefID (Recommended):** `@e1`, `@e2`, `@e3`
- Most reliable method - stable across page changes
- Get RefIDs from `/snapshot` endpoint
- Valid for 5 minutes after snapshot
- Example: `"identifier": "@e1"`
- Works with multi-strategy fallback for robustness
2. **CSS Selector:** `#id`, `.class`, `button[type="submit"]`
- Standard CSS selectors
- Example: `"identifier": "#login-button"`
3. **XPath:** `//button[@id='login']`, `//a[contains(text(), 'Submit')]`
- XPath expressions for complex queries
- Example: `"identifier": "//button[@id='login']"`
4. **Text Content:** `Login`, `Sign Up`, `Submit`
- Searches buttons and links with matching text
- Example: `"identifier": "Login"`
5. **ARIA Label:** Elements with `aria-label` attribute
- Automatically searched
## Guidelines
**Before starting:**
- Call `GET /help` if you're unsure about available commands or their parameters
- Ensure browser is started (if not, it will auto-start on first operation)
**During automation:**
- **Always call `/snapshot` after navigation** to get page structure and RefIDs
- **Prefer RefIDs** (like `@e1`) over CSS selectors for reliability and stability
- **Re-snapshot after page changes** to get updated RefIDs
- **Use `/wait`** for dynamic content that loads asynchronously
- **Check element states** before interaction (visible, enabled)
- **Use `/batch`** for multiple sequential operations to improve efficiency
**Error handling:**
- If operation fails, check element identifier and try different format
- For timeout errors, increase timeout value
- If element not found, call `/snapshot` again to refresh page structure
- Explain errors clearly to user with suggested solutions
**Data extraction:**
- Use `fields` parameter to specify what to extract: `["text", "href", "src"]`
- Set `multiple: true` to extract from multiple elements
- Format extracted data in a readable way for user
## Complete Workflow Example
**Scenario:** User wants to login to a website
```
User: "Please log in to example.com with username 'john' and password 'secret123'"
```
**Your Actions:**
**Step 1:** Navigate to login page
```bash
POST http://localhost:8080/api/v1/executor/navigate
{"url": "https://example.com/login"}
```
**Step 2:** Get page structure
```bash
GET http://localhost:8080/api/v1/executor/snapshot
```
Response:
```
Clickable Elements:
@e1 Login (role: button)
Input Elements:
@e2 Username (role: textbox)
@e3 Password (role: textbox)
```
**Step 3:** Enter username
```bash
POST http://localhost:8080/api/v1/executor/type
{"identifier": "@e2", "text": "john"}
```
**Step 4:** Enter password
```bash
POST http://localhost:8080/api/v1/executor/type
{"identifier": "@e3", "text": "secret123"}
```
**Step 5:** Click login button
```bash
POST http://localhost:8080/api/v1/executor/click
{"identifier": "@e1"}
```
**Step 6:** Wait for login success (optional)
```bash
POST http://localhost:8080/api/v1/executor/wait
{"identifier": ".welcome-message", "state": "visible", "timeout": 10}
```
**Step 7:** Inform user
```
"Successfully logged in to example.com!"
```
## Batch Operation Example
**Scenario:** Fill out a form with multiple fields
Instead of making 5 separate API calls, use one batch operation:
```bash
curl -X POST 'http://localhost:8080/api/v1/executor/batch' \
-H 'Content-Type: application/json' \
-d '{
"operations": [
{
"type": "navigate",
"params": {"url": "https://example.com/form"},
"stop_on_error": true
},
{
"type": "type",
"params": {"identifier": "#name", "text": "John Doe"},
"stop_on_error": true
},
{
"type": "type",
"params": {"identifier": "#email", "text": "john@example.com"},
"stop_on_error": true
},
{
"type": "select",
"params": {"identifier": "#country", "value": "United States"},
"stop_on_error": true
},
{
"type": "click",
"params": {"identifier": "#submit"},
"stop_on_error": true
}
]
}'
```
## Best Practices
1. **Discovery first:** If unsure, call `/help` or `/help?command=` to learn about commands
2. **Structure first:** Always call `/snapshot` after navigation to understand the page
3. **Use accessibility indices:** They're more reliable than CSS selectors (elements might have dynamic classes)
4. **Wait for dynamic content:** Use `/wait` before interacting with elements that load asynchronously
5. **Batch when possible:** Use `/batch` for multiple sequential operations
6. **Handle errors gracefully:** Provide clear explanations and suggestions when operations fail
7. **Verify results:** After operations, check if desired outcome was achieved
## Common Scenarios
### Form Filling
1. Navigate to form page
2. Get accessibility snapshot to find input elements and their RefIDs
3. Use `/type` for each field: `@e1`, `@e2`, etc.
4. Use `/select` for dropdowns
5. Click submit button using its RefID
### Data Scraping
1. Navigate to target page
2. Wait for content to load with `/wait`
3. Use `/extract` with CSS selector and `multiple: true`
4. Specify fields to extract: `["text", "href", "src"]`
### Search Operations
1. Navigate to search page
2. Get accessibility snapshot to locate search input
3. Type search query into input
4. Press Enter or click search button
5. Wait for results
6. Extract results data
### Login Automation
1. Navigate to login page
2. Get accessibility snapshot to find RefIDs
3. Type username: `@e2`
4. Type password: `@e3`
5. Click login button: `@e1`
6. Wait for success indicator
## Important Notes
- Browser must be running (it will auto-start on first operation if needed)
- Operations are executed on the **currently active browser tab**
- Accessibility snapshot updates after each navigation and click operation
- All timeouts are in seconds
- Use `wait_visible: true` (default) for reliable element interaction
- Replace `localhost:8080` with actual API host address
- Authentication required: use `X-BrowserWing-Key` header or JWT token
## Troubleshooting
**Element not found:**
- Call `/snapshot` to see available elements
- Try different identifier format (accessibility index, CSS selector, text)
- Check if page has finished loading
**Timeout errors:**
- Increase timeout value in request
- Check if element actually appears on page
- Use `/wait` with appropriate state before interaction
**Extraction returns empty:**
- Verify CSS selector matches target elements
- Check if content has loaded (use `/wait` first)
- Try different extraction fields or type
## Quick Reference
```bash
# Discover commands
GET localhost:8080/api/v1/executor/help
# Navigate
POST localhost:8080/api/v1/executor/navigate {"url": "..."}
# Get page structure
GET localhost:8080/api/v1/executor/snapshot
# Click element
POST localhost:8080/api/v1/executor/click {"identifier": "@e1"}
# Type text
POST localhost:8080/api/v1/executor/type {"identifier": "@e3", "text": "..."}
# Extract data
POST localhost:8080/api/v1/executor/extract {"selector": "...", "fields": [...], "multiple": true}
```
## Response Format
All operations return:
```json
{
"success": true,
"message": "Operation description",
"timestamp": "2026-01-15T10:30:00Z",
"data": {
// Operation-specific data
}
}
```
**Error response:**
```json
{
"error": "error.operationFailed",
"detail": "Detailed error message"
}
```
================================================
FILE: apps/memos-local-openclaw/skill/memos-memory-guide/SKILL.md
================================================
---
name: memos-memory-guide
description: "Use the MemOS Local memory system to search and use the user's past conversations. Use this skill whenever the user refers to past chats, their own preferences or history, or when you need to answer from prior context. When auto-recall returns nothing (long or unclear user query), generate your own short search query and call memory_search. Available tools: memory_search, memory_get, memory_write_public, task_summary, skill_get, skill_search, skill_install, skill_publish, skill_unpublish, memory_timeline, memory_viewer."
---
# MemOS Local Memory — Agent Guide
This skill describes how to use the MemOS memory tools so you can reliably search and use the user's long-term conversation history, share knowledge across agents, and discover public skills.
## How memory is provided each turn
- **Automatic recall (hook):** At the start of each turn, the system runs a memory search using the user's current message and injects relevant past memories into your context. You do not need to call any tool for that.
- **When that is not enough:** If the user's message is very long, vague, or the automatic search returns **no memories**, you should **generate your own short, focused query** and call `memory_search` yourself.
- **Memory isolation:** Each agent can only see its own memories and memories marked as `public`. Other agents' private memories are invisible to you.
## Tools — what they do and when to call
### memory_search
- **What it does:** Search long-term conversation memory for past conversations, user preferences, decisions, and experiences. Returns relevant excerpts with `chunkId` and optionally `task_id`. Only returns memories belonging to the current agent or marked as public.
- **When to call:**
- The automatic recall did not run or returned nothing.
- The user's query is long or unclear — **generate a short query yourself** and call `memory_search(query="...")`.
- You need to search with a different angle (e.g. filter by `role='user'`).
- **Parameters:**
- `query` (string, **required**) — Natural language search query.
- `maxResults` (number, optional) — Max results, default 20, max 20.
- `minScore` (number, optional) — Minimum score 0–1, default 0.45, floor 0.35.
- `role` (string, optional) — Filter by role: `'user'`, `'assistant'`, or `'tool'`. Use `'user'` to find what the user said.
### memory_get
- **What it does:** Get the full original text of a memory chunk. Use to verify exact details from a search hit.
- **When to call:** A `memory_search` hit looks relevant but you need to see the complete original content, not just the summary/excerpt.
- **Parameters:**
- `chunkId` (string, **required**) — The chunkId from a search hit.
- `maxChars` (number, optional) — Max characters to return (default 4000, max 12000).
### memory_write_public
- **What it does:** Write a piece of information to public memory. Public memories are visible to all agents during `memory_search`. Use for shared knowledge, team decisions, or cross-agent coordination information.
- **When to call:** In multi-agent or collaborative scenarios, when you have persistent information useful to everyone (e.g. shared decisions, conventions, configurations, workflows). Do not write session-only or purely private content.
- **Parameters:**
- `content` (string, **required**) — The content to write to public memory.
- `summary` (string, optional) — Short summary of the content.
### task_summary
- **What it does:** Get the detailed summary of a complete task: title, status, narrative summary, and related skills. Use when `memory_search` returns a hit with a `task_id` and you need the full story. Preserves critical information: URLs, file paths, commands, error codes, step-by-step instructions.
- **When to call:** A `memory_search` hit included a `task_id` and you need the full context of that task.
- **Parameters:**
- `taskId` (string, **required**) — The task_id from a memory_search hit.
### skill_get
- **What it does:** Retrieve a proven skill (experience guide) by `skillId` or by `taskId`. If you pass a `taskId`, the system will find the associated skill automatically.
- **When to call:** A search hit has a `task_id` and the task has a "how to do this again" guide. Use this to follow the same approach or reuse steps.
- **Parameters:**
- `skillId` (string, optional) — Direct skill ID.
- `taskId` (string, optional) — Task ID — will look up the skill linked to this task.
- At least one of `skillId` or `taskId` must be provided.
### skill_search
- **What it does:** Search available skills by natural language. Searches your own skills, public skills, or both — controlled by the `scope` parameter.
- **When to call:** The current task requires a capability or guide you don't have. Use `skill_search` to find one first; after finding it, use `skill_get` to read it, then `skill_install` to load it for future turns.
- **Parameters:**
- `query` (string, **required**) — Natural language description of the needed skill.
- `scope` (string, optional) — Search scope: `'mix'` (default, self + public), `'self'` (own only), `'public'` (public only).
### skill_install
- **What it does:** Install a learned skill into the agent workspace so it becomes permanently available. After installation, the skill will be loaded automatically in future sessions.
- **When to call:** After `skill_get` when the skill is useful for ongoing use.
- **Parameters:**
- `skillId` (string, **required**) — The skill ID to install.
### skill_publish
- **What it does:** Make a skill public so other agents can discover and install it via `skill_search`.
- **When to call:** You have a useful skill that other agents could benefit from, and you want to share it.
- **Parameters:**
- `skillId` (string, **required**) — The skill ID to publish.
### skill_unpublish
- **What it does:** Make a skill private again. Other agents will no longer be able to discover it.
- **When to call:** You want to stop sharing a previously published skill.
- **Parameters:**
- `skillId` (string, **required**) — The skill ID to unpublish.
### memory_timeline
- **What it does:** Expand context around a memory search hit. Pass the `chunkId` from a search result to read the surrounding conversation messages.
- **When to call:** A `memory_search` hit is relevant but you need the surrounding dialogue.
- **Parameters:**
- `chunkId` (string, **required**) — The chunkId from a memory_search hit.
- `window` (number, optional) — Context window ±N messages, default 2.
### memory_viewer
- **What it does:** Show the MemOS Memory Viewer URL. Call this when the user asks how to view, browse, manage, or check their memories. Returns the URL the user can open in their browser.
- **When to call:** The user asks where to see or manage their memories.
- **Parameters:** None.
## Quick decision flow
1. **No memories in context or auto-recall reported nothing**
→ Call `memory_search(query="...")` with a **self-generated short query**.
2. **Need to see the full original text of a search hit**
→ Call `memory_get(chunkId="...")`.
3. **Search returned hits with `task_id` and you need full context**
→ Call `task_summary(taskId="...")`.
4. **Task has an experience guide you want to follow**
→ Call `skill_get(taskId="...")` or `skill_get(skillId="...")`. Optionally `skill_install(skillId="...")` for future use.
5. **You need the exact surrounding conversation of a hit**
→ Call `memory_timeline(chunkId="...")`.
6. **You need a capability/guide that you don't have**
→ Call `skill_search(query="...", scope="mix")` to discover available skills.
7. **You have shared knowledge useful to all agents**
→ Call `memory_write_public(content="...")` to persist it in public memory.
8. **You want to share/stop sharing a skill with other agents**
→ Call `skill_publish(skillId="...")` or `skill_unpublish(skillId="...")`.
9. **User asks where to see or manage their memories**
→ Call `memory_viewer()` and share the URL.
## Writing good search queries
- Prefer **short, focused** queries (a few words or one clear question).
- Use **concrete terms**: names, topics, tools, or decisions.
- If the user's message is long, **derive one or two sub-queries** rather than pasting the whole message.
- Use `role='user'` when you specifically want to find what the user said.
## Memory ownership and agent isolation
Each memory is tagged with an `owner` (e.g. `agent:main`, `agent:sales-bot`). This is handled **automatically** — you do not need to pass any owner parameter.
- **Your memories:** All tools (`memory_search`, `memory_get`, `memory_timeline`) automatically scope queries to your agent's own memories.
- **Public memories:** Memories marked as `public` are visible to all agents. Use `memory_write_public` to write shared knowledge.
- **Cross-agent isolation:** You cannot see memories owned by other agents (unless they are public).
- **How it works:** The system identifies your agent ID from the OpenClaw runtime context and applies owner filtering automatically on every search, recall, and retrieval.
================================================
FILE: apps/memos-local-openclaw/src/capture/index.ts
================================================
import type { ConversationMessage, Role, Logger } from "../types";
const SKIP_ROLES: Set = new Set(["system"]);
const SYSTEM_BOILERPLATE_RE = /^A new session was started via \/new or \/reset\b/;
const SELF_TOOLS = new Set([
"memory_search",
"memory_timeline",
"memory_get",
"memory_viewer",
"memory_write_public",
"skill_search",
"skill_publish",
"skill_unpublish",
]);
// OpenClaw inbound metadata sentinels — these are AI-facing prefixes,
// not user content. Must be stripped before storing as memory.
const INBOUND_META_SENTINELS = [
"Conversation info (untrusted metadata):",
"Sender (untrusted metadata):",
"Thread starter (untrusted, for context):",
"Replied message (untrusted, for context):",
"Forwarded message context (untrusted metadata):",
"Chat history since last reply (untrusted, for context):",
];
const SENTINEL_FAST_RE = new RegExp(
INBOUND_META_SENTINELS.map((s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")).join("|"),
);
const ENVELOPE_PREFIX_RE =
/^\s*\[(?:Mon|Tue|Wed|Thu|Fri|Sat|Sun)\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}(?::\d{2})?\s+[A-Z]{3}[+-]\d{1,2}\]\s*/;
/**
* Extract writable messages from a conversation turn.
*
* Stores the user's actual text — strips only OpenClaw's injected metadata
* prefixes (Sender info, conversation context, etc.) which are not user content.
* Only skips: system prompts and our own memory tool results (prevents loop).
*/
export function captureMessages(
messages: Array<{ role: string; content: string; toolName?: string }>,
sessionKey: string,
turnId: string,
evidenceTag: string,
log: Logger,
owner?: string,
): ConversationMessage[] {
const now = Date.now();
const result: ConversationMessage[] = [];
for (const msg of messages) {
const role = msg.role as Role;
if (SKIP_ROLES.has(role)) continue;
if (!msg.content || msg.content.trim().length === 0) continue;
if (role === "tool" && msg.toolName && SELF_TOOLS.has(msg.toolName)) {
log.debug(`Skipping self-tool result: ${msg.toolName}`);
continue;
}
if (role === "user" && SYSTEM_BOILERPLATE_RE.test(msg.content.trim())) {
log.debug(`Skipping system boilerplate: ${msg.content.slice(0, 60)}...`);
continue;
}
let content = msg.content;
if (role === "user") {
content = stripInboundMetadata(content);
} else {
content = stripThinkingTags(content);
content = stripEvidenceWrappers(content, evidenceTag);
}
if (!content.trim()) continue;
result.push({
role,
content,
timestamp: now,
turnId,
sessionKey,
toolName: role === "tool" ? msg.toolName : undefined,
owner: owner ?? "agent:main",
});
}
log.debug(`Captured ${result.length}/${messages.length} messages for session=${sessionKey} turn=${turnId} owner=${owner ?? "agent:main"}`);
return result;
}
/**
* Strip OpenClaw-injected inbound metadata blocks from user messages.
*
* These blocks have the shape:
* Sender (untrusted metadata):
* ```json
* { "label": "...", "id": "..." }
* ```
*
* Also strips the envelope timestamp prefix like "[Tue 2026-03-03 21:58 GMT+8] "
*/
export function stripInboundMetadata(text: string): string {
let cleaned = stripMemoryInjection(text);
cleaned = stripEnvelopePrefix(cleaned);
// Strip OpenClaw envelope tags: [message_id: ...], [[reply_to_current]], etc.
cleaned = cleaned.replace(/\[message_id:\s*[a-f0-9-]+\]/gi, "");
cleaned = cleaned.replace(/\[\[reply_to_current\]\]/gi, "");
if (!SENTINEL_FAST_RE.test(cleaned)) {
return stripEnvelopePrefix(cleaned).trim();
}
const lines = cleaned.split("\n");
const result: string[] = [];
let inMetaBlock = false;
let inFencedJson = false;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
const trimmed = line.trim();
if (!inMetaBlock && INBOUND_META_SENTINELS.some((s) => s === trimmed)) {
if (lines[i + 1]?.trim() === "```json") {
inMetaBlock = true;
inFencedJson = false;
continue;
}
continue;
}
if (inMetaBlock) {
if (!inFencedJson && trimmed === "```json") {
inFencedJson = true;
continue;
}
if (inFencedJson && trimmed === "```") {
inMetaBlock = false;
inFencedJson = false;
continue;
}
continue;
}
result.push(line);
}
return stripEnvelopePrefix(result.join("\n")).trim();
}
/** Strip …][\s\S]*?<\/think>\s*/gi;
function stripThinkingTags(text: string): string {
return text.replace(THINKING_TAG_RE, "");
}
function stripEnvelopePrefix(text: string): string {
return text.replace(ENVELOPE_PREFIX_RE, "");
}
/**
* Strip memory-system injections that get prepended to user messages:
* - ...
* - === MemOS LONG-TERM MEMORY ... ===\n...MANDATORY...
* - [MemOS Auto-Recall] Found N relevant memories:...
* - ## Memory system\n\nNo memories were automatically recalled...
*/
function stripMemoryInjection(text: string): string {
let cleaned = text;
// ...
const mcStart = cleaned.indexOf("");
if (mcStart !== -1) {
const mcEnd = cleaned.indexOf("");
if (mcEnd !== -1) {
cleaned = cleaned.slice(0, mcStart) + cleaned.slice(mcEnd + "".length);
} else {
cleaned = cleaned.slice(0, mcStart);
}
cleaned = cleaned.trim();
}
// === MemOS LONG-TERM MEMORY (retrieved from past conversations) ===\n...\nMANDATORY...
cleaned = cleaned.replace(
/=== MemOS LONG-TERM MEMORY[\s\S]*?(?:MANDATORY[^\n]*\n?|(?=\n{2,}))/gi,
"",
).trim();
// [MemOS Auto-Recall] Found N relevant memories:\n...
cleaned = cleaned.replace(
/\[MemOS Auto-Recall\][^\n]*\n(?:(?:\d+\.\s+\[(?:USER|ASSISTANT)[^\n]*\n?)*)/gi,
"",
).trim();
// ## Memory system\n\nNo memories were automatically recalled...
cleaned = cleaned.replace(
/## Memory system\n+No memories were automatically recalled[^\n]*(?:\n[^\n]*memory_search[^\n]*)*/gi,
"",
).trim();
// Old format: ## Retrieved memories from past conversations\n\nCRITICAL INSTRUCTION:...
const recallIdx = cleaned.indexOf("## Retrieved memories from past conversations");
if (recallIdx !== -1) {
const before = cleaned.slice(0, recallIdx);
const after = cleaned.slice(recallIdx);
const tsMatch = after.match(/\n\[(?:Mon|Tue|Wed|Thu|Fri|Sat|Sun)\s+\d{4}-\d{2}-\d{2}/);
if (tsMatch && tsMatch.index != null) {
cleaned = (before + after.slice(tsMatch.index)).trim();
} else {
cleaned = before.trim();
}
}
// prependContext format: ## User's conversation history (from memory system)\n...
// Ends at last "Current time:" line or last chunkId= line, whichever comes later.
const prependIdx = cleaned.indexOf("## User's conversation history (from memory system)");
if (prependIdx !== -1) {
const before = cleaned.slice(0, prependIdx);
const after = cleaned.slice(prependIdx);
// Find the last anchor line that belongs to the injected block
const currentTimeMatch = after.match(/Current time:[^\n]*/g);
const chunkIdMatch = after.match(/chunkId="[^"]*"/g);
let cutPos = 0;
if (currentTimeMatch) {
const lastCt = after.lastIndexOf(currentTimeMatch[currentTimeMatch.length - 1]);
const lineEnd = after.indexOf("\n", lastCt);
cutPos = Math.max(cutPos, lineEnd !== -1 ? lineEnd + 1 : after.length);
}
if (chunkIdMatch) {
const lastCk = after.lastIndexOf(chunkIdMatch[chunkIdMatch.length - 1]);
const lineEnd = after.indexOf("\n", lastCk);
cutPos = Math.max(cutPos, lineEnd !== -1 ? lineEnd + 1 : after.length);
}
if (cutPos === 0) {
// No anchors found; remove everything from the header onward
cleaned = before.trim();
} else {
cleaned = (before + after.slice(cutPos)).trim();
}
}
// New format: ...\n\n📝 Related memories:...
const memosTagIdx = cleaned.indexOf("");
if (memosTagIdx !== -1) {
const before = cleaned.slice(0, memosTagIdx);
const after = cleaned.slice(memosTagIdx);
const tsMatch = after.match(/\n\[(?:Mon|Tue|Wed|Thu|Fri|Sat|Sun)\s+\d{4}-\d{2}-\d{2}/);
if (tsMatch && tsMatch.index != null) {
cleaned = (before + after.slice(tsMatch.index)).trim();
} else {
cleaned = before.trim();
}
}
return cleaned;
}
function stripEvidenceWrappers(text: string, evidenceTag: string): string {
const tag = evidenceTag.trim();
if (!tag) return text;
const escapedTag = tag.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const wrapperRe = new RegExp(`\\[${escapedTag}\\][\\s\\S]*?\\[\\/${escapedTag}\\]`, "g");
return text
.replace(wrapperRe, "")
.replace(/[ \t]{2,}/g, " ")
.replace(/\s+([,.;:!?])/g, "$1")
.replace(/\n{3,}/g, "\n\n")
.trim();
}
================================================
FILE: apps/memos-local-openclaw/src/config.ts
================================================
import * as path from "path";
import { DEFAULTS, type MemosLocalConfig, type PluginContext, type Logger } from "./types";
const ENV_RE = /\$\{([A-Z_][A-Z0-9_]*)\}/g;
function resolveEnvVars(value: string): string {
return value.replace(ENV_RE, (_, name) => process.env[name] ?? "");
}
function deepResolveEnv(obj: T): T {
if (typeof obj === "string") return resolveEnvVars(obj) as unknown as T;
if (Array.isArray(obj)) return obj.map(deepResolveEnv) as unknown as T;
if (obj && typeof obj === "object") {
const out: Record = {};
for (const [k, v] of Object.entries(obj)) {
out[k] = deepResolveEnv(v);
}
return out as T;
}
return obj;
}
export function resolveConfig(raw: Partial | undefined, stateDir: string): MemosLocalConfig {
const cfg = deepResolveEnv(raw ?? {});
const telemetryEnvVar = process.env.TELEMETRY_ENABLED;
const telemetryEnabled =
cfg.telemetry?.enabled ??
(telemetryEnvVar === "false" || telemetryEnvVar === "0" ? false : true);
return {
...cfg,
storage: {
dbPath: cfg.storage?.dbPath ?? path.join(stateDir, "memos-local", "memos.db"),
},
recall: {
maxResultsDefault: cfg.recall?.maxResultsDefault ?? DEFAULTS.maxResultsDefault,
maxResultsMax: cfg.recall?.maxResultsMax ?? DEFAULTS.maxResultsMax,
minScoreDefault: cfg.recall?.minScoreDefault ?? DEFAULTS.minScoreDefault,
minScoreFloor: cfg.recall?.minScoreFloor ?? DEFAULTS.minScoreFloor,
rrfK: cfg.recall?.rrfK ?? DEFAULTS.rrfK,
mmrLambda: cfg.recall?.mmrLambda ?? DEFAULTS.mmrLambda,
recencyHalfLifeDays: cfg.recall?.recencyHalfLifeDays ?? DEFAULTS.recencyHalfLifeDays,
vectorSearchMaxChunks: cfg.recall?.vectorSearchMaxChunks ?? DEFAULTS.vectorSearchMaxChunks,
},
dedup: {
similarityThreshold: cfg.dedup?.similarityThreshold ?? DEFAULTS.dedupSimilarityThreshold,
},
capture: {
evidenceWrapperTag: cfg.capture?.evidenceWrapperTag ?? DEFAULTS.evidenceWrapperTag,
},
telemetry: {
enabled: telemetryEnabled,
},
};
}
export function buildContext(
stateDir: string,
workspaceDir: string,
rawConfig: Partial | undefined,
log?: Logger,
): PluginContext {
const defaultLog: Logger = {
debug: (...args) => console.debug("[memos-local]", ...args),
info: (...args) => console.info("[memos-local]", ...args),
warn: (...args) => console.warn("[memos-local]", ...args),
error: (...args) => console.error("[memos-local]", ...args),
};
return {
stateDir,
workspaceDir,
config: resolveConfig(rawConfig, stateDir),
log: log ?? defaultLog,
};
}
================================================
FILE: apps/memos-local-openclaw/src/embedding/index.ts
================================================
import type { EmbeddingConfig, Logger } from "../types";
import { embedOpenAI } from "./providers/openai";
import { embedGemini } from "./providers/gemini";
import { embedCohere, embedCohereQuery } from "./providers/cohere";
import { embedVoyage } from "./providers/voyage";
import { embedMistral } from "./providers/mistral";
import { embedLocal } from "./local";
import { modelHealth } from "../ingest/providers";
export class Embedder {
constructor(
private cfg: EmbeddingConfig | undefined,
private log: Logger,
) {}
get provider(): string {
return this.cfg?.provider ?? "local";
}
get dimensions(): number {
if (this.provider === "local") return 384;
return this.cfg?.dimensions ?? 1536;
}
async embed(texts: string[]): Promise {
const batchSize = this.cfg?.batchSize ?? 32;
const results: number[][] = [];
for (let i = 0; i < texts.length; i += batchSize) {
const batch = texts.slice(i, i + batchSize);
const vecs = await this.embedBatch(batch);
results.push(...vecs);
}
return results;
}
async embedQuery(text: string): Promise {
if (this.provider === "cohere" && this.cfg) {
return embedCohereQuery(text, this.cfg, this.log);
}
const vecs = await this.embedBatch([text]);
return vecs[0];
}
private async embedBatch(texts: string[]): Promise {
const provider = this.provider;
const cfg = this.cfg;
const modelInfo = `${provider}/${cfg?.model ?? "default"}`;
try {
let result: number[][];
switch (provider) {
case "openai":
case "openai_compatible":
case "azure_openai":
case "zhipu":
case "siliconflow":
case "bailian":
result = await embedOpenAI(texts, cfg!, this.log); break;
case "gemini":
result = await embedGemini(texts, cfg!, this.log); break;
case "cohere":
result = await embedCohere(texts, cfg!, this.log); break;
case "mistral":
result = await embedMistral(texts, cfg!, this.log); break;
case "voyage":
result = await embedVoyage(texts, cfg!, this.log); break;
case "local":
default:
result = await embedLocal(texts, this.log); break;
}
modelHealth.recordSuccess("embedding", modelInfo);
return result;
} catch (err) {
modelHealth.recordError("embedding", modelInfo, String(err));
if (provider !== "local") {
this.log.warn(`Embedding provider '${provider}' failed, falling back to local: ${err}`);
return await embedLocal(texts, this.log);
}
throw err;
}
}
}
================================================
FILE: apps/memos-local-openclaw/src/embedding/local.ts
================================================
import type { Logger } from "../types";
import { DEFAULTS } from "../types";
let extractorPromise: Promise | null = null;
function getExtractor(log: Logger): Promise {
if (extractorPromise) return extractorPromise;
extractorPromise = (async () => {
log.info("Loading local embedding model (first call may download ~23MB)...");
const { pipeline } = await import("@huggingface/transformers");
const ext = await pipeline("feature-extraction", DEFAULTS.localEmbeddingModel, {
dtype: "q8",
device: "cpu",
});
log.info("Local embedding model ready");
return ext;
})().catch((err) => {
extractorPromise = null;
throw err;
});
return extractorPromise;
}
export async function embedLocal(texts: string[], log: Logger): Promise {
const ext = await getExtractor(log);
const results: number[][] = [];
for (const text of texts) {
const output = await ext(text, { pooling: "mean", normalize: true });
results.push(Array.from(output.data as Float32Array).slice(0, DEFAULTS.localEmbeddingDimensions));
}
return results;
}
================================================
FILE: apps/memos-local-openclaw/src/embedding/providers/cohere.ts
================================================
import type { EmbeddingConfig, Logger } from "../../types";
export async function embedCohere(
texts: string[],
cfg: EmbeddingConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.cohere.ai/v1/embed";
const model = cfg.model ?? "embed-english-v3.0";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
texts,
model,
input_type: "search_document",
truncate: "END",
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Cohere embedding failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { embeddings: number[][] };
return json.embeddings;
}
export async function embedCohereQuery(
text: string,
cfg: EmbeddingConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.cohere.ai/v1/embed";
const model = cfg.model ?? "embed-english-v3.0";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
texts: [text],
model,
input_type: "search_query",
truncate: "END",
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Cohere query embedding failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { embeddings: number[][] };
return json.embeddings[0];
}
================================================
FILE: apps/memos-local-openclaw/src/embedding/providers/gemini.ts
================================================
import type { EmbeddingConfig, Logger } from "../../types";
export async function embedGemini(
texts: string[],
cfg: EmbeddingConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "text-embedding-004";
const endpoint =
cfg.endpoint ??
`https://generativelanguage.googleapis.com/v1beta/models/${model}:batchEmbedContents`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const url = `${endpoint}?key=${cfg.apiKey}`;
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
requests: texts.map((text) => ({
model: `models/${model}`,
content: { parts: [{ text }] },
})),
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Gemini embedding failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as {
embeddings: Array<{ values: number[] }>;
};
return json.embeddings.map((e) => e.values);
}
================================================
FILE: apps/memos-local-openclaw/src/embedding/providers/mistral.ts
================================================
import type { EmbeddingConfig, Logger } from "../../types";
export async function embedMistral(
texts: string[],
cfg: EmbeddingConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.mistral.ai/v1/embeddings";
const model = cfg.model ?? "mistral-embed";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({ input: texts, model, encoding_format: "float" }),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Mistral embedding failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as {
data: Array<{ embedding: number[] }>;
};
return json.data.map((d) => d.embedding);
}
================================================
FILE: apps/memos-local-openclaw/src/embedding/providers/openai.ts
================================================
import type { EmbeddingConfig, Logger } from "../../types";
export async function embedOpenAI(
texts: string[],
cfg: EmbeddingConfig,
log: Logger,
): Promise {
const endpoint = normalizeEmbeddingEndpoint(cfg.endpoint ?? "https://api.openai.com/v1/embeddings");
const model = cfg.model ?? "text-embedding-3-small";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({ input: texts, model }),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`OpenAI embedding failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as {
data: Array<{ embedding: number[] }>;
};
return json.data.map((d) => d.embedding);
}
/**
* Normalize endpoint: if user provides a base_url (e.g. https://host/v1)
* without the /embeddings suffix, append it automatically.
*/
function normalizeEmbeddingEndpoint(url: string): string {
const stripped = url.replace(/\/+$/, "");
if (stripped.endsWith("/embeddings")) return stripped;
return `${stripped}/embeddings`;
}
================================================
FILE: apps/memos-local-openclaw/src/embedding/providers/voyage.ts
================================================
import type { EmbeddingConfig, Logger } from "../../types";
export async function embedVoyage(
texts: string[],
cfg: EmbeddingConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.voyageai.com/v1/embeddings";
const model = cfg.model ?? "voyage-2";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({ input: texts, model }),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Voyage embedding failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as {
data: Array<{ embedding: number[] }>;
};
return json.data.map((d) => d.embedding);
}
================================================
FILE: apps/memos-local-openclaw/src/index.ts
================================================
import { v4 as uuid } from "uuid";
import { buildContext } from "./config";
import { ensureSqliteBinding } from "./storage/ensure-binding";
import { SqliteStore } from "./storage/sqlite";
import { Embedder } from "./embedding";
import { IngestWorker } from "./ingest/worker";
import { RecallEngine } from "./recall/engine";
import { captureMessages } from "./capture";
import { createMemorySearchTool, createMemoryTimelineTool, createMemoryGetTool } from "./tools";
import type { MemosLocalConfig, ToolDefinition, Logger } from "./types";
export interface MemosLocalPlugin {
id: string;
tools: ToolDefinition[];
onConversationTurn: (messages: Array<{ role: string; content: string }>, sessionKey?: string, owner?: string) => void;
/** Wait for all pending ingest operations to complete. */
flush: () => Promise;
shutdown: () => Promise;
}
export interface PluginInitOptions {
stateDir?: string;
workspaceDir?: string;
config?: Partial;
log?: Logger;
}
/**
* Initialize the memos-local plugin.
*
* Typical usage inside OpenClaw plugin lifecycle:
*
* ```ts
* import { initPlugin } from "@memos/local-openclaw";
*
* export default function activate(ctx) {
* const plugin = initPlugin({
* stateDir: ctx.stateDir,
* workspaceDir: ctx.workspaceDir,
* config: ctx.pluginConfig,
* log: ctx.log,
* });
* ctx.registerTools(plugin.tools);
* ctx.onConversationTurn((msgs, session) => {
* plugin.onConversationTurn(msgs, session);
* });
* ctx.onDeactivate(() => plugin.shutdown());
* }
* ```
*/
export function initPlugin(opts: PluginInitOptions = {}): MemosLocalPlugin {
const stateDir = opts.stateDir ?? defaultStateDir();
const workspaceDir = opts.workspaceDir ?? process.cwd();
const ctx = buildContext(stateDir, workspaceDir, opts.config, opts.log);
ctx.log.info("Initializing memos-local plugin...");
ensureSqliteBinding(ctx.log);
const store = new SqliteStore(ctx.config.storage!.dbPath!, ctx.log);
const embedder = new Embedder(ctx.config.embedding, ctx.log);
const worker = new IngestWorker(store, embedder, ctx);
const engine = new RecallEngine(store, embedder, ctx);
const tools: ToolDefinition[] = [
createMemorySearchTool(engine),
createMemoryTimelineTool(store),
createMemoryGetTool(store),
];
ctx.log.info(`Plugin ready. DB: ${ctx.config.storage!.dbPath}, Embedding: ${embedder.provider}`);
return {
id: "memos-local",
tools,
onConversationTurn(
messages: Array<{ role: string; content: string }>,
sessionKey?: string,
owner?: string,
): void {
const session = sessionKey ?? "default";
const turnId = uuid();
const tag = ctx.config.capture?.evidenceWrapperTag ?? "STORED_MEMORY";
const captured = captureMessages(messages, session, turnId, tag, ctx.log, owner);
if (captured.length > 0) {
worker.enqueue(captured);
}
},
async flush(): Promise {
await worker.flush();
},
async shutdown(): Promise {
ctx.log.info("Shutting down memos-local plugin...");
await worker.flush();
store.close();
},
};
}
function defaultStateDir(): string {
const home = process.env.HOME ?? process.env.USERPROFILE ?? "/tmp";
return `${home}/.openclaw`;
}
// Re-export types for consumers
export type { MemosLocalConfig, ToolDefinition, SearchResult, SearchHit, TimelineResult, GetResult } from "./types";
================================================
FILE: apps/memos-local-openclaw/src/ingest/chunker.ts
================================================
export interface RawChunk {
content: string;
kind: "paragraph";
}
const MAX_CHUNK_CHARS = 3000;
const MIN_CHUNK_CHARS = 40;
const IDEAL_CHUNK_CHARS = 1500;
const FENCED_CODE_RE = /^(`{3,})[^\n]*\n[\s\S]*?^\1\s*$/gm;
const FUNC_OPEN_RE =
/^[ \t]*(?:(?:export\s+)?(?:async\s+)?(?:function|class|const\s+\w+\s*=\s*(?:\([^)]*\)|[^=])*=>)|(?:def |class )|(?:func |fn |pub\s+fn )|(?:public |private |protected |static )+.*\{)\s*$/;
const BLOCK_CLOSE_RE = /^[ \t]*[}\]]\s*;?\s*$/;
const ERROR_STACK_RE =
/(?:(?:Error|Exception|Traceback)[^\n]*\n(?:\s+at\s+[^\n]+\n?|.*File "[^\n]+\n?|.*line \d+[^\n]*\n?){2,})/gm;
const LIST_BLOCK_RE = /(?:^[\s]*[-*•]\s+.+\n?){3,}/gm;
const COMMAND_LINE_RE = /^(?:\$|>|#)\s+.+$/gm;
/**
* Semantic-aware chunking:
* 1. Extract fenced code blocks as whole units (never split inside)
* 2. Detect unfenced code regions by brace-matching (functions/classes kept intact)
* 3. Extract error stacks, list blocks, command lines as separate chunks
* 4. Split remaining prose at paragraph boundaries (double newline)
* 5. Merge short adjacent chunks
*/
export function chunkText(text: string): RawChunk[] {
let remaining = text;
const slots: Array<{ placeholder: string; content: string }> = [];
let counter = 0;
function ph(content: string): string {
const tag = `\x00SLOT_${counter++}\x00`;
slots.push({ placeholder: tag, content: content.trim() });
return tag;
}
remaining = remaining.replace(FENCED_CODE_RE, (m) => ph(m));
remaining = extractBraceBlocks(remaining, ph);
const structural: RegExp[] = [ERROR_STACK_RE, LIST_BLOCK_RE, COMMAND_LINE_RE];
for (const re of structural) {
remaining = remaining.replace(re, (m) => ph(m));
}
const raw: RawChunk[] = [];
const sections = remaining.split(/\n{2,}/);
for (const sec of sections) {
const trimmed = sec.trim();
if (!trimmed) continue;
if (trimmed.includes("\x00SLOT_")) {
const parts = trimmed.split(/(\x00SLOT_\d+\x00)/);
for (const part of parts) {
const slot = slots.find((s) => s.placeholder === part);
if (slot) {
raw.push({ content: slot.content, kind: "paragraph" });
} else if (part.trim().length >= MIN_CHUNK_CHARS) {
raw.push({ content: part.trim(), kind: "paragraph" });
}
}
} else if (trimmed.length >= MIN_CHUNK_CHARS) {
raw.push({ content: trimmed, kind: "paragraph" });
}
}
for (const s of slots) {
if (!raw.some((c) => c.content === s.content)) {
raw.push({ content: s.content, kind: "paragraph" });
}
}
const merged = mergeSmallChunks(raw);
const final = splitOversized(merged);
return final.length > 0 ? final : [{ content: text.trim(), kind: "paragraph" }];
}
/**
* Detect function/class bodies that aren't inside fenced blocks.
* Tracks brace depth to keep complete blocks together.
*/
function extractBraceBlocks(
text: string,
ph: (content: string) => string,
): string {
const lines = text.split("\n");
const result: string[] = [];
let blockLines: string[] = [];
let depth = 0;
let inBlock = false;
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (line.includes("\x00SLOT_")) {
if (inBlock) {
blockLines.push(line);
} else {
result.push(line);
}
continue;
}
if (!inBlock && FUNC_OPEN_RE.test(line)) {
inBlock = true;
blockLines = [line];
depth = countBraces(line);
if (depth <= 0) depth = 1;
continue;
}
if (inBlock) {
blockLines.push(line);
depth += countBraces(line);
if (depth <= 0 || (BLOCK_CLOSE_RE.test(line) && depth <= 0)) {
const block = blockLines.join("\n");
if (block.trim().length >= MIN_CHUNK_CHARS) {
result.push(ph(block));
} else {
result.push(block);
}
inBlock = false;
blockLines = [];
depth = 0;
}
} else {
result.push(line);
}
}
if (blockLines.length > 0) {
const block = blockLines.join("\n");
if (block.trim().length >= MIN_CHUNK_CHARS) {
result.push(ph(block));
} else {
result.push(block);
}
}
return result.join("\n");
}
function countBraces(line: string): number {
let d = 0;
for (const ch of line) {
if (ch === "{" || ch === "(") d++;
else if (ch === "}" || ch === ")") d--;
}
return d;
}
function mergeSmallChunks(chunks: RawChunk[]): RawChunk[] {
if (chunks.length <= 1) return chunks;
const merged: RawChunk[] = [];
let buf: RawChunk | null = null;
for (const c of chunks) {
if (!buf) {
buf = { ...c };
continue;
}
const bothSmall = buf.content.length < IDEAL_CHUNK_CHARS && c.content.length < IDEAL_CHUNK_CHARS;
const mergedLen = buf.content.length + c.content.length + 2;
if (bothSmall && mergedLen <= MAX_CHUNK_CHARS) {
buf.content = buf.content + "\n\n" + c.content;
} else {
merged.push(buf);
buf = { ...c };
}
}
if (buf) merged.push(buf);
return merged;
}
function splitOversized(chunks: RawChunk[]): RawChunk[] {
const result: RawChunk[] = [];
for (const c of chunks) {
if (c.content.length <= MAX_CHUNK_CHARS) {
result.push(c);
continue;
}
result.push(...splitAtSentenceBoundary(c.content));
}
return result;
}
function splitAtSentenceBoundary(text: string): RawChunk[] {
const sentences = text.match(/[^.!?。!?\n]+(?:[.!?。!?]+|\n{2,})/g) ?? [text];
const result: RawChunk[] = [];
let buf = "";
for (const s of sentences) {
if (buf.length + s.length > MAX_CHUNK_CHARS && buf.length > 0) {
result.push({ content: buf.trim(), kind: "paragraph" });
buf = "";
}
buf += s;
}
if (buf.trim().length >= MIN_CHUNK_CHARS) {
result.push({ content: buf.trim(), kind: "paragraph" });
}
return result;
}
================================================
FILE: apps/memos-local-openclaw/src/ingest/dedup.ts
================================================
import { cosineSimilarity } from "../storage/vector";
import type { SqliteStore } from "../storage/sqlite";
import type { Logger } from "../types";
/**
* Check if a new summary embedding is a near-duplicate of any
* existing embedding. If similarity >= threshold, return the
* existing chunk ID to merge/update instead of creating a new entry.
*
* PRD §4.4: dedup threshold 0.92–0.95
*/
export function findDuplicate(
store: SqliteStore,
newVec: number[],
threshold: number,
log: Logger,
ownerFilter?: string[],
): string | null {
const all = store.getAllEmbeddings(ownerFilter);
let bestId: string | null = null;
let bestScore = 0;
for (const { chunkId, vector } of all) {
const sim = cosineSimilarity(newVec, vector);
if (sim > bestScore) {
bestScore = sim;
bestId = chunkId;
}
}
if (bestId && bestScore >= threshold) {
log.debug(`Dedup: found duplicate chunk=${bestId} sim=${bestScore.toFixed(4)}`);
return bestId;
}
return null;
}
/**
* Find Top-N most similar chunks above a threshold.
* Used for smart dedup: retrieve candidates, then ask LLM to judge.
*/
export function findTopSimilar(
store: SqliteStore,
newVec: number[],
threshold: number,
topN: number,
log: Logger,
ownerFilter?: string[],
): Array<{ chunkId: string; score: number }> {
const all = store.getAllEmbeddings(ownerFilter);
const scored: Array<{ chunkId: string; score: number }> = [];
for (const { chunkId, vector } of all) {
const sim = cosineSimilarity(newVec, vector);
if (sim >= threshold) {
scored.push({ chunkId, score: sim });
}
}
scored.sort((a, b) => b.score - a.score);
const result = scored.slice(0, topN);
if (result.length > 0) {
log.debug(`findTopSimilar: found ${result.length} candidates above ${threshold} (best=${result[0].score.toFixed(4)})`);
}
return result;
}
================================================
FILE: apps/memos-local-openclaw/src/ingest/providers/anthropic.ts
================================================
import type { SummarizerConfig, Logger } from "../../types";
const SYSTEM_PROMPT = `You generate a retrieval-friendly title.
Return exactly one noun phrase that names the topic AND its key details.
Requirements:
- Same language as input
- Keep proper nouns, API/function names, specific parameters, versions, error codes
- Include WHO/WHAT/WHERE details when present (e.g. person name + event, tool name + what it does)
- Prefer concrete topic words over generic words
- No verbs unless unavoidable
- No generic endings like:
功能说明、使用说明、简介、介绍、用途、summary、overview、basics
- Chinese: 10-50 characters (aim for 15-30)
- Non-Chinese: 5-15 words (aim for 8-12)
- Output title only`;
const TASK_SUMMARY_PROMPT = `You create a DETAILED task summary from a multi-turn conversation. This summary will be the ONLY record of this conversation, so it must preserve ALL important information.
## LANGUAGE RULE (HIGHEST PRIORITY)
Detect the PRIMARY language of the user's messages. If most user messages are Chinese, ALL output (title, goal, steps, result, details) MUST be in Chinese. If English, output in English. NEVER mix. This rule overrides everything below.
Output EXACTLY this structure:
📌 Title / 标题
A short, descriptive title (10-30 characters). Same language as user messages.
🎯 Goal / 目标
One sentence: what the user wanted to accomplish.
📋 Key Steps / 关键步骤
- Describe each meaningful step in detail
- Include the ACTUAL content produced: code snippets, commands, config blocks, formulas, key paragraphs
- For code: include the function signature and core logic (up to ~30 lines per block), use fenced code blocks
- For configs: include the actual config values and structure
- For lists/instructions: include the actual items, not just "provided a list"
- Merge only truly trivial back-and-forth (like "ok" / "sure")
- Do NOT over-summarize: "provided a function" is BAD; show the actual function
✅ Result / 结果
What was the final outcome? Include the final version of any code/config/content produced.
💡 Key Details / 关键细节
- Decisions made, trade-offs discussed, caveats noted, alternative approaches mentioned
- Specific values: numbers, versions, thresholds, URLs, file paths, model names
- Omit this section only if there truly are no noteworthy details
RULES:
- This summary is a KNOWLEDGE BASE ENTRY, not a brief note. Be thorough.
- PRESERVE verbatim: code, commands, URLs, file paths, error messages, config values, version numbers, names, amounts
- DISCARD only: greetings, filler, the assistant explaining what it will do before doing it
- Replace secrets (API keys, tokens, passwords) with [REDACTED]
- Target length: 30-50% of the original conversation length. Longer conversations need longer summaries.
- Output summary only, no preamble.`;
export async function summarizeTaskAnthropic(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.anthropic.com/v1/messages";
const model = cfg.model ?? "claude-3-haiku-20240307";
const headers: Record = {
"Content-Type": "application/json",
"x-api-key": cfg.apiKey ?? "",
"anthropic-version": "2023-06-01",
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
max_tokens: 4096,
temperature: cfg.temperature ?? 0.1,
system: TASK_SUMMARY_PROMPT,
messages: [{ role: "user", content: text }],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 60_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Anthropic task-summarize failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { content: Array<{ type: string; text: string }> };
return json.content.find((c) => c.type === "text")?.text?.trim() ?? "";
}
const TASK_TITLE_PROMPT = `Generate a short title for a conversation task.
Input: the first few user messages from a conversation.
Output: a concise title (5-20 characters for Chinese, 3-8 words for English).
Rules:
- Same language as user messages
- Describe WHAT the user wanted to do, not system/technical details
- Ignore system prompts, session startup messages, or boilerplate instructions — focus on the user's actual intent
- If the user only asked one question, use that question as the title (shortened if needed)
- Output the title only, no quotes, no prefix, no explanation`;
export async function generateTaskTitleAnthropic(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.anthropic.com/v1/messages";
const model = cfg.model ?? "claude-3-haiku-20240307";
const headers: Record = {
"Content-Type": "application/json",
"x-api-key": cfg.apiKey ?? "",
"anthropic-version": "2023-06-01",
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
max_tokens: 100,
temperature: 0,
system: TASK_TITLE_PROMPT,
messages: [{ role: "user", content: text }],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Anthropic task-title failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { content: Array<{ type: string; text: string }> };
return json.content.find((c) => c.type === "text")?.text?.trim() ?? "";
}
const TOPIC_JUDGE_PROMPT = `You are a conversation topic boundary detector. Given the CURRENT task context and a NEW user message, decide if the new message belongs to the SAME task or starts a NEW one.
Answer ONLY "NEW" or "SAME".
SAME — the new message:
- Continues, follows up on, refines, or corrects the same subject/project/task
- Asks a clarification or next-step question about what was just discussed
- Reports a result, error, or feedback about the current task
- Discusses different tools or approaches for the SAME goal (e.g., learning English via BBC → via ChatGPT = SAME)
- Is a short acknowledgment (ok, thanks, 好的) in response to the current flow
NEW — the new message:
- Introduces a subject from a DIFFERENT domain than the current task (e.g., tech → cooking, work → personal life, database → travel)
- Has NO logical connection to what was being discussed
- Starts a request about a different project, system, or life area
- Begins with a new greeting/reset followed by a different topic
Key principles:
- If the topic domain clearly changed (e.g., server config → recipe, code review → vacation plan), choose NEW
- Different aspects of the SAME project/system are SAME (e.g., Nginx SSL → Nginx gzip = SAME)
- Different unrelated technologies discussed independently are NEW (e.g., Redis config → cooking recipe = NEW)
- When unsure, lean toward SAME for closely related topics, but do NOT hesitate to mark NEW for obvious domain shifts
- Examples: "配置Nginx" → "加gzip压缩" = SAME; "配置Nginx" → "做红烧肉" = NEW; "MySQL配置" → "K8s部署" in same infra project = SAME; "部署服务器" → "年会安排" = NEW
Output exactly one word: NEW or SAME`;
export async function judgeNewTopicAnthropic(
currentContext: string,
newMessage: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.anthropic.com/v1/messages";
const model = cfg.model ?? "claude-3-haiku-20240307";
const headers: Record = {
"Content-Type": "application/json",
"x-api-key": cfg.apiKey ?? "",
"anthropic-version": "2023-06-01",
...cfg.headers,
};
const userContent = `CURRENT TASK CONTEXT:\n${currentContext}\n\n---\n\nNEW USER MESSAGE:\n${newMessage}`;
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
max_tokens: 10,
temperature: 0,
system: TOPIC_JUDGE_PROMPT,
messages: [{ role: "user", content: userContent }],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Anthropic topic-judge failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { content: Array<{ type: string; text: string }> };
const answer = json.content.find((c) => c.type === "text")?.text?.trim().toUpperCase() ?? "";
log.debug(`Topic judge result: "${answer}"`);
return answer.startsWith("NEW");
}
const FILTER_RELEVANT_PROMPT = `You are a memory relevance judge.
Given a QUERY and CANDIDATE memories, decide: does each candidate's content contain information that would HELP ANSWER the query?
CORE QUESTION: "If I include this memory, will it help produce a better answer?"
- YES → include
- NO → exclude
RULES:
1. A candidate is relevant if its content provides facts, context, or data that directly supports answering the query.
2. A candidate that merely shares the same broad topic/domain but contains NO useful information for answering is NOT relevant.
3. If NO candidate can help answer the query, return {"relevant":[],"sufficient":false} — do NOT force-pick the "least irrelevant" one.
OUTPUT — JSON only:
{"relevant":[1,3],"sufficient":true}
- "relevant": candidate numbers whose content helps answer the query. [] if none can help.
- "sufficient": true only if the selected memories fully answer the query.`;
import type { FilterResult } from "./openai";
export type { FilterResult } from "./openai";
export async function filterRelevantAnthropic(
query: string,
candidates: Array<{ index: number; role: string; content: string; time?: string }>,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.anthropic.com/v1/messages";
const model = cfg.model ?? "claude-3-haiku-20240307";
const headers: Record = {
"Content-Type": "application/json",
"x-api-key": cfg.apiKey ?? "",
"anthropic-version": "2023-06-01",
...cfg.headers,
};
const candidateText = candidates
.map((c) => {
const timeTag = c.time ? ` (${c.time})` : "";
return `${c.index}. [${c.role}]${timeTag}\n ${c.content}`;
})
.join("\n");
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
max_tokens: 200,
temperature: 0,
system: FILTER_RELEVANT_PROMPT,
messages: [{ role: "user", content: `QUERY: ${query}\n\nCANDIDATES:\n${candidateText}` }],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Anthropic filter-relevant failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { content: Array<{ type: string; text: string }> };
const raw = json.content.find((c) => c.type === "text")?.text?.trim() ?? "{}";
log.debug(`filterRelevant raw LLM response: "${raw}"`);
return parseFilterResult(raw, log);
}
function parseFilterResult(raw: string, log: Logger): FilterResult {
try {
const match = raw.match(/\{[\s\S]*\}/);
if (match) {
const obj = JSON.parse(match[0]);
if (obj && Array.isArray(obj.relevant)) {
return {
relevant: obj.relevant.filter((n: any) => typeof n === "number"),
sufficient: obj.sufficient === true,
};
}
}
} catch {}
log.warn(`filterRelevant: failed to parse LLM output: "${raw}", fallback to all+insufficient`);
return { relevant: [], sufficient: false };
}
export async function summarizeAnthropic(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.anthropic.com/v1/messages";
const model = cfg.model ?? "claude-3-haiku-20240307";
const headers: Record = {
"Content-Type": "application/json",
"x-api-key": cfg.apiKey ?? "",
"anthropic-version": "2023-06-01",
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
max_tokens: 100,
temperature: cfg.temperature ?? 0,
system: SYSTEM_PROMPT,
messages: [{ role: "user", content: `[TEXT TO SUMMARIZE]\n${text}\n[/TEXT TO SUMMARIZE]` }],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Anthropic summarize failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as {
content: Array<{ type: string; text: string }>;
};
return json.content.find((c) => c.type === "text")?.text?.trim() ?? "";
}
// ─── Smart Dedup ───
import { DEDUP_JUDGE_PROMPT, parseDedupResult } from "./openai";
import type { DedupResult } from "./openai";
export type { DedupResult } from "./openai";
export async function judgeDedupAnthropic(
newSummary: string,
candidates: Array<{ index: number; summary: string; chunkId: string }>,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = cfg.endpoint ?? "https://api.anthropic.com/v1/messages";
const model = cfg.model ?? "claude-3-haiku-20240307";
const headers: Record = {
"Content-Type": "application/json",
"x-api-key": cfg.apiKey ?? "",
"anthropic-version": "2023-06-01",
...cfg.headers,
};
const candidateText = candidates.map((c) => `${c.index}. ${c.summary}`).join("\n");
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
max_tokens: 300,
temperature: 0,
system: DEDUP_JUDGE_PROMPT,
messages: [{ role: "user", content: `NEW MEMORY:\n${newSummary}\n\nEXISTING MEMORIES:\n${candidateText}` }],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Anthropic dedup-judge failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { content: Array<{ type: string; text: string }> };
const raw = json.content.find((c) => c.type === "text")?.text?.trim() ?? "{}";
return parseDedupResult(raw, log);
}
================================================
FILE: apps/memos-local-openclaw/src/ingest/providers/bedrock.ts
================================================
import type { SummarizerConfig, Logger } from "../../types";
const SYSTEM_PROMPT = `You generate a retrieval-friendly title.
Return exactly one noun phrase that names the topic AND its key details.
Requirements:
- Same language as input
- Keep proper nouns, API/function names, specific parameters, versions, error codes
- Include WHO/WHAT/WHERE details when present (e.g. person name + event, tool name + what it does)
- Prefer concrete topic words over generic words
- No verbs unless unavoidable
- No generic endings like:
功能说明、使用说明、简介、介绍、用途、summary、overview、basics
- Chinese: 10-50 characters (aim for 15-30)
- Non-Chinese: 5-15 words (aim for 8-12)
- Output title only`;
const TASK_SUMMARY_PROMPT = `You create a DETAILED task summary from a multi-turn conversation. This summary will be the ONLY record of this conversation, so it must preserve ALL important information.
## LANGUAGE RULE (HIGHEST PRIORITY)
Detect the PRIMARY language of the user's messages. If most user messages are Chinese, ALL output (title, goal, steps, result, details) MUST be in Chinese. If English, output in English. NEVER mix. This rule overrides everything below.
Output EXACTLY this structure:
📌 Title / 标题
A short, descriptive title (10-30 characters). Same language as user messages.
🎯 Goal / 目标
One sentence: what the user wanted to accomplish.
📋 Key Steps / 关键步骤
- Describe each meaningful step in detail
- Include the ACTUAL content produced: code snippets, commands, config blocks, formulas, key paragraphs
- For code: include the function signature and core logic (up to ~30 lines per block), use fenced code blocks
- For configs: include the actual config values and structure
- For lists/instructions: include the actual items, not just "provided a list"
- Merge only truly trivial back-and-forth (like "ok" / "sure")
- Do NOT over-summarize: "provided a function" is BAD; show the actual function
✅ Result / 结果
What was the final outcome? Include the final version of any code/config/content produced.
💡 Key Details / 关键细节
- Decisions made, trade-offs discussed, caveats noted, alternative approaches mentioned
- Specific values: numbers, versions, thresholds, URLs, file paths, model names
- Omit this section only if there truly are no noteworthy details
RULES:
- This summary is a KNOWLEDGE BASE ENTRY, not a brief note. Be thorough.
- PRESERVE verbatim: code, commands, URLs, file paths, error messages, config values, version numbers, names, amounts
- DISCARD only: greetings, filler, the assistant explaining what it will do before doing it
- Replace secrets (API keys, tokens, passwords) with [REDACTED]
- Target length: 30-50% of the original conversation length. Longer conversations need longer summaries.
- Output summary only, no preamble.`;
export async function summarizeTaskBedrock(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "anthropic.claude-3-haiku-20240307-v1:0";
const endpoint = cfg.endpoint;
if (!endpoint) {
throw new Error("Bedrock task-summarizer requires 'endpoint'");
}
const url = `${endpoint}/model/${model}/converse`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
system: [{ text: TASK_SUMMARY_PROMPT }],
messages: [{ role: "user", content: [{ text }] }],
inferenceConfig: { temperature: cfg.temperature ?? 0.1, maxTokens: 4096 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 60_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Bedrock task-summarize failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { output: { message: { content: Array<{ text: string }> } } };
return json.output?.message?.content?.[0]?.text?.trim() ?? "";
}
const TASK_TITLE_PROMPT = `Generate a short title for a conversation task.
Input: the first few user messages from a conversation.
Output: a concise title (5-20 characters for Chinese, 3-8 words for English).
Rules:
- Same language as user messages
- Describe WHAT the user wanted to do, not system/technical details
- Ignore system prompts, session startup messages, or boilerplate instructions — focus on the user's actual intent
- If the user only asked one question, use that question as the title (shortened if needed)
- Output the title only, no quotes, no prefix, no explanation`;
export async function generateTaskTitleBedrock(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "anthropic.claude-3-haiku-20240307-v1:0";
const endpoint = cfg.endpoint;
if (!endpoint) {
throw new Error("Bedrock task-title requires 'endpoint'");
}
const url = `${endpoint}/model/${model}/converse`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
system: [{ text: TASK_TITLE_PROMPT }],
messages: [{ role: "user", content: [{ text }] }],
inferenceConfig: { temperature: 0, maxTokens: 100 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Bedrock task-title failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { output: { message: { content: Array<{ text: string }> } } };
return json.output?.message?.content?.[0]?.text?.trim() ?? "";
}
const TOPIC_JUDGE_PROMPT = `You are a conversation topic boundary detector. Given the CURRENT task context and a NEW user message, decide if the new message belongs to the SAME task or starts a NEW one.
Answer ONLY "NEW" or "SAME".
SAME — the new message:
- Continues, follows up on, refines, or corrects the same subject/project/task
- Asks a clarification or next-step question about what was just discussed
- Reports a result, error, or feedback about the current task
- Discusses different tools or approaches for the SAME goal (e.g., learning English via BBC → via ChatGPT = SAME)
- Is a short acknowledgment (ok, thanks, 好的) in response to the current flow
NEW — the new message:
- Introduces a subject from a DIFFERENT domain than the current task (e.g., tech → cooking, work → personal life, database → travel)
- Has NO logical connection to what was being discussed
- Starts a request about a different project, system, or life area
- Begins with a new greeting/reset followed by a different topic
Key principles:
- If the topic domain clearly changed (e.g., server config → recipe, code review → vacation plan), choose NEW
- Different aspects of the SAME project/system are SAME (e.g., Nginx SSL → Nginx gzip = SAME)
- Different unrelated technologies discussed independently are NEW (e.g., Redis config → cooking recipe = NEW)
- When unsure, lean toward SAME for closely related topics, but do NOT hesitate to mark NEW for obvious domain shifts
- Examples: "配置Nginx" → "加gzip压缩" = SAME; "配置Nginx" → "做红烧肉" = NEW; "MySQL配置" → "K8s部署" in same infra project = SAME; "部署服务器" → "年会安排" = NEW
Output exactly one word: NEW or SAME`;
export async function judgeNewTopicBedrock(
currentContext: string,
newMessage: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "anthropic.claude-3-haiku-20240307-v1:0";
const endpoint = cfg.endpoint;
if (!endpoint) {
throw new Error("Bedrock topic-judge requires 'endpoint'");
}
const url = `${endpoint}/model/${model}/converse`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const userContent = `CURRENT TASK CONTEXT:\n${currentContext}\n\n---\n\nNEW USER MESSAGE:\n${newMessage}`;
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
system: [{ text: TOPIC_JUDGE_PROMPT }],
messages: [{ role: "user", content: [{ text: userContent }] }],
inferenceConfig: { temperature: 0, maxTokens: 10 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Bedrock topic-judge failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { output: { message: { content: Array<{ text: string }> } } };
const answer = json.output?.message?.content?.[0]?.text?.trim().toUpperCase() ?? "";
log.debug(`Topic judge result: "${answer}"`);
return answer.startsWith("NEW");
}
const FILTER_RELEVANT_PROMPT = `You are a memory relevance judge.
Given a QUERY and CANDIDATE memories, decide: does each candidate's content contain information that would HELP ANSWER the query?
CORE QUESTION: "If I include this memory, will it help produce a better answer?"
- YES → include
- NO → exclude
RULES:
1. A candidate is relevant if its content provides facts, context, or data that directly supports answering the query.
2. A candidate that merely shares the same broad topic/domain but contains NO useful information for answering is NOT relevant.
3. If NO candidate can help answer the query, return {"relevant":[],"sufficient":false} — do NOT force-pick the "least irrelevant" one.
OUTPUT — JSON only:
{"relevant":[1,3],"sufficient":true}
- "relevant": candidate numbers whose content helps answer the query. [] if none can help.
- "sufficient": true only if the selected memories fully answer the query.`;
import type { FilterResult } from "./openai";
export type { FilterResult } from "./openai";
export async function filterRelevantBedrock(
query: string,
candidates: Array<{ index: number; role: string; content: string; time?: string }>,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "anthropic.claude-3-haiku-20240307-v1:0";
const endpoint = cfg.endpoint;
if (!endpoint) {
throw new Error("Bedrock filter-relevant requires 'endpoint'");
}
const url = `${endpoint}/model/${model}/converse`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const candidateText = candidates
.map((c) => {
const timeTag = c.time ? ` (${c.time})` : "";
return `${c.index}. [${c.role}]${timeTag}\n ${c.content}`;
})
.join("\n");
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
system: [{ text: FILTER_RELEVANT_PROMPT }],
messages: [{ role: "user", content: [{ text: `QUERY: ${query}\n\nCANDIDATES:\n${candidateText}` }] }],
inferenceConfig: { temperature: 0, maxTokens: 200 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Bedrock filter-relevant failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { output: { message: { content: Array<{ text: string }> } } };
const raw = json.output?.message?.content?.[0]?.text?.trim() ?? "{}";
log.debug(`filterRelevant raw LLM response: "${raw}"`);
return parseFilterResult(raw, log);
}
function parseFilterResult(raw: string, log: Logger): FilterResult {
try {
const match = raw.match(/\{[\s\S]*\}/);
if (match) {
const obj = JSON.parse(match[0]);
if (obj && Array.isArray(obj.relevant)) {
return {
relevant: obj.relevant.filter((n: any) => typeof n === "number"),
sufficient: obj.sufficient === true,
};
}
}
} catch {}
log.warn(`filterRelevant: failed to parse LLM output: "${raw}", fallback to all+insufficient`);
return { relevant: [], sufficient: false };
}
export async function summarizeBedrock(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "anthropic.claude-3-haiku-20240307-v1:0";
const endpoint = cfg.endpoint;
if (!endpoint) {
throw new Error("Bedrock summarizer requires 'endpoint' to be set (e.g. https://bedrock-runtime.us-east-1.amazonaws.com)");
}
const url = `${endpoint}/model/${model}/converse`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
system: [{ text: SYSTEM_PROMPT }],
messages: [{ role: "user", content: [{ text: `[TEXT TO SUMMARIZE]\n${text}\n[/TEXT TO SUMMARIZE]` }] }],
inferenceConfig: {
temperature: cfg.temperature ?? 0,
maxTokens: 100,
},
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Bedrock summarize failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as {
output: { message: { content: Array<{ text: string }> } };
};
return json.output?.message?.content?.[0]?.text?.trim() ?? "";
}
// ─── Smart Dedup ───
import { DEDUP_JUDGE_PROMPT, parseDedupResult } from "./openai";
import type { DedupResult } from "./openai";
export type { DedupResult } from "./openai";
export async function judgeDedupBedrock(
newSummary: string,
candidates: Array<{ index: number; summary: string; chunkId: string }>,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "anthropic.claude-3-haiku-20240307-v1:0";
const endpoint = cfg.endpoint;
if (!endpoint) throw new Error("Bedrock dedup-judge requires 'endpoint'");
const url = `${endpoint}/model/${model}/converse`;
const headers: Record = { "Content-Type": "application/json", ...cfg.headers };
const candidateText = candidates.map((c) => `${c.index}. ${c.summary}`).join("\n");
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
system: [{ text: DEDUP_JUDGE_PROMPT }],
messages: [{ role: "user", content: [{ text: `NEW MEMORY:\n${newSummary}\n\nEXISTING MEMORIES:\n${candidateText}` }] }],
inferenceConfig: { temperature: 0, maxTokens: 300 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Bedrock dedup-judge failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { output: { message: { content: Array<{ text: string }> } } };
const raw = json.output?.message?.content?.[0]?.text?.trim() ?? "{}";
return parseDedupResult(raw, log);
}
================================================
FILE: apps/memos-local-openclaw/src/ingest/providers/gemini.ts
================================================
import type { SummarizerConfig, Logger } from "../../types";
const SYSTEM_PROMPT = `You generate a retrieval-friendly title.
Return exactly one noun phrase that names the topic AND its key details.
Requirements:
- Same language as input
- Keep proper nouns, API/function names, specific parameters, versions, error codes
- Include WHO/WHAT/WHERE details when present (e.g. person name + event, tool name + what it does)
- Prefer concrete topic words over generic words
- No verbs unless unavoidable
- No generic endings like:
功能说明、使用说明、简介、介绍、用途、summary、overview、basics
- Chinese: 10-50 characters (aim for 15-30)
- Non-Chinese: 5-15 words (aim for 8-12)
- Output title only`;
const TASK_SUMMARY_PROMPT = `You create a DETAILED task summary from a multi-turn conversation. This summary will be the ONLY record of this conversation, so it must preserve ALL important information.
## LANGUAGE RULE (HIGHEST PRIORITY)
Detect the PRIMARY language of the user's messages. If most user messages are Chinese, ALL output (title, goal, steps, result, details) MUST be in Chinese. If English, output in English. NEVER mix. This rule overrides everything below.
Output EXACTLY this structure:
📌 Title / 标题
A short, descriptive title (10-30 characters). Same language as user messages.
🎯 Goal / 目标
One sentence: what the user wanted to accomplish.
📋 Key Steps / 关键步骤
- Describe each meaningful step in detail
- Include the ACTUAL content produced: code snippets, commands, config blocks, formulas, key paragraphs
- For code: include the function signature and core logic (up to ~30 lines per block), use fenced code blocks
- For configs: include the actual config values and structure
- For lists/instructions: include the actual items, not just "provided a list"
- Merge only truly trivial back-and-forth (like "ok" / "sure")
- Do NOT over-summarize: "provided a function" is BAD; show the actual function
✅ Result / 结果
What was the final outcome? Include the final version of any code/config/content produced.
💡 Key Details / 关键细节
- Decisions made, trade-offs discussed, caveats noted, alternative approaches mentioned
- Specific values: numbers, versions, thresholds, URLs, file paths, model names
- Omit this section only if there truly are no noteworthy details
RULES:
- This summary is a KNOWLEDGE BASE ENTRY, not a brief note. Be thorough.
- PRESERVE verbatim: code, commands, URLs, file paths, error messages, config values, version numbers, names, amounts
- DISCARD only: greetings, filler, the assistant explaining what it will do before doing it
- Replace secrets (API keys, tokens, passwords) with [REDACTED]
- Target length: 30-50% of the original conversation length. Longer conversations need longer summaries.
- Output summary only, no preamble.`;
export async function summarizeTaskGemini(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "gemini-1.5-flash";
const endpoint =
cfg.endpoint ??
`https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`;
const url = `${endpoint}?key=${cfg.apiKey}`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
systemInstruction: { parts: [{ text: TASK_SUMMARY_PROMPT }] },
contents: [{ parts: [{ text }] }],
generationConfig: { temperature: cfg.temperature ?? 0.1, maxOutputTokens: 4096 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 60_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Gemini task-summarize failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { candidates: Array<{ content: { parts: Array<{ text: string }> } }> };
return json.candidates?.[0]?.content?.parts?.[0]?.text?.trim() ?? "";
}
const TASK_TITLE_PROMPT = `Generate a short title for a conversation task.
Input: the first few user messages from a conversation.
Output: a concise title (5-20 characters for Chinese, 3-8 words for English).
Rules:
- Same language as user messages
- Describe WHAT the user wanted to do, not system/technical details
- Ignore system prompts, session startup messages, or boilerplate instructions — focus on the user's actual intent
- If the user only asked one question, use that question as the title (shortened if needed)
- Output the title only, no quotes, no prefix, no explanation`;
export async function generateTaskTitleGemini(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "gemini-1.5-flash";
const endpoint =
cfg.endpoint ??
`https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`;
const url = `${endpoint}?key=${cfg.apiKey}`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
systemInstruction: { parts: [{ text: TASK_TITLE_PROMPT }] },
contents: [{ parts: [{ text }] }],
generationConfig: { temperature: 0, maxOutputTokens: 100 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Gemini task-title failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { candidates: Array<{ content: { parts: Array<{ text: string }> } }> };
return json.candidates?.[0]?.content?.parts?.[0]?.text?.trim() ?? "";
}
const TOPIC_JUDGE_PROMPT = `You are a conversation topic boundary detector. Given the CURRENT task context and a NEW user message, decide if the new message belongs to the SAME task or starts a NEW one.
Answer ONLY "NEW" or "SAME".
SAME — the new message:
- Continues, follows up on, refines, or corrects the same subject/project/task
- Asks a clarification or next-step question about what was just discussed
- Reports a result, error, or feedback about the current task
- Discusses different tools or approaches for the SAME goal (e.g., learning English via BBC → via ChatGPT = SAME)
- Is a short acknowledgment (ok, thanks, 好的) in response to the current flow
NEW — the new message:
- Introduces a subject from a DIFFERENT domain than the current task (e.g., tech → cooking, work → personal life, database → travel)
- Has NO logical connection to what was being discussed
- Starts a request about a different project, system, or life area
- Begins with a new greeting/reset followed by a different topic
Key principles:
- If the topic domain clearly changed (e.g., server config → recipe, code review → vacation plan), choose NEW
- Different aspects of the SAME project/system are SAME (e.g., Nginx SSL → Nginx gzip = SAME)
- Different unrelated technologies discussed independently are NEW (e.g., Redis config → cooking recipe = NEW)
- When unsure, lean toward SAME for closely related topics, but do NOT hesitate to mark NEW for obvious domain shifts
- Examples: "配置Nginx" → "加gzip压缩" = SAME; "配置Nginx" → "做红烧肉" = NEW; "MySQL配置" → "K8s部署" in same infra project = SAME; "部署服务器" → "年会安排" = NEW
Output exactly one word: NEW or SAME`;
export async function judgeNewTopicGemini(
currentContext: string,
newMessage: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "gemini-1.5-flash";
const endpoint =
cfg.endpoint ??
`https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`;
const url = `${endpoint}?key=${cfg.apiKey}`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const userContent = `CURRENT TASK CONTEXT:\n${currentContext}\n\n---\n\nNEW USER MESSAGE:\n${newMessage}`;
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
systemInstruction: { parts: [{ text: TOPIC_JUDGE_PROMPT }] },
contents: [{ parts: [{ text: userContent }] }],
generationConfig: { temperature: 0, maxOutputTokens: 10 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Gemini topic-judge failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { candidates: Array<{ content: { parts: Array<{ text: string }> } }> };
const answer = json.candidates?.[0]?.content?.parts?.[0]?.text?.trim().toUpperCase() ?? "";
log.debug(`Topic judge result: "${answer}"`);
return answer.startsWith("NEW");
}
const FILTER_RELEVANT_PROMPT = `You are a memory relevance judge.
Given a QUERY and CANDIDATE memories, decide: does each candidate's content contain information that would HELP ANSWER the query?
CORE QUESTION: "If I include this memory, will it help produce a better answer?"
- YES → include
- NO → exclude
RULES:
1. A candidate is relevant if its content provides facts, context, or data that directly supports answering the query.
2. A candidate that merely shares the same broad topic/domain but contains NO useful information for answering is NOT relevant.
3. If NO candidate can help answer the query, return {"relevant":[],"sufficient":false} — do NOT force-pick the "least irrelevant" one.
OUTPUT — JSON only:
{"relevant":[1,3],"sufficient":true}
- "relevant": candidate numbers whose content helps answer the query. [] if none can help.
- "sufficient": true only if the selected memories fully answer the query.`;
import type { FilterResult } from "./openai";
export type { FilterResult } from "./openai";
export async function filterRelevantGemini(
query: string,
candidates: Array<{ index: number; role: string; content: string; time?: string }>,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "gemini-1.5-flash";
const endpoint =
cfg.endpoint ??
`https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`;
const url = `${endpoint}?key=${cfg.apiKey}`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const candidateText = candidates
.map((c) => {
const timeTag = c.time ? ` (${c.time})` : "";
return `${c.index}. [${c.role}]${timeTag}\n ${c.content}`;
})
.join("\n");
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
systemInstruction: { parts: [{ text: FILTER_RELEVANT_PROMPT }] },
contents: [{ parts: [{ text: `QUERY: ${query}\n\nCANDIDATES:\n${candidateText}` }] }],
generationConfig: { temperature: 0, maxOutputTokens: 200 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Gemini filter-relevant failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { candidates: Array<{ content: { parts: Array<{ text: string }> } }> };
const raw = json.candidates?.[0]?.content?.parts?.[0]?.text?.trim() ?? "{}";
log.debug(`filterRelevant raw LLM response: "${raw}"`);
return parseFilterResult(raw, log);
}
function parseFilterResult(raw: string, log: Logger): FilterResult {
try {
const match = raw.match(/\{[\s\S]*\}/);
if (match) {
const obj = JSON.parse(match[0]);
if (obj && Array.isArray(obj.relevant)) {
return {
relevant: obj.relevant.filter((n: any) => typeof n === "number"),
sufficient: obj.sufficient === true,
};
}
}
} catch {}
log.warn(`filterRelevant: failed to parse LLM output: "${raw}", fallback to all+insufficient`);
return { relevant: [], sufficient: false };
}
export async function summarizeGemini(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "gemini-1.5-flash";
const endpoint =
cfg.endpoint ??
`https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`;
const url = `${endpoint}?key=${cfg.apiKey}`;
const headers: Record = {
"Content-Type": "application/json",
...cfg.headers,
};
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
systemInstruction: { parts: [{ text: SYSTEM_PROMPT }] },
contents: [{ parts: [{ text: `[TEXT TO SUMMARIZE]\n${text}\n[/TEXT TO SUMMARIZE]` }] }],
generationConfig: { temperature: cfg.temperature ?? 0, maxOutputTokens: 100 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Gemini summarize failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as {
candidates: Array<{ content: { parts: Array<{ text: string }> } }>;
};
return json.candidates?.[0]?.content?.parts?.[0]?.text?.trim() ?? "";
}
// ─── Smart Dedup ───
import { DEDUP_JUDGE_PROMPT, parseDedupResult } from "./openai";
import type { DedupResult } from "./openai";
export type { DedupResult } from "./openai";
export async function judgeDedupGemini(
newSummary: string,
candidates: Array<{ index: number; summary: string; chunkId: string }>,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const model = cfg.model ?? "gemini-1.5-flash";
const endpoint = cfg.endpoint ?? `https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`;
const url = `${endpoint}?key=${cfg.apiKey}`;
const headers: Record = { "Content-Type": "application/json", ...cfg.headers };
const candidateText = candidates.map((c) => `${c.index}. ${c.summary}`).join("\n");
const resp = await fetch(url, {
method: "POST",
headers,
body: JSON.stringify({
systemInstruction: { parts: [{ text: DEDUP_JUDGE_PROMPT }] },
contents: [{ parts: [{ text: `NEW MEMORY:\n${newSummary}\n\nEXISTING MEMORIES:\n${candidateText}` }] }],
generationConfig: { temperature: 0, maxOutputTokens: 300 },
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`Gemini dedup-judge failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { candidates: Array<{ content: { parts: Array<{ text: string }> } }> };
const raw = json.candidates?.[0]?.content?.parts?.[0]?.text?.trim() ?? "{}";
return parseDedupResult(raw, log);
}
================================================
FILE: apps/memos-local-openclaw/src/ingest/providers/index.ts
================================================
import * as fs from "fs";
import * as path from "path";
import type { SummarizerConfig, SummaryProvider, Logger } from "../../types";
import { summarizeOpenAI, summarizeTaskOpenAI, judgeNewTopicOpenAI, filterRelevantOpenAI, judgeDedupOpenAI } from "./openai";
import type { FilterResult, DedupResult } from "./openai";
export type { FilterResult, DedupResult } from "./openai";
import { summarizeAnthropic, summarizeTaskAnthropic, generateTaskTitleAnthropic, judgeNewTopicAnthropic, filterRelevantAnthropic, judgeDedupAnthropic } from "./anthropic";
import { summarizeGemini, summarizeTaskGemini, generateTaskTitleGemini, judgeNewTopicGemini, filterRelevantGemini, judgeDedupGemini } from "./gemini";
import { summarizeBedrock, summarizeTaskBedrock, generateTaskTitleBedrock, judgeNewTopicBedrock, filterRelevantBedrock, judgeDedupBedrock } from "./bedrock";
/**
* Detect provider type from provider key name or base URL.
*/
function detectProvider(
providerKey: string | undefined,
baseUrl: string,
): SummaryProvider {
const key = providerKey?.toLowerCase() ?? "";
const url = baseUrl.toLowerCase();
if (key.includes("anthropic") || url.includes("anthropic")) return "anthropic";
if (key.includes("gemini") || url.includes("generativelanguage.googleapis.com")) {
return "gemini";
}
if (key.includes("bedrock") || url.includes("bedrock")) return "bedrock";
return "openai_compatible";
}
/**
* Return the correct endpoint for a given provider and base URL.
*/
function normalizeEndpointForProvider(
provider: SummaryProvider,
baseUrl: string,
): string {
const stripped = baseUrl.replace(/\/+$/, "");
if (provider === "anthropic") {
if (stripped.endsWith("/v1/messages")) return stripped;
return `${stripped}/v1/messages`;
}
if (stripped.endsWith("/chat/completions")) return stripped;
if (stripped.endsWith("/completions")) return stripped;
return `${stripped}/chat/completions`;
}
/**
* Build a SummarizerConfig from OpenClaw's native model configuration (openclaw.json).
* This serves as the final fallback when both strongCfg and plugin summarizer fail or are absent.
*/
function loadOpenClawFallbackConfig(log: Logger): SummarizerConfig | undefined {
try {
const home = process.env.HOME ?? process.env.USERPROFILE ?? "";
const ocHome = process.env.OPENCLAW_STATE_DIR || path.join(home, ".openclaw");
const cfgPath = path.join(ocHome, "openclaw.json");
if (!fs.existsSync(cfgPath)) return undefined;
const raw = JSON.parse(fs.readFileSync(cfgPath, "utf-8"));
const agentModel: string | undefined = raw?.agents?.defaults?.model?.primary;
if (!agentModel) return undefined;
const [providerKey, modelId] = agentModel.includes("/")
? agentModel.split("/", 2)
: [undefined, agentModel];
const providerCfg = providerKey
? raw?.models?.providers?.[providerKey]
: Object.values(raw?.models?.providers ?? {})[0] as any;
if (!providerCfg) return undefined;
const baseUrl: string | undefined = providerCfg.baseUrl;
const apiKey: string | undefined = providerCfg.apiKey;
if (!baseUrl || !apiKey) return undefined;
const provider = detectProvider(providerKey, baseUrl);
const endpoint = normalizeEndpointForProvider(provider, baseUrl);
log.debug(`OpenClaw fallback model: ${modelId} via ${baseUrl} (${provider})`);
return {
provider,
endpoint,
apiKey,
model: modelId,
};
} catch (err) {
log.debug(`Failed to load OpenClaw fallback config: ${err}`);
return undefined;
}
}
// ─── Model Health Tracking ───
export interface ModelHealthEntry {
role: string;
status: "ok" | "degraded" | "error" | "unknown";
lastSuccess: number | null;
lastError: number | null;
lastErrorMessage: string | null;
consecutiveErrors: number;
model: string | null;
failedModel: string | null;
}
class ModelHealthTracker {
private state = new Map();
private pendingErrors = new Map();
recordSuccess(role: string, model: string): void {
const entry = this.getOrCreate(role);
const pending = this.pendingErrors.get(role);
if (pending) {
entry.status = "degraded";
entry.lastError = Date.now();
entry.lastErrorMessage = pending.error.length > 300 ? pending.error.slice(0, 300) + "..." : pending.error;
entry.failedModel = pending.model;
this.pendingErrors.delete(role);
} else {
entry.status = "ok";
}
entry.lastSuccess = Date.now();
entry.consecutiveErrors = 0;
entry.model = model;
}
recordError(role: string, model: string, error: string): void {
const entry = this.getOrCreate(role);
entry.lastError = Date.now();
entry.lastErrorMessage = error.length > 300 ? error.slice(0, 300) + "..." : error;
entry.consecutiveErrors++;
entry.failedModel = model;
entry.status = "error";
this.pendingErrors.set(role, { model, error: entry.lastErrorMessage });
}
getAll(): ModelHealthEntry[] {
return [...this.state.values()];
}
private getOrCreate(role: string): ModelHealthEntry {
let entry = this.state.get(role);
if (!entry) {
entry = { role, status: "unknown", lastSuccess: null, lastError: null, lastErrorMessage: null, consecutiveErrors: 0, model: null, failedModel: null };
this.state.set(role, entry);
}
return entry;
}
}
export const modelHealth = new ModelHealthTracker();
export class Summarizer {
private strongCfg: SummarizerConfig | undefined;
private fallbackCfg: SummarizerConfig | undefined;
constructor(
private cfg: SummarizerConfig | undefined,
private log: Logger,
strongCfg?: SummarizerConfig,
) {
this.strongCfg = strongCfg;
this.fallbackCfg = loadOpenClawFallbackConfig(log);
}
/**
* Ordered config chain: strongCfg → cfg → fallbackCfg (OpenClaw native model).
* Returns configs that are defined, in priority order.
*/
private getConfigChain(): SummarizerConfig[] {
const chain: SummarizerConfig[] = [];
if (this.strongCfg) chain.push(this.strongCfg);
if (this.cfg) chain.push(this.cfg);
if (this.fallbackCfg) chain.push(this.fallbackCfg);
return chain;
}
/**
* Try calling fn with each config in the chain until one succeeds.
* Returns undefined if all fail.
*/
private async tryChain(
label: string,
fn: (cfg: SummarizerConfig) => Promise,
): Promise {
const chain = this.getConfigChain();
for (let i = 0; i < chain.length; i++) {
const modelInfo = `${chain[i].provider}/${chain[i].model ?? "?"}`;
try {
const result = await fn(chain[i]);
modelHealth.recordSuccess(label, modelInfo);
return result;
} catch (err) {
const level = i < chain.length - 1 ? "warn" : "error";
this.log[level](`${label} failed (${modelInfo}), ${i < chain.length - 1 ? "trying next" : "no more fallbacks"}: ${err}`);
modelHealth.recordError(label, modelInfo, String(err));
}
}
return undefined;
}
async summarize(text: string): Promise {
const cleaned = stripMarkdown(text).trim();
if (wordCount(cleaned) <= 10) {
return cleaned;
}
if (!this.cfg && !this.fallbackCfg) {
return ruleFallback(cleaned);
}
const accept = (s: string | undefined): s is string =>
!!s && s.length > 0 && s.length < cleaned.length;
let llmCalled = false;
try {
const result = await this.tryChain("summarize", (cfg) => callSummarize(cfg, text, this.log));
llmCalled = true;
const resultCleaned = result ? stripMarkdown(result).trim() : undefined;
if (accept(resultCleaned)) {
return resultCleaned;
}
if (resultCleaned !== undefined && resultCleaned !== null) {
const len: number = (resultCleaned as string).length;
this.log.warn(`summarize: result (${len}) >= input (${cleaned.length}), retrying`);
}
} catch (err) {
this.log.warn(`summarize primary failed: ${err}`);
}
const fallback = this.fallbackCfg ?? this.cfg;
if (fallback) {
try {
const retry = await callSummarize(fallback, text, this.log);
llmCalled = true;
const retryCleaned = retry ? stripMarkdown(retry).trim() : undefined;
if (accept(retryCleaned)) {
modelHealth.recordSuccess("summarize", `${fallback.provider}/${fallback.model ?? "?"}`);
return retryCleaned;
}
} catch (err) {
this.log.warn(`summarize fallback retry failed: ${err}`);
}
}
return llmCalled ? cleaned : ruleFallback(cleaned);
}
async summarizeTask(text: string): Promise {
if (!this.cfg && !this.fallbackCfg) {
return taskFallback(text);
}
const result = await this.tryChain("summarizeTask", (cfg) => callSummarizeTask(cfg, text, this.log));
return result ?? taskFallback(text);
}
async generateTaskTitle(text: string): Promise {
if (!this.cfg && !this.fallbackCfg) return "";
const result = await this.tryChain("generateTaskTitle", (cfg) => callGenerateTaskTitle(cfg, text, this.log));
return result ?? "";
}
async judgeNewTopic(currentContext: string, newMessage: string): Promise {
const chain: SummarizerConfig[] = [];
if (this.strongCfg) chain.push(this.strongCfg);
if (this.fallbackCfg) chain.push(this.fallbackCfg);
if (chain.length === 0 && this.cfg) chain.push(this.cfg);
if (chain.length === 0) return null;
for (let i = 0; i < chain.length; i++) {
const modelInfo = `${chain[i].provider}/${chain[i].model ?? "?"}`;
try {
const result = await callTopicJudge(chain[i], currentContext, newMessage, this.log);
modelHealth.recordSuccess("judgeNewTopic", modelInfo);
return result;
} catch (err) {
const level = i < chain.length - 1 ? "warn" : "error";
this.log[level](`judgeNewTopic failed (${modelInfo}), ${i < chain.length - 1 ? "trying next" : "no more fallbacks"}: ${err}`);
modelHealth.recordError("judgeNewTopic", modelInfo, String(err));
}
}
return null;
}
async filterRelevant(
query: string,
candidates: Array<{ index: number; role: string; content: string; time?: string }>,
): Promise {
if (!this.cfg && !this.fallbackCfg) return null;
if (candidates.length === 0) return { relevant: [], sufficient: true };
const result = await this.tryChain("filterRelevant", (cfg) => callFilterRelevant(cfg, query, candidates, this.log));
return result ?? null;
}
async judgeDedup(
newSummary: string,
candidates: Array<{ index: number; summary: string; chunkId: string }>,
): Promise {
if (!this.cfg && !this.fallbackCfg) return null;
if (candidates.length === 0) return null;
const result = await this.tryChain("judgeDedup", (cfg) => callJudgeDedup(cfg, newSummary, candidates, this.log));
return result ?? { action: "NEW", reason: "all_models_failed" };
}
getStrongConfig(): SummarizerConfig | undefined {
return this.strongCfg;
}
}
// ─── Dispatch helpers ───
function callSummarize(cfg: SummarizerConfig, text: string, log: Logger): Promise {
switch (cfg.provider) {
case "openai":
case "openai_compatible":
case "azure_openai":
case "zhipu":
case "siliconflow":
case "bailian":
case "cohere":
case "mistral":
case "voyage":
return summarizeOpenAI(text, cfg, log);
case "anthropic":
return summarizeAnthropic(text, cfg, log);
case "gemini":
return summarizeGemini(text, cfg, log);
case "bedrock":
return summarizeBedrock(text, cfg, log);
default:
throw new Error(`Unknown summarizer provider: ${cfg.provider}`);
}
}
function callSummarizeTask(cfg: SummarizerConfig, text: string, log: Logger): Promise {
switch (cfg.provider) {
case "openai":
case "openai_compatible":
case "azure_openai":
case "zhipu":
case "siliconflow":
case "bailian":
case "cohere":
case "mistral":
case "voyage":
return summarizeTaskOpenAI(text, cfg, log);
case "anthropic":
return summarizeTaskAnthropic(text, cfg, log);
case "gemini":
return summarizeTaskGemini(text, cfg, log);
case "bedrock":
return summarizeTaskBedrock(text, cfg, log);
default:
throw new Error(`Unknown summarizer provider: ${cfg.provider}`);
}
}
function callGenerateTaskTitle(cfg: SummarizerConfig, text: string, log: Logger): Promise {
switch (cfg.provider) {
case "openai":
case "openai_compatible":
case "azure_openai":
case "zhipu":
case "siliconflow":
case "bailian":
case "cohere":
case "mistral":
case "voyage":
return generateTaskTitleOpenAI(text, cfg, log);
case "anthropic":
return generateTaskTitleAnthropic(text, cfg, log);
case "gemini":
return generateTaskTitleGemini(text, cfg, log);
case "bedrock":
return generateTaskTitleBedrock(text, cfg, log);
default:
throw new Error(`Unknown summarizer provider: ${cfg.provider}`);
}
}
function callTopicJudge(cfg: SummarizerConfig, currentContext: string, newMessage: string, log: Logger): Promise {
switch (cfg.provider) {
case "openai":
case "openai_compatible":
case "azure_openai":
case "zhipu":
case "siliconflow":
case "bailian":
case "cohere":
case "mistral":
case "voyage":
return judgeNewTopicOpenAI(currentContext, newMessage, cfg, log);
case "anthropic":
return judgeNewTopicAnthropic(currentContext, newMessage, cfg, log);
case "gemini":
return judgeNewTopicGemini(currentContext, newMessage, cfg, log);
case "bedrock":
return judgeNewTopicBedrock(currentContext, newMessage, cfg, log);
default:
throw new Error(`Unknown summarizer provider: ${cfg.provider}`);
}
}
function callFilterRelevant(cfg: SummarizerConfig, query: string, candidates: Array<{ index: number; role: string; content: string; time?: string }>, log: Logger): Promise {
switch (cfg.provider) {
case "openai":
case "openai_compatible":
case "azure_openai":
case "zhipu":
case "siliconflow":
case "bailian":
case "cohere":
case "mistral":
case "voyage":
return filterRelevantOpenAI(query, candidates, cfg, log);
case "anthropic":
return filterRelevantAnthropic(query, candidates, cfg, log);
case "gemini":
return filterRelevantGemini(query, candidates, cfg, log);
case "bedrock":
return filterRelevantBedrock(query, candidates, cfg, log);
default:
throw new Error(`Unknown summarizer provider: ${cfg.provider}`);
}
}
function callJudgeDedup(cfg: SummarizerConfig, newSummary: string, candidates: Array<{ index: number; summary: string; chunkId: string }>, log: Logger): Promise {
switch (cfg.provider) {
case "openai":
case "openai_compatible":
case "azure_openai":
case "zhipu":
case "siliconflow":
case "bailian":
case "cohere":
case "mistral":
case "voyage":
return judgeDedupOpenAI(newSummary, candidates, cfg, log);
case "anthropic":
return judgeDedupAnthropic(newSummary, candidates, cfg, log);
case "gemini":
return judgeDedupGemini(newSummary, candidates, cfg, log);
case "bedrock":
return judgeDedupBedrock(newSummary, candidates, cfg, log);
default:
throw new Error(`Unknown summarizer provider: ${cfg.provider}`);
}
}
// ─── Fallbacks ───
function ruleFallback(text: string): string {
const lines = text.split("\n").filter((l) => l.trim().length > 5);
return (lines[0] ?? text).trim();
}
function taskFallback(text: string): string {
const lines = text.split("\n").filter((l) => l.trim().length > 10);
return lines.slice(0, 30).join("\n").slice(0, 2000);
}
function stripMarkdown(text: string): string {
return text
.replace(/\*\*([^*]+)\*\*/g, "$1")
.replace(/\*([^*]+)\*/g, "$1")
.replace(/^#{1,6}\s+/gm, "")
.replace(/`([^`]+)`/g, "$1")
.replace(/\[([^\]]+)\]\([^)]+\)/g, "$1")
.trim();
}
/** Count "words": CJK characters count as 1 word each, latin words separated by spaces. */
function wordCount(text: string): number {
let count = 0;
const cjk = /[\u4e00-\u9fff\u3400-\u4dbf\uf900-\ufaff]/g;
const cjkMatches = text.match(cjk);
if (cjkMatches) count += cjkMatches.length;
const noCjk = text.replace(cjk, " ").trim();
if (noCjk) count += noCjk.split(/\s+/).filter(Boolean).length;
return count;
}
================================================
FILE: apps/memos-local-openclaw/src/ingest/providers/openai.ts
================================================
import type { SummarizerConfig, Logger } from "../../types";
const SYSTEM_PROMPT = `You generate a retrieval-friendly title.
Return exactly one noun phrase that names the topic AND its key details.
Requirements:
- Same language as input
- Keep proper nouns, API/function names, specific parameters, versions, error codes
- Include WHO/WHAT/WHERE details when present (e.g. person name + event, tool name + what it does)
- Prefer concrete topic words over generic words
- No verbs unless unavoidable
- No generic endings like:
功能说明、使用说明、简介、介绍、用途、summary、overview、basics
- Chinese: 10-50 characters (aim for 15-30)
- Non-Chinese: 5-15 words (aim for 8-12)
- Output title only`;
const TASK_SUMMARY_PROMPT = `You create a DETAILED task summary from a multi-turn conversation. This summary will be the ONLY record of this conversation, so it must preserve ALL important information.
## LANGUAGE RULE (HIGHEST PRIORITY)
Detect the PRIMARY language of the user's messages. If most user messages are Chinese, ALL output (title, goal, steps, result, details) MUST be in Chinese. If English, output in English. NEVER mix. This rule overrides everything below.
Output EXACTLY this structure:
📌 Title / 标题
A short, descriptive title (10-30 characters). Same language as user messages.
🎯 Goal / 目标
One sentence: what the user wanted to accomplish.
📋 Key Steps / 关键步骤
- Describe each meaningful step in detail
- Include the ACTUAL content produced: code snippets, commands, config blocks, formulas, key paragraphs
- For code: include the function signature and core logic (up to ~30 lines per block), use fenced code blocks
- For configs: include the actual config values and structure
- For lists/instructions: include the actual items, not just "provided a list"
- Merge only truly trivial back-and-forth (like "ok" / "sure")
- Do NOT over-summarize: "provided a function" is BAD; show the actual function
✅ Result / 结果
What was the final outcome? Include the final version of any code/config/content produced.
💡 Key Details / 关键细节
- Decisions made, trade-offs discussed, caveats noted, alternative approaches mentioned
- Specific values: numbers, versions, thresholds, URLs, file paths, model names
- Omit this section only if there truly are no noteworthy details
RULES:
- This summary is a KNOWLEDGE BASE ENTRY, not a brief note. Be thorough.
- PRESERVE verbatim: code, commands, URLs, file paths, error messages, config values, version numbers, names, amounts
- DISCARD only: greetings, filler, the assistant explaining what it will do before doing it
- Replace secrets (API keys, tokens, passwords) with [REDACTED]
- Target length: 30-50% of the original conversation length. Longer conversations need longer summaries.
- Output summary only, no preamble.`;
export async function summarizeTaskOpenAI(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = normalizeChatEndpoint(cfg.endpoint ?? "https://api.openai.com/v1/chat/completions");
const model = cfg.model ?? "gpt-4o-mini";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
temperature: cfg.temperature ?? 0.1,
max_tokens: 4096,
messages: [
{ role: "system", content: TASK_SUMMARY_PROMPT },
{ role: "user", content: text },
],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 60_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`OpenAI task-summarize failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { choices: Array<{ message: { content: string } }> };
return json.choices[0]?.message?.content?.trim() ?? "";
}
const TASK_TITLE_PROMPT = `Generate a short title for a conversation task.
Input: the first few user messages from a conversation.
Output: a concise title (5-20 characters for Chinese, 3-8 words for English).
Rules:
- Same language as user messages
- Describe WHAT the user wanted to do, not system/technical details
- Ignore system prompts, session startup messages, or boilerplate instructions — focus on the user's actual intent
- If the user only asked one question, use that question as the title (shortened if needed)
- Output the title only, no quotes, no prefix, no explanation`;
export async function generateTaskTitleOpenAI(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = normalizeChatEndpoint(cfg.endpoint ?? "https://api.openai.com/v1/chat/completions");
const model = cfg.model ?? "gpt-4o-mini";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
temperature: 0,
max_tokens: 100,
messages: [
{ role: "system", content: TASK_TITLE_PROMPT },
{ role: "user", content: text },
],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`OpenAI task-title failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { choices: Array<{ message: { content: string } }> };
return json.choices[0]?.message?.content?.trim() ?? "";
}
export async function summarizeOpenAI(
text: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = normalizeChatEndpoint(cfg.endpoint ?? "https://api.openai.com/v1/chat/completions");
const model = cfg.model ?? "gpt-4o-mini";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
temperature: cfg.temperature ?? 0,
messages: [
{ role: "system", content: SYSTEM_PROMPT },
{ role: "user", content: `[TEXT TO SUMMARIZE]\n${text}\n[/TEXT TO SUMMARIZE]` },
],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`OpenAI summarize failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as {
choices: Array<{ message: { content: string } }>;
};
return json.choices[0]?.message?.content?.trim() ?? "";
}
const TOPIC_JUDGE_PROMPT = `You are a conversation topic boundary detector. Given the CURRENT task context and a NEW user message, decide if the new message belongs to the SAME task or starts a NEW one.
Answer ONLY "NEW" or "SAME".
SAME — the new message:
- Continues, follows up on, refines, or corrects the same subject/project/task
- Asks a clarification or next-step question about what was just discussed
- Reports a result, error, or feedback about the current task
- Discusses different tools or approaches for the SAME goal (e.g., learning English via BBC → via ChatGPT = SAME)
- Is a short acknowledgment (ok, thanks, 好的) in response to the current flow
NEW — the new message:
- Introduces a subject from a DIFFERENT domain than the current task (e.g., tech → cooking, work → personal life, database → travel)
- Has NO logical connection to what was being discussed
- Starts a request about a different project, system, or life area
- Begins with a new greeting/reset followed by a different topic
Key principles:
- If the topic domain clearly changed (e.g., server config → recipe, code review → vacation plan), choose NEW
- Different aspects of the SAME project/system are SAME (e.g., Nginx SSL → Nginx gzip = SAME)
- Different unrelated technologies discussed independently are NEW (e.g., Redis config → cooking recipe = NEW)
- When unsure, lean toward SAME for closely related topics, but do NOT hesitate to mark NEW for obvious domain shifts
- Examples: "配置Nginx" → "加gzip压缩" = SAME; "配置Nginx" → "做红烧肉" = NEW; "MySQL配置" → "K8s部署" in same infra project = SAME; "部署服务器" → "年会安排" = NEW
Output exactly one word: NEW or SAME`;
export async function judgeNewTopicOpenAI(
currentContext: string,
newMessage: string,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = normalizeChatEndpoint(cfg.endpoint ?? "https://api.openai.com/v1/chat/completions");
const model = cfg.model ?? "gpt-4o-mini";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const userContent = `CURRENT TASK CONTEXT:\n${currentContext}\n\n---\n\nNEW USER MESSAGE:\n${newMessage}`;
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
temperature: 0,
max_tokens: 10,
messages: [
{ role: "system", content: TOPIC_JUDGE_PROMPT },
{ role: "user", content: userContent },
],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`OpenAI topic-judge failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { choices: Array<{ message: { content: string } }> };
const answer = json.choices[0]?.message?.content?.trim().toUpperCase() ?? "";
log.debug(`Topic judge result: "${answer}"`);
return answer.startsWith("NEW");
}
const FILTER_RELEVANT_PROMPT = `You are a memory relevance judge.
Given a QUERY and CANDIDATE memories, decide: does each candidate's content contain information that would HELP ANSWER the query?
CORE QUESTION: "If I include this memory, will it help produce a better answer?"
- YES → include
- NO → exclude
RULES:
1. A candidate is relevant if its content provides facts, context, or data that directly supports answering the query.
2. A candidate that merely shares the same broad topic/domain but contains NO useful information for answering is NOT relevant.
3. If NO candidate can help answer the query, return {"relevant":[],"sufficient":false} — do NOT force-pick the "least irrelevant" one.
OUTPUT — JSON only:
{"relevant":[1,3],"sufficient":true}
- "relevant": candidate numbers whose content helps answer the query. [] if none can help.
- "sufficient": true only if the selected memories fully answer the query.`;
export interface FilterResult {
relevant: number[];
sufficient: boolean;
}
export async function filterRelevantOpenAI(
query: string,
candidates: Array<{ index: number; role: string; content: string; time?: string }>,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = normalizeChatEndpoint(cfg.endpoint ?? "https://api.openai.com/v1/chat/completions");
const model = cfg.model ?? "gpt-4o-mini";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const candidateText = candidates
.map((c) => {
const timeTag = c.time ? ` (${c.time})` : "";
return `${c.index}. [${c.role}]${timeTag}\n ${c.content}`;
})
.join("\n");
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
temperature: 0,
max_tokens: 200,
messages: [
{ role: "system", content: FILTER_RELEVANT_PROMPT },
{ role: "user", content: `QUERY: ${query}\n\nCANDIDATES:\n${candidateText}` },
],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`OpenAI filter-relevant failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { choices: Array<{ message: { content: string } }> };
const raw = json.choices[0]?.message?.content?.trim() ?? "{}";
log.debug(`filterRelevant raw LLM response: "${raw}"`);
return parseFilterResult(raw, log);
}
function parseFilterResult(raw: string, log: Logger): FilterResult {
try {
const match = raw.match(/\{[\s\S]*\}/);
if (match) {
const obj = JSON.parse(match[0]);
if (obj && Array.isArray(obj.relevant)) {
return {
relevant: obj.relevant.filter((n: any) => typeof n === "number"),
sufficient: obj.sufficient === true,
};
}
}
} catch {}
log.warn(`filterRelevant: failed to parse LLM output: "${raw}", fallback to all+insufficient`);
return { relevant: [], sufficient: false };
}
// ─── Smart Dedup: judge whether new memory is DUPLICATE / UPDATE / NEW ───
export const DEDUP_JUDGE_PROMPT = `You are a memory deduplication system.
LANGUAGE RULE (MUST FOLLOW): You MUST reply in the SAME language as the input memories. 如果输入是中文,reason 和 mergedSummary 必须用中文。If input is English, reply in English. This applies to ALL text fields in your JSON output.
Given a NEW memory summary and several EXISTING memory summaries, determine the relationship.
For each EXISTING memory, the NEW memory is either:
- "DUPLICATE": NEW conveys the same intent/meaning as an EXISTING memory, even if worded differently. Examples: "请告诉我你的名字" vs "你希望我怎么称呼你"; "新会话已开始" vs "New session started"; greetings with minor variations. If the core information/intent is the same, it IS a duplicate.
- "UPDATE": NEW contains meaningful additional information that supplements an EXISTING memory (new data, status change, concrete detail not present before)
- "NEW": NEW covers a genuinely different topic/event with no semantic overlap
IMPORTANT: Lean toward DUPLICATE when memories share the same intent, topic, or factual content. Only choose NEW when the topics are truly unrelated. Repetitive conversational patterns (greetings, session starts, identity questions, capability descriptions) across different sessions should be treated as DUPLICATE.
Pick the BEST match among all candidates. If none match well, choose "NEW".
Output a single JSON object (reason and mergedSummary MUST match input language):
- If DUPLICATE: {"action":"DUPLICATE","targetIndex":2,"reason":"与已有记忆意图相同"}
- If UPDATE: {"action":"UPDATE","targetIndex":3,"reason":"新记忆补充了额外细节","mergedSummary":"合并后的完整摘要,保留新旧所有信息"}
- If NEW: {"action":"NEW","reason":"不同主题,无关联"}
Output ONLY the JSON object, no other text.`;
export interface DedupResult {
action: "DUPLICATE" | "UPDATE" | "NEW";
targetIndex?: number;
reason: string;
mergedSummary?: string;
}
export async function judgeDedupOpenAI(
newSummary: string,
candidates: Array<{ index: number; summary: string; chunkId: string }>,
cfg: SummarizerConfig,
log: Logger,
): Promise {
const endpoint = normalizeChatEndpoint(cfg.endpoint ?? "https://api.openai.com/v1/chat/completions");
const model = cfg.model ?? "gpt-4o-mini";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const candidateText = candidates
.map((c) => `${c.index}. ${c.summary}`)
.join("\n");
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
temperature: 0,
max_tokens: 300,
messages: [
{ role: "system", content: DEDUP_JUDGE_PROMPT },
{ role: "user", content: `NEW MEMORY:\n${newSummary}\n\nEXISTING MEMORIES:\n${candidateText}` },
],
}),
signal: AbortSignal.timeout(cfg.timeoutMs ?? 15_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`OpenAI dedup-judge failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { choices: Array<{ message: { content: string } }> };
const raw = json.choices[0]?.message?.content?.trim() ?? "{}";
return parseDedupResult(raw, log);
}
export function parseDedupResult(raw: string, log: Logger): DedupResult {
try {
const match = raw.match(/\{[\s\S]*\}/);
if (match) {
const obj = JSON.parse(match[0]);
if (obj && typeof obj.action === "string") {
return {
action: obj.action as DedupResult["action"],
targetIndex: typeof obj.targetIndex === "number" ? obj.targetIndex : undefined,
reason: obj.reason || "",
mergedSummary: obj.mergedSummary || undefined,
};
}
}
} catch {}
log.warn(`judgeDedup: failed to parse LLM output: "${raw}", fallback to NEW`);
return { action: "NEW", reason: "parse_failed" };
}
function normalizeChatEndpoint(url: string): string {
const stripped = url.replace(/\/+$/, "");
if (stripped.endsWith("/chat/completions")) return stripped;
if (stripped.endsWith("/completions")) return stripped;
return `${stripped}/chat/completions`;
}
================================================
FILE: apps/memos-local-openclaw/src/ingest/task-processor.ts
================================================
import { v4 as uuid } from "uuid";
import type { SqliteStore } from "../storage/sqlite";
import type { PluginContext, Task, Chunk } from "../types";
import { DEFAULTS } from "../types";
import { Summarizer } from "./providers";
const TRIVIAL_PATTERNS = [
/^(test|testing|hello|hi|hey|ok|okay|yes|no|yeah|nope|sure|thanks|thank you|thx|ping|pong|哈哈|好的|嗯|是的|不是|谢谢|你好|测试)\s*[.!?。!?]*$/,
/^(aaa+|bbb+|xxx+|zzz+|123+|asdf+|qwer+|haha+|lol+|hmm+)\s*$/,
/^[\s\p{P}\p{S}]*$/u,
];
const SKIP_REASONS = {
noChunks: "该任务没有对话内容,已自动跳过。",
} as const;
/**
* Asynchronous task-level processor.
*
* After each ingestion batch, checks whether the current conversation
* constitutes a "new task" compared to the previous one. If so:
* 1. Finalizes the previous task (generates a detailed summary).
* 2. Creates a new active task for incoming chunks.
*
* Task boundary detection:
* - Session change → always new task
* - Time gap > 2h → always new task
* - LLM judges whether new user message starts a different topic
*/
export class TaskProcessor {
private summarizer: Summarizer;
private processing = false;
private pendingEvents: Array<{ sessionKey: string; latestTimestamp: number; owner: string }> = [];
private drainPromise: Promise | null = null;
private onTaskCompletedCallback?: (task: Task) => void;
constructor(
private store: SqliteStore,
private ctx: PluginContext,
) {
const strongCfg = ctx.config.skillEvolution?.summarizer;
this.summarizer = new Summarizer(ctx.config.summarizer, ctx.log, strongCfg);
}
onTaskCompleted(cb: (task: Task) => void): void {
this.onTaskCompletedCallback = cb;
}
/**
* Called after new chunks are ingested.
* Determines if a new task boundary was crossed and handles transition.
*/
async onChunksIngested(sessionKey: string, latestTimestamp: number, owner?: string): Promise {
const resolvedOwner = owner ?? "agent:main";
this.ctx.log.debug(`TaskProcessor.onChunksIngested called session=${sessionKey} ts=${latestTimestamp} owner=${resolvedOwner} processing=${this.processing}`);
this.pendingEvents.push({ sessionKey, latestTimestamp, owner: resolvedOwner });
if (!this.drainPromise) {
this.drainPromise = this.drainPending();
}
await this.drainPromise;
}
private async drainPending(): Promise {
this.processing = true;
try {
while (this.pendingEvents.length > 0) {
const next = this.pendingEvents.shift()!;
try {
await this.detectAndProcess(next.sessionKey, next.latestTimestamp, next.owner);
} catch (err) {
this.ctx.log.error(`TaskProcessor error: ${err}`);
}
}
} finally {
this.processing = false;
this.drainPromise = null;
}
}
private async detectAndProcess(sessionKey: string, latestTimestamp: number, owner: string): Promise {
this.ctx.log.debug(`TaskProcessor.detectAndProcess session=${sessionKey} owner=${owner}`);
const allActive = this.store.getAllActiveTasks(owner);
for (const t of allActive) {
if (t.sessionKey !== sessionKey) {
this.ctx.log.info(`Session changed: finalizing task=${t.id} from session=${t.sessionKey} (owner=${owner})`);
await this.finalizeTask(t);
}
}
let activeTask = this.store.getActiveTask(sessionKey, owner);
this.ctx.log.debug(`TaskProcessor.detectAndProcess activeTask=${activeTask?.id ?? "none"} owner=${owner}`);
if (!activeTask) {
// Create a new empty task — do NOT assign all chunks yet.
// processChunksIncrementally will assign them one turn at a time with boundary checks.
activeTask = await this.createNewTaskReturn(sessionKey, latestTimestamp, owner);
}
await this.processChunksIncrementally(activeTask, sessionKey, latestTimestamp, owner);
}
/**
* Process unassigned chunks one user-turn at a time.
*
* Strategy:
* - Need at least 1 user turn in the current task before starting LLM judgment
* (0 turns = no reference point for comparison).
* - Each subsequent user turn is individually checked against the full task context.
* - Time gap > 2h always triggers a split regardless of topic.
*/
private async processChunksIncrementally(
activeTask: Task,
sessionKey: string,
latestTimestamp: number,
owner: string,
): Promise {
const unassigned = this.store.getUnassignedChunks(sessionKey);
if (unassigned.length === 0) return;
const taskChunks = this.store.getChunksByTask(activeTask.id);
// Time gap check against the earliest unassigned chunk
if (taskChunks.length > 0) {
const lastTaskTs = Math.max(...taskChunks.map((c) => c.createdAt));
const firstUnassignedTs = Math.min(...unassigned.map((c) => c.createdAt));
const gap = firstUnassignedTs - lastTaskTs;
if (gap > DEFAULTS.taskIdleTimeoutMs) {
this.ctx.log.info(
`Task boundary: time gap ${Math.round(gap / 60000)}min > ${Math.round(DEFAULTS.taskIdleTimeoutMs / 60000)}min`,
);
await this.finalizeTask(activeTask);
const newTask = await this.createNewTaskReturn(sessionKey, latestTimestamp, owner);
// Recurse with the new empty task so remaining unassigned chunks get boundary-checked too
return this.processChunksIncrementally(newTask, sessionKey, latestTimestamp, owner);
}
}
const turns = this.groupIntoTurns(unassigned);
if (turns.length === 0) {
this.assignChunksToTask(unassigned, activeTask.id);
return;
}
let currentTask = activeTask;
let currentTaskChunks = [...taskChunks];
for (let i = 0; i < turns.length; i++) {
const turn = turns[i];
const userChunk = turn.find((c) => c.role === "user");
if (!userChunk) {
this.assignChunksToTask(turn, currentTask.id);
currentTaskChunks = currentTaskChunks.concat(turn);
continue;
}
// Time gap check per turn
if (currentTaskChunks.length > 0) {
const lastTs = Math.max(...currentTaskChunks.map((c) => c.createdAt));
if (userChunk.createdAt - lastTs > DEFAULTS.taskIdleTimeoutMs) {
this.ctx.log.info(`Task boundary at turn ${i}: time gap ${Math.round((userChunk.createdAt - lastTs) / 60000)}min`);
await this.finalizeTask(currentTask);
currentTask = await this.createNewTaskReturn(sessionKey, userChunk.createdAt, owner);
currentTaskChunks = [];
this.assignChunksToTask(turn, currentTask.id);
currentTaskChunks = currentTaskChunks.concat(turn);
continue;
}
}
// Need at least 1 user turn before we can meaningfully judge topic shifts
const existingUserCount = currentTaskChunks.filter((c) => c.role === "user").length;
if (existingUserCount < 1) {
this.assignChunksToTask(turn, currentTask.id);
currentTaskChunks = currentTaskChunks.concat(turn);
continue;
}
// LLM topic judgment — check this single user message against full task context
const context = this.buildContextSummary(currentTaskChunks);
const newMsg = userChunk.content.slice(0, 500);
this.ctx.log.info(`Topic judge: "${newMsg.slice(0, 60)}" vs ${existingUserCount} user turns`);
const isNew = await this.summarizer.judgeNewTopic(context, newMsg);
this.ctx.log.info(`Topic judge result: ${isNew === null ? "null(fallback)" : isNew ? "NEW" : "SAME"}`);
if (isNew === null) {
this.assignChunksToTask(turn, currentTask.id);
currentTaskChunks = currentTaskChunks.concat(turn);
continue;
}
if (isNew) {
this.ctx.log.info(`Task boundary at turn ${i}: LLM judged new topic. Msg: "${newMsg.slice(0, 80)}..."`);
await this.finalizeTask(currentTask);
currentTask = await this.createNewTaskReturn(sessionKey, userChunk.createdAt, owner);
currentTaskChunks = [];
}
this.assignChunksToTask(turn, currentTask.id);
currentTaskChunks = currentTaskChunks.concat(turn);
}
this.store.updateTask(currentTask.id, { endedAt: undefined });
}
/**
* Group chunks into user-turns: each turn starts with a user message
* and includes all subsequent non-user messages until the next user message.
*/
private groupIntoTurns(chunks: Chunk[]): Chunk[][] {
const turns: Chunk[][] = [];
let current: Chunk[] = [];
for (const c of chunks) {
if (c.role === "user" && current.length > 0) {
turns.push(current);
current = [];
}
current.push(c);
}
if (current.length > 0) turns.push(current);
return turns;
}
/**
* Build context from existing task chunks for the LLM topic judge.
* Includes both the task's opening topic and recent exchanges,
* so the LLM understands both what the task was originally about
* and where the conversation currently is.
*
* For user messages, include full content (up to 500 chars) since
* they carry the topic signal. For assistant messages, use summary
* or truncated content since they mostly elaborate.
*/
private buildContextSummary(chunks: Chunk[]): string {
const conversational = chunks.filter((c) => c.role === "user" || c.role === "assistant");
if (conversational.length === 0) return "";
const formatChunk = (c: Chunk) => {
const label = c.role === "user" ? "User" : "Assistant";
const maxLen = c.role === "user" ? 500 : 200;
const text = c.summary || c.content.slice(0, maxLen);
return `[${label}]: ${text}`;
};
if (conversational.length <= 10) {
return conversational.map(formatChunk).join("\n");
}
const opening = conversational.slice(0, 6).map(formatChunk);
const recent = conversational.slice(-4).map(formatChunk);
return [
"--- Task opening ---",
...opening,
"--- Recent exchanges ---",
...recent,
].join("\n");
}
private async createNewTaskReturn(sessionKey: string, timestamp: number, owner: string = "agent:main"): Promise {
const taskId = uuid();
const task: Task = {
id: taskId,
sessionKey,
title: "",
summary: "",
status: "active",
owner,
startedAt: timestamp,
endedAt: null,
updatedAt: timestamp,
};
this.store.insertTask(task);
this.ctx.log.info(`Created new task=${taskId} session=${sessionKey}`);
return task;
}
private async createNewTask(sessionKey: string, timestamp: number, owner: string = "agent:main"): Promise {
const task = await this.createNewTaskReturn(sessionKey, timestamp, owner);
this.assignUnassignedChunks(sessionKey, task.id);
}
private assignChunksToTask(chunks: Chunk[], taskId: string): void {
for (const chunk of chunks) {
this.store.setChunkTaskId(chunk.id, taskId);
}
if (chunks.length > 0) {
this.ctx.log.debug(`Assigned ${chunks.length} chunks to task=${taskId}`);
}
}
private assignUnassignedChunks(sessionKey: string, taskId: string): void {
const unassigned = this.store.getUnassignedChunks(sessionKey);
this.assignChunksToTask(unassigned, taskId);
}
async finalizeTask(task: Task): Promise {
const chunks = this.store.getChunksByTask(task.id);
const fallbackTitle = chunks.length > 0 ? this.extractTitle(chunks) : "";
if (chunks.length === 0) {
this.ctx.log.info(`Task ${task.id} skipped: no chunks`);
this.store.updateTask(task.id, { title: fallbackTitle, summary: SKIP_REASONS.noChunks, status: "skipped", endedAt: Date.now() });
return;
}
const skipReason = this.shouldSkipSummary(chunks);
if (skipReason) {
const skipTitle = await this.generateTitle(chunks, fallbackTitle);
this.ctx.log.info(`Task ${task.id} skipped: ${skipReason} (chunks=${chunks.length}, title="${skipTitle}")`);
const reason = this.humanReadableSkipReason(skipReason, chunks);
this.store.updateTask(task.id, { title: skipTitle, summary: reason, status: "skipped", endedAt: Date.now() });
return;
}
const conversationText = this.buildConversationText(chunks);
let summary: string;
try {
summary = await this.summarizer.summarizeTask(conversationText);
} catch (err) {
this.ctx.log.warn(`Task summary generation failed for task=${task.id}: ${err}`);
summary = this.fallbackSummary(chunks);
}
const { title: llmTitle, body } = this.parseTitleFromSummary(summary);
const title = llmTitle || await this.generateTitle(chunks, fallbackTitle);
this.store.updateTask(task.id, {
title,
summary: body,
status: "completed",
endedAt: Date.now(),
});
this.ctx.log.info(
`Finalized task=${task.id} title="${title}" chunks=${chunks.length} summaryLen=${body.length}`,
);
if (this.onTaskCompletedCallback) {
const finalized = this.store.getTask(task.id);
if (finalized) {
try {
this.onTaskCompletedCallback(finalized);
} catch (err) {
this.ctx.log.warn(`TaskProcessor onTaskCompleted callback error: ${err}`);
}
}
}
}
/**
* Determine if a task is too trivial to warrant an LLM summary call.
* Returns a skip reason string, or null if summary should proceed.
*
* Skip conditions (any one triggers skip):
* 1. Total chunks < 4 — too few messages to form a meaningful task
* 2. Real conversation turns < 2 — no back-and-forth dialogue
* 3. No user messages — purely system/tool generated, no user intent
* 4. Total content < 200 chars — not enough substance
* 5. User content is trivial/test data — "hello", "test", "ok" etc.
* 6. All messages are tool results — automated output, no conversation
* 7. High content repetition — user repeated the same thing (debug loops)
*/
private shouldSkipSummary(chunks: Chunk[]): string | null {
const userChunks = chunks.filter((c) => c.role === "user");
const assistantChunks = chunks.filter((c) => c.role === "assistant");
const toolChunks = chunks.filter((c) => c.role === "tool");
// 1. Too few chunks
if (chunks.length < 4) {
return `too few chunks (${chunks.length} < 4 minimum)`;
}
// 2. Not enough real conversation turns (need at least 2 user-assistant exchanges)
const turns = Math.min(userChunks.length, assistantChunks.length);
if (turns < 2) {
return `too few conversation turns (${turns} < 2 minimum)`;
}
// 3. No user messages at all — purely automated
if (userChunks.length === 0) {
return "no user messages — task appears to be automated/system-generated";
}
// 4. Total content too short
// CJK characters carry more info per char, so use a lower threshold
const totalContentLen = chunks.reduce((sum, c) => sum + c.content.length, 0);
const hasCJK = /[\u4e00-\u9fff\u3040-\u30ff\uac00-\ud7af]/.test(
userChunks[0]?.content ?? "",
);
const minContentLen = hasCJK ? 80 : 200;
if (totalContentLen < minContentLen) {
return `content too short (${totalContentLen} chars < ${minContentLen} minimum)`;
}
// 5. User content is trivial/test data
const userContent = userChunks.map((c) => c.content).join("\n");
if (this.looksLikeTrivialContent(userContent)) {
return "user content appears to be test/trivial data";
}
// 6. Assistant content is also trivial (both sides are low-value)
const assistantContent = assistantChunks.map((c) => c.content).join("\n");
if (this.looksLikeTrivialContent(userContent + "\n" + assistantContent)) {
return "conversation content (both user and assistant) appears trivial";
}
// 7. Almost all messages are tool results with minimal user interaction
if (toolChunks.length > 0 && toolChunks.length >= chunks.length * 0.7 && userChunks.length <= 1) {
return `dominated by tool results (${toolChunks.length}/${chunks.length} chunks) with minimal user input`;
}
// 8. High repetition — user keeps saying the same thing
if (userChunks.length >= 3) {
const uniqueUserMsgs = new Set(userChunks.map((c) => c.content.trim().toLowerCase()));
const uniqueRatio = uniqueUserMsgs.size / userChunks.length;
if (uniqueRatio < 0.4) {
return `high content repetition (${uniqueUserMsgs.size} unique out of ${userChunks.length} user messages)`;
}
}
return null;
}
private looksLikeTrivialContent(text: string): boolean {
const lines = text.toLowerCase().split(/\n/).map((l) => l.trim()).filter(Boolean);
if (lines.length === 0) return true;
const trivialCount = lines.filter((line) => {
if (line.length < 5) return true;
if (TRIVIAL_PATTERNS.some((p) => p.test(line))) return true;
return false;
}).length;
return trivialCount / lines.length > 0.7;
}
private buildConversationText(chunks: Chunk[]): string {
const lines: string[] = [];
for (const c of chunks) {
const roleLabel = c.role === "user" ? "User" : c.role === "assistant" ? "Assistant" : c.role;
lines.push(`[${roleLabel}]: ${c.content}`);
}
return lines.join("\n\n");
}
/**
* Extract the LLM-generated title from the summary output.
* The LLM is prompted to output "📌 Title\n" as the first section.
* Returns the title and the remaining body (with the title section stripped).
*/
private parseTitleFromSummary(summary: string): { title: string; body: string } {
const titleMatch = summary.match(/📌\s*(?:Title|标题)\s*\n(.+)/);
if (titleMatch) {
const title = titleMatch[1].trim();
const body = summary.replace(/📌\s*(?:Title|标题)\s*\n.+\n?/, "").trim();
return { title, body };
}
return { title: "", body: summary };
}
private async generateTitle(chunks: Chunk[], fallback: string): Promise {
try {
const userChunks = chunks.filter((c) => c.role === "user");
const titleInput = userChunks
.slice(0, 3)
.map((c) => c.content.trim())
.join("\n\n");
if (!titleInput) return fallback || "Untitled Task";
const title = await this.summarizer.generateTaskTitle(titleInput);
return title || fallback || "Untitled Task";
} catch (err) {
this.ctx.log.warn(`generateTitle failed: ${err}`);
return fallback || "Untitled Task";
}
}
private extractTitle(chunks: Chunk[]): string {
const firstUser = chunks.find((c) => {
if (c.role !== "user") return false;
const t = c.content.trim();
if (t.length > 200) return false;
if (/session.startup|Session Startup|\/new|\/reset/i.test(t)) return false;
return true;
});
if (!firstUser) return "Untitled Task";
return firstUser.content.trim().slice(0, 80);
}
private humanReadableSkipReason(reason: string, chunks: Chunk[]): string {
const userCount = chunks.filter((c) => c.role === "user").length;
const assistantCount = chunks.filter((c) => c.role === "assistant").length;
if (reason.includes("too few chunks")) {
return `对话内容过少(${chunks.length} 条消息),不足以生成有效摘要。至少需要 4 条消息。`;
}
if (reason.includes("too few conversation turns")) {
return `对话轮次不足(${Math.min(userCount, assistantCount)} 轮),需要至少 2 轮完整的问答交互才能生成摘要。`;
}
if (reason.includes("no user messages")) {
return "该任务没有用户消息,仅包含系统或工具自动生成的内容。";
}
if (reason.includes("content too short")) {
return "对话内容过短,信息量不足以生成有意义的摘要。";
}
if (reason.includes("trivial")) {
return "对话内容为简单问候或测试数据(如 hello、test、ok),无需生成摘要。";
}
if (reason.includes("tool results")) {
return "该任务主要由工具执行结果组成,缺少足够的用户交互内容。";
}
if (reason.includes("repetition")) {
return "对话中存在大量重复内容,无法提取有效信息生成摘要。";
}
return `对话未达到生成摘要的条件:${reason}`;
}
private fallbackSummary(chunks: Chunk[]): string {
const title = this.extractTitle(chunks);
const summaries = chunks
.filter((c) => c.summary)
.map((c) => `- ${c.summary}`);
const lines = [
`🎯 Goal`,
title,
``,
`📋 Key Steps`,
...summaries.slice(0, 20),
];
return lines.join("\n");
}
}
================================================
FILE: apps/memos-local-openclaw/src/ingest/worker.ts
================================================
import { v4 as uuid } from "uuid";
import { createHash } from "crypto";
import type { ConversationMessage, Chunk, PluginContext } from "../types";
import type { SqliteStore } from "../storage/sqlite";
import type { Embedder } from "../embedding";
import { Summarizer } from "./providers";
import { findDuplicate, findTopSimilar } from "./dedup";
import { TaskProcessor } from "./task-processor";
export class IngestWorker {
private summarizer: Summarizer;
private taskProcessor: TaskProcessor;
private queue: ConversationMessage[] = [];
private processing = false;
private flushResolvers: Array<() => void> = [];
constructor(
private store: SqliteStore,
private embedder: Embedder,
private ctx: PluginContext,
) {
this.summarizer = new Summarizer(ctx.config.summarizer, ctx.log);
this.taskProcessor = new TaskProcessor(store, ctx);
}
getTaskProcessor(): TaskProcessor { return this.taskProcessor; }
enqueue(messages: ConversationMessage[]): void {
this.queue.push(...messages);
if (!this.processing) {
this.processQueue().catch((err) => {
this.ctx.log.error(`Ingest worker error: ${err}`);
this.processing = false;
});
}
}
/** Wait until all queued messages have been processed. */
async flush(): Promise {
if (this.queue.length === 0 && !this.processing) return;
return new Promise((resolve) => {
this.flushResolvers.push(resolve);
});
}
private async processQueue(): Promise {
this.processing = true;
try {
while (this.queue.length > 0) {
const t0 = performance.now();
const batchSize = this.queue.length;
let lastSessionKey: string | undefined;
let lastOwner: string | undefined;
let lastTimestamp = 0;
let stored = 0;
let skipped = 0;
let merged = 0;
let duplicated = 0;
let errors = 0;
const resultLines: string[] = [];
const inputDetails: Array<{ role: string; content: string }> = [];
while (this.queue.length > 0) {
const msg = this.queue.shift()!;
inputDetails.push({ role: msg.role, content: msg.content });
try {
const result = await this.ingestMessage(msg);
lastSessionKey = msg.sessionKey;
lastOwner = msg.owner ?? "agent:main";
lastTimestamp = Math.max(lastTimestamp, msg.timestamp);
if (result === "skipped") {
skipped++;
resultLines.push(JSON.stringify({ role: msg.role, action: "exact-dup", summary: "", content: msg.content }));
} else if (result.action === "stored") {
stored++;
resultLines.push(JSON.stringify({ role: msg.role, action: "stored", summary: result.summary ?? "", content: msg.content }));
} else if (result.action === "duplicate") {
duplicated++;
resultLines.push(JSON.stringify({ role: msg.role, action: "dedup", reason: result.reason ?? "similar", summary: result.summary ?? "", content: msg.content }));
} else if (result.action === "merged") {
merged++;
resultLines.push(JSON.stringify({ role: msg.role, action: "merged", summary: result.summary ?? "", content: msg.content }));
}
} catch (err) {
errors++;
resultLines.push(JSON.stringify({ role: msg.role, action: "error", summary: "", content: msg.content }));
this.ctx.log.error(`Failed to ingest message turn=${msg.turnId}: ${err}`);
}
}
const dur = performance.now() - t0;
if (stored + merged > 0 || skipped > 0 || duplicated > 0) {
this.store.recordToolCall("memory_add", dur, errors === 0);
try {
const inputInfo = {
session: lastSessionKey,
messages: batchSize,
details: inputDetails,
};
const stats = [`stored=${stored}`, skipped > 0 ? `skipped=${skipped}` : null, duplicated > 0 ? `dedup=${duplicated}` : null, merged > 0 ? `merged=${merged}` : null, errors > 0 ? `errors=${errors}` : null].filter(Boolean).join(", ");
this.store.recordApiLog("memory_add", inputInfo, `${stats}\n${resultLines.join("\n")}`, dur, errors === 0);
} catch (_) { /* best-effort */ }
}
if (lastSessionKey) {
this.ctx.log.debug(`Calling TaskProcessor.onChunksIngested session=${lastSessionKey} ts=${lastTimestamp} owner=${lastOwner}`);
try {
await this.taskProcessor.onChunksIngested(lastSessionKey, lastTimestamp, lastOwner);
} catch (err) {
this.ctx.log.error(`TaskProcessor post-ingest error: ${err}`);
}
}
}
} finally {
this.processing = false;
for (const resolve of this.flushResolvers) resolve();
this.flushResolvers = [];
}
}
private async ingestMessage(msg: ConversationMessage): Promise<
"skipped" | { action: "stored" | "duplicate" | "merged"; summary?: string; reason?: string }
> {
return await this.storeChunk(msg, msg.content, "paragraph", 0);
}
private async storeChunk(
msg: ConversationMessage,
content: string,
kind: Chunk["kind"],
seq: number,
): Promise<{ action: "stored" | "duplicate" | "merged"; chunkId?: string; summary?: string; targetChunkId?: string; reason?: string }> {
const chunkId = uuid();
let summary = await this.summarizer.summarize(content);
let embedding: number[] | null = null;
try {
[embedding] = await this.embedder.embed([summary]);
} catch (err) {
this.ctx.log.warn(`Embedding failed for chunk=${chunkId}, storing without vector: ${err}`);
}
let dedupStatus: "active" | "duplicate" | "merged" = "active";
let dedupTarget: string | null = null;
let dedupReason: string | null = null;
let mergedFromOld: string | null = null;
let mergeCount = 0;
let mergeHistory = "[]";
// Fast path: exact content_hash match within same owner (agent dimension)
const chunkOwner = msg.owner ?? "agent:main";
const existingByHash = this.store.findActiveChunkByHash(content, chunkOwner);
if (existingByHash) {
this.ctx.log.debug(`Exact-dup (owner=${chunkOwner}): hash match → existing=${existingByHash}`);
this.store.recordMergeHit(existingByHash, "DUPLICATE", "exact content hash match");
dedupStatus = "duplicate";
dedupTarget = existingByHash;
dedupReason = "exact content hash match";
}
// Smart dedup: find Top-5 similar chunks, then ask LLM to judge
if (dedupStatus === "active" && embedding) {
const similarThreshold = this.ctx.config.dedup?.similarityThreshold ?? 0.80;
const dedupOwnerFilter = msg.owner ? [msg.owner] : undefined;
const topSimilar = findTopSimilar(this.store, embedding, similarThreshold, 5, this.ctx.log, dedupOwnerFilter);
if (topSimilar.length > 0) {
const candidates = topSimilar.map((s, i) => {
const chunk = this.store.getChunk(s.chunkId);
return {
index: i + 1,
summary: chunk?.summary ?? "",
chunkId: s.chunkId,
};
}).filter(c => c.summary);
if (candidates.length > 0) {
const dedupResult = await this.summarizer.judgeDedup(summary, candidates);
if (dedupResult && dedupResult.action === "DUPLICATE" && dedupResult.targetIndex) {
const targetChunkId = candidates[dedupResult.targetIndex - 1]?.chunkId;
if (targetChunkId) {
this.store.recordMergeHit(targetChunkId, "DUPLICATE", dedupResult.reason);
dedupStatus = "duplicate";
dedupTarget = targetChunkId;
dedupReason = dedupResult.reason;
this.ctx.log.debug(`Smart dedup: DUPLICATE → target=${targetChunkId}, storing with status=duplicate, reason: ${dedupResult.reason}`);
}
}
if (dedupStatus === "active" && dedupResult && dedupResult.action === "UPDATE" && dedupResult.targetIndex && dedupResult.mergedSummary) {
const targetChunkId = candidates[dedupResult.targetIndex - 1]?.chunkId;
if (targetChunkId) {
const oldChunk = this.store.getChunk(targetChunkId);
const oldSummary = oldChunk?.summary ?? "";
this.store.recordMergeHit(targetChunkId, "UPDATE", dedupResult.reason, oldSummary, dedupResult.mergedSummary);
summary = dedupResult.mergedSummary;
try {
const [newEmb] = await this.embedder.embed([summary]);
if (newEmb) embedding = newEmb;
} catch (err) {
this.ctx.log.warn(`Re-embed after merge failed: ${err}`);
}
this.store.markDedupStatus(targetChunkId, "merged", chunkId, dedupResult.reason);
this.store.deleteEmbedding(targetChunkId);
mergedFromOld = targetChunkId;
dedupReason = dedupResult.reason;
// Inherit merge history from the old chunk
if (oldChunk) {
const oldHistory = JSON.parse(oldChunk.mergeHistory || "[]");
oldHistory.push({
action: "merge",
at: Date.now(),
reason: dedupResult.reason,
from: oldSummary,
to: dedupResult.mergedSummary,
sourceChunkId: targetChunkId,
});
mergeHistory = JSON.stringify(oldHistory);
mergeCount = (oldChunk.mergeCount || 0) + 1;
}
this.ctx.log.debug(`Smart dedup: UPDATE → old chunk=${targetChunkId} retired, new chunk=${chunkId} gets merged summary (mergeCount=${mergeCount}), reason: ${dedupResult.reason}`);
}
}
if (dedupStatus === "active") {
this.ctx.log.debug(`Smart dedup: NEW — creating active chunk (reason: ${dedupResult?.reason ?? "no_result"})`);
}
}
}
}
const chunk: Chunk = {
id: chunkId,
sessionKey: msg.sessionKey,
turnId: msg.turnId,
seq,
role: msg.role,
content,
kind,
summary,
embedding: null,
taskId: null,
skillId: null,
owner: msg.owner ?? "agent:main",
dedupStatus,
dedupTarget,
dedupReason,
mergeCount: mergeCount,
lastHitAt: null,
mergeHistory: mergeHistory,
createdAt: msg.timestamp,
updatedAt: msg.timestamp,
};
this.store.insertChunk(chunk);
if (embedding && dedupStatus === "active") {
this.store.upsertEmbedding(chunkId, embedding);
}
this.ctx.log.debug(`Stored chunk=${chunkId} kind=${kind} role=${msg.role} dedup=${dedupStatus} len=${content.length} hasVec=${!!embedding && dedupStatus === "active"}`);
if (dedupStatus === "duplicate") {
return { action: "duplicate", summary, targetChunkId: dedupTarget ?? undefined, reason: dedupReason ?? undefined };
}
if (mergedFromOld) {
return { action: "merged", chunkId, summary, targetChunkId: mergedFromOld, reason: dedupReason ?? undefined };
}
return { action: "stored", chunkId, summary };
}
}
================================================
FILE: apps/memos-local-openclaw/src/recall/engine.ts
================================================
import type { SqliteStore } from "../storage/sqlite";
import type { Embedder } from "../embedding";
import type { PluginContext, SearchHit, SearchResult, SkillSearchHit, Skill } from "../types";
import { vectorSearch, cosineSimilarity } from "../storage/vector";
import { rrfFuse } from "./rrf";
import { mmrRerank } from "./mmr";
import { applyRecencyDecay } from "./recency";
import { Summarizer } from "../ingest/providers";
export type SkillSearchScope = "mix" | "self" | "public";
export interface RecallOptions {
query?: string;
maxResults?: number;
minScore?: number;
role?: string;
ownerFilter?: string[];
}
const MAX_RECENT_QUERIES = 20;
export class RecallEngine {
private recentQueries: Array<{ query: string; maxResults: number; minScore: number; hitCount: number }> = [];
constructor(
private store: SqliteStore,
private embedder: Embedder,
private ctx: PluginContext,
) {}
async search(opts: RecallOptions): Promise {
const recallCfg = this.ctx.config.recall!;
const maxResults = Math.min(
opts.maxResults ?? recallCfg.maxResultsDefault!,
recallCfg.maxResultsMax!,
);
const minScore = opts.minScore ?? recallCfg.minScoreDefault!;
const query = opts.query ?? "";
const roleFilter = opts.role;
const repeatNote = this.checkRepeat(query, maxResults, minScore);
const candidatePool = maxResults * 5;
const ownerFilter = opts.ownerFilter;
// Step 1: Gather candidates from FTS, vector search, and pattern search
const ftsCandidates = query
? this.store.ftsSearch(query, candidatePool, ownerFilter)
: [];
let vecCandidates: Array<{ chunkId: string; score: number }> = [];
if (query) {
try {
const queryVec = await this.embedder.embedQuery(query);
const maxChunks = recallCfg.vectorSearchMaxChunks && recallCfg.vectorSearchMaxChunks > 0
? recallCfg.vectorSearchMaxChunks
: undefined;
vecCandidates = vectorSearch(this.store, queryVec, candidatePool, maxChunks, ownerFilter);
} catch (err) {
this.ctx.log.warn(`Vector search failed, using FTS only: ${err}`);
}
}
// Step 1b: Pattern search (LIKE-based) as fallback for short terms that
// trigram FTS cannot match (trigram requires >= 3 chars).
const shortTerms = query
.replace(/[."""(){}[\]*:^~!@#$%&\\/<>,;'`??。,!、:""''()【】《》]/g, " ")
.split(/\s+/)
.filter((t) => t.length === 2);
const patternHits = shortTerms.length > 0
? this.store.patternSearch(shortTerms, { limit: candidatePool })
: [];
const patternRanked = patternHits.map((h, i) => ({
id: h.chunkId,
score: 1 / (i + 1),
}));
// Step 2: RRF fusion
const ftsRanked = ftsCandidates.map((c) => ({ id: c.chunkId, score: c.score }));
const vecRanked = vecCandidates.map((c) => ({ id: c.chunkId, score: c.score }));
const rrfScores = rrfFuse([ftsRanked, vecRanked, patternRanked], recallCfg.rrfK);
if (rrfScores.size === 0) {
this.recordQuery(query, maxResults, minScore, 0);
return {
hits: [],
meta: {
usedMinScore: minScore,
usedMaxResults: maxResults,
totalCandidates: 0,
note: repeatNote ?? "No candidates found for the given query.",
},
};
}
// Step 3: MMR re-ranking
const rrfList = [...rrfScores.entries()]
.map(([id, score]) => ({ id, score }))
.sort((a, b) => b.score - a.score);
const mmrResults = mmrRerank(rrfList, this.store, recallCfg.mmrLambda, maxResults * 2);
// Step 4: Time decay
const withTs = mmrResults.map((r) => {
const chunk = this.store.getChunk(r.id);
return { ...r, createdAt: chunk?.createdAt ?? 0 };
});
const decayed = applyRecencyDecay(withTs, recallCfg.recencyHalfLifeDays);
// Step 5: Apply relative threshold on raw scores, then normalize to [0,1]
const sorted = [...decayed].sort((a, b) => b.score - a.score);
const topScore = sorted.length > 0 ? sorted[0].score : 0;
const absoluteFloor = topScore * minScore * 0.3;
// When role filter is active, keep a larger pool before slicing so we don't
// discard target-role candidates that rank below non-target ones.
const preSliceLimit = roleFilter ? maxResults * 5 : maxResults;
const filtered = sorted
.filter((d) => d.score >= absoluteFloor)
.slice(0, preSliceLimit);
const displayMax = filtered.length > 0 ? filtered[0].score : 1;
const normalized = filtered.map((d) => ({
...d,
score: d.score / displayMax,
}));
// Step 6: Build hits (with optional role filter), applying maxResults cap at the end
const hits: SearchHit[] = [];
for (const candidate of normalized) {
if (hits.length >= maxResults) break;
const chunk = this.store.getChunk(candidate.id);
if (!chunk) continue;
if (roleFilter && chunk.role !== roleFilter) continue;
const excerpt = (chunk.mergeCount ?? 0) > 0 ? chunk.summary : makeExcerpt(chunk.content);
hits.push({
summary: chunk.summary,
original_excerpt: excerpt,
ref: {
sessionKey: chunk.sessionKey,
chunkId: chunk.id,
turnId: chunk.turnId,
seq: chunk.seq,
},
score: Math.round(candidate.score * 1000) / 1000,
taskId: chunk.taskId,
skillId: chunk.skillId,
source: {
ts: chunk.createdAt,
role: chunk.role,
sessionKey: chunk.sessionKey,
},
});
}
this.recordQuery(query, maxResults, minScore, hits.length);
return {
hits,
meta: {
usedMinScore: minScore,
usedMaxResults: maxResults,
totalCandidates: rrfScores.size,
...(repeatNote ? { note: repeatNote } : {}),
},
};
}
/**
* PRD §6.1: Detect repeated identical/similar queries and produce a
* warning note so the model knows to vary its approach.
*/
private checkRepeat(query: string, maxResults: number, minScore: number): string | undefined {
const normalized = query.toLowerCase().trim();
if (!normalized) return undefined;
const dup = this.recentQueries.find(
(q) => q.query === normalized && q.maxResults === maxResults && q.minScore === minScore,
);
if (dup) {
if (dup.hitCount === 0) {
return "This exact query with the same parameters was already tried and returned 0 results. Try rephrasing with different keywords, or adjust maxResults/minScore.";
}
return "This exact query with the same parameters was already executed. Consider varying the query or expanding parameters to get different results.";
}
return undefined;
}
private recordQuery(query: string, maxResults: number, minScore: number, hitCount: number): void {
const normalized = query.toLowerCase().trim();
if (!normalized) return;
this.recentQueries = this.recentQueries.filter(
(q) => !(q.query === normalized && q.maxResults === maxResults && q.minScore === minScore),
);
this.recentQueries.push({ query: normalized, maxResults, minScore, hitCount });
if (this.recentQueries.length > MAX_RECENT_QUERIES) {
this.recentQueries.shift();
}
}
async searchSkills(query: string, scope: SkillSearchScope, currentOwner: string): Promise {
const RRF_K = 60;
const TOP_CANDIDATES = 20;
// FTS on name + description
const ftsCandidates = this.store.skillFtsSearch(query, TOP_CANDIDATES, scope, currentOwner);
// Vector search on description embedding
let vecCandidates: Array<{ skillId: string; score: number }> = [];
try {
const queryVec = await this.embedder.embedQuery(query);
const allEmb = this.store.getSkillEmbeddings(scope, currentOwner);
vecCandidates = allEmb.map((row) => ({
skillId: row.skillId,
score: cosineSimilarity(queryVec, row.vector),
}));
vecCandidates.sort((a, b) => b.score - a.score);
vecCandidates = vecCandidates.slice(0, TOP_CANDIDATES);
} catch (err) {
this.ctx.log.warn(`Skill vector search failed, using FTS only: ${err}`);
}
// RRF fusion
const ftsRanked = ftsCandidates.map((c) => ({ id: c.skillId, score: c.score }));
const vecRanked = vecCandidates.map((c) => ({ id: c.skillId, score: c.score }));
const rrfScores = rrfFuse([ftsRanked, vecRanked], RRF_K);
if (rrfScores.size === 0) return [];
const sorted = [...rrfScores.entries()]
.map(([id, score]) => ({ id, score }))
.sort((a, b) => b.score - a.score)
.slice(0, TOP_CANDIDATES);
// Load skill details for LLM judgment
const candidateSkills: Array<{ skill: Skill; rrfScore: number }> = [];
for (const item of sorted) {
const skill = this.store.getSkill(item.id);
if (skill) candidateSkills.push({ skill, rrfScore: item.score });
}
if (candidateSkills.length === 0) return [];
// LLM relevance judgment
const summarizer = new Summarizer(this.ctx.config.summarizer, this.ctx.log);
const relevantIndices = await this.judgeSkillRelevance(summarizer, query, candidateSkills);
return relevantIndices.map((idx) => {
const { skill, rrfScore } = candidateSkills[idx];
return {
skillId: skill.id,
name: skill.name,
description: skill.description,
owner: skill.owner,
visibility: skill.visibility,
score: rrfScore,
reason: "relevant",
};
});
}
private async judgeSkillRelevance(
summarizer: Summarizer,
query: string,
candidates: Array<{ skill: Skill; rrfScore: number }>,
): Promise {
const candidateList = candidates.map((c, i) => ({
index: i,
role: "skill" as const,
content: `[${c.skill.name}] ${c.skill.description}`,
}));
try {
const result = await summarizer.filterRelevant(query, candidateList);
if (result && result.relevant.length > 0) {
return result.relevant.map((r) => r);
}
} catch (err) {
this.ctx.log.warn(`Skill relevance judgment failed, returning all: ${err}`);
}
// Fallback: return all candidates
return candidates.map((_, i) => i);
}
}
function makeExcerpt(content: string): string {
return content;
}
================================================
FILE: apps/memos-local-openclaw/src/recall/mmr.ts
================================================
import { cosineSimilarity } from "../storage/vector";
import type { SqliteStore } from "../storage/sqlite";
/**
* Maximal Marginal Relevance (PRD §5.3)
*
* Re-ranks candidates to balance relevance with diversity,
* preventing top-K results from being too similar.
*
* MMR = λ · sim(q, d) - (1-λ) · max(sim(d, d_selected))
*/
export function mmrRerank(
candidates: Array<{ id: string; score: number }>,
store: SqliteStore,
lambda: number = 0.7,
topK: number = 20,
): Array<{ id: string; score: number }> {
if (candidates.length <= 1) return candidates;
const embeddings = new Map();
for (const c of candidates) {
const vec = store.getEmbedding(c.id);
if (vec) embeddings.set(c.id, vec);
}
const selected: Array<{ id: string; score: number }> = [];
const remaining = [...candidates];
while (selected.length < topK && remaining.length > 0) {
let bestIdx = 0;
let bestMmr = -Infinity;
for (let i = 0; i < remaining.length; i++) {
const cand = remaining[i];
const candVec = embeddings.get(cand.id);
let maxSimToSelected = 0;
if (candVec && selected.length > 0) {
for (const s of selected) {
const sVec = embeddings.get(s.id);
if (sVec) {
const sim = cosineSimilarity(candVec, sVec);
maxSimToSelected = Math.max(maxSimToSelected, sim);
}
}
}
const mmrScore = lambda * cand.score - (1 - lambda) * maxSimToSelected;
if (mmrScore > bestMmr) {
bestMmr = mmrScore;
bestIdx = i;
}
}
const chosen = remaining.splice(bestIdx, 1)[0];
// Preserve original RRF score for downstream filtering;
// MMR only determines selection order, not the score value.
selected.push({ id: chosen.id, score: chosen.score });
}
return selected;
}
================================================
FILE: apps/memos-local-openclaw/src/recall/recency.ts
================================================
/**
* Time decay scoring (PRD §5.3)
*
* Applies exponential decay based on document age, biasing towards
* more recent memories. Uses configurable half-life (default 14 days).
*
* decay(t) = 0.5 ^ (age_days / half_life)
* final = base_score * (alpha + (1-alpha) * decay)
*
* alpha=0.3 ensures old but highly relevant results are not zeroed out.
*/
export function applyRecencyDecay(
candidates: Array<{ id: string; score: number; createdAt: number }>,
halfLifeDays: number = 14,
now?: number,
): Array<{ id: string; score: number }> {
const currentTime = now ?? Date.now();
const halfLifeMs = halfLifeDays * 24 * 60 * 60 * 1000;
const alpha = 0.3;
return candidates.map((c) => {
const ageMs = Math.max(0, currentTime - c.createdAt);
const decay = Math.pow(0.5, ageMs / halfLifeMs);
const adjustedScore = c.score * (alpha + (1 - alpha) * decay);
return { id: c.id, score: adjustedScore };
});
}
================================================
FILE: apps/memos-local-openclaw/src/recall/rrf.ts
================================================
/**
* Reciprocal Rank Fusion (PRD §5.2)
*
* Merges ranked lists from different retrieval sources (FTS, vector)
* into a single ranking. Handles score scale mismatch between BM25
* and cosine similarity.
*
* RRF(d) = Σ 1 / (k + rank_i(d))
* where k is a constant (default 60) and rank_i is the rank in list i.
*/
export interface RankedItem {
id: string;
score: number;
}
export function rrfFuse(
lists: RankedItem[][],
k: number = 60,
): Map {
const scores = new Map();
for (const list of lists) {
for (let rank = 0; rank < list.length; rank++) {
const item = list[rank];
const prev = scores.get(item.id) ?? 0;
scores.set(item.id, prev + 1 / (k + rank + 1));
}
}
return scores;
}
================================================
FILE: apps/memos-local-openclaw/src/shared/llm-call.ts
================================================
import * as fs from "fs";
import * as path from "path";
import type { SummarizerConfig, SummaryProvider, Logger, PluginContext } from "../types";
/**
* Detect provider type from provider key name or base URL.
*/
function detectProvider(providerKey: string | undefined, baseUrl: string): SummaryProvider {
const key = providerKey?.toLowerCase() ?? "";
const url = baseUrl.toLowerCase();
if (key.includes("anthropic") || url.includes("anthropic")) return "anthropic";
if (key.includes("gemini") || url.includes("generativelanguage.googleapis.com")) {
return "gemini";
}
if (key.includes("bedrock") || url.includes("bedrock")) return "bedrock";
return "openai_compatible";
}
/**
* Return the correct default endpoint for a given provider.
*/
function defaultEndpointForProvider(provider: SummaryProvider, baseUrl: string): string {
const stripped = baseUrl.replace(/\/+$/, "");
if (provider === "anthropic") {
if (stripped.endsWith("/v1/messages")) return stripped;
return `${stripped}/v1/messages`;
}
// OpenAI-compatible providers
if (stripped.endsWith("/chat/completions")) return stripped;
if (stripped.endsWith("/completions")) return stripped;
return `${stripped}/chat/completions`;
}
/**
* Build a SummarizerConfig from OpenClaw's native model configuration (openclaw.json).
* Final fallback when both strongCfg and plugin summarizer fail or are absent.
*/
export function loadOpenClawFallbackConfig(log: Logger): SummarizerConfig | undefined {
try {
const home = process.env.HOME ?? process.env.USERPROFILE ?? "";
const cfgPath = path.join(home, ".openclaw", "openclaw.json");
if (!fs.existsSync(cfgPath)) return undefined;
const raw = JSON.parse(fs.readFileSync(cfgPath, "utf-8"));
const agentModel: string | undefined = raw?.agents?.defaults?.model?.primary;
if (!agentModel) return undefined;
const [providerKey, modelId] = agentModel.includes("/")
? agentModel.split("/", 2)
: [undefined, agentModel];
const providerCfg = providerKey
? raw?.models?.providers?.[providerKey]
: Object.values(raw?.models?.providers ?? {})[0] as any;
if (!providerCfg) return undefined;
const baseUrl: string | undefined = providerCfg.baseUrl;
const apiKey: string | undefined = providerCfg.apiKey;
if (!baseUrl || !apiKey) return undefined;
const provider = detectProvider(providerKey, baseUrl);
const endpoint = defaultEndpointForProvider(provider, baseUrl);
log.debug(`OpenClaw fallback model: ${modelId} via ${baseUrl} (${provider})`);
return {
provider,
endpoint,
apiKey,
model: modelId,
};
} catch (err) {
log.debug(`Failed to load OpenClaw fallback config: ${err}`);
return undefined;
}
}
/**
* Build the ordered fallback chain for skill-related LLM calls:
* skillEvolution.summarizer → plugin summarizer → OpenClaw native model
*/
export function buildSkillConfigChain(ctx: PluginContext): SummarizerConfig[] {
const chain: SummarizerConfig[] = [];
const skillCfg = ctx.config.skillEvolution?.summarizer;
const pluginCfg = ctx.config.summarizer;
const fallbackCfg = loadOpenClawFallbackConfig(ctx.log);
if (skillCfg) chain.push(skillCfg);
if (pluginCfg && pluginCfg !== skillCfg) chain.push(pluginCfg);
if (fallbackCfg) chain.push(fallbackCfg);
return chain;
}
export interface LLMCallOptions {
maxTokens?: number;
temperature?: number;
timeoutMs?: number;
}
function normalizeOpenAIEndpoint(url: string): string {
const stripped = url.replace(/\/+$/, "");
if (stripped.endsWith("/chat/completions")) return stripped;
if (stripped.endsWith("/completions")) return stripped;
return `${stripped}/chat/completions`;
}
function normalizeAnthropicEndpoint(url: string): string {
const stripped = url.replace(/\/+$/, "");
if (stripped.endsWith("/v1/messages")) return stripped;
if (stripped.endsWith("/messages")) return stripped;
return `${stripped}/v1/messages`;
}
function isAnthropicProvider(cfg: SummarizerConfig): boolean {
return cfg.provider === "anthropic";
}
/**
* Make a single LLM call with the given config. Throws on failure.
* Dispatches to Anthropic or OpenAI-compatible format based on provider.
*/
export async function callLLMOnce(
cfg: SummarizerConfig,
prompt: string,
opts: LLMCallOptions = {},
): Promise {
if (isAnthropicProvider(cfg)) {
return callLLMOnceAnthropic(cfg, prompt, opts);
}
return callLLMOnceOpenAI(cfg, prompt, opts);
}
async function callLLMOnceAnthropic(
cfg: SummarizerConfig,
prompt: string,
opts: LLMCallOptions = {},
): Promise {
const endpoint = normalizeAnthropicEndpoint(
cfg.endpoint ?? "https://api.anthropic.com/v1/messages",
);
const model = cfg.model ?? "claude-3-haiku-20240307";
const headers: Record = {
"Content-Type": "application/json",
"x-api-key": cfg.apiKey ?? "",
"anthropic-version": "2023-06-01",
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
temperature: opts.temperature ?? 0.1,
max_tokens: opts.maxTokens ?? 1024,
messages: [{ role: "user", content: prompt }],
}),
signal: AbortSignal.timeout(opts.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`LLM call failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { content: Array<{ type: string; text: string }> };
return json.content.find((c) => c.type === "text")?.text?.trim() ?? "";
}
async function callLLMOnceOpenAI(
cfg: SummarizerConfig,
prompt: string,
opts: LLMCallOptions = {},
): Promise {
const endpoint = normalizeOpenAIEndpoint(
cfg.endpoint ?? "https://api.openai.com/v1/chat/completions",
);
const model = cfg.model ?? "gpt-4o-mini";
const headers: Record = {
"Content-Type": "application/json",
Authorization: `Bearer ${cfg.apiKey}`,
...cfg.headers,
};
const resp = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify({
model,
temperature: opts.temperature ?? 0.1,
max_tokens: opts.maxTokens ?? 1024,
messages: [{ role: "user", content: prompt }],
}),
signal: AbortSignal.timeout(opts.timeoutMs ?? 30_000),
});
if (!resp.ok) {
const body = await resp.text();
throw new Error(`LLM call failed (${resp.status}): ${body}`);
}
const json = (await resp.json()) as { choices: Array<{ message: { content: string } }> };
return json.choices[0]?.message?.content?.trim() ?? "";
}
/**
* Call LLM with fallback chain: tries each config in order until one succeeds.
* Returns the result string, or throws if ALL configs fail.
*/
export async function callLLMWithFallback(
chain: SummarizerConfig[],
prompt: string,
log: Logger,
label: string,
opts: LLMCallOptions = {},
): Promise {
if (chain.length === 0) {
throw new Error(`${label}: no LLM config available`);
}
for (let i = 0; i < chain.length; i++) {
try {
return await callLLMOnce(chain[i], prompt, opts);
} catch (err) {
const modelInfo = `${chain[i].provider ?? "?"}/${chain[i].model ?? "?"}`;
if (i < chain.length - 1) {
log.warn(`${label} failed (${modelInfo}), trying next fallback: ${err}`);
} else {
log.error(`${label} failed (${modelInfo}), no more fallbacks: ${err}`);
throw err;
}
}
}
throw new Error(`${label}: all models failed`);
}
================================================
FILE: apps/memos-local-openclaw/src/skill/bundled-memory-guide.ts
================================================
/**
* Bundled MemOS memory-guide skill content.
* Reads from skill/memos-memory-guide/SKILL.md at runtime (single source of truth).
*/
import * as fs from "fs";
import * as path from "path";
const skillPath = path.join(__dirname, "..", "..", "skill", "memos-memory-guide", "SKILL.md");
export const MEMORY_GUIDE_SKILL_MD: string = fs.readFileSync(skillPath, "utf-8");
================================================
FILE: apps/memos-local-openclaw/src/skill/evaluator.ts
================================================
import type { Chunk, Task, Skill, PluginContext } from "../types";
import { DEFAULTS } from "../types";
import { buildSkillConfigChain, callLLMWithFallback } from "../shared/llm-call";
export interface CreateEvalResult {
shouldGenerate: boolean;
reason: string;
suggestedName: string;
suggestedTags: string[];
confidence: number;
}
export interface UpgradeEvalResult {
shouldUpgrade: boolean;
upgradeType: "refine" | "extend" | "fix";
dimensions: string[];
reason: string;
mergeStrategy: string;
confidence: number;
}
const CREATE_EVAL_PROMPT = `You are a strict experience evaluation expert. Based on the completed task record below, decide whether this task contains **reusable, transferable** experience worth distilling into a "skill".
A skill is a reusable guide that helps an AI agent handle **the same type of task** better in the future. The key question is: "Will someone likely need to do this exact type of thing again?"
STRICT criteria — must meet ALL of:
1. **Repeatable**: The task type is likely to recur (not a one-off personal conversation)
2. **Transferable**: The approach/solution would help others facing the same problem
3. **Technical depth**: Contains non-trivial steps, commands, code, configs, or diagnostic reasoning
Worth distilling (must meet criteria above AND at least ONE below):
- Solves a recurring technical problem with a specific approach/workflow
- Went through trial-and-error (wrong approach then corrected) — the learning is valuable
- Involves non-obvious usage of specific tools, APIs, or frameworks
- Contains debugging/troubleshooting with diagnostic reasoning
- Shows how to combine multiple tools/services to accomplish a technical goal
- Contains deployment, configuration, or infrastructure setup steps
- Demonstrates a reusable data processing or automation pipeline
NOT worth distilling (if ANY matches, return shouldGenerate=false):
- Pure factual Q&A with no process ("what is TCP", "what's the capital of France")
- Single-turn simple answers with no workflow
- Conversation too fragmented or incoherent to extract a clear process
- One-off personal tasks: identity confirmation, preference setting, self-introduction
- Casual chat, opinion discussion, news commentary, brainstorming without actionable output
- Simple information lookup or summarization (e.g. "summarize this article", "explain X concept")
- Organizing/listing personal information (work history, resume, contacts)
- Generic product/system overviews without specific operational steps
- Tasks where the "steps" are just the AI answering questions (no real workflow)
Task title: {TITLE}
Task summary:
{SUMMARY}
LANGUAGE RULE (MUST FOLLOW): Detect the language of the task title/summary. If it is Chinese, the "reason" field MUST be in Chinese. If English, reason in English. Only "suggestedName" stays in English kebab-case. 如果任务标题/摘要是中文,reason 必须用中文。
Reply in JSON only, no extra text:
{
"shouldGenerate": boolean,
"reason": "brief explanation (same language as input)",
"suggestedName": "kebab-case-name",
"suggestedTags": ["tag1", "tag2"],
"confidence": 0.0-1.0
}`;
const UPGRADE_EVAL_PROMPT = `You are a skill upgrade evaluation expert.
Existing skill (v{VERSION}):
Name: {SKILL_NAME}
Content:
{SKILL_CONTENT}
Newly completed task:
Title: {TITLE}
Summary:
{SUMMARY}
Does the new task bring substantive improvements to the existing skill?
Worth upgrading (any one qualifies):
1. Faster — shorter path discovered
2. More elegant — cleaner, follows best practices better
3. More convenient — fewer dependencies or complexity
4. Fewer tokens — less exploration/trial-and-error needed
5. More accurate — corrects wrong parameters/steps in old skill
6. More robust — adds edge cases, error handling
7. New scenario — covers a variant the old skill didn't
8. Fixes outdated info — old skill has stale information
NOT worth upgrading:
- New task is identical to existing skill
- New task's approach is worse than existing skill
- Differences are trivial
LANGUAGE RULE: "reason" and "mergeStrategy" MUST use the SAME language as the task title/summary. Chinese input → Chinese output. English input → English output.
Reply in JSON only, no extra text:
{
"shouldUpgrade": boolean,
"upgradeType": "refine" | "extend" | "fix",
"dimensions": ["faster", "more_elegant", "more_convenient", "fewer_tokens", "more_accurate", "more_robust", "new_scenario", "fix_outdated"],
"reason": "what new value the task brings (same language as input)",
"mergeStrategy": "which specific parts need updating (same language as input)",
"confidence": 0.0-1.0
}`;
export class SkillEvaluator {
constructor(private ctx: PluginContext) {}
passesRuleFilter(chunks: Chunk[], task: Task): { pass: boolean; skipReason: string } {
const minChunks = this.ctx.config.skillEvolution?.minChunksForEval ?? DEFAULTS.skillMinChunksForEval;
if (chunks.length < minChunks) {
return { pass: false, skipReason: `chunks不足 (${chunks.length} < ${minChunks})` };
}
if (task.status === "skipped") {
return { pass: false, skipReason: "task状态为skipped" };
}
if (task.summary.length < 100) {
return { pass: false, skipReason: `summary过短 (${task.summary.length} < 100)` };
}
const userChunks = chunks.filter(c => c.role === "user");
if (userChunks.length === 0) {
return { pass: false, skipReason: "无用户消息" };
}
const assistantChunks = chunks.filter(c => c.role === "assistant");
if (assistantChunks.length === 0) {
return { pass: false, skipReason: "无助手回复" };
}
return { pass: true, skipReason: "" };
}
async evaluateCreate(task: Task): Promise {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) {
return { shouldGenerate: false, reason: "no LLM configured", suggestedName: "", suggestedTags: [], confidence: 0 };
}
const prompt = CREATE_EVAL_PROMPT
.replace("{TITLE}", task.title)
.replace("{SUMMARY}", task.summary.slice(0, 3000));
try {
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillEvaluator.create");
return this.parseJSON(raw, {
shouldGenerate: false, reason: "parse failed", suggestedName: "", suggestedTags: [], confidence: 0,
});
} catch (err) {
this.ctx.log.warn(`SkillEvaluator.evaluateCreate failed: ${err}`);
return { shouldGenerate: false, reason: `error: ${err}`, suggestedName: "", suggestedTags: [], confidence: 0 };
}
}
async evaluateUpgrade(task: Task, skill: Skill, skillContent: string): Promise {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) {
return { shouldUpgrade: false, upgradeType: "refine", dimensions: [], reason: "no LLM configured", mergeStrategy: "", confidence: 0 };
}
const prompt = UPGRADE_EVAL_PROMPT
.replace("{VERSION}", String(skill.version))
.replace("{SKILL_NAME}", skill.name)
.replace("{SKILL_CONTENT}", skillContent.slice(0, 4000))
.replace("{TITLE}", task.title)
.replace("{SUMMARY}", task.summary.slice(0, 3000));
try {
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillEvaluator.upgrade");
return this.parseJSON(raw, {
shouldUpgrade: false, upgradeType: "refine", dimensions: [], reason: "parse failed", mergeStrategy: "", confidence: 0,
});
} catch (err) {
this.ctx.log.warn(`SkillEvaluator.evaluateUpgrade failed: ${err}`);
return { shouldUpgrade: false, upgradeType: "refine", dimensions: [], reason: `error: ${err}`, mergeStrategy: "", confidence: 0 };
}
}
private parseJSON(raw: string, fallback: T): T {
const jsonMatch = raw.match(/\{[\s\S]*\}/);
if (!jsonMatch) return fallback;
try {
return JSON.parse(jsonMatch[0]) as T;
} catch {
return fallback;
}
}
}
================================================
FILE: apps/memos-local-openclaw/src/skill/evolver.ts
================================================
import * as fs from "fs";
import * as path from "path";
import type { SqliteStore } from "../storage/sqlite";
import type { RecallEngine } from "../recall/engine";
import type { Embedder } from "../embedding";
import { cosineSimilarity } from "../storage/vector";
import type { Task, Skill, Chunk, PluginContext } from "../types";
import { DEFAULTS } from "../types";
import { SkillEvaluator } from "./evaluator";
import { SkillGenerator } from "./generator";
import { SkillUpgrader } from "./upgrader";
import { SkillInstaller } from "./installer";
import { buildSkillConfigChain, callLLMWithFallback } from "../shared/llm-call";
export type SkillEvolvedCallback = (skillName: string, upgradeType: "created" | "upgraded") => void;
export class SkillEvolver {
private evaluator: SkillEvaluator;
private generator: SkillGenerator;
private upgrader: SkillUpgrader;
private installer: SkillInstaller;
private processing = false;
private queue: Task[] = [];
onSkillEvolved: SkillEvolvedCallback | null = null;
constructor(
private store: SqliteStore,
private engine: RecallEngine,
private ctx: PluginContext,
private embedder?: Embedder,
) {
this.evaluator = new SkillEvaluator(ctx);
this.generator = new SkillGenerator(store, engine, ctx, embedder);
this.upgrader = new SkillUpgrader(store, ctx);
this.installer = new SkillInstaller(store, ctx);
}
async recoverOrphanedTasks(): Promise {
const orphaned = this.store.getTasksBySkillStatus(["queued", "generating"]);
if (orphaned.length === 0) return 0;
this.ctx.log.info(`SkillEvolver: recovering ${orphaned.length} orphaned tasks (queued/generating from previous run)`);
for (const task of orphaned) {
try {
await this.processOne(task);
} catch (err) {
this.ctx.log.error(`SkillEvolver: recovery failed for task ${task.id}: ${err}`);
}
}
return orphaned.length;
}
async onTaskCompleted(task: Task): Promise {
const enabled = this.ctx.config.skillEvolution?.enabled ?? DEFAULTS.skillEvolutionEnabled;
const autoEval = this.ctx.config.skillEvolution?.autoEvaluate ?? DEFAULTS.skillAutoEvaluate;
if (!enabled || !autoEval) return;
if (this.processing) {
this.ctx.log.debug(`SkillEvolver: busy, queuing task ${task.id} (queue=${this.queue.length})`);
this.store.setTaskSkillMeta(task.id, { skillStatus: "queued", skillReason: `排队中,前方还有 ${this.queue.length + 1} 个任务` });
this.queue.push(task);
return;
}
await this.drain(task);
}
private async drain(task: Task): Promise {
this.processing = true;
try {
await this.processOne(task);
while (this.queue.length > 0) {
const next = this.queue.shift()!;
await this.processOne(next);
}
} finally {
this.processing = false;
}
}
private async processOne(task: Task): Promise {
try {
await this.process(task);
} catch (err) {
this.ctx.log.error(`SkillEvolver error for task ${task.id}: ${err}`);
this.store.setTaskSkillMeta(task.id, { skillStatus: "skipped", skillReason: `Error: ${err}` });
}
}
private async process(task: Task): Promise {
const chunks = this.store.getChunksByTask(task.id);
const { pass, skipReason } = this.evaluator.passesRuleFilter(chunks, task);
if (!pass) {
this.ctx.log.debug(`SkillEvolver: task ${task.id} skipped by rule filter: ${skipReason} (chunks=${chunks.length})`);
this.store.setTaskSkillMeta(task.id, { skillStatus: "skipped", skillReason: skipReason });
return;
}
const relatedSkill = await this.findRelatedSkill(task);
if (relatedSkill) {
await this.handleExistingSkill(task, chunks, relatedSkill);
} else {
await this.handleNewSkill(task, chunks);
}
}
/** Max candidates to send to LLM for relevance judgment. */
private static readonly RELATED_SKILL_CANDIDATE_TOP = 10;
/**
* Search for an existing skill that is HIGHLY related to the given task.
*
* 1. Collect top 50 skill candidates by FTS + vector similarity (relaxed thresholds).
* 2. Call LLM with task title/summary and each skill's name/description; strict rule:
* only output ONE skill index if the task clearly belongs to that skill's domain;
* otherwise output 0 (do not force a match).
*/
private async findRelatedSkill(task: Task): Promise {
const query = task.summary.slice(0, 600);
const owner = task.owner ?? "agent:main";
// Relaxed thresholds to gather a larger candidate pool; LLM will do strict filtering
const VEC_FLOOR = 0.35;
const TOP_N = SkillEvolver.RELATED_SKILL_CANDIDATE_TOP;
type Candidate = { skill: Skill; vecScore: number; ftsScore: number; combined: number };
const candidateMap = new Map();
// 1. FTS on skill name + description (take more candidates)
try {
const ftsHits = this.store.skillFtsSearch(query, TOP_N, "mix", owner);
for (const hit of ftsHits) {
const skill = this.store.getSkill(hit.skillId);
if (skill && (skill.status === "active" || skill.status === "draft")) {
candidateMap.set(skill.id, { skill, vecScore: 0, ftsScore: hit.score, combined: 0 });
}
}
} catch (err) {
this.ctx.log.warn(`SkillEvolver: skill FTS search failed: ${err}`);
}
// 2. Vector similarity: include all skills above a low floor to rank them
if (this.embedder) {
try {
const queryVec = await this.embedder.embedQuery(query);
const allSkillEmb = this.store.getSkillEmbeddings("mix", owner);
for (const row of allSkillEmb) {
const sim = cosineSimilarity(queryVec, row.vector);
if (sim >= VEC_FLOOR) {
const existing = candidateMap.get(row.skillId);
if (existing) {
existing.vecScore = sim;
} else {
const skill = this.store.getSkill(row.skillId);
if (skill && (skill.status === "active" || skill.status === "draft")) {
candidateMap.set(skill.id, { skill, vecScore: sim, ftsScore: 0, combined: 0 });
}
}
}
}
} catch (err) {
this.ctx.log.warn(`SkillEvolver: skill vector search failed: ${err}`);
}
}
if (candidateMap.size === 0) return null;
for (const c of candidateMap.values()) {
c.combined = c.vecScore * 0.7 + c.ftsScore * 0.3;
}
const sorted = [...candidateMap.values()]
.sort((a, b) => b.combined - a.combined)
.slice(0, TOP_N);
if (sorted.length === 0) return null;
// 3. LLM strict relevance judgment: only one skill if HIGHLY related, else none
const selectedSkill = await this.judgeSkillRelatedToTask(task, sorted);
if (selectedSkill) {
this.ctx.log.debug(`SkillEvolver: LLM selected related skill "${selectedSkill.name}" for task "${task.title}"`);
} else {
this.ctx.log.debug(`SkillEvolver: LLM found no highly related skill for task "${task.title}" (${sorted.length} candidates)`);
}
return selectedSkill;
}
/**
* Ask LLM to pick at most ONE skill that is HIGHLY relevant to the task.
* Strict rule: only return a skill if the task clearly belongs to that skill's domain; otherwise return null.
*/
private async judgeSkillRelatedToTask(
task: Task,
candidates: Array<{ skill: Skill; vecScore: number; ftsScore: number; combined: number }>,
): Promise {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) {
this.ctx.log.warn("SkillEvolver: no LLM config available, skipping skill relevance judgment");
return null;
}
const taskTitle = task.title || "(no title)";
const taskSummary = task.summary.slice(0, 800);
const skillList = candidates
.map((c, i) => `${i + 1}. [${c.skill.name}]\n ${(c.skill.description || "").slice(0, 300)}`)
.join("\n\n");
const prompt = `You are a strict judge: decide whether a completed TASK should be merged into an EXISTING SKILL. The task and the skill must be in the SAME domain/topic — e.g. same type of problem, same tool, same workflow. Loose or tangential relevance is NOT enough.
TASK TITLE: ${taskTitle}
TASK SUMMARY:
${taskSummary}
CANDIDATE SKILLS (index, name, description):
${skillList}
RULES:
- Output exactly ONE skill index (1 to ${candidates.length}) ONLY if the task's experience clearly belongs to that skill's domain. Same topic, same kind of work.
- If no skill is clearly relevant (different domain, or only loosely related), output 0. When in doubt, output 0.
- Do not force a match. "Movie recommendation" task must not match "Weather query" or "Legal discussion" skill even if both exist in the list.
LANGUAGE RULE: "reason" MUST use the SAME language as the task title/summary. Chinese input → Chinese reason.
Reply with JSON only, no other text:
{"selectedIndex": 0, "reason": "brief explanation (same language as input)"}
Use selectedIndex 0 when none is highly relevant.`;
try {
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillEvolver.judgeRelated", { temperature: 0, maxTokens: 256 });
const parsed = this.parseJudgeSkillResult(raw, candidates.length);
if (parsed.selectedIndex >= 1 && parsed.selectedIndex <= candidates.length) {
return candidates[parsed.selectedIndex - 1].skill;
}
return null;
} catch (err) {
this.ctx.log.warn(`SkillEvolver: LLM skill relevance judgment failed: ${err}`);
return null;
}
}
private parseJudgeSkillResult(raw: string, maxIndex: number): { selectedIndex: number; reason: string } {
const fallback = { selectedIndex: 0, reason: "parse failed" };
const match = raw.match(/\{[\s\S]*\}/);
if (!match) return fallback;
try {
const obj = JSON.parse(match[0]) as { selectedIndex?: number; reason?: string };
const idx = typeof obj.selectedIndex === "number" ? obj.selectedIndex : 0;
const reason = typeof obj.reason === "string" ? obj.reason : "";
if (idx < 0 || idx > maxIndex) return { selectedIndex: 0, reason: reason || "out of range" };
return { selectedIndex: idx, reason };
} catch {
return fallback;
}
}
private async handleExistingSkill(task: Task, chunks: Chunk[], skill: Skill): Promise {
// Verify skill still exists in DB (may have been manually deleted)
const freshSkill = this.store.getSkill(skill.id);
if (!freshSkill) {
this.ctx.log.warn(`SkillEvolver: skill "${skill.name}" (${skill.id}) no longer exists, treating as new`);
await this.handleNewSkill(task, chunks);
return;
}
const skillContent = this.readSkillContent(freshSkill);
if (!skillContent) {
this.ctx.log.warn(`SkillEvolver: cannot read skill "${freshSkill.name}" content, treating as new`);
await this.handleNewSkill(task, chunks);
return;
}
const minConfidence = this.ctx.config.skillEvolution?.minConfidence ?? DEFAULTS.skillMinConfidence;
const evalResult = await this.evaluator.evaluateUpgrade(task, freshSkill, skillContent);
if (evalResult.shouldUpgrade && evalResult.confidence >= minConfidence) {
this.ctx.log.info(`SkillEvolver: upgrading skill "${freshSkill.name}" — ${evalResult.reason}`);
const { upgraded } = await this.upgrader.upgrade(task, freshSkill, evalResult);
this.markChunksWithSkill(chunks, freshSkill.id);
if (upgraded) {
this.store.linkTaskSkill(task.id, freshSkill.id, "evolved_from", freshSkill.version + 1);
this.installer.syncIfInstalled(freshSkill.name);
this.onSkillEvolved?.(freshSkill.name, "upgraded");
} else {
this.store.linkTaskSkill(task.id, freshSkill.id, "applied_to", freshSkill.version);
}
} else if (evalResult.confidence < 0.3) {
this.ctx.log.info(
`SkillEvolver: skill "${freshSkill.name}" has low relevance (confidence=${evalResult.confidence}), ` +
`falling back to new skill evaluation for task "${task.title}"`,
);
await this.handleNewSkill(task, chunks);
} else {
this.ctx.log.debug(`SkillEvolver: skill "${freshSkill.name}" not worth upgrading (confidence=${evalResult.confidence})`);
this.markChunksWithSkill(chunks, freshSkill.id);
this.store.linkTaskSkill(task.id, freshSkill.id, "applied_to", freshSkill.version);
}
}
private async handleNewSkill(task: Task, chunks: Chunk[]): Promise {
const minConfidence = this.ctx.config.skillEvolution?.minConfidence ?? DEFAULTS.skillMinConfidence;
const evalResult = await this.evaluator.evaluateCreate(task);
if (evalResult.shouldGenerate && evalResult.confidence >= minConfidence) {
this.ctx.log.info(`SkillEvolver: generating new skill "${evalResult.suggestedName}" — ${evalResult.reason}`);
this.store.setTaskSkillMeta(task.id, { skillStatus: "generating", skillReason: evalResult.reason });
const skill = await this.generator.generate(task, chunks, evalResult);
this.markChunksWithSkill(chunks, skill.id);
this.store.linkTaskSkill(task.id, skill.id, "generated_from", 1);
this.store.setTaskSkillMeta(task.id, { skillStatus: "generated", skillReason: evalResult.reason });
this.onSkillEvolved?.(skill.name, "created");
const autoInstall = this.ctx.config.skillEvolution?.autoInstall ?? DEFAULTS.skillAutoInstall;
if (autoInstall && skill.status === "active") {
this.installer.install(skill.id);
}
} else {
const reason = evalResult.reason || `confidence不足 (${evalResult.confidence} < ${minConfidence})`;
this.ctx.log.debug(`SkillEvolver: task "${task.title}" not worth generating skill — ${reason}`);
this.store.setTaskSkillMeta(task.id, { skillStatus: "not_generated", skillReason: reason });
}
}
private markChunksWithSkill(chunks: Chunk[], skillId: string): void {
for (const chunk of chunks) {
this.store.setChunkSkillId(chunk.id, skillId);
}
this.ctx.log.debug(`SkillEvolver: marked ${chunks.length} chunks with skill_id=${skillId}`);
}
private readSkillContent(skill: Skill): string | null {
const filePath = path.join(skill.dirPath, "SKILL.md");
try {
if (fs.existsSync(filePath)) {
return fs.readFileSync(filePath, "utf-8");
}
} catch { /* fall through */ }
const sv = this.store.getLatestSkillVersion(skill.id);
return sv?.content ?? null;
}
}
================================================
FILE: apps/memos-local-openclaw/src/skill/generator.ts
================================================
import { v4 as uuid } from "uuid";
import * as fs from "fs";
import * as path from "path";
import type { SqliteStore } from "../storage/sqlite";
import type { RecallEngine } from "../recall/engine";
import type { Embedder } from "../embedding";
import type { Chunk, Task, Skill, PluginContext, SkillGenerateOutput } from "../types";
import { DEFAULTS } from "../types";
import type { CreateEvalResult } from "./evaluator";
import { SkillValidator } from "./validator";
import { buildSkillConfigChain, callLLMWithFallback } from "../shared/llm-call";
// ─── Step 1: Generate SKILL.md ───
// Based on Anthropic skill-creator principles:
// - Progressive disclosure (metadata ~100 words → body <500 lines → resources on demand)
// - Description as primary trigger mechanism — write it "pushy"
// - Explain WHY, not pile up MUST/NEVER
// - Imperative form, keep it concise
// - Generalize from the specific task, don't over-fit
const STEP1_SKILL_MD_PROMPT = `You are a Skill creation expert. Your job is to distill a completed task's execution record into a reusable SKILL.md file.
This Skill is special: it comes from real execution experience — every step was actually run, every pitfall was actually encountered and resolved.
## Core principles (follow strictly but do NOT include these in output)
### Progressive disclosure
- The frontmatter description (~100 words) is ALWAYS in the agent's context — it must be self-sufficient for deciding whether to use this skill.
- The SKILL.md body loads when triggered — keep it under 400 lines, focused, no fluff.
- If the task involved large configs/scripts, mention them but DON'T inline everything — just reference that scripts/ or references/ may contain them.
### Description as trigger mechanism
The description field decides whether the agent activates this skill. Write it "proactively":
- Don't just say what it does — list the situations, keywords, and phrasings that should trigger it.
- Claude/agents tend to under-trigger skills. Counter this by being explicit about when to use it.
- Bad: "How to deploy Node.js to Docker"
- Good: "How to containerize and deploy a Node.js application using Docker. Use when the user mentions Docker deployment, Dockerfile writing, container builds, multi-stage builds, port mapping, .dockerignore, image optimization, CI/CD container pipelines, or any task involving packaging a Node/JS backend into a container — even if they don't say 'Docker' explicitly but describe wanting to 'package the app for production' or 'run it anywhere'."
### Writing style
- Use imperative form
- Explain WHY for each step, not just HOW — today's LLMs respond better to reasoning than rigid rules
- Seeing yourself write ALWAYS or NEVER in caps is a yellow flag — rephrase with reasoning instead
- Generalize from the specific task so the skill works for similar future scenarios, don't over-fit to this exact project
- Keep real commands/code/config from the task record — these are verified to work
### Language matching (CRITICAL)
You MUST write the ENTIRE skill in the SAME language as the user's messages in the task record.
- If the user wrote in Chinese → the skill title, description, all prose sections MUST be in Chinese
- If the user wrote in English → write in English
- If mixed → use the language that appears most in the user's messages
- The "name" field in frontmatter should still use English kebab-case (it's a machine identifier)
- But "description", section headings, step explanations, pitfall descriptions — ALL must match the user's language
- Code/commands stay in their original language (they are language-agnostic)
DO NOT default to English. Look at the task record below and match its language.
## Output format
Output ONLY the complete SKILL.md content. No extra text before or after.
---
name: "{NAME}"
description: "{A natural, proactive description. 60-120 words. Cover what it does + multiple phrasings/scenarios that should trigger it. Be pushy about triggering — list keywords, alternative descriptions, edge-case phrasings.}"
metadata: {{ "openclaw": {{ "emoji": "{emoji}" }} }}
---
# {Title — clear, action-oriented}
{One sentence: what this skill helps you do and why it's valuable}
## When to use this skill
{2-4 bullet points describing the scenarios. Focus on the user's INTENT, not just keywords. Example: "When you need to get a Node app running reliably in a container and want to avoid common pitfalls like bloated images or missing health checks."}
## Steps
{Numbered or sectioned steps extracted from the task. EVERY step actually performed must be included — do NOT skip or generalize away concrete steps like "configure security groups", "set environment variables", etc. For each step:
1. What to do (keep inline code short — if a step involves a long script or config, write a brief summary here and say "see scripts/ for the complete script")
2. Why this matters (one sentence explaining the reasoning)
Keep the actual commands/code from the task — they're verified. But avoid duplicating large code blocks that will also appear in scripts/ — reference them instead.}
## Pitfalls and solutions
{What went wrong during the task and how it was fixed. Format:
❌ Wrong approach → Why it fails → ✅ Correct approach
These are the most valuable parts — real debugging experience.}
## Key code and configuration
{Complete, verified code blocks and config files. Don't summarize code — keep it complete and runnable.}
## Environment and prerequisites
{Versions, dependencies, permissions, OS requirements — anything needed to reproduce.}
## Companion files
{If the skill comes with automation scripts or reference docs, list them here so the reader knows they exist:
- \`scripts/\` — brief description of what this script does
- \`references/\` — brief description of what this reference covers
If no companion files exist, omit this section entirely.}
## Task record
Task title: {TITLE}
Task summary:
{SUMMARY}
Conversation highlights:
{CONVERSATION}`;
// ─── Step 2: Extract scripts ───
const STEP2_SCRIPTS_PROMPT = `Based on the following SKILL.md and task record, extract reusable automation scripts.
Rules:
- Only extract if the task record contains concrete shell commands, Python scripts, or TypeScript code that form a complete, reusable automation.
- Each script must be self-contained and runnable.
- If there are no automatable scripts (e.g., the task was mostly manual steps or config editing), return an empty array.
- Don't fabricate scripts — only extract what was actually used in the task.
- The script should COMPLEMENT the SKILL.md, not duplicate it. If SKILL.md already has the steps in detail, the script should be the automation version. If SKILL.md references the script, the script should contain the full implementation.
- The script filename should be descriptive (e.g., "deploy.sh", "configure_openclaw.sh", "setup_security_group.sh").
SKILL.md:
{SKILL_CONTENT}
Task conversation highlights:
{CONVERSATION}
Reply with a JSON array only. No extra text:
[
{{ "filename": "deploy.sh", "content": "#!/bin/bash\\n..." }},
{{ "filename": "setup.py", "content": "..." }}
]
If no scripts should be extracted, reply with: []`;
// ─── Step 3: Generate evals ───
const STEP3_EVALS_PROMPT = `Based on the following skill, generate realistic test prompts that should trigger this skill.
Requirements:
- Write 3-4 test prompts that a real user would type
- Mix of direct and indirect phrasings (some obviously match the skill, some are edge cases)
- Include realistic details: file paths, project names, specific error messages
- Mix formal and casual tones, include some with typos or shorthand
- Each prompt should be complex enough that the agent would need the skill (not simple Q&A)
- Write expectations that are specific and verifiable
- LANGUAGE RULE: Write prompts and expectations in the SAME language as the skill content. If the skill is in Chinese, write Chinese test prompts. If English, write English.
Skill:
{SKILL_CONTENT}
Reply with a JSON array only:
[
{{
"id": 1,
"prompt": "A realistic user message that should trigger this skill",
"expectations": ["Specific expected behavior 1", "Specific expected behavior 2"],
"trigger_confidence": "high|medium"
}}
]`;
// ─── Step 2b: Extract references ───
const STEP2B_REFS_PROMPT = `Based on the following SKILL.md and task record, extract reference documentation worth preserving.
Rules:
- Only extract if the task involved important API docs, configuration references, or technical notes that would be useful for future similar tasks.
- Each reference should be a standalone markdown document.
- Don't duplicate what's already in SKILL.md — references are for deeper detail.
- If there's nothing worth extracting, return an empty array.
- LANGUAGE RULE: Write reference content in the SAME language as the SKILL.md and task record. Chinese input → Chinese output.
SKILL.md:
{SKILL_CONTENT}
Task conversation highlights:
{CONVERSATION}
Reply with a JSON array only:
[
{{ "filename": "api-notes.md", "content": "# API Reference\\n..." }}
]
If no references should be extracted, reply with: []`;
export class SkillGenerator {
private validator: SkillValidator;
private embedder: Embedder | null = null;
constructor(
private store: SqliteStore,
private engine: RecallEngine,
private ctx: PluginContext,
embedder?: Embedder,
) {
this.validator = new SkillValidator(ctx);
this.embedder = embedder ?? null;
}
async generate(task: Task, chunks: Chunk[], evalResult: CreateEvalResult): Promise {
const conversationText = this.buildConversationText(chunks);
// ── Step 1: Generate SKILL.md (primary, largest output) ──
this.ctx.log.info(`SkillGenerator: Step 1/4 — generating SKILL.md for "${evalResult.suggestedName}"`);
let skillMdContent = await this.step1GenerateSkillMd(task, conversationText, evalResult);
const skillsStoreDir = path.join(this.ctx.stateDir, "skills-store");
const dirPath = path.join(skillsStoreDir, evalResult.suggestedName);
fs.mkdirSync(dirPath, { recursive: true });
fs.writeFileSync(path.join(dirPath, "SKILL.md"), skillMdContent, "utf-8");
// ── Step 2: Extract scripts (parallel with refs) ──
this.ctx.log.info(`SkillGenerator: Step 2/4 — extracting scripts and references`);
const [scripts, references] = await Promise.all([
this.step2ExtractScripts(skillMdContent, conversationText),
this.step2bExtractReferences(skillMdContent, conversationText),
]);
if (scripts.length > 0) {
const scriptsDir = path.join(dirPath, "scripts");
fs.mkdirSync(scriptsDir, { recursive: true });
for (const s of scripts) {
fs.writeFileSync(path.join(scriptsDir, s.filename), s.content, "utf-8");
}
}
if (references.length > 0) {
const refsDir = path.join(dirPath, "references");
fs.mkdirSync(refsDir, { recursive: true });
for (const r of references) {
fs.writeFileSync(path.join(refsDir, r.filename), r.content, "utf-8");
}
}
// Ensure SKILL.md has companion files section
if (scripts.length > 0 || references.length > 0) {
const hasCompanionSection = /## Companion files|## 附属文件|## 辅助文件/.test(skillMdContent);
if (!hasCompanionSection) {
const companionLines: string[] = ["\n\n## Companion files\n"];
for (const s of scripts) {
companionLines.push(`- \`scripts/${s.filename}\` — automation script`);
}
for (const r of references) {
companionLines.push(`- \`references/${r.filename}\` — reference documentation`);
}
skillMdContent += companionLines.join("\n");
fs.writeFileSync(path.join(dirPath, "SKILL.md"), skillMdContent, "utf-8");
}
}
// ── Step 3: Generate evals ──
this.ctx.log.info(`SkillGenerator: Step 3/4 — generating eval test cases`);
const evals = await this.step3GenerateEvals(skillMdContent);
if (evals.length > 0) {
const evalsDir = path.join(dirPath, "evals");
fs.mkdirSync(evalsDir, { recursive: true });
fs.writeFileSync(
path.join(evalsDir, "evals.json"),
JSON.stringify({ skill_name: evalResult.suggestedName, evals }, null, 2),
"utf-8",
);
}
// ── Step 4: Validate + verify evals ──
this.ctx.log.info(`SkillGenerator: Step 4/4 — validating and verifying`);
const validation = await this.validator.validate(dirPath);
const evalVerification = await this.verifyEvals(evals);
const description = this.parseDescription(skillMdContent);
const status = validation.qualityScore !== null && validation.qualityScore < 6 ? "draft" as const : "active" as const;
const skillId = uuid();
const now = Date.now();
const skill: Skill = {
id: skillId,
name: evalResult.suggestedName,
description,
version: 1,
status,
tags: JSON.stringify(evalResult.suggestedTags),
sourceType: "task",
dirPath,
installed: 0,
owner: "agent:main",
visibility: "private",
qualityScore: validation.qualityScore,
createdAt: now,
updatedAt: now,
};
this.store.insertSkill(skill);
if (description && this.embedder) {
try {
const [descEmb] = await this.embedder.embed([description]);
if (descEmb) this.store.upsertSkillEmbedding(skillId, descEmb);
} catch (err) {
this.ctx.log.warn(`SkillGenerator: embedding for description failed: ${err}`);
}
}
this.store.insertSkillVersion({
id: uuid(),
skillId,
version: 1,
content: skillMdContent,
changelog: `Initial generation from task "${task.title}"`,
changeSummary: `首次从任务"${task.title}"的实际执行记录中提炼生成。${description ? `该技能涵盖:${description.slice(0, 200)}` : ""}${scripts.length > 0 ? ` 包含 ${scripts.length} 个辅助脚本。` : ""}${evals.length > 0 ? ` 附带 ${evals.length} 个测试用例(${evalVerification.hitCount}/${evals.length} 通过命中验证)。` : ""}`,
upgradeType: "create",
sourceTaskId: task.id,
metrics: JSON.stringify({
dimensions: [],
confidence: evalResult.confidence,
scripts: scripts.map(s => s.filename),
references: references.map(r => r.filename),
evalCount: evals.length,
evalVerification,
validation: {
errors: validation.errors,
warnings: validation.warnings,
suggestions: validation.suggestions,
},
}),
qualityScore: validation.qualityScore,
createdAt: now,
});
if (validation.warnings.length > 0) {
this.ctx.log.info(`Skill "${skill.name}" validation warnings: ${validation.warnings.join("; ")}`);
}
this.ctx.log.info(
`Skill generated: "${skill.name}" v1 [${status}] score=${validation.qualityScore ?? "N/A"} `
+ `scripts=${scripts.length} refs=${references.length} evals=${evals.length} `
+ `evalHits=${evalVerification.hitCount}/${evals.length} `
+ `from task "${task.title}"`,
);
return skill;
}
// ─── Step 1: SKILL.md generation ───
private detectLanguage(text: string): string {
const cjk = text.match(/[\u4e00-\u9fff\u3400-\u4dbf]/g)?.length ?? 0;
const total = text.replace(/\s+/g, "").length || 1;
if (cjk / total > 0.15) return "Chinese (中文)";
return "English";
}
private async step1GenerateSkillMd(task: Task, conversationText: string, evalResult: CreateEvalResult): Promise {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) throw new Error("No LLM configured for skill generation");
const lang = this.detectLanguage(conversationText);
const langInstruction = `\n\n⚠️ LANGUAGE REQUIREMENT: The task record is in ${lang}. You MUST write ALL prose content (description, headings, explanations, pitfalls) in ${lang}. Only the "name" field stays in English kebab-case.\n`;
const prompt = STEP1_SKILL_MD_PROMPT
.replace("{NAME}", evalResult.suggestedName)
.replace("{TITLE}", task.title)
.replace("{SUMMARY}", task.summary.slice(0, 5000))
.replace("{CONVERSATION}", conversationText.slice(0, 12000))
+ langInstruction;
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillGenerator.step1", { maxTokens: 6000, temperature: 0.2, timeoutMs: 120_000 });
const trimmed = raw.trim();
if (trimmed.startsWith("---")) return trimmed;
const fmStart = trimmed.indexOf("---");
if (fmStart !== -1) return trimmed.slice(fmStart);
return trimmed;
}
// ─── Step 2: Extract scripts ───
private async step2ExtractScripts(
skillContent: string,
conversationText: string,
): Promise> {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) return [];
const prompt = STEP2_SCRIPTS_PROMPT
.replace("{SKILL_CONTENT}", skillContent.slice(0, 4000))
.replace("{CONVERSATION}", conversationText.slice(0, 6000));
try {
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillGenerator.scripts", { maxTokens: 3000, temperature: 0.1, timeoutMs: 120_000 });
return this.parseJSONArray<{ filename: string; content: string }>(raw);
} catch (err) {
this.ctx.log.warn(`SkillGenerator: script extraction failed: ${err}`);
return [];
}
}
// ─── Step 2b: Extract references ───
private async step2bExtractReferences(
skillContent: string,
conversationText: string,
): Promise> {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) return [];
const prompt = STEP2B_REFS_PROMPT
.replace("{SKILL_CONTENT}", skillContent.slice(0, 4000))
.replace("{CONVERSATION}", conversationText.slice(0, 6000));
try {
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillGenerator.refs", { maxTokens: 3000, temperature: 0.1, timeoutMs: 120_000 });
return this.parseJSONArray<{ filename: string; content: string }>(raw);
} catch (err) {
this.ctx.log.warn(`SkillGenerator: reference extraction failed: ${err}`);
return [];
}
}
// ─── Step 3: Generate evals ───
private async step3GenerateEvals(
skillContent: string,
): Promise> {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) return [];
const lang = this.detectLanguage(skillContent);
const prompt = STEP3_EVALS_PROMPT
.replace("{SKILL_CONTENT}", skillContent.slice(0, 4000))
+ `\n\n⚠️ LANGUAGE: Write test prompts and expectations in ${lang}, matching the skill's language.\n`;
try {
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillGenerator.evals", { maxTokens: 2000, temperature: 0.3, timeoutMs: 120_000 });
return this.parseJSONArray(raw);
} catch (err) {
this.ctx.log.warn(`SkillGenerator: eval generation failed: ${err}`);
return [];
}
}
// ─── Step 4: Verify evals via memory search ───
private async verifyEvals(
evals: Array<{ id: number; prompt: string; expectations: string[] }>,
): Promise<{ hitCount: number; results: Array<{ evalId: number; hit: boolean; topScore: number }> }> {
const results: Array<{ evalId: number; hit: boolean; topScore: number }> = [];
let hitCount = 0;
for (const ev of evals.slice(0, 4)) {
try {
const searchResult = await this.engine.search({
query: ev.prompt,
maxResults: 5,
minScore: 0.3,
});
const topScore = searchResult.hits.length > 0 ? searchResult.hits[0].score : 0;
const hasSkillHit = searchResult.hits.some(h => h.skillId != null);
const hit = searchResult.hits.length > 0 && topScore >= 0.4;
if (hit) hitCount++;
results.push({ evalId: ev.id, hit, topScore });
this.ctx.log.debug(
`SkillGenerator eval verify: "${ev.prompt.slice(0, 50)}..." → `
+ `hits=${searchResult.hits.length} topScore=${topScore.toFixed(3)} skillHit=${hasSkillHit}`,
);
} catch (err) {
this.ctx.log.warn(`SkillGenerator: eval verification failed for eval ${ev.id}: ${err}`);
results.push({ evalId: ev.id, hit: false, topScore: 0 });
}
}
return { hitCount, results };
}
// ─── Helpers ───
private parseJSONArray(raw: string): T[] {
const match = raw.match(/\[[\s\S]*\]/);
if (!match) return [];
try {
const arr = JSON.parse(match[0]);
return Array.isArray(arr) ? arr : [];
} catch {
this.ctx.log.warn("SkillGenerator: JSON array parse failed");
return [];
}
}
private buildConversationText(chunks: Chunk[]): string {
const lines: string[] = [];
for (const c of chunks) {
if (c.role !== "user" && c.role !== "assistant") continue;
const roleLabel = c.role === "user" ? "User" : "Assistant";
lines.push(`[${roleLabel}]: ${c.content}`);
}
return lines.join("\n\n");
}
private parseDescription(content: string): string {
const match = content.match(/description:\s*"([^"]+)"/);
if (match) return match[1];
const match2 = content.match(/description:\s*'([^']+)'/);
if (match2) return match2[1];
return "";
}
}
================================================
FILE: apps/memos-local-openclaw/src/skill/installer.ts
================================================
import * as fs from "fs";
import * as path from "path";
import type { SqliteStore } from "../storage/sqlite";
import type { PluginContext } from "../types";
export class SkillInstaller {
private workspaceSkillsDir: string;
constructor(
private store: SqliteStore,
private ctx: PluginContext,
) {
this.workspaceSkillsDir = path.join(ctx.workspaceDir, "skills");
}
install(skillId: string): { installed: boolean; path: string; message: string } {
const skill = this.store.getSkill(skillId);
if (!skill) return { installed: false, path: "", message: "Skill not found" };
if (!fs.existsSync(skill.dirPath)) {
return { installed: false, path: "", message: `Skill directory not found: ${skill.dirPath}` };
}
const dstDir = path.join(this.workspaceSkillsDir, skill.name);
fs.mkdirSync(dstDir, { recursive: true });
fs.cpSync(skill.dirPath, dstDir, { recursive: true });
this.store.updateSkill(skillId, { installed: 1 });
this.ctx.log.info(`Skill installed: "${skill.name}" v${skill.version} → ${dstDir}`);
return {
installed: true,
path: dstDir,
message: `Skill "${skill.name}" v${skill.version} installed`,
};
}
uninstall(skillId: string): void {
const skill = this.store.getSkill(skillId);
if (!skill) return;
const dstDir = path.join(this.workspaceSkillsDir, skill.name);
if (fs.existsSync(dstDir)) {
fs.rmSync(dstDir, { recursive: true });
}
this.store.updateSkill(skillId, { installed: 0 });
this.ctx.log.info(`Skill uninstalled: "${skill.name}"`);
}
syncIfInstalled(skillName: string): void {
const skill = this.store.getSkillByName(skillName);
if (!skill || !skill.installed) return;
const dstDir = path.join(this.workspaceSkillsDir, skill.name);
if (fs.existsSync(dstDir) && fs.existsSync(skill.dirPath)) {
fs.cpSync(skill.dirPath, dstDir, { recursive: true });
this.ctx.log.info(`Skill synced: "${skill.name}" v${skill.version} → workspace`);
}
}
}
================================================
FILE: apps/memos-local-openclaw/src/skill/upgrader.ts
================================================
import { v4 as uuid } from "uuid";
import * as fs from "fs";
import * as path from "path";
import type { SqliteStore } from "../storage/sqlite";
import type { Task, Skill, PluginContext } from "../types";
import type { UpgradeEvalResult } from "./evaluator";
import { SkillValidator } from "./validator";
import { buildSkillConfigChain, callLLMWithFallback } from "../shared/llm-call";
const UPGRADE_PROMPT = `You are a Skill upgrade expert. You're merging new real-world execution experience into an existing Skill to make it better.
Remember: this is based on ACTUAL execution — the new task was really run, errors were really encountered and fixed. This makes the upgrade valuable.
## Core principles (follow strictly but do NOT include in output)
### Progressive disclosure
- Keep the frontmatter description as the primary trigger mechanism (~60-120 words, proactive — see below)
- SKILL.md body should stay under 400 lines total
- If content grows too large, consider moving deep details to references/ and just pointing to them
### Description as trigger
The description decides whether the agent activates this skill. Write it "proactively":
- Cover what it does + situations/keywords/phrasings that should trigger it
- Be explicit about edge cases — "even if the user doesn't say X explicitly but describes Y"
- If the new task reveals new trigger scenarios, ADD them to the description
### Writing style
- Imperative form
- Explain WHY for each step — reasoning beats rigid rules
- Avoid ALWAYS/NEVER in caps — rephrase with reasoning instead
- Generalize from specific tasks
- Keep verified commands/code/config from both old and new tasks
- CRITICAL: Match the language of the skill and task record. If the existing skill or the new task record is in Chinese, write ALL upgraded content in Chinese. If English, write in English. Only the "name" field stays in English kebab-case. DO NOT default to English.
## Existing skill (v{VERSION}):
{SKILL_CONTENT}
## Upgrade context
- Type: {UPGRADE_TYPE}
- Dimensions improved: {DIMENSIONS}
- Reason: {REASON}
- Merge strategy: {MERGE_STRATEGY}
## New task record
Title: {TITLE}
Summary:
{SUMMARY}
## Merge rules
1. Preserve all valid core content from the existing skill — upgrades should ADD value, not lose it
2. Merge new experience strategically:
- Better approach found → replace old, keep old as "Alternative approach" if it's still valid
- New scenario discovered → add a new section (don't replace unrelated content)
- Bug/error corrected → replace directly, add to "Pitfalls and solutions" section
- Performance improvement → update steps, note the improvement in why-reasoning
3. Update description if new scenarios/keywords/triggers need coverage
4. Update "When to use this skill" section if the new task reveals new use cases
5. If a "Pitfalls and solutions" section exists, append new pitfalls; if it doesn't exist, create it
6. Total length ≤ 400 lines — if approaching limit, move detailed configs/references to references/
7. Add version comment at end:
## Output format
Output the complete upgraded SKILL.md (with full frontmatter), then on a new line write:
---CHANGELOG---
{one-line changelog title}
---CHANGE_SUMMARY---
{A 3-5 sentence summary in the same language as the skill. Cover: (1) What specifically was changed and what triggered the change, (2) What concrete new capability or improvement this version brings, (3) What real problem from the new task this solves. Write for a human reader who wants to quickly understand the value of this upgrade.}`;
export class SkillUpgrader {
private validator: SkillValidator;
constructor(
private store: SqliteStore,
private ctx: PluginContext,
) {
this.validator = new SkillValidator(ctx);
}
async upgrade(task: Task, skill: Skill, evalResult: UpgradeEvalResult): Promise<{ upgraded: boolean; qualityScore: number | null }> {
const currentContent = this.readCurrentContent(skill);
if (!currentContent) {
this.ctx.log.warn(`SkillUpgrader: could not read content for "${skill.name}"`);
return { upgraded: false, qualityScore: null };
}
const { newContent, changelog, changeSummary } = await this.callUpgradeLLM(task, skill, currentContent, evalResult);
if (!newContent || newContent.length < 100) {
this.ctx.log.warn(`SkillUpgrader: generated content too short for "${skill.name}", skipping`);
return { upgraded: false, qualityScore: null };
}
fs.writeFileSync(path.join(skill.dirPath, "SKILL.md"), newContent, "utf-8");
const validation = await this.validator.validate(skill.dirPath, {
previousContent: currentContent,
});
if (!validation.valid) {
this.ctx.log.warn(`SkillUpgrader: validation failed for "${skill.name}", reverting: ${validation.errors.join("; ")}`);
fs.writeFileSync(path.join(skill.dirPath, "SKILL.md"), currentContent, "utf-8");
return { upgraded: false, qualityScore: null };
}
const newVersion = skill.version + 1;
const newDescription = this.parseDescription(newContent) || skill.description;
const newStatus = validation.qualityScore !== null && validation.qualityScore < 6 ? "draft" as const : skill.status;
this.store.updateSkill(skill.id, {
description: newDescription,
version: newVersion,
status: newStatus,
qualityScore: validation.qualityScore,
updatedAt: Date.now(),
});
this.store.insertSkillVersion({
id: uuid(),
skillId: skill.id,
version: newVersion,
content: newContent,
changelog: changelog || `Upgraded from task "${task.title}"`,
changeSummary: changeSummary || `基于任务"${task.title}"的执行记录进行了版本升级。`,
upgradeType: evalResult.upgradeType,
sourceTaskId: task.id,
metrics: JSON.stringify({
dimensions: evalResult.dimensions,
confidence: evalResult.confidence,
validation: {
errors: validation.errors,
warnings: validation.warnings,
suggestions: validation.suggestions,
},
}),
qualityScore: validation.qualityScore,
createdAt: Date.now(),
});
if (validation.warnings.length > 0) {
this.ctx.log.info(`Skill "${skill.name}" upgrade warnings: ${validation.warnings.join("; ")}`);
}
this.ctx.log.info(
`Skill upgraded: "${skill.name}" v${skill.version} → v${newVersion} [${newStatus}] score=${validation.qualityScore ?? "N/A"}`,
);
return { upgraded: true, qualityScore: validation.qualityScore };
}
private readCurrentContent(skill: Skill): string | null {
const filePath = path.join(skill.dirPath, "SKILL.md");
try {
return fs.readFileSync(filePath, "utf-8");
} catch {
const sv = this.store.getLatestSkillVersion(skill.id);
return sv?.content ?? null;
}
}
private async callUpgradeLLM(
task: Task,
skill: Skill,
currentContent: string,
evalResult: UpgradeEvalResult,
): Promise<{ newContent: string; changelog: string; changeSummary: string }> {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) throw new Error("No LLM configured for skill upgrade");
const newVersion = skill.version + 1;
const detectLang = (text: string): string => {
const cjk = text.match(/[\u4e00-\u9fff\u3400-\u4dbf]/g)?.length ?? 0;
const total = text.replace(/\s+/g, "").length || 1;
return (cjk / total > 0.15) ? "Chinese (中文)" : "English";
};
const lang = detectLang(task.summary + currentContent);
const langInstruction = `\n\n⚠️ LANGUAGE REQUIREMENT: The content is in ${lang}. You MUST write ALL prose (description, headings, explanations, pitfalls, changelog, change summary) in ${lang}. Only the "name" field stays in English kebab-case.\n`;
const prompt = UPGRADE_PROMPT
.replace("{VERSION}", String(skill.version))
.replace("{SKILL_CONTENT}", currentContent.slice(0, 6000))
.replace("{UPGRADE_TYPE}", evalResult.upgradeType)
.replace("{DIMENSIONS}", evalResult.dimensions.join(", "))
.replace("{REASON}", evalResult.reason)
.replace("{MERGE_STRATEGY}", evalResult.mergeStrategy)
.replace("{TITLE}", task.title)
.replace("{SUMMARY}", task.summary.slice(0, 4000))
.replace("{NEW_VERSION}", String(newVersion))
.replace("{TASK_ID}", task.id)
+ langInstruction;
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillUpgrader.upgrade", { maxTokens: 6000, temperature: 0.2, timeoutMs: 90_000 });
const changelogSep = raw.indexOf("---CHANGELOG---");
if (changelogSep !== -1) {
const newContent = raw.slice(0, changelogSep).trim();
const afterChangelog = raw.slice(changelogSep + "---CHANGELOG---".length).trim();
const summarySep = afterChangelog.indexOf("---CHANGE_SUMMARY---");
if (summarySep !== -1) {
const changelog = afterChangelog.slice(0, summarySep).trim();
const changeSummary = afterChangelog.slice(summarySep + "---CHANGE_SUMMARY---".length).trim();
return { newContent, changelog, changeSummary };
}
return { newContent, changelog: afterChangelog, changeSummary: "" };
}
return { newContent: raw, changelog: "", changeSummary: "" };
}
private parseDescription(content: string): string {
const match = content.match(/description:\s*"([^"]+)"/);
if (match) return match[1];
const match2 = content.match(/description:\s*'([^']+)'/);
if (match2) return match2[1];
return "";
}
}
================================================
FILE: apps/memos-local-openclaw/src/skill/validator.ts
================================================
import * as fs from "fs";
import * as path from "path";
import type { PluginContext } from "../types";
import { DEFAULTS } from "../types";
import { buildSkillConfigChain, callLLMWithFallback } from "../shared/llm-call";
export interface ValidationResult {
valid: boolean;
qualityScore: number | null;
errors: string[];
warnings: string[];
suggestions: string[];
}
export class SkillValidator {
constructor(private ctx: PluginContext) {}
/**
* Format validation (no LLM needed) + optional LLM quality assessment.
* Returns combined result with score 0-10.
*/
async validate(dirPath: string, opts?: { skipLLM?: boolean; previousContent?: string }): Promise {
const result: ValidationResult = {
valid: true,
qualityScore: null,
errors: [],
warnings: [],
suggestions: [],
};
this.validateFormat(dirPath, result);
if (!result.valid) return result;
if (opts?.previousContent) {
this.regressionCheck(dirPath, opts.previousContent, result);
}
if (!opts?.skipLLM) {
try {
await this.assessQuality(dirPath, result);
} catch (err) {
this.ctx.log.warn(`SkillValidator: LLM quality assessment failed: ${err}`);
result.warnings.push(`Quality assessment skipped: ${err}`);
}
}
return result;
}
private validateFormat(dirPath: string, result: ValidationResult): void {
const skillMdPath = path.join(dirPath, "SKILL.md");
if (!fs.existsSync(skillMdPath)) {
result.valid = false;
result.errors.push("SKILL.md not found");
return;
}
const content = fs.readFileSync(skillMdPath, "utf-8");
if (!content.trim()) {
result.valid = false;
result.errors.push("SKILL.md is empty");
return;
}
const fmMatch = content.match(/^---\s*\n([\s\S]*?)\n---/);
if (!fmMatch) {
result.valid = false;
result.errors.push("YAML frontmatter missing (expected --- ... ---)");
return;
}
const frontmatter = fmMatch[1];
const nameMatch = frontmatter.match(/^name:\s*["']?(.+?)["']?\s*$/m);
if (!nameMatch || !nameMatch[1].trim()) {
result.valid = false;
result.errors.push("Frontmatter missing 'name' field");
return;
}
const name = nameMatch[1].trim();
if (name.length > 64) {
result.errors.push(`Name too long (${name.length} chars, max 64)`);
result.valid = false;
}
if (!/^[a-z0-9][a-z0-9-]*[a-z0-9]$/.test(name) && name.length > 1) {
result.warnings.push(`Name "${name}" is not strict kebab-case`);
}
const descMatch = frontmatter.match(/^description:\s*["']?([\s\S]*?)["']?\s*$/m);
if (!descMatch || !descMatch[1].trim()) {
result.valid = false;
result.errors.push("Frontmatter missing 'description' field");
return;
}
const desc = descMatch[1].trim();
if (desc.length > 1024) {
result.warnings.push(`Description too long (${desc.length} chars, max 1024)`);
}
const maxLines = this.ctx.config.skillEvolution?.maxSkillLines ?? DEFAULTS.skillMaxLines;
const lineCount = content.split("\n").length;
if (lineCount > maxLines) {
result.warnings.push(`Content exceeds ${maxLines} lines (has ${lineCount})`);
}
if (content.length < 200) {
result.warnings.push("Content seems very short (< 200 chars)");
}
}
/**
* Check that an upgrade doesn't lose significant content from the previous version.
*/
private regressionCheck(dirPath: string, previousContent: string, result: ValidationResult): void {
const skillMdPath = path.join(dirPath, "SKILL.md");
const newContent = fs.readFileSync(skillMdPath, "utf-8");
const prevLines = previousContent.split("\n").length;
const newLines = newContent.split("\n").length;
if (newLines < prevLines * 0.7 && prevLines > 20) {
result.warnings.push(
`Content shrank significantly: ${prevLines} → ${newLines} lines (${Math.round((1 - newLines / prevLines) * 100)}% reduction)`,
);
}
const prevSections = (previousContent.match(/^##\s+.+$/gm) || []).map(s => s.replace(/^##\s+/, "").trim().toLowerCase());
const newSections = (newContent.match(/^##\s+.+$/gm) || []).map(s => s.replace(/^##\s+/, "").trim().toLowerCase());
const missingSections = prevSections.filter(s => !newSections.some(ns => ns.includes(s) || s.includes(ns)));
if (missingSections.length > 0) {
result.warnings.push(`Sections may have been lost: ${missingSections.join(", ")}`);
}
}
private async assessQuality(dirPath: string, result: ValidationResult): Promise {
const chain = buildSkillConfigChain(this.ctx);
if (chain.length === 0) return;
const skillMdPath = path.join(dirPath, "SKILL.md");
const content = fs.readFileSync(skillMdPath, "utf-8");
const prompt = QUALITY_PROMPT.replace("{SKILL_CONTENT}", content.slice(0, 6000));
try {
const raw = await callLLMWithFallback(chain, prompt, this.ctx.log, "SkillValidator.quality");
const jsonMatch = raw.match(/\{[\s\S]*\}/);
if (!jsonMatch) return;
const assessment = JSON.parse(jsonMatch[0]) as {
score: number;
strengths: string[];
weaknesses: string[];
suggestions: string[];
};
result.qualityScore = Math.max(0, Math.min(10, assessment.score));
if (assessment.suggestions) {
result.suggestions.push(...assessment.suggestions);
}
if (assessment.weaknesses) {
result.warnings.push(...assessment.weaknesses);
}
if (result.qualityScore < 6) {
result.warnings.push(`Quality score ${result.qualityScore}/10 is below threshold, marked as draft`);
}
} catch (err) {
this.ctx.log.warn(`SkillValidator: quality assessment failed: ${err}`);
}
}
}
const QUALITY_PROMPT = `You are a skill quality reviewer. Evaluate the following SKILL.md and give a score from 0 to 10.
Criteria:
1. Clarity: Are the steps clear and actionable? (0-2 pts)
2. Completeness: Does it cover scenarios, pitfalls, and key code? (0-2 pts)
3. Reusability: Can this skill be applied to similar future tasks? (0-2 pts)
4. Accuracy: Are commands, code, and configurations correct? (0-2 pts)
5. Structure: Is the format well-organized with proper sections? (0-2 pts)
SKILL.md:
{SKILL_CONTENT}
LANGUAGE RULE: "strengths", "weaknesses", and "suggestions" MUST use the SAME language as the SKILL.md content. Chinese skill → Chinese feedback. English skill → English feedback.
Reply in JSON only:
{
"score": 0-10,
"strengths": ["what's good (same language as skill)"],
"weaknesses": ["what's lacking (same language as skill)"],
"suggestions": ["how to improve (same language as skill)"]
}`;
================================================
FILE: apps/memos-local-openclaw/src/storage/ensure-binding.ts
================================================
import { existsSync, mkdirSync, copyFileSync } from "fs";
import { execSync } from "child_process";
import path from "path";
import { createRequire } from "module";
const require = createRequire(import.meta.url);
/**
* Ensure the better-sqlite3 native binary is available.
*
* OpenClaw installs plugins with `--ignore-scripts`, which skips
* the native compilation step. This function checks for the binary
* and restores it from bundled prebuilds if missing.
*/
export function ensureSqliteBinding(log?: { info: (msg: string) => void; warn: (msg: string) => void }): void {
const bsqlPkg = require.resolve("better-sqlite3/package.json");
const bsqlDir = path.dirname(bsqlPkg);
const bindingPath = path.join(bsqlDir, "build", "Release", "better_sqlite3.node");
if (existsSync(bindingPath)) return;
const platform = `${process.platform}-${process.arch}`;
const pluginRoot = path.resolve(path.dirname(new URL(import.meta.url).pathname), "..", "..");
const prebuildSrc = path.join(pluginRoot, "prebuilds", platform, "better_sqlite3.node");
if (existsSync(prebuildSrc)) {
log?.info(`[ensure-binding] Copying prebuild for ${platform}...`);
mkdirSync(path.dirname(bindingPath), { recursive: true });
copyFileSync(prebuildSrc, bindingPath);
log?.info(`[ensure-binding] Prebuild installed successfully.`);
return;
}
log?.warn(`[ensure-binding] No prebuild for ${platform}, attempting npm rebuild...`);
try {
const installDir = path.resolve(bsqlDir, "..", "..");
execSync("npm rebuild better-sqlite3", {
cwd: installDir,
stdio: "pipe",
timeout: 180_000,
});
if (existsSync(bindingPath)) {
log?.info(`[ensure-binding] Rebuilt better-sqlite3 successfully.`);
return;
}
} catch { /* fall through */ }
throw new Error(
`better-sqlite3 native binary not found for ${platform}.\n` +
`Prebuild not bundled and npm rebuild failed.\n` +
`Fix: cd ${path.resolve(bsqlDir, "..", "..")} && npm rebuild better-sqlite3`,
);
}
================================================
FILE: apps/memos-local-openclaw/src/storage/sqlite.ts
================================================
import Database from "better-sqlite3";
import { createHash } from "crypto";
import * as fs from "fs";
import * as path from "path";
import type { Chunk, ChunkRef, DedupStatus, Task, TaskStatus, Skill, SkillStatus, SkillVisibility, SkillVersion, TaskSkillLink, TaskSkillRelation, Logger } from "../types";
export class SqliteStore {
private db: Database.Database;
constructor(dbPath: string, private log: Logger) {
fs.mkdirSync(path.dirname(dbPath), { recursive: true });
this.db = new Database(dbPath);
this.db.pragma("journal_mode = WAL");
this.db.pragma("foreign_keys = ON");
this.migrate();
}
// ─── Schema ───
private migrate(): void {
this.db.exec(`
CREATE TABLE IF NOT EXISTS chunks (
id TEXT PRIMARY KEY,
session_key TEXT NOT NULL,
turn_id TEXT NOT NULL,
seq INTEGER NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL,
kind TEXT NOT NULL DEFAULT 'paragraph',
summary TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_chunks_session
ON chunks(session_key);
CREATE INDEX IF NOT EXISTS idx_chunks_turn
ON chunks(session_key, turn_id, seq);
CREATE INDEX IF NOT EXISTS idx_chunks_created
ON chunks(created_at);
CREATE INDEX IF NOT EXISTS idx_chunks_session_created
ON chunks(session_key, created_at, seq);
CREATE VIRTUAL TABLE IF NOT EXISTS chunks_fts USING fts5(
summary,
content,
content='chunks',
content_rowid='rowid',
tokenize='trigram'
);
CREATE TRIGGER IF NOT EXISTS chunks_ai AFTER INSERT ON chunks BEGIN
INSERT INTO chunks_fts(rowid, summary, content)
VALUES (new.rowid, new.summary, new.content);
END;
CREATE TRIGGER IF NOT EXISTS chunks_ad AFTER DELETE ON chunks BEGIN
INSERT INTO chunks_fts(chunks_fts, rowid, summary, content)
VALUES ('delete', old.rowid, old.summary, old.content);
END;
CREATE TRIGGER IF NOT EXISTS chunks_au AFTER UPDATE ON chunks BEGIN
INSERT INTO chunks_fts(chunks_fts, rowid, summary, content)
VALUES ('delete', old.rowid, old.summary, old.content);
INSERT INTO chunks_fts(rowid, summary, content)
VALUES (new.rowid, new.summary, new.content);
END;
CREATE TABLE IF NOT EXISTS embeddings (
chunk_id TEXT PRIMARY KEY REFERENCES chunks(id) ON DELETE CASCADE,
vector BLOB NOT NULL,
dimensions INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS viewer_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
event_type TEXT NOT NULL,
created_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_viewer_events_created ON viewer_events(created_at);
CREATE INDEX IF NOT EXISTS idx_viewer_events_type ON viewer_events(event_type);
CREATE TABLE IF NOT EXISTS tasks (
id TEXT PRIMARY KEY,
session_key TEXT NOT NULL,
title TEXT NOT NULL DEFAULT '',
summary TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT 'active',
started_at INTEGER NOT NULL,
ended_at INTEGER,
updated_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_tasks_session ON tasks(session_key);
CREATE INDEX IF NOT EXISTS idx_tasks_status ON tasks(status);
`);
this.migrateTaskId();
this.migrateContentHash();
this.migrateSkillTables();
this.migrateSkillId();
this.migrateSkillQualityScore();
this.migrateTaskSkillMeta();
this.migrateToolCalls();
this.migrateMergeFields();
this.migrateApiLogs();
this.migrateDedupStatus();
this.migrateChunksIndexesForRecall();
this.migrateOwnerFields();
this.migrateSkillVisibility();
this.migrateSkillEmbeddingsAndFts();
this.migrateFtsToTrigram();
this.log.debug("Database schema initialized");
}
private migrateChunksIndexesForRecall(): void {
this.db.exec("CREATE INDEX IF NOT EXISTS idx_chunks_dedup_created ON chunks(dedup_status, created_at DESC)");
}
private migrateOwnerFields(): void {
const chunkCols = this.db.prepare("PRAGMA table_info(chunks)").all() as Array<{ name: string }>;
if (!chunkCols.some((c) => c.name === "owner")) {
this.db.exec("ALTER TABLE chunks ADD COLUMN owner TEXT NOT NULL DEFAULT 'agent:main'");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_chunks_owner ON chunks(owner)");
this.log.info("Migrated: added owner column to chunks");
}
const taskCols = this.db.prepare("PRAGMA table_info(tasks)").all() as Array<{ name: string }>;
if (!taskCols.some((c) => c.name === "owner")) {
this.db.exec("ALTER TABLE tasks ADD COLUMN owner TEXT NOT NULL DEFAULT 'agent:main'");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_tasks_owner ON tasks(owner)");
this.log.info("Migrated: added owner column to tasks");
}
}
private migrateSkillVisibility(): void {
const cols = this.db.prepare("PRAGMA table_info(skills)").all() as Array<{ name: string }>;
if (!cols.some((c) => c.name === "owner")) {
this.db.exec("ALTER TABLE skills ADD COLUMN owner TEXT NOT NULL DEFAULT 'agent:main'");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_skills_owner ON skills(owner)");
this.log.info("Migrated: added owner column to skills");
}
if (!cols.some((c) => c.name === "visibility")) {
this.db.exec("ALTER TABLE skills ADD COLUMN visibility TEXT NOT NULL DEFAULT 'private'");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_skills_visibility ON skills(visibility)");
this.log.info("Migrated: added visibility column to skills");
}
}
private migrateSkillEmbeddingsAndFts(): void {
this.db.exec(`
CREATE TABLE IF NOT EXISTS skill_embeddings (
skill_id TEXT PRIMARY KEY REFERENCES skills(id) ON DELETE CASCADE,
vector BLOB NOT NULL,
dimensions INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE VIRTUAL TABLE IF NOT EXISTS skills_fts USING fts5(
name,
description,
content='skills',
content_rowid='rowid',
tokenize='trigram'
);
`);
try {
this.db.exec(`
CREATE TRIGGER IF NOT EXISTS skills_ai AFTER INSERT ON skills BEGIN
INSERT INTO skills_fts(rowid, name, description)
VALUES (new.rowid, new.name, new.description);
END;
CREATE TRIGGER IF NOT EXISTS skills_ad AFTER DELETE ON skills BEGIN
INSERT INTO skills_fts(skills_fts, rowid, name, description)
VALUES ('delete', old.rowid, old.name, old.description);
END;
CREATE TRIGGER IF NOT EXISTS skills_au AFTER UPDATE ON skills BEGIN
INSERT INTO skills_fts(skills_fts, rowid, name, description)
VALUES ('delete', old.rowid, old.name, old.description);
INSERT INTO skills_fts(rowid, name, description)
VALUES (new.rowid, new.name, new.description);
END;
`);
} catch {
// triggers may already exist
}
// Backfill FTS for existing skills
try {
const count = (this.db.prepare("SELECT COUNT(*) as c FROM skills_fts").get() as { c: number }).c;
const skillCount = (this.db.prepare("SELECT COUNT(*) as c FROM skills").get() as { c: number }).c;
if (count === 0 && skillCount > 0) {
this.db.exec("INSERT INTO skills_fts(rowid, name, description) SELECT rowid, name, description FROM skills");
this.log.info(`Migrated: backfilled skills_fts for ${skillCount} skills`);
}
} catch { /* best-effort */ }
}
private migrateFtsToTrigram(): void {
// Check if chunks_fts still uses the old tokenizer (porter unicode61)
try {
const row = this.db.prepare(
"SELECT sql FROM sqlite_master WHERE name='chunks_fts'"
).get() as { sql: string } | undefined;
if (row && row.sql && !row.sql.includes("trigram")) {
this.log.info("Migrating chunks_fts from porter/unicode61 to trigram tokenizer...");
this.db.exec("DROP TRIGGER IF EXISTS chunks_ai");
this.db.exec("DROP TRIGGER IF EXISTS chunks_ad");
this.db.exec("DROP TRIGGER IF EXISTS chunks_au");
this.db.exec("DROP TABLE IF EXISTS chunks_fts");
this.db.exec(`
CREATE VIRTUAL TABLE chunks_fts USING fts5(
summary, content, content='chunks', content_rowid='rowid',
tokenize='trigram'
)
`);
this.db.exec(`
CREATE TRIGGER chunks_ai AFTER INSERT ON chunks BEGIN
INSERT INTO chunks_fts(rowid, summary, content) VALUES (new.rowid, new.summary, new.content);
END;
CREATE TRIGGER chunks_ad AFTER DELETE ON chunks BEGIN
INSERT INTO chunks_fts(chunks_fts, rowid, summary, content) VALUES ('delete', old.rowid, old.summary, old.content);
END;
CREATE TRIGGER chunks_au AFTER UPDATE ON chunks BEGIN
INSERT INTO chunks_fts(chunks_fts, rowid, summary, content) VALUES ('delete', old.rowid, old.summary, old.content);
INSERT INTO chunks_fts(rowid, summary, content) VALUES (new.rowid, new.summary, new.content);
END
`);
this.db.exec("INSERT INTO chunks_fts(rowid, summary, content) SELECT rowid, summary, content FROM chunks");
const count = (this.db.prepare("SELECT COUNT(*) as c FROM chunks_fts").get() as { c: number }).c;
this.log.info(`Migrated chunks_fts to trigram: ${count} rows indexed`);
}
} catch (err) {
this.log.warn(`Failed to migrate chunks_fts to trigram: ${err}`);
}
// Same for skills_fts
try {
const row = this.db.prepare(
"SELECT sql FROM sqlite_master WHERE name='skills_fts'"
).get() as { sql: string } | undefined;
if (row && row.sql && !row.sql.includes("trigram")) {
this.log.info("Migrating skills_fts to trigram tokenizer...");
this.db.exec("DROP TRIGGER IF EXISTS skills_ai");
this.db.exec("DROP TRIGGER IF EXISTS skills_ad");
this.db.exec("DROP TRIGGER IF EXISTS skills_au");
this.db.exec("DROP TABLE IF EXISTS skills_fts");
this.db.exec(`
CREATE VIRTUAL TABLE skills_fts USING fts5(
name, description, content='skills', content_rowid='rowid',
tokenize='trigram'
)
`);
this.db.exec(`
CREATE TRIGGER skills_ai AFTER INSERT ON skills BEGIN
INSERT INTO skills_fts(rowid, name, description) VALUES (new.rowid, new.name, new.description);
END;
CREATE TRIGGER skills_ad AFTER DELETE ON skills BEGIN
INSERT INTO skills_fts(skills_fts, rowid, name, description) VALUES ('delete', old.rowid, old.name, old.description);
END;
CREATE TRIGGER skills_au AFTER UPDATE ON skills BEGIN
INSERT INTO skills_fts(skills_fts, rowid, name, description) VALUES ('delete', old.rowid, old.name, old.description);
INSERT INTO skills_fts(rowid, name, description) VALUES (new.rowid, new.name, new.description);
END
`);
this.db.exec("INSERT INTO skills_fts(rowid, name, description) SELECT rowid, name, description FROM skills");
this.log.info("Migrated skills_fts to trigram");
}
} catch (err) {
this.log.warn(`Failed to migrate skills_fts to trigram: ${err}`);
}
}
private migrateTaskId(): void {
const cols = this.db.prepare("PRAGMA table_info(chunks)").all() as Array<{ name: string }>;
if (!cols.some((c) => c.name === "task_id")) {
this.db.exec("ALTER TABLE chunks ADD COLUMN task_id TEXT REFERENCES tasks(id)");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_chunks_task ON chunks(task_id)");
this.log.info("Migrated: added task_id column to chunks");
}
}
private migrateContentHash(): void {
const cols = this.db.prepare("PRAGMA table_info(chunks)").all() as Array<{ name: string }>;
if (!cols.some((c) => c.name === "content_hash")) {
this.db.exec("ALTER TABLE chunks ADD COLUMN content_hash TEXT");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_chunks_dedup ON chunks(session_key, role, content_hash)");
// Backfill existing rows
const rows = this.db.prepare("SELECT id, content FROM chunks WHERE content_hash IS NULL").all() as Array<{ id: string; content: string }>;
const updateStmt = this.db.prepare("UPDATE chunks SET content_hash = ? WHERE id = ?");
for (const r of rows) {
updateStmt.run(contentHash(r.content), r.id);
}
if (rows.length > 0) {
this.log.info(`Migrated: backfilled content_hash for ${rows.length} chunks`);
}
}
}
private migrateSkillTables(): void {
this.db.exec(`
CREATE TABLE IF NOT EXISTS skills (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
description TEXT NOT NULL DEFAULT '',
version INTEGER NOT NULL DEFAULT 1,
status TEXT NOT NULL DEFAULT 'active',
tags TEXT NOT NULL DEFAULT '[]',
source_type TEXT NOT NULL DEFAULT 'task',
dir_path TEXT NOT NULL DEFAULT '',
installed INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_skills_status ON skills(status);
CREATE INDEX IF NOT EXISTS idx_skills_name ON skills(name);
CREATE TABLE IF NOT EXISTS skill_versions (
id TEXT PRIMARY KEY,
skill_id TEXT NOT NULL REFERENCES skills(id),
version INTEGER NOT NULL,
content TEXT NOT NULL,
changelog TEXT NOT NULL DEFAULT '',
upgrade_type TEXT NOT NULL DEFAULT 'create',
source_task_id TEXT,
metrics TEXT NOT NULL DEFAULT '{}',
created_at INTEGER NOT NULL,
UNIQUE(skill_id, version)
);
CREATE INDEX IF NOT EXISTS idx_skill_versions_skill ON skill_versions(skill_id);
CREATE TABLE IF NOT EXISTS task_skills (
task_id TEXT NOT NULL REFERENCES tasks(id),
skill_id TEXT NOT NULL REFERENCES skills(id),
relation TEXT NOT NULL DEFAULT 'generated_from',
version_at INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL,
PRIMARY KEY (task_id, skill_id)
);
`);
}
private migrateSkillId(): void {
const cols = this.db.prepare("PRAGMA table_info(chunks)").all() as Array<{ name: string }>;
if (!cols.some((c) => c.name === "skill_id")) {
this.db.exec("ALTER TABLE chunks ADD COLUMN skill_id TEXT");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_chunks_skill ON chunks(skill_id)");
this.log.info("Migrated: added skill_id column to chunks");
}
}
private migrateSkillQualityScore(): void {
const skillCols = this.db.prepare("PRAGMA table_info(skills)").all() as Array<{ name: string }>;
if (!skillCols.some((c) => c.name === "quality_score")) {
this.db.exec("ALTER TABLE skills ADD COLUMN quality_score REAL");
this.log.info("Migrated: added quality_score column to skills");
}
const versionCols = this.db.prepare("PRAGMA table_info(skill_versions)").all() as Array<{ name: string }>;
if (!versionCols.some((c) => c.name === "quality_score")) {
this.db.exec("ALTER TABLE skill_versions ADD COLUMN quality_score REAL");
this.log.info("Migrated: added quality_score column to skill_versions");
}
if (!versionCols.some((c) => c.name === "change_summary")) {
this.db.exec("ALTER TABLE skill_versions ADD COLUMN change_summary TEXT NOT NULL DEFAULT ''");
this.log.info("Migrated: added change_summary column to skill_versions");
}
}
private migrateTaskSkillMeta(): void {
const cols = this.db.prepare("PRAGMA table_info(tasks)").all() as Array<{ name: string }>;
if (!cols.some((c) => c.name === "skill_status")) {
this.db.exec("ALTER TABLE tasks ADD COLUMN skill_status TEXT DEFAULT NULL");
this.db.exec("ALTER TABLE tasks ADD COLUMN skill_reason TEXT DEFAULT NULL");
this.log.info("Migrated: added skill_status/skill_reason columns to tasks");
}
}
setTaskSkillMeta(taskId: string, meta: { skillStatus: string; skillReason: string }): void {
this.db.prepare("UPDATE tasks SET skill_status = ?, skill_reason = ?, updated_at = ? WHERE id = ?")
.run(meta.skillStatus, meta.skillReason, Date.now(), taskId);
}
getTasksBySkillStatus(statuses: string[]): Task[] {
const placeholders = statuses.map(() => "?").join(",");
const rows = this.db.prepare(
`SELECT * FROM tasks WHERE skill_status IN (${placeholders}) AND status = 'completed' ORDER BY updated_at ASC`,
).all(...statuses) as TaskRow[];
return rows.map(rowToTask);
}
private migrateMergeFields(): void {
const cols = this.db.prepare("PRAGMA table_info(chunks)").all() as Array<{ name: string }>;
if (!cols.some((c) => c.name === "merge_count")) {
this.db.exec("ALTER TABLE chunks ADD COLUMN merge_count INTEGER NOT NULL DEFAULT 0");
this.db.exec("ALTER TABLE chunks ADD COLUMN last_hit_at INTEGER");
this.db.exec("ALTER TABLE chunks ADD COLUMN merge_history TEXT NOT NULL DEFAULT '[]'");
this.log.info("Migrated: added merge_count/last_hit_at/merge_history columns to chunks");
}
}
private migrateApiLogs(): void {
this.db.exec(`
CREATE TABLE IF NOT EXISTS api_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
tool_name TEXT NOT NULL,
input_data TEXT NOT NULL DEFAULT '{}',
output_data TEXT NOT NULL DEFAULT '',
duration_ms INTEGER NOT NULL DEFAULT 0,
success INTEGER NOT NULL DEFAULT 1,
called_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_api_logs_at ON api_logs(called_at);
CREATE INDEX IF NOT EXISTS idx_api_logs_name ON api_logs(tool_name);
`);
}
private migrateDedupStatus(): void {
const cols = this.db.prepare("PRAGMA table_info(chunks)").all() as Array<{ name: string }>;
if (!cols.some((c) => c.name === "dedup_status")) {
this.db.exec("ALTER TABLE chunks ADD COLUMN dedup_status TEXT NOT NULL DEFAULT 'active'");
this.db.exec("ALTER TABLE chunks ADD COLUMN dedup_target TEXT DEFAULT NULL");
this.db.exec("ALTER TABLE chunks ADD COLUMN dedup_reason TEXT DEFAULT NULL");
this.db.exec("CREATE INDEX IF NOT EXISTS idx_chunks_dedup_status ON chunks(dedup_status)");
this.log.info("Migrated: added dedup_status/dedup_target/dedup_reason columns to chunks");
}
}
recordApiLog(toolName: string, input: unknown, output: string, durationMs: number, success: boolean): void {
const inputStr = typeof input === "string" ? input : JSON.stringify(input ?? {});
this.db.prepare(
"INSERT INTO api_logs (tool_name, input_data, output_data, duration_ms, success, called_at) VALUES (?, ?, ?, ?, ?, ?)",
).run(toolName, inputStr, output, Math.round(durationMs), success ? 1 : 0, Date.now());
}
getApiLogs(limit: number = 50, offset: number = 0, toolFilter?: string): {
logs: Array<{ id: number; toolName: string; input: string; output: string; durationMs: number; success: boolean; calledAt: number }>;
total: number;
} {
const whereClause = toolFilter ? " WHERE tool_name = ?" : "";
const filterParams: unknown[] = toolFilter ? [toolFilter] : [];
const countRow = this.db.prepare("SELECT COUNT(*) as c FROM api_logs" + whereClause).get(...filterParams) as { c: number };
const rows = this.db.prepare(
"SELECT id, tool_name, input_data, output_data, duration_ms, success, called_at FROM api_logs" +
whereClause + " ORDER BY called_at DESC LIMIT ? OFFSET ?",
).all(...filterParams, limit, offset) as Array<{
id: number; tool_name: string; input_data: string; output_data: string;
duration_ms: number; success: number; called_at: number;
}>;
return {
logs: rows.map((r) => ({
id: r.id,
toolName: r.tool_name,
input: r.input_data,
output: r.output_data,
durationMs: r.duration_ms,
success: r.success === 1,
calledAt: r.called_at,
})),
total: countRow.c,
};
}
getApiLogToolNames(): string[] {
const rows = this.db.prepare("SELECT DISTINCT tool_name FROM api_logs ORDER BY tool_name").all() as Array<{ tool_name: string }>;
return rows.map((r) => r.tool_name);
}
recordMergeHit(chunkId: string, action: "DUPLICATE" | "UPDATE", reason: string, oldSummary?: string, newSummary?: string): void {
const chunk = this.getChunk(chunkId);
if (!chunk) return;
const history = JSON.parse(chunk.mergeHistory || "[]") as any[];
const entry: Record = { at: Date.now(), action, reason };
if (action === "UPDATE" && oldSummary && newSummary) {
entry.from = oldSummary;
entry.to = newSummary;
}
history.push(entry);
this.db.prepare(`
UPDATE chunks SET merge_count = merge_count + 1, last_hit_at = ?, merge_history = ?, updated_at = ?
WHERE id = ?
`).run(Date.now(), JSON.stringify(history), Date.now(), chunkId);
}
updateChunkSummaryAndContent(chunkId: string, newSummary: string, appendContent: string): void {
this.db.prepare(`
UPDATE chunks SET summary = ?, content = content || ? || ?, updated_at = ? WHERE id = ?
`).run(newSummary, "\n\n---\n\n", appendContent, Date.now(), chunkId);
}
private migrateToolCalls(): void {
this.db.exec(`
CREATE TABLE IF NOT EXISTS tool_calls (
id INTEGER PRIMARY KEY AUTOINCREMENT,
tool_name TEXT NOT NULL,
duration_ms INTEGER NOT NULL,
success INTEGER NOT NULL DEFAULT 1,
called_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_tool_calls_at ON tool_calls(called_at);
CREATE INDEX IF NOT EXISTS idx_tool_calls_name ON tool_calls(tool_name);
`);
}
recordToolCall(toolName: string, durationMs: number, success: boolean): void {
this.db.prepare(
"INSERT INTO tool_calls (tool_name, duration_ms, success, called_at) VALUES (?, ?, ?, ?)",
).run(toolName, Math.round(durationMs), success ? 1 : 0, Date.now());
}
getToolMetrics(minutes: number): {
tools: string[];
series: Array<{ minute: string; [tool: string]: number | string }>;
aggregated: Array<{ tool: string; totalCalls: number; avgMs: number; p95Ms: number; errorCount: number }>;
} {
const since = Date.now() - minutes * 60 * 1000;
const rows = this.db.prepare(
`SELECT tool_name,
duration_ms,
success,
strftime('%Y-%m-%d %H:%M', called_at/1000, 'unixepoch', 'localtime') as minute_key
FROM tool_calls
WHERE called_at >= ?
ORDER BY called_at`,
).all(since) as Array<{ tool_name: string; duration_ms: number; success: number; minute_key: string }>;
const toolSet = new Set();
const minuteMap = new Map>();
const aggMap = new Map();
for (const r of rows) {
toolSet.add(r.tool_name);
if (!aggMap.has(r.tool_name)) aggMap.set(r.tool_name, { durations: [], errors: 0 });
const agg = aggMap.get(r.tool_name)!;
agg.durations.push(r.duration_ms);
if (!r.success) agg.errors++;
if (!minuteMap.has(r.minute_key)) minuteMap.set(r.minute_key, new Map());
const toolMap = minuteMap.get(r.minute_key)!;
if (!toolMap.has(r.tool_name)) toolMap.set(r.tool_name, { total: 0, count: 0 });
const entry = toolMap.get(r.tool_name)!;
entry.total += r.duration_ms;
entry.count++;
}
const tools = Array.from(toolSet).sort();
const allMinutes: string[] = [];
if (minutes > 0) {
const startMinute = new Date(since);
startMinute.setSeconds(0, 0);
const now = new Date();
for (let t = startMinute.getTime(); t <= now.getTime(); t += 60000) {
const d = new Date(t);
const pad = (n: number) => String(n).padStart(2, "0");
allMinutes.push(`${d.getFullYear()}-${pad(d.getMonth() + 1)}-${pad(d.getDate())} ${pad(d.getHours())}:${pad(d.getMinutes())}`);
}
}
const series = allMinutes.map((m) => {
const entry: { minute: string; [tool: string]: number | string } = { minute: m };
const toolMap = minuteMap.get(m);
for (const t of tools) {
const data = toolMap?.get(t);
entry[t] = data ? Math.round(data.total / data.count) : 0;
}
return entry;
});
const p95 = (arr: number[]) => {
if (arr.length === 0) return 0;
const sorted = [...arr].sort((a, b) => a - b);
return sorted[Math.floor(sorted.length * 0.95)] ?? sorted[sorted.length - 1];
};
const aggregated = tools.map((t) => {
const agg = aggMap.get(t)!;
return {
tool: t,
totalCalls: agg.durations.length,
avgMs: Math.round(agg.durations.reduce((s, v) => s + v, 0) / agg.durations.length),
p95Ms: p95(agg.durations),
errorCount: agg.errors,
};
});
return { tools, series, aggregated };
}
/** Record a viewer API call for analytics (list, search, etc.). */
recordViewerEvent(eventType: string): void {
this.db.prepare("INSERT INTO viewer_events (event_type, created_at) VALUES (?, ?)").run(eventType, Date.now());
}
/**
* Return metrics for the last N days: writes per day (from chunks), viewer calls per day.
*/
getMetrics(days: number): {
writesPerDay: Array<{ date: string; count: number }>;
viewerCallsPerDay: Array<{ date: string; list: number; search: number; total: number }>;
totals: { memories: number; sessions: number; embeddings: number; todayWrites: number; todayViewerCalls: number };
} {
const since = Date.now() - days * 86400 * 1000;
const now = new Date();
const todayStart = new Date(now.getFullYear(), now.getMonth(), now.getDate()).getTime();
const writesRows = this.db
.prepare(
`SELECT date(created_at/1000, 'unixepoch', 'localtime') as d, COUNT(*) as c
FROM chunks WHERE created_at >= ? GROUP BY d ORDER BY d`,
)
.all(since) as Array<{ d: string; c: number }>;
const writesPerDay = writesRows.map((r) => ({ date: r.d, count: r.c }));
const eventsRows = this.db
.prepare(
`SELECT date(created_at/1000, 'unixepoch', 'localtime') as d, event_type, COUNT(*) as c
FROM viewer_events WHERE created_at >= ? GROUP BY d, event_type ORDER BY d`,
)
.all(since) as Array<{ d: string; event_type: string; c: number }>;
const byDate = new Map();
for (const r of eventsRows) {
let row = byDate.get(r.d);
if (!row) {
row = { list: 0, search: 0 };
byDate.set(r.d, row);
}
if (r.event_type === "list") row.list += r.c;
else if (r.event_type === "search") row.search += r.c;
}
const viewerCallsPerDay = Array.from(byDate.entries())
.sort((a, b) => a[0].localeCompare(b[0]))
.map(([date, v]) => ({ date, list: v.list, search: v.search, total: v.list + v.search }));
const totalChunks = (this.db.prepare("SELECT COUNT(*) as c FROM chunks").get() as { c: number }).c;
const totalSessions = (this.db.prepare("SELECT COUNT(DISTINCT session_key) as c FROM chunks").get() as { c: number }).c;
const totalEmbeddings = (this.db.prepare("SELECT COUNT(*) as c FROM embeddings").get() as { c: number }).c;
const todayWrites = (this.db.prepare("SELECT COUNT(*) as c FROM chunks WHERE created_at >= ?").get(todayStart) as { c: number }).c;
const todayViewerCalls = (this.db.prepare("SELECT COUNT(*) as c FROM viewer_events WHERE created_at >= ?").get(todayStart) as { c: number }).c;
return {
writesPerDay,
viewerCallsPerDay,
totals: {
memories: totalChunks,
sessions: totalSessions,
embeddings: totalEmbeddings,
todayWrites,
todayViewerCalls,
},
};
}
// ─── Write ───
insertChunk(chunk: Chunk): void {
const stmt = this.db.prepare(`
INSERT OR REPLACE INTO chunks (id, session_key, turn_id, seq, role, content, kind, summary, task_id, content_hash, owner, dedup_status, dedup_target, dedup_reason, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
stmt.run(
chunk.id,
chunk.sessionKey,
chunk.turnId,
chunk.seq,
chunk.role,
chunk.content,
chunk.kind,
chunk.summary,
chunk.taskId,
contentHash(chunk.content),
chunk.owner ?? "agent:main",
chunk.dedupStatus ?? "active",
chunk.dedupTarget ?? null,
chunk.dedupReason ?? null,
chunk.createdAt,
chunk.updatedAt,
);
}
markDedupStatus(chunkId: string, status: "duplicate" | "merged", targetChunkId: string | null, reason: string): void {
this.db.prepare(
"UPDATE chunks SET dedup_status = ?, dedup_target = ?, dedup_reason = ?, updated_at = ? WHERE id = ?",
).run(status, targetChunkId, reason, Date.now(), chunkId);
}
updateSummary(chunkId: string, summary: string): void {
this.db.prepare("UPDATE chunks SET summary = ?, updated_at = ? WHERE id = ?").run(
summary,
Date.now(),
chunkId,
);
}
upsertEmbedding(chunkId: string, vector: number[]): void {
const buf = Buffer.from(new Float32Array(vector).buffer);
this.db.prepare(`
INSERT OR REPLACE INTO embeddings (chunk_id, vector, dimensions, updated_at)
VALUES (?, ?, ?, ?)
`).run(chunkId, buf, vector.length, Date.now());
}
deleteEmbedding(chunkId: string): void {
this.db.prepare("DELETE FROM embeddings WHERE chunk_id = ?").run(chunkId);
}
// ─── Read ───
getChunk(chunkId: string): Chunk | null {
const row = this.db.prepare("SELECT * FROM chunks WHERE id = ?").get(chunkId) as ChunkRow | undefined;
return row ? rowToChunk(row) : null;
}
getChunkForOwners(chunkId: string, ownerFilter?: string[]): Chunk | null {
if (!ownerFilter || ownerFilter.length === 0) return this.getChunk(chunkId);
const placeholders = ownerFilter.map(() => "?").join(",");
const row = this.db.prepare(
`SELECT * FROM chunks WHERE id = ? AND owner IN (${placeholders}) LIMIT 1`,
).get(chunkId, ...ownerFilter) as ChunkRow | undefined;
return row ? rowToChunk(row) : null;
}
getChunksByRef(ref: ChunkRef, ownerFilter?: string[]): Chunk | null {
return this.getChunkForOwners(ref.chunkId, ownerFilter);
}
getNeighborChunks(sessionKey: string, turnId: string, seq: number, window: number, ownerFilter?: string[]): Chunk[] {
let sql = `
SELECT * FROM chunks
WHERE session_key = ?`;
const params: any[] = [sessionKey];
if (ownerFilter && ownerFilter.length > 0) {
const placeholders = ownerFilter.map(() => "?").join(",");
sql += ` AND owner IN (${placeholders})`;
params.push(...ownerFilter);
}
sql += `
ORDER BY created_at, seq
`;
const allRows = this.db.prepare(sql).all(...params) as ChunkRow[];
const targetIdx = allRows.findIndex(
(r) => r.turn_id === turnId && r.seq === seq,
);
if (targetIdx === -1) return [];
const radius = window * 3;
const start = Math.max(0, targetIdx - radius);
const end = Math.min(allRows.length, targetIdx + radius + 1);
return allRows.slice(start, end).map(rowToChunk);
}
// ─── FTS Search ───
ftsSearch(query: string, limit: number, ownerFilter?: string[]): Array<{ chunkId: string; score: number }> {
const sanitized = sanitizeFtsQuery(query);
if (!sanitized) return [];
try {
let sql = `
SELECT c.id as chunk_id, rank
FROM chunks_fts f
JOIN chunks c ON c.rowid = f.rowid
WHERE chunks_fts MATCH ? AND c.dedup_status = 'active'`;
const params: any[] = [sanitized];
if (ownerFilter && ownerFilter.length > 0) {
const placeholders = ownerFilter.map(() => "?").join(",");
sql += ` AND c.owner IN (${placeholders})`;
params.push(...ownerFilter);
}
sql += ` ORDER BY rank LIMIT ?`;
params.push(limit);
const rows = this.db.prepare(sql).all(...params) as Array<{ chunk_id: string; rank: number }>;
if (rows.length === 0) return [];
const maxAbsRank = Math.max(...rows.map((r) => Math.abs(r.rank)));
return rows.map((r) => ({
chunkId: r.chunk_id,
score: maxAbsRank > 0 ? Math.abs(r.rank) / maxAbsRank : 0,
}));
} catch {
this.log.warn(`FTS query failed for: "${sanitized}", returning empty`);
return [];
}
}
// ─── Pattern Search (LIKE-based, for CJK text where FTS tokenization is weak) ───
patternSearch(patterns: string[], opts: { role?: string; limit?: number } = {}): Array<{ chunkId: string; content: string; role: string; createdAt: number }> {
if (patterns.length === 0) return [];
const limit = opts.limit ?? 10;
const conditions = patterns.map(() => "c.content LIKE ?");
const whereClause = conditions.join(" OR ");
const roleClause = opts.role ? " AND c.role = ?" : "";
const params: (string | number)[] = patterns.map(p => `%${p}%`);
if (opts.role) params.push(opts.role);
params.push(limit);
try {
const rows = this.db.prepare(`
SELECT c.id as chunk_id, c.content, c.role, c.created_at
FROM chunks c
WHERE (${whereClause})${roleClause} AND c.dedup_status = 'active'
ORDER BY c.created_at DESC
LIMIT ?
`).all(...params) as Array<{ chunk_id: string; content: string; role: string; created_at: number }>;
return rows.map(r => ({
chunkId: r.chunk_id,
content: r.content,
role: r.role,
createdAt: r.created_at,
}));
} catch {
return [];
}
}
// ─── Vector Search ───
getAllEmbeddings(ownerFilter?: string[]): Array<{ chunkId: string; vector: number[] }> {
let sql = `SELECT e.chunk_id, e.vector, e.dimensions FROM embeddings e
JOIN chunks c ON c.id = e.chunk_id
WHERE c.dedup_status = 'active'`;
const params: any[] = [];
if (ownerFilter && ownerFilter.length > 0) {
const placeholders = ownerFilter.map(() => "?").join(",");
sql += ` AND c.owner IN (${placeholders})`;
params.push(...ownerFilter);
}
const rows = this.db.prepare(sql).all(...params) as Array<{ chunk_id: string; vector: Buffer; dimensions: number }>;
return rows.map((r) => ({
chunkId: r.chunk_id,
vector: Array.from(new Float32Array(r.vector.buffer, r.vector.byteOffset, r.dimensions)),
}));
}
getRecentEmbeddings(limit: number, ownerFilter?: string[]): Array<{ chunkId: string; vector: number[] }> {
if (limit <= 0) return this.getAllEmbeddings(ownerFilter);
let sql = `SELECT e.chunk_id, e.vector, e.dimensions
FROM chunks c
JOIN embeddings e ON e.chunk_id = c.id
WHERE c.dedup_status = 'active'`;
const params: any[] = [];
if (ownerFilter && ownerFilter.length > 0) {
const placeholders = ownerFilter.map(() => "?").join(",");
sql += ` AND c.owner IN (${placeholders})`;
params.push(...ownerFilter);
}
sql += ` ORDER BY c.created_at DESC LIMIT ?`;
params.push(limit);
const rows = this.db.prepare(sql).all(...params) as Array<{ chunk_id: string; vector: Buffer; dimensions: number }>;
return rows.map((r) => ({
chunkId: r.chunk_id,
vector: Array.from(new Float32Array(r.vector.buffer, r.vector.byteOffset, r.dimensions)),
}));
}
getEmbedding(chunkId: string): number[] | null {
const row = this.db.prepare(
"SELECT vector, dimensions FROM embeddings WHERE chunk_id = ?",
).get(chunkId) as { vector: Buffer; dimensions: number } | undefined;
if (!row) return null;
return Array.from(new Float32Array(row.vector.buffer, row.vector.byteOffset, row.dimensions));
}
// ─── Update ───
updateChunk(chunkId: string, fields: { summary?: string; content?: string; role?: string; kind?: string; owner?: string }): boolean {
const sets: string[] = [];
const params: unknown[] = [];
if (fields.summary !== undefined) {
sets.push("summary = ?");
params.push(fields.summary);
}
if (fields.content !== undefined) {
sets.push("content = ?");
params.push(fields.content);
}
if (fields.role !== undefined) {
sets.push("role = ?");
params.push(fields.role);
}
if (fields.kind !== undefined) {
sets.push("kind = ?");
params.push(fields.kind);
}
if (fields.owner !== undefined) {
sets.push("owner = ?");
params.push(fields.owner);
}
if (sets.length === 0) return false;
sets.push("updated_at = ?");
params.push(Date.now());
params.push(chunkId);
const result = this.db.prepare(
`UPDATE chunks SET ${sets.join(", ")} WHERE id = ?`,
).run(...params);
return result.changes > 0;
}
/**
* Find user-role chunks that contain system-injected content that should
* have been stripped before storage. Returns chunk IDs and a preview.
*/
findPollutedUserChunks(): Array<{ id: string; preview: string; reason: string }> {
const results: Array<{ id: string; preview: string; reason: string }> = [];
const patterns: Array<{ sql: string; reason: string }> = [
{ sql: "content LIKE '%%'", reason: "memory_context injection" },
{ sql: "content LIKE '%=== MemOS LONG-TERM MEMORY%'", reason: "MemOS legacy injection" },
{ sql: "content LIKE '%[MemOS Auto-Recall]%'", reason: "MemOS Auto-Recall injection" },
{ sql: "content LIKE '%## Memory system%No memories were automatically recalled%'", reason: "Memory system no-recall hint" },
{ sql: "content LIKE '%## Retrieved memories from past conversations%CRITICAL INSTRUCTION%'", reason: "prependContext recall injection" },
{ sql: "content LIKE '%VERIFIED facts the user previously shared%'", reason: "VERIFIED facts injection" },
{ sql: "content LIKE '%%'", reason: "memos_system_instruction injection" },
{ sql: "content LIKE '%📝 Related memories:%'", reason: "Related memories injection" },
];
for (const { sql, reason } of patterns) {
const rows = this.db.prepare(
`SELECT id, substr(content, 1, 120) AS preview FROM chunks WHERE role = 'user' AND ${sql}`,
).all() as Array<{ id: string; preview: string }>;
for (const row of rows) {
results.push({ id: row.id, preview: row.preview, reason });
}
}
return results;
}
/**
* Find user chunks where user+assistant content was mixed together
* (separated by \n\n---\n), and truncate to keep only the user's part.
*/
fixMixedUserChunks(): number {
const rows = this.db.prepare(
`SELECT id, content FROM chunks WHERE role = 'user'
AND content LIKE '%' || char(10) || char(10) || '---' || char(10) || '%'
AND length(content) > 300`,
).all() as Array<{ id: string; content: string }>;
let fixed = 0;
for (const { id, content } of rows) {
const dashIdx = content.indexOf("\n\n---\n");
if (dashIdx > 5) {
const userPart = content.slice(0, dashIdx).trim();
if (userPart.length >= 5 && userPart.length < content.length) {
this.db.prepare("UPDATE chunks SET content = ?, updated_at = ? WHERE id = ?")
.run(userPart, Date.now(), id);
fixed++;
}
}
}
return fixed;
}
// ─── Delete ───
deleteChunk(chunkId: string): boolean {
const result = this.db.prepare("DELETE FROM chunks WHERE id = ?").run(chunkId);
return result.changes > 0;
}
deleteSession(sessionKey: string): number {
const result = this.db.prepare("DELETE FROM chunks WHERE session_key = ?").run(sessionKey);
return result.changes;
}
deleteAll(): number {
this.db.exec("PRAGMA foreign_keys = OFF");
const tables = [
"task_skills",
"skill_embeddings",
"skill_versions",
"skills",
"embeddings",
"chunks",
"tasks",
"viewer_events",
"api_logs",
"tool_calls",
];
for (const table of tables) {
try {
this.db.prepare(`DELETE FROM ${table}`).run();
} catch (err) {
this.log.warn(`deleteAll: failed to clear ${table}: ${err}`);
}
}
this.db.exec("PRAGMA foreign_keys = ON");
const remaining = this.countChunks();
return remaining === 0 ? 1 : 0;
}
deleteTask(taskId: string): boolean {
this.db.prepare("DELETE FROM task_skills WHERE task_id = ?").run(taskId);
this.db.prepare("UPDATE chunks SET task_id = NULL WHERE task_id = ?").run(taskId);
const result = this.db.prepare("DELETE FROM tasks WHERE id = ?").run(taskId);
return result.changes > 0;
}
deleteSkill(skillId: string): boolean {
this.db.prepare("DELETE FROM task_skills WHERE skill_id = ?").run(skillId);
this.db.prepare("DELETE FROM skill_versions WHERE skill_id = ?").run(skillId);
this.db.prepare("DELETE FROM skill_embeddings WHERE skill_id = ?").run(skillId);
this.db.prepare("UPDATE chunks SET skill_id = NULL WHERE skill_id = ?").run(skillId);
const result = this.db.prepare("DELETE FROM skills WHERE id = ?").run(skillId);
return result.changes > 0;
}
// ─── Task CRUD ───
insertTask(task: Task): void {
this.db.prepare(`
INSERT OR REPLACE INTO tasks (id, session_key, title, summary, status, owner, started_at, ended_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(task.id, task.sessionKey, task.title, task.summary, task.status, task.owner ?? "agent:main", task.startedAt, task.endedAt, task.updatedAt);
}
getTask(taskId: string): Task | null {
const row = this.db.prepare("SELECT * FROM tasks WHERE id = ?").get(taskId) as TaskRow | undefined;
return row ? rowToTask(row) : null;
}
getActiveTask(sessionKey: string, owner?: string): Task | null {
if (owner) {
const row = this.db.prepare(
"SELECT * FROM tasks WHERE session_key = ? AND status = 'active' AND owner = ? ORDER BY started_at DESC LIMIT 1",
).get(sessionKey, owner) as TaskRow | undefined;
return row ? rowToTask(row) : null;
}
const row = this.db.prepare(
"SELECT * FROM tasks WHERE session_key = ? AND status = 'active' ORDER BY started_at DESC LIMIT 1",
).get(sessionKey) as TaskRow | undefined;
return row ? rowToTask(row) : null;
}
hasTaskForSession(sessionKey: string): boolean {
const row = this.db.prepare(
"SELECT 1 FROM tasks WHERE session_key = ? LIMIT 1",
).get(sessionKey);
return !!row;
}
hasSkillForSessionTask(sessionKey: string): boolean {
const row = this.db.prepare(
"SELECT 1 FROM task_skills ts JOIN tasks t ON ts.task_id = t.id WHERE t.session_key = ? LIMIT 1",
).get(sessionKey);
return !!row;
}
getCompletedTasksForSession(sessionKey: string): Task[] {
const rows = this.db.prepare(
"SELECT * FROM tasks WHERE session_key = ? AND status = 'completed'",
).all(sessionKey) as TaskRow[];
return rows.map(rowToTask);
}
getAllActiveTasks(owner?: string): Task[] {
if (owner) {
const rows = this.db.prepare(
"SELECT * FROM tasks WHERE status = 'active' AND owner = ? ORDER BY started_at DESC",
).all(owner) as TaskRow[];
return rows.map(rowToTask);
}
const rows = this.db.prepare(
"SELECT * FROM tasks WHERE status = 'active' ORDER BY started_at DESC",
).all() as TaskRow[];
return rows.map(rowToTask);
}
updateTask(taskId: string, fields: { title?: string; summary?: string; status?: TaskStatus; endedAt?: number }): boolean {
const sets: string[] = [];
const params: unknown[] = [];
if (fields.title !== undefined) { sets.push("title = ?"); params.push(fields.title); }
if (fields.summary !== undefined) { sets.push("summary = ?"); params.push(fields.summary); }
if (fields.status !== undefined) { sets.push("status = ?"); params.push(fields.status); }
if (fields.endedAt !== undefined) { sets.push("ended_at = ?"); params.push(fields.endedAt); }
if (sets.length === 0) return false;
sets.push("updated_at = ?");
params.push(Date.now());
params.push(taskId);
const result = this.db.prepare(`UPDATE tasks SET ${sets.join(", ")} WHERE id = ?`).run(...params);
return result.changes > 0;
}
getChunksByTask(taskId: string): Chunk[] {
const rows = this.db.prepare("SELECT * FROM chunks WHERE task_id = ? ORDER BY created_at, seq").all(taskId) as ChunkRow[];
return rows.map(rowToChunk);
}
listTasks(opts: { status?: string; limit?: number; offset?: number; owner?: string } = {}): { tasks: Task[]; total: number } {
const conditions: string[] = [];
const params: unknown[] = [];
if (opts.status) { conditions.push("status = ?"); params.push(opts.status); }
if (opts.owner) { conditions.push("owner = ?"); params.push(opts.owner); }
const whereClause = conditions.length > 0 ? `WHERE ${conditions.join(" AND ")}` : "";
const countRow = this.db.prepare(`SELECT COUNT(*) as c FROM tasks ${whereClause}`).get(...params) as { c: number };
const total = countRow.c;
const limit = opts.limit ?? 50;
const offset = opts.offset ?? 0;
const rows = this.db.prepare(
`SELECT * FROM tasks ${whereClause} ORDER BY started_at DESC LIMIT ? OFFSET ?`,
).all(...params, limit, offset) as TaskRow[];
return { tasks: rows.map(rowToTask), total };
}
countChunksByTask(taskId: string): number {
const row = this.db.prepare("SELECT COUNT(*) as c FROM chunks WHERE task_id = ?").get(taskId) as { c: number };
return row.c;
}
setChunkTaskId(chunkId: string, taskId: string): void {
this.db.prepare("UPDATE chunks SET task_id = ?, updated_at = ? WHERE id = ?").run(taskId, Date.now(), chunkId);
}
getUnassignedChunks(sessionKey: string, owner?: string): Chunk[] {
if (owner) {
const rows = this.db.prepare(
"SELECT * FROM chunks WHERE session_key = ? AND task_id IS NULL AND owner = ? ORDER BY created_at, seq",
).all(sessionKey, owner) as ChunkRow[];
return rows.map(rowToChunk);
}
const rows = this.db.prepare(
"SELECT * FROM chunks WHERE session_key = ? AND task_id IS NULL ORDER BY created_at, seq",
).all(sessionKey) as ChunkRow[];
return rows.map(rowToChunk);
}
/**
* Check if a chunk with the same (session_key, role, content_hash) already exists.
* Uses indexed content_hash for O(1) lookup to prevent duplicate ingestion
* when agent_end sends the full conversation history every turn.
*/
chunkExistsByContent(sessionKey: string, role: string, content: string): boolean {
const hash = contentHash(content);
const row = this.db.prepare(
"SELECT 1 FROM chunks WHERE session_key = ? AND role = ? AND content_hash = ? LIMIT 1",
).get(sessionKey, role, hash);
return !!row;
}
/**
* Find an active chunk with the same content_hash within the same owner (agent dimension).
* Returns the existing chunk ID if found, null otherwise.
*/
findActiveChunkByHash(content: string, owner?: string): string | null {
const hash = contentHash(content);
// Check ANY existing chunk with the same hash (regardless of dedup_status)
// to prevent re-creating duplicates when all prior copies have been marked duplicate/merged.
if (owner) {
const row = this.db.prepare(
"SELECT id FROM chunks WHERE content_hash = ? AND owner = ? ORDER BY CASE dedup_status WHEN 'active' THEN 0 ELSE 1 END LIMIT 1",
).get(hash, owner) as { id: string } | undefined;
return row?.id ?? null;
}
const row = this.db.prepare(
"SELECT id FROM chunks WHERE content_hash = ? ORDER BY CASE dedup_status WHEN 'active' THEN 0 ELSE 1 END LIMIT 1",
).get(hash) as { id: string } | undefined;
return row?.id ?? null;
}
// ─── Util ───
getRecentChunkIds(limit: number): string[] {
const rows = this.db.prepare(
"SELECT id FROM chunks ORDER BY created_at DESC LIMIT ?",
).all(limit) as Array<{ id: string }>;
return rows.map((r) => r.id);
}
countChunks(): number {
const row = this.db.prepare("SELECT COUNT(*) AS cnt FROM chunks").get() as { cnt: number };
return row.cnt;
}
// ─── Skill CRUD ───
insertSkill(skill: Skill): void {
this.db.prepare(`
INSERT OR REPLACE INTO skills (id, name, description, version, status, tags, source_type, dir_path, installed, owner, visibility, quality_score, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(skill.id, skill.name, skill.description, skill.version, skill.status, skill.tags, skill.sourceType, skill.dirPath, skill.installed, skill.owner ?? "agent:main", skill.visibility ?? "private", skill.qualityScore, skill.createdAt, skill.updatedAt);
}
getSkill(skillId: string): Skill | null {
const row = this.db.prepare("SELECT * FROM skills WHERE id = ?").get(skillId) as SkillRow | undefined;
return row ? rowToSkill(row) : null;
}
getSkillByName(name: string): Skill | null {
const row = this.db.prepare("SELECT * FROM skills WHERE name = ?").get(name) as SkillRow | undefined;
return row ? rowToSkill(row) : null;
}
updateSkill(skillId: string, fields: { description?: string; version?: number; status?: SkillStatus; installed?: number; qualityScore?: number | null; updatedAt?: number }): void {
const sets: string[] = [];
const params: unknown[] = [];
if (fields.description !== undefined) { sets.push("description = ?"); params.push(fields.description); }
if (fields.version !== undefined) { sets.push("version = ?"); params.push(fields.version); }
if (fields.status !== undefined) { sets.push("status = ?"); params.push(fields.status); }
if (fields.installed !== undefined) { sets.push("installed = ?"); params.push(fields.installed); }
if (fields.qualityScore !== undefined) { sets.push("quality_score = ?"); params.push(fields.qualityScore); }
if (sets.length === 0) return;
sets.push("updated_at = ?");
params.push(fields.updatedAt ?? Date.now());
params.push(skillId);
this.db.prepare(`UPDATE skills SET ${sets.join(", ")} WHERE id = ?`).run(...params);
}
listSkills(opts: { status?: string } = {}): Skill[] {
const cond = opts.status ? "WHERE status = ?" : "";
const params = opts.status ? [opts.status] : [];
const rows = this.db.prepare(`SELECT * FROM skills ${cond} ORDER BY updated_at DESC`).all(...params) as SkillRow[];
return rows.map(rowToSkill);
}
// ─── Skill Visibility & Embeddings ───
setSkillVisibility(skillId: string, visibility: SkillVisibility): void {
this.db.prepare("UPDATE skills SET visibility = ?, updated_at = ? WHERE id = ?")
.run(visibility, Date.now(), skillId);
}
upsertSkillEmbedding(skillId: string, vector: number[]): void {
const buf = Buffer.from(new Float32Array(vector).buffer);
this.db.prepare(`
INSERT OR REPLACE INTO skill_embeddings (skill_id, vector, dimensions, updated_at)
VALUES (?, ?, ?, ?)
`).run(skillId, buf, vector.length, Date.now());
}
getSkillEmbedding(skillId: string): number[] | null {
const row = this.db.prepare(
"SELECT vector, dimensions FROM skill_embeddings WHERE skill_id = ?",
).get(skillId) as { vector: Buffer; dimensions: number } | undefined;
if (!row) return null;
return Array.from(new Float32Array(row.vector.buffer, row.vector.byteOffset, row.dimensions));
}
getSkillEmbeddings(scope: "self" | "public" | "mix", currentOwner: string): Array<{ skillId: string; vector: number[] }> {
let sql = `SELECT se.skill_id, se.vector, se.dimensions
FROM skill_embeddings se
JOIN skills s ON s.id = se.skill_id
WHERE s.status = 'active'`;
const params: any[] = [];
if (scope === "self") {
sql += ` AND s.owner = ?`;
params.push(currentOwner);
} else if (scope === "public") {
sql += ` AND s.visibility = 'public'`;
} else {
sql += ` AND (s.owner = ? OR s.visibility = 'public')`;
params.push(currentOwner);
}
const rows = this.db.prepare(sql).all(...params) as Array<{ skill_id: string; vector: Buffer; dimensions: number }>;
return rows.map((r) => ({
skillId: r.skill_id,
vector: Array.from(new Float32Array(r.vector.buffer, r.vector.byteOffset, r.dimensions)),
}));
}
skillFtsSearch(query: string, limit: number, scope: "self" | "public" | "mix", currentOwner: string): Array<{ skillId: string; score: number }> {
const sanitized = sanitizeFtsQuery(query);
if (!sanitized) return [];
try {
let sql = `
SELECT s.id as skill_id, rank
FROM skills_fts f
JOIN skills s ON s.rowid = f.rowid
WHERE skills_fts MATCH ? AND s.status = 'active'`;
const params: any[] = [sanitized];
if (scope === "self") {
sql += ` AND s.owner = ?`;
params.push(currentOwner);
} else if (scope === "public") {
sql += ` AND s.visibility = 'public'`;
} else {
sql += ` AND (s.owner = ? OR s.visibility = 'public')`;
params.push(currentOwner);
}
sql += ` ORDER BY rank LIMIT ?`;
params.push(limit);
const rows = this.db.prepare(sql).all(...params) as Array<{ skill_id: string; rank: number }>;
if (rows.length === 0) return [];
const maxAbsRank = Math.max(...rows.map((r) => Math.abs(r.rank)));
return rows.map((r) => ({
skillId: r.skill_id,
score: maxAbsRank > 0 ? Math.abs(r.rank) / maxAbsRank : 0,
}));
} catch {
this.log.warn(`Skill FTS query failed for: "${sanitized}", returning empty`);
return [];
}
}
listPublicSkills(): Skill[] {
const rows = this.db.prepare("SELECT * FROM skills WHERE visibility = 'public' AND status = 'active' ORDER BY updated_at DESC").all() as SkillRow[];
return rows.map(rowToSkill);
}
// ─── Skill Versions ───
insertSkillVersion(sv: SkillVersion): void {
this.db.prepare(`
INSERT OR REPLACE INTO skill_versions (id, skill_id, version, content, changelog, change_summary, upgrade_type, source_task_id, metrics, quality_score, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(sv.id, sv.skillId, sv.version, sv.content, sv.changelog, sv.changeSummary, sv.upgradeType, sv.sourceTaskId, sv.metrics, sv.qualityScore, sv.createdAt);
}
getLatestSkillVersion(skillId: string): SkillVersion | null {
const row = this.db.prepare("SELECT * FROM skill_versions WHERE skill_id = ? ORDER BY version DESC LIMIT 1").get(skillId) as SkillVersionRow | undefined;
return row ? rowToSkillVersion(row) : null;
}
getSkillVersions(skillId: string): SkillVersion[] {
const rows = this.db.prepare("SELECT * FROM skill_versions WHERE skill_id = ? ORDER BY version DESC").all(skillId) as SkillVersionRow[];
return rows.map(rowToSkillVersion);
}
getSkillVersion(skillId: string, version: number): SkillVersion | null {
const row = this.db.prepare("SELECT * FROM skill_versions WHERE skill_id = ? AND version = ?").get(skillId, version) as SkillVersionRow | undefined;
return row ? rowToSkillVersion(row) : null;
}
// ─── Task-Skill Links ───
linkTaskSkill(taskId: string, skillId: string, relation: TaskSkillRelation, versionAt: number): void {
const skillExists = this.db.prepare("SELECT 1 FROM skills WHERE id = ?").get(skillId);
if (!skillExists) return;
const taskExists = this.db.prepare("SELECT 1 FROM tasks WHERE id = ?").get(taskId);
if (!taskExists) return;
this.db.prepare(`
INSERT OR REPLACE INTO task_skills (task_id, skill_id, relation, version_at, created_at)
VALUES (?, ?, ?, ?, ?)
`).run(taskId, skillId, relation, versionAt, Date.now());
}
getSkillsByTask(taskId: string): Array<{ skill: Skill; relation: TaskSkillRelation; versionAt: number }> {
const rows = this.db.prepare(`
SELECT s.*, ts.relation, ts.version_at
FROM task_skills ts JOIN skills s ON s.id = ts.skill_id
WHERE ts.task_id = ?
`).all(taskId) as Array;
return rows.map(r => ({
skill: rowToSkill(r),
relation: r.relation as TaskSkillRelation,
versionAt: r.version_at,
}));
}
getTasksBySkill(skillId: string): Array<{ task: Task; relation: TaskSkillRelation }> {
const rows = this.db.prepare(`
SELECT t.*, ts.relation
FROM task_skills ts JOIN tasks t ON t.id = ts.task_id
WHERE ts.skill_id = ?
ORDER BY t.started_at DESC
`).all(skillId) as Array;
return rows.map(r => ({
task: rowToTask(r),
relation: r.relation as TaskSkillRelation,
}));
}
countSkills(status?: string): number {
const cond = status ? "WHERE status = ?" : "";
const params = status ? [status] : [];
const row = this.db.prepare(`SELECT COUNT(*) as c FROM skills ${cond}`).get(...params) as { c: number };
return row.c;
}
// ─── Chunk-Skill ───
setChunkSkillId(chunkId: string, skillId: string): void {
this.db.prepare("UPDATE chunks SET skill_id = ?, updated_at = ? WHERE id = ?").run(skillId, Date.now(), chunkId);
}
getDistinctSessionKeys(): string[] {
return (this.db.prepare("SELECT DISTINCT session_key FROM chunks ORDER BY session_key").all() as Array<{ session_key: string }>)
.map(r => r.session_key);
}
getSessionOwnerMap(sessionKeys: string[]): Map {
const result = new Map();
if (sessionKeys.length === 0) return result;
const placeholders = sessionKeys.map(() => "?").join(",");
const rows = this.db.prepare(
`SELECT session_key, owner FROM chunks WHERE session_key IN (${placeholders}) AND owner IS NOT NULL GROUP BY session_key`,
).all(...sessionKeys) as Array<{ session_key: string; owner: string }>;
for (const r of rows) result.set(r.session_key, r.owner);
return result;
}
close(): void {
this.db.close();
}
}
// ─── FTS helpers ───
/**
* Sanitize user input for FTS5 MATCH queries.
* Strip FTS operators and special characters, then join tokens
* with implicit AND (space-separated) for safe querying.
*/
function sanitizeFtsQuery(raw: string): string {
const tokens = raw
.replace(/[."""(){}[\]*:^~!@#$%&\\/<>,;'`-]/g, " ")
.split(/\s+/)
.map((t) => t.trim().replace(/^-+|-+$/g, ""))
.filter((t) => t.length > 1)
.filter((t) => !FTS_RESERVED.has(t.toUpperCase()));
return tokens.join(" ");
}
const FTS_RESERVED = new Set(["AND", "OR", "NOT", "NEAR"]);
// ─── Internal helpers ───
interface ChunkRow {
id: string;
session_key: string;
turn_id: string;
seq: number;
role: string;
content: string;
kind: string;
summary: string;
task_id: string | null;
skill_id: string | null;
owner: string;
dedup_status: string;
dedup_target: string | null;
dedup_reason: string | null;
merge_count: number;
last_hit_at: number | null;
merge_history: string;
created_at: number;
updated_at: number;
}
function rowToChunk(row: ChunkRow): Chunk {
return {
id: row.id,
sessionKey: row.session_key,
turnId: row.turn_id,
seq: row.seq,
role: row.role as Chunk["role"],
content: row.content,
kind: row.kind as Chunk["kind"],
summary: row.summary,
embedding: null,
taskId: row.task_id,
skillId: row.skill_id ?? null,
owner: row.owner ?? "agent:main",
dedupStatus: (row.dedup_status ?? "active") as DedupStatus,
dedupTarget: row.dedup_target ?? null,
dedupReason: row.dedup_reason ?? null,
mergeCount: row.merge_count ?? 0,
lastHitAt: row.last_hit_at ?? null,
mergeHistory: row.merge_history ?? "[]",
createdAt: row.created_at,
updatedAt: row.updated_at,
};
}
interface TaskRow {
id: string;
session_key: string;
title: string;
summary: string;
status: string;
owner: string;
started_at: number;
ended_at: number | null;
updated_at: number;
}
function rowToTask(row: TaskRow): Task {
return {
id: row.id,
sessionKey: row.session_key,
title: row.title,
summary: row.summary,
status: row.status as Task["status"],
owner: row.owner ?? "agent:main",
startedAt: row.started_at,
endedAt: row.ended_at,
updatedAt: row.updated_at,
};
}
interface SkillRow {
id: string;
name: string;
description: string;
version: number;
status: string;
tags: string;
source_type: string;
dir_path: string;
installed: number;
owner: string;
visibility: string;
quality_score: number | null;
created_at: number;
updated_at: number;
}
function rowToSkill(row: SkillRow): Skill {
return {
id: row.id,
name: row.name,
description: row.description,
version: row.version,
status: row.status as Skill["status"],
tags: row.tags,
sourceType: row.source_type as Skill["sourceType"],
dirPath: row.dir_path,
installed: row.installed,
owner: row.owner ?? "agent:main",
visibility: (row.visibility ?? "private") as Skill["visibility"],
qualityScore: row.quality_score ?? null,
createdAt: row.created_at,
updatedAt: row.updated_at,
};
}
interface SkillVersionRow {
id: string;
skill_id: string;
version: number;
content: string;
changelog: string;
change_summary: string;
upgrade_type: string;
source_task_id: string | null;
metrics: string;
quality_score: number | null;
created_at: number;
}
function rowToSkillVersion(row: SkillVersionRow): SkillVersion {
return {
id: row.id,
skillId: row.skill_id,
version: row.version,
content: row.content,
changelog: row.changelog,
changeSummary: row.change_summary ?? "",
upgradeType: row.upgrade_type as SkillVersion["upgradeType"],
sourceTaskId: row.source_task_id,
metrics: row.metrics,
qualityScore: row.quality_score ?? null,
createdAt: row.created_at,
};
}
function contentHash(content: string): string {
return createHash("sha256").update(content).digest("hex").slice(0, 16);
}
================================================
FILE: apps/memos-local-openclaw/src/storage/vector.ts
================================================
import type { SqliteStore } from "./sqlite";
export function cosineSimilarity(a: number[], b: number[]): number {
if (a.length !== b.length) return 0;
let dot = 0;
let normA = 0;
let normB = 0;
for (let i = 0; i < a.length; i++) {
dot += a[i] * b[i];
normA += a[i] * a[i];
normB += b[i] * b[i];
}
const denom = Math.sqrt(normA) * Math.sqrt(normB);
return denom === 0 ? 0 : dot / denom;
}
export interface VectorHit {
chunkId: string;
score: number;
}
/**
* Brute-force vector search over stored embeddings.
* When maxChunks > 0, only searches the most recent maxChunks chunks (uses index; avoids full scan as data grows).
*/
export function vectorSearch(
store: SqliteStore,
queryVec: number[],
topK: number,
maxChunks?: number,
ownerFilter?: string[],
): VectorHit[] {
const all = maxChunks != null && maxChunks > 0
? store.getRecentEmbeddings(maxChunks, ownerFilter)
: store.getAllEmbeddings(ownerFilter);
const scored: VectorHit[] = all.map((row) => ({
chunkId: row.chunkId,
score: cosineSimilarity(queryVec, row.vector),
}));
scored.sort((a, b) => b.score - a.score);
return scored.slice(0, topK);
}
================================================
FILE: apps/memos-local-openclaw/src/telemetry.ts
================================================
/**
* Telemetry module — anonymous usage analytics via Aliyun ARMS RUM.
*
* Privacy-first design:
* - Enabled by default with anonymous data only; opt-out via TELEMETRY_ENABLED=false
* - Uses a random anonymous ID persisted locally (no PII)
* - Never sends memory content, queries, or any user data
* - Only sends aggregate counts, tool names, latencies, and version info
*/
import * as fs from "fs";
import * as path from "path";
import * as os from "os";
import { v4 as uuidv4 } from "uuid";
import type { Logger } from "./types";
export interface TelemetryConfig {
enabled?: boolean;
}
const ARMS_ENDPOINT =
"https://proj-xtrace-e218d9316b328f196a3c640cc7ca84-cn-hangzhou.cn-hangzhou.log.aliyuncs.com" +
"/rum/web/v2" +
"?workspace=default-cms-1026429231103299-cn-hangzhou" +
"&service_id=a3u72ukxmr@066657d42a13a9a9f337f";
const ARMS_PID = "a3u72ukxmr@066657d42a13a9a9f337f";
const ARMS_ENV = "prod";
const FLUSH_AT = 10;
const FLUSH_INTERVAL_MS = 30_000;
const SEND_TIMEOUT_MS = 30_000;
const SESSION_TTL_MS = 30 * 60_000; // 30 min inactivity → new session
interface ArmsEvent {
event_type: "custom";
type: string;
name: string;
group: string;
value: number;
properties: Record;
timestamp: number;
event_id: string;
times: number;
}
export class Telemetry {
private distinctId: string;
private enabled: boolean;
private pluginVersion: string;
private log: Logger;
private dailyPingSent = false;
private dailyPingDate = "";
private buffer: ArmsEvent[] = [];
private flushTimer: ReturnType | null = null;
private sessionId: string;
private firstSeenDate: string;
constructor(config: TelemetryConfig, stateDir: string, pluginVersion: string, log: Logger) {
this.log = log;
this.pluginVersion = pluginVersion;
this.enabled = config.enabled !== false;
this.distinctId = this.loadOrCreateAnonymousId(stateDir);
this.firstSeenDate = this.loadOrCreateFirstSeen(stateDir);
this.sessionId = this.loadOrCreateSessionId(stateDir);
if (!this.enabled) {
this.log.debug("Telemetry disabled (opt-out via TELEMETRY_ENABLED=false)");
return;
}
this.flushTimer = setInterval(() => this.flush(), FLUSH_INTERVAL_MS);
if (this.flushTimer.unref) this.flushTimer.unref();
this.log.debug("Telemetry initialized (ARMS)");
}
private loadOrCreateAnonymousId(stateDir: string): string {
const newDir = path.join(stateDir, "memos-local");
const oldDir = path.join(stateDir, "memos-lite");
const idFile = path.join(newDir, ".anonymous-id");
const oldIdFile = path.join(oldDir, ".anonymous-id");
try {
const existing = fs.readFileSync(idFile, "utf-8").trim();
if (existing.length > 10) return existing;
} catch {}
try {
const existing = fs.readFileSync(oldIdFile, "utf-8").trim();
if (existing.length > 10) return existing;
} catch {}
const newId = uuidv4();
try {
fs.mkdirSync(path.dirname(idFile), { recursive: true });
fs.writeFileSync(idFile, newId, "utf-8");
} catch {}
return newId;
}
private loadOrCreateSessionId(stateDir: string): string {
const filePath = path.join(stateDir, "memos-local", ".session");
try {
const raw = fs.readFileSync(filePath, "utf-8").trim();
const sep = raw.indexOf("|");
if (sep > 0) {
const ts = parseInt(raw.slice(0, sep), 10);
const id = raw.slice(sep + 1);
if (id.length > 10 && Date.now() - ts < SESSION_TTL_MS) {
this.touchSession(filePath, id);
return id;
}
}
} catch {}
const newId = uuidv4();
this.touchSession(filePath, newId);
return newId;
}
private touchSession(filePath: string, id: string): void {
try {
fs.mkdirSync(path.dirname(filePath), { recursive: true });
fs.writeFileSync(filePath, `${Date.now()}|${id}`, "utf-8");
} catch {}
}
private loadOrCreateFirstSeen(stateDir: string): string {
const filePath = path.join(stateDir, "memos-local", ".first-seen");
try {
const existing = fs.readFileSync(filePath, "utf-8").trim();
if (existing.length === 10) return existing;
} catch {}
const today = new Date().toISOString().slice(0, 10);
try {
fs.mkdirSync(path.dirname(filePath), { recursive: true });
fs.writeFileSync(filePath, today, "utf-8");
} catch {}
return today;
}
private capture(event: string, properties?: Record): void {
if (!this.enabled) return;
const safeProps: Record = {
plugin_version: this.pluginVersion,
os: os.platform(),
os_version: os.release(),
node_version: process.version,
arch: os.arch(),
};
if (properties) {
for (const [k, v] of Object.entries(properties)) {
if (typeof v === "string" || typeof v === "number" || typeof v === "boolean") {
safeProps[k] = v;
}
}
}
this.buffer.push({
event_type: "custom",
type: "memos_plugin",
name: event,
group: "memos_local",
value: 1,
properties: safeProps,
timestamp: Date.now(),
event_id: uuidv4(),
times: 1,
});
if (this.buffer.length >= FLUSH_AT) {
this.flush();
}
}
private buildPayload(events: ArmsEvent[]): Record {
return {
app: {
id: ARMS_PID,
env: ARMS_ENV,
version: this.pluginVersion,
type: "node",
},
user: { id: this.distinctId },
session: { id: this.sessionId },
net: {},
view: { id: "plugin", name: "memos-local-openclaw" },
events,
_v: "1.0.0",
};
}
private async flush(): Promise {
if (this.buffer.length === 0) return;
const batch = this.buffer.splice(0);
const payload = this.buildPayload(batch);
try {
const resp = await fetch(ARMS_ENDPOINT, {
method: "POST",
headers: { "Content-Type": "text/plain" },
body: JSON.stringify(payload),
signal: AbortSignal.timeout(SEND_TIMEOUT_MS),
});
this.log.debug(`Telemetry flush: ${batch.length} events → ${resp.status}`);
} catch (err) {
this.log.debug(`Telemetry flush failed: ${err}`);
}
}
// ─── Public event methods ───
trackPluginStarted(embeddingProvider: string, summarizerProvider: string): void {
this.capture("plugin_started", {
embedding_provider: embeddingProvider,
summarizer_provider: summarizerProvider,
});
this.maybeSendDailyPing();
}
trackToolCalled(toolName: string, latencyMs: number, success: boolean): void {
this.capture(toolName, {
latency_ms: Math.round(latencyMs),
success,
});
}
trackMemoryIngested(chunkCount: number): void {
this.capture("memory_ingested", {
chunk_count: chunkCount,
});
}
trackSkillInstalled(skillName: string): void {
this.capture("skill_installed", {
skill_name: skillName,
});
}
trackSkillEvolved(skillName: string, upgradeType: "created" | "upgraded"): void {
this.capture("skill_evolved", {
skill_name: skillName,
upgrade_type: upgradeType,
});
}
trackViewerOpened(): void {
this.capture("viewer_opened");
}
trackAutoRecall(hitCount: number, latencyMs: number): void {
this.capture("memory_search", {
auto: true,
hit_count: hitCount,
latency_ms: Math.round(latencyMs),
});
}
trackError(source: string, errorType: string): void {
this.capture("plugin_error", {
error_source: source,
error_type: errorType,
});
}
private maybeSendDailyPing(): void {
const today = new Date().toISOString().slice(0, 10);
if (this.dailyPingSent && this.dailyPingDate === today) return;
this.dailyPingSent = true;
this.dailyPingDate = today;
this.capture("daily_active", {
first_seen_date: this.firstSeenDate,
});
}
async shutdown(): Promise {
if (this.flushTimer) {
clearInterval(this.flushTimer);
this.flushTimer = null;
}
await this.flush();
}
}
================================================
FILE: apps/memos-local-openclaw/src/tools/index.ts
================================================
export { createMemorySearchTool } from "./memory-search";
export { createMemoryTimelineTool } from "./memory-timeline";
export { createMemoryGetTool } from "./memory-get";
================================================
FILE: apps/memos-local-openclaw/src/tools/memory-get.ts
================================================
import type { SqliteStore } from "../storage/sqlite";
import type { ToolDefinition, GetResult, ChunkRef } from "../types";
import { DEFAULTS } from "../types";
function resolveOwnerFilter(owner: unknown): string[] {
const resolvedOwner = typeof owner === "string" && owner.trim().length > 0 ? owner : "agent:main";
return resolvedOwner === "public" ? ["public"] : [resolvedOwner, "public"];
}
export function createMemoryGetTool(store: SqliteStore): ToolDefinition {
return {
name: "memory_get",
description:
"Retrieve the full original text of a specific memory chunk. Use after memory_search or memory_timeline " +
"when you need to see the complete content (not just the excerpt). Useful for verifying exact details.",
inputSchema: {
type: "object",
properties: {
ref: {
type: "object",
description: "Reference object from a memory_search hit or memory_timeline entry.",
properties: {
sessionKey: { type: "string" },
chunkId: { type: "string" },
turnId: { type: "string" },
seq: { type: "number" },
},
required: ["sessionKey", "chunkId", "turnId", "seq"],
},
maxChars: {
type: "number",
description: `Maximum characters to return (default ${DEFAULTS.getMaxCharsDefault}, max ${DEFAULTS.getMaxCharsMax}).`,
},
},
required: ["ref"],
},
handler: async (input) => {
const ref = input.ref as ChunkRef;
const maxChars = Math.min(
(input.maxChars as number) ?? DEFAULTS.getMaxCharsDefault,
DEFAULTS.getMaxCharsMax,
);
const chunk = store.getChunksByRef(ref, resolveOwnerFilter(input.owner));
if (!chunk) {
return { error: `Chunk not found: ${ref.chunkId}` };
}
const content = chunk.content;
const result: GetResult = {
content,
ref: {
sessionKey: chunk.sessionKey,
chunkId: chunk.id,
turnId: chunk.turnId,
seq: chunk.seq,
},
source: {
ts: chunk.createdAt,
role: chunk.role,
sessionKey: chunk.sessionKey,
},
};
return result;
},
};
}
================================================
FILE: apps/memos-local-openclaw/src/tools/memory-search.ts
================================================
import type { RecallEngine } from "../recall/engine";
import type { ToolDefinition } from "../types";
function resolveOwnerFilter(owner: unknown): string[] {
const resolvedOwner = typeof owner === "string" && owner.trim().length > 0 ? owner : "agent:main";
return resolvedOwner === "public" ? ["public"] : [resolvedOwner, "public"];
}
export function createMemorySearchTool(engine: RecallEngine): ToolDefinition {
return {
name: "memory_search",
description:
"Search stored conversation memories. Returns matching entries with summary, original_excerpt (evidence), score, and ref for follow-up with memory_timeline or memory_get. " +
"Default: top 6 results, minScore 0.45. Increase maxResults to 12/20 or lower minScore to 0.35 if initial results are insufficient.",
inputSchema: {
type: "object",
properties: {
query: {
type: "string",
description: "Natural language search query. Include specific entities, commands, or error messages for better recall.",
},
maxResults: {
type: "number",
description: "Maximum number of results (default 6, max 20).",
},
minScore: {
type: "number",
description: "Minimum relevance score threshold 0-1 (default 0.45, floor 0.35).",
},
},
},
handler: async (input) => {
const result = await engine.search({
query: (input.query as string) ?? "",
maxResults: input.maxResults as number | undefined,
minScore: input.minScore as number | undefined,
ownerFilter: resolveOwnerFilter(input.owner),
});
return result;
},
};
}
================================================
FILE: apps/memos-local-openclaw/src/tools/memory-timeline.ts
================================================
import type { SqliteStore } from "../storage/sqlite";
import type { ToolDefinition, TimelineResult, TimelineEntry, ChunkRef } from "../types";
import { DEFAULTS } from "../types";
function resolveOwnerFilter(owner: unknown): string[] {
const resolvedOwner = typeof owner === "string" && owner.trim().length > 0 ? owner : "agent:main";
return resolvedOwner === "public" ? ["public"] : [resolvedOwner, "public"];
}
export function createMemoryTimelineTool(store: SqliteStore): ToolDefinition {
return {
name: "memory_timeline",
description:
"Retrieve neighboring context around a memory reference. Use after memory_search to expand context " +
"around a specific hit. Provides adjacent conversation chunks marked as before/current/after.",
inputSchema: {
type: "object",
properties: {
ref: {
type: "object",
description: "Reference object from a memory_search hit (must contain sessionKey, chunkId, turnId, seq).",
properties: {
sessionKey: { type: "string" },
chunkId: { type: "string" },
turnId: { type: "string" },
seq: { type: "number" },
},
required: ["sessionKey", "chunkId", "turnId", "seq"],
},
window: {
type: "number",
description: "Number of turns/chunks to include before and after (default ±2).",
},
},
required: ["ref"],
},
handler: async (input) => {
const ref = input.ref as ChunkRef;
const window = (input.window as number) ?? DEFAULTS.timelineWindowDefault;
const ownerFilter = resolveOwnerFilter(input.owner);
const anchorChunk = store.getChunksByRef(ref, ownerFilter);
if (!anchorChunk) {
return { entries: [], anchorRef: ref } satisfies TimelineResult;
}
const neighbors = store.getNeighborChunks(
ref.sessionKey,
ref.turnId,
ref.seq,
window,
ownerFilter,
);
const entries: TimelineEntry[] = neighbors.map((chunk) => {
let relation: TimelineEntry["relation"] = "before";
if (chunk.id === ref.chunkId) {
relation = "current";
} else if (chunk.createdAt > anchorChunk.createdAt) {
relation = "after";
}
return {
excerpt: chunk.content.slice(0, DEFAULTS.excerptMaxChars),
ref: {
sessionKey: chunk.sessionKey,
chunkId: chunk.id,
turnId: chunk.turnId,
seq: chunk.seq,
},
role: chunk.role,
ts: chunk.createdAt,
relation,
};
});
const result: TimelineResult = {
entries,
anchorRef: ref,
};
return result;
},
};
}
================================================
FILE: apps/memos-local-openclaw/src/types.ts
================================================
// ─── Role & Message ───
export type Role = "user" | "assistant" | "system" | "tool";
export interface ConversationMessage {
role: Role;
content: string;
timestamp: number;
turnId: string;
sessionKey: string;
toolName?: string;
owner?: string;
}
// ─── Chunk & Storage ───
export type DedupStatus = "active" | "duplicate" | "merged";
export interface Chunk {
id: string;
sessionKey: string;
turnId: string;
seq: number;
role: Role;
content: string;
kind: ChunkKind;
summary: string;
embedding: number[] | null;
taskId: string | null;
skillId: string | null;
owner: string;
dedupStatus: DedupStatus;
dedupTarget: string | null;
dedupReason: string | null;
mergeCount: number;
lastHitAt: number | null;
mergeHistory: string;
createdAt: number;
updatedAt: number;
}
// ─── Task ───
export type TaskStatus = "active" | "completed" | "skipped";
export interface Task {
id: string;
sessionKey: string;
title: string;
summary: string;
status: TaskStatus;
owner: string;
startedAt: number;
endedAt: number | null;
updatedAt: number;
}
export type ChunkKind = "paragraph";
export interface ChunkRef {
sessionKey: string;
chunkId: string;
turnId: string;
seq: number;
}
// ─── Search / Recall ───
export interface SearchHit {
summary: string;
original_excerpt: string;
ref: ChunkRef;
score: number;
taskId: string | null;
skillId: string | null;
owner?: string;
source: {
ts: number;
role: Role;
sessionKey: string;
};
}
export interface SkillSearchHit {
skillId: string;
name: string;
description: string;
owner: string;
visibility: SkillVisibility;
score: number;
reason: string;
}
export interface SearchResult {
hits: SearchHit[];
meta: {
usedMinScore: number;
usedMaxResults: number;
totalCandidates: number;
note?: string;
};
}
export interface TimelineEntry {
excerpt: string;
ref: ChunkRef;
role: Role;
ts: number;
relation: "before" | "current" | "after";
}
export interface TimelineResult {
entries: TimelineEntry[];
anchorRef: ChunkRef;
}
export interface GetResult {
content: string;
ref: ChunkRef;
source: {
ts: number;
role: Role;
sessionKey: string;
};
}
// ─── Candidate (internal) ───
export interface RankedCandidate {
chunkId: string;
ftsScore: number | null;
vecScore: number | null;
rrfScore: number;
mmrScore: number;
recencyScore: number;
finalScore: number;
}
// ─── Provider ───
export type SummaryProvider =
| "openai"
| "openai_compatible"
| "anthropic"
| "gemini"
| "azure_openai"
| "bedrock"
| "zhipu"
| "siliconflow"
| "bailian"
| "cohere"
| "mistral"
| "voyage";
export type EmbeddingProvider =
| "openai"
| "openai_compatible"
| "gemini"
| "azure_openai"
| "cohere"
| "mistral"
| "voyage"
| "local";
export interface ProviderConfig {
provider: string;
endpoint?: string;
apiKey?: string;
model?: string;
headers?: Record;
timeoutMs?: number;
temperature?: number;
}
export interface SummarizerConfig extends ProviderConfig {
provider: SummaryProvider;
}
export interface EmbeddingConfig extends ProviderConfig {
provider: EmbeddingProvider;
batchSize?: number;
dimensions?: number;
retry?: number;
}
// ─── Skill ───
export type SkillStatus = "active" | "archived" | "draft";
export type SkillUpgradeType = "create" | "refine" | "extend" | "fix";
export type TaskSkillRelation = "generated_from" | "evolved_from" | "applied_to";
export type SkillVisibility = "private" | "public";
export interface Skill {
id: string;
name: string;
description: string;
version: number;
status: SkillStatus;
tags: string;
sourceType: "task" | "manual";
dirPath: string;
installed: number;
owner: string;
visibility: SkillVisibility;
qualityScore: number | null;
createdAt: number;
updatedAt: number;
}
export interface SkillVersion {
id: string;
skillId: string;
version: number;
content: string;
changelog: string;
changeSummary: string;
upgradeType: SkillUpgradeType;
sourceTaskId: string | null;
metrics: string;
qualityScore: number | null;
createdAt: number;
}
export interface SkillGenerateOutput {
skill_md: string;
scripts: Array<{ filename: string; content: string }>;
references: Array<{ filename: string; content: string }>;
evals: Array<{ id: number; prompt: string; expectations: string[] }>;
}
export interface TaskSkillLink {
taskId: string;
skillId: string;
relation: TaskSkillRelation;
versionAt: number;
createdAt: number;
}
// ─── Plugin Config ───
export interface SkillEvolutionConfig {
enabled?: boolean;
autoEvaluate?: boolean;
minChunksForEval?: number;
minConfidence?: number;
maxSkillLines?: number;
autoInstall?: boolean;
summarizer?: SummarizerConfig;
}
export interface TelemetryConfig {
enabled?: boolean;
}
export interface MemosLocalConfig {
summarizer?: SummarizerConfig;
embedding?: EmbeddingConfig;
storage?: {
dbPath?: string;
};
recall?: {
maxResultsDefault?: number;
maxResultsMax?: number;
minScoreDefault?: number;
minScoreFloor?: number;
rrfK?: number;
mmrLambda?: number;
recencyHalfLifeDays?: number;
/** Cap vector search to this many most recent chunks. 0 = no cap (search all; may get slower with 200k+ chunks). If you set a cap for performance, use a large value (e.g. 200000–300000) so older memories are still in the window; FTS always searches all. */
vectorSearchMaxChunks?: number;
};
dedup?: {
similarityThreshold?: number;
};
capture?: {
evidenceWrapperTag?: string;
};
skillEvolution?: SkillEvolutionConfig;
telemetry?: TelemetryConfig;
}
// ─── Defaults ───
export const DEFAULTS = {
maxResultsDefault: 6,
maxResultsMax: 20,
minScoreDefault: 0.45,
minScoreFloor: 0.35,
rrfK: 60,
mmrLambda: 0.7,
recencyHalfLifeDays: 14,
vectorSearchMaxChunks: 0,
dedupSimilarityThreshold: 0.80,
evidenceWrapperTag: "STORED_MEMORY",
excerptMinChars: 200,
excerptMaxChars: 500,
getMaxCharsDefault: 2000,
getMaxCharsMax: 8000,
timelineWindowDefault: 2,
localEmbeddingModel: "Xenova/all-MiniLM-L6-v2",
localEmbeddingDimensions: 384,
toolResultMaxChars: 2000,
taskIdleTimeoutMs: 2 * 60 * 60 * 1000, // 2 hour gap → new task
taskSummaryMaxTokens: 2000,
skillEvolutionEnabled: true,
skillAutoEvaluate: true,
skillMinChunksForEval: 6,
skillMinConfidence: 0.7,
skillMaxLines: 400,
skillAutoInstall: false,
} as const;
// ─── Plugin Hooks (OpenClaw integration) ───
export interface PluginContext {
stateDir: string;
workspaceDir: string;
config: MemosLocalConfig;
log: Logger;
}
export interface Logger {
debug(msg: string, ...args: unknown[]): void;
info(msg: string, ...args: unknown[]): void;
warn(msg: string, ...args: unknown[]): void;
error(msg: string, ...args: unknown[]): void;
}
export interface ToolDefinition {
name: string;
description: string;
inputSchema: Record;
handler: (input: Record) => Promise;
}
================================================
FILE: apps/memos-local-openclaw/src/update-check.ts
================================================
/**
* Channel-aware update check against npm registry dist-tags.
* - Prerelease users (e.g. 1.0.2-beta.x) compare against beta tag only (semver gt).
* - Stable users compare against latest tag only (semver gt).
* - Beta users get optional stableChannel hint to install @latest when stable exists.
*/
import * as semver from "semver";
export interface UpdateCheckResult {
updateAvailable: boolean;
current: string;
/** Version on the channel we compared against (beta tag or latest tag). */
latest: string;
packageName: string;
/** Channel used for the primary comparison. */
channel: "beta" | "latest";
/** Full install command (includes @beta when updating on beta channel). */
installCommand: string;
/** When current is prerelease and registry has a stable latest — how to switch to stable. */
stableChannel?: { version: string; installCommand: string };
}
function isPrerelease(v: string): boolean {
return semver.prerelease(v) != null;
}
/**
* Fetch registry package doc and compute update state.
*/
export async function computeUpdateCheck(
packageName: string,
current: string,
fetchImpl: typeof fetch,
timeoutMs = 8_000,
): Promise {
if (!semver.valid(current)) return null;
const url = `https://registry.npmjs.org/${encodeURIComponent(packageName)}`;
const resp = await fetchImpl(url, { signal: AbortSignal.timeout(timeoutMs) });
if (!resp.ok) return null;
const data = (await resp.json()) as { "dist-tags"?: Record };
const tags = data["dist-tags"] ?? {};
const latestTag = tags.latest;
const betaTag = tags.beta;
const onBeta = isPrerelease(current);
let updateAvailable = false;
let channel: "beta" | "latest" = "latest";
let targetVersion = current;
let installCommand = `openclaw plugins install ${packageName}`;
if (onBeta) {
channel = "beta";
// Beta users: only compare against beta tag; never suggest "updating" to stable via gt confusion.
if (betaTag && semver.valid(betaTag) && semver.gt(betaTag, current)) {
updateAvailable = true;
targetVersion = betaTag;
installCommand = `openclaw plugins install ${packageName}@beta`;
} else {
targetVersion = betaTag && semver.valid(betaTag) ? betaTag : current;
if (betaTag && semver.valid(betaTag) && semver.eq(betaTag, current)) {
installCommand = `openclaw plugins install ${packageName}@beta`;
}
}
} else {
// Stable users: compare against latest only.
if (latestTag && semver.valid(latestTag) && semver.gt(latestTag, current)) {
updateAvailable = true;
targetVersion = latestTag;
installCommand = `openclaw plugins install ${packageName}`;
} else {
targetVersion = latestTag && semver.valid(latestTag) ? latestTag : current;
}
}
// Beta user + stable exists on latest: optional hint to switch to stable (not counted as "update").
let stableChannel: UpdateCheckResult["stableChannel"];
if (onBeta && latestTag && semver.valid(latestTag) && !isPrerelease(latestTag)) {
stableChannel = {
version: latestTag,
installCommand: `openclaw plugins install ${packageName}@latest`,
};
}
return {
updateAvailable,
current,
latest: targetVersion,
packageName,
channel,
installCommand,
stableChannel,
};
}
================================================
FILE: apps/memos-local-openclaw/src/viewer/html.ts
================================================
export function viewerHTML(pluginVersion?: string): string {
const vBadge = pluginVersion ? `v${pluginVersion}` : '';
return `
OpenClaw Memory - Powered by MemOS
OpenClaw Memory
Powered by MemOS
Set a password to protect your memories
OpenClaw Memory
Powered by MemOS
Enter your password to access memories
1
Open Terminal
Run the following command to get your reset token (use the pattern below so you get the line that contains the token):
In the output, find password reset token: a1b2c3d4e5f6... (plain line or inside JSON). Copy the 32-character hex string after the colon.
3
Paste & reset
Paste the token below and set your new password.
OpenClaw Memory${vBadge}
-
Memories
-
Sessions
-
Embeddings
-
Days
Sessions
\u{1F50D}
-Total Tasks
-Active
-Completed
-Skipped
Related Memories
-Total Skills
-Active
-Draft
-Installed
-Public
Skill Files
SKILL.md Content
Version History
Related Tasks
Range
-
Total Memories
-
Writes Today
-
Sessions
-
Embeddings
\u{1F4CA}Memory Writes per Day
\u26A1Tool Response Time(per minute avg)
Model Configuration
\u{1F4CA}Model Health
Loading model status...
\u{1F4E1}Embedding Model
\u{1F9E0}Summarizer Model
\u{1F527}Skill Evolution
Skill Dedicated Model
If not configured, the main Summarizer Model above will be used for skill generation. Configure a dedicated model here for higher quality skill output.
\u{1F4CA}Telemetry
Anonymous usage analytics to help improve the plugin. Only sends tool names, latencies, and version info. No memory content, queries, or personal data is ever sent.
\u{1F4BE}General
Requires restart to take effect
\u2713 Saved
Some changes require restarting the OpenClaw gateway to take effect.
\u{1F4E5}Import OpenClaw Memory
Migrate your existing OpenClaw built-in memories and conversation history into this plugin. The import process uses smart deduplication to avoid duplicates.
Three ways to use:
\u2460 Import memories only (fast) — Click "Start Import" to quickly migrate all memory chunks and conversations. No task/skill generation. Suitable when you just need the raw data.
\u2461 Import + generate tasks & skills (slow, serial) — After importing memories, enable "Generate Tasks" and/or "Trigger Skill Evolution" below to analyze conversations one by one. This takes longer as each session is processed by LLM sequentially.
\u2462 Import first, generate later (flexible) — Import memories now, then come back anytime to start task/skill generation. You can pause the generation at any point and resume later — it will pick up where you left off, only processing sessions that haven't been handled yet.
\u26A0 Configuration Required
Please configure both Embedding Model and Summarizer Model in Settings before importing. These are required for processing memories.
Memory Index (SQLite)
0
Conversation History
0
Concurrent agents
\u26A0 Increasing concurrency raises LLM API call frequency, which may trigger rate limits and cause failures.
\u{1F9E0} Optional: Generate Tasks & Skills
This step is completely optional. The import above has already stored raw memory data. Here you can further analyze imported conversations to generate structured task summaries and evolve reusable skills. Processing is serial (one session at a time) and may take a while. You can stop at any time and resume later — it will only process sessions not yet handled.
Concurrent agents
\u26A0 Increasing concurrency raises LLM API call frequency, which may trigger rate limits and cause failures.
交互式演示记忆导入、智能检索和 Viewer 管理的完整流程。所有数据均为模拟,无需安装即可体验。Interactive demo of memory import, smart retrieval, and Viewer management. All data is simulated — no installation required.
🦐 记忆导入🦐 Memory Import
🔍 智能检索🔍 Smart Retrieval
📊 Viewer 管理📊 Viewer Dashboard
1
扫描记忆Scan
2
导入迁移Import
3
导入完成Complete
Memory Viewer — Import
🦐
导入 OpenClaw 记忆Import OpenClaw Memories
将 OpenClaw 内置的记忆数据和对话历史迁移到智能记忆系统。Migrate OpenClaw's built-in memory data and conversation history to the intelligent memory system.
USER帮我查一下阿里云ECS安全组怎么配置,需要开放6333端口给Qdrant使用How to configure Alibaba Cloud ECS security groups? Need to open port 6333 for Qdrant🦐03/04 10:41
ASST安全组配置需要在ECS控制台中设置入站和出站规则,包括端口范围、协议类型、授权对象等。具体步骤如下:1. 登录ECS控制台 → 网络与安全 → 安全组;2. 选择实例所在安全组 → 添加规则;3. 协议类型选 TCP,端口范围填 6333/6333,授权对象填你的客户端 IPSecurity group configuration requires setting inbound and outbound rules in ECS console, including port range, protocol type, and authorization target. Steps: 1. Login ECS console → Network & Security → Security Groups; 2. Select instance security group → Add Rule; 3. Protocol TCP, port 6333/6333, authorize your client IP🦐03/04 10:41
USER红烧肉怎么做?要那种入口即化的How to make braised pork? The melt-in-mouth kind🦐03/04 09:42
ASST经典红烧肉做法:五花肉切块冷水下锅焯水,热锅炒糖色至枣红色,加入五花肉翻炒上色,加生抽老抽料酒,小火慢炖1.5小时。入口即化的关键:小火慢炖、糖色不要炒过头、焯水后冰水激一下Classic braised pork recipe: cut pork belly into cubes, blanch in cold water, stir-fry sugar caramel to dark red, add pork belly and coat evenly, add soy sauce and cooking wine, simmer on low heat for 1.5 hours. Key to melt-in-mouth: low heat slow simmer, don't over-caramelize sugar, ice bath after blanching03/04 09:42
USER帮我整理一下我的工作经历,用于更新简历Help me organize my work experience for resume update03/05 09:07
1 - 5 / 597
4总任务Total
3已完成Completed
1进行中Active
0跳过Skipped
阿里云ECS安全组设置与Qdrant集成Alibaba Cloud ECS Security Groups & Qdrant Integration
已完成Completed📄 98 条记忆📄 98 memories03/04
用户请求帮助配置阿里云ECS安全组以支持Qdrant向量数据库的集成部署,涉及端口开放、防火墙规则设置和安全最佳实践。User requested help configuring Alibaba Cloud ECS security groups for Qdrant vector database integration deployment, involving port opening, firewall rules, and security best practices.
红烧肉做法Braised Pork Recipe
已完成Completed📄 4 条记忆📄 4 memories03/04
用户询问红烧肉的详细做法,助手提供了从选材到烹饪的完整步骤,包括入口即化的关键技巧。User asked for detailed braised pork recipe. Assistant provided complete steps from ingredient selection to cooking, including key tips for melt-in-mouth texture.
工作经历整理Work Experience Summary
已完成Completed📄 8 条记忆📄 8 memories03/05
整理和结构化用户的工作经历信息,用于简历和职业规划。涵盖2018-2021阿里云高级工程师及近期AI Agent项目。Organized and structured user's work experience for resume and career planning. Covers 2018-2021 Alibaba Cloud senior engineer and recent AI Agent projects.
用户探索 OpenClaw 的插件系统和 MemOS 记忆管理功能,包括安装配置、Viewer 使用和记忆迁移。User explored OpenClaw's plugin system and MemOS memory management features, including installation, Viewer usage, and memory migration.
2总技能Total
🧠 memos-memory-guide
生效中Activev1质量: 8.5Quality: 8.5已安装Installed
Agent 记忆工具使用指南 — 指导 Agent 何时使用 memory_search、memory_timeline、task_summary、skill_get 等工具,自动优化召回策略。Agent memory tool usage guide — guides the Agent on when to use memory_search, memory_timeline, task_summary, skill_get tools, auto-optimizing recall strategy.
⚡ cloud-infrastructure-setup
生效中Activev2质量: 7.8Quality: 7.8已安装Installed
从多次云基础设施配置对话中提炼的技能 — 安全组配置、端口管理、服务部署的标准化流程与踩坑警告。Skill distilled from multiple cloud infrastructure conversations — standardized processes for security groups, port management, service deployment, and pitfall warnings.
为 OpenClaw 提供完全本地化的持久记忆、智能任务总结、技能自动进化和多智能体协同。npm 一键安装,支持分级模型配置。Fully local persistent memory, smart task summarization, auto skill evolution, and multi-agent collaboration for OpenClaw. One-command install, tiered model support.
完全本地化:数据存于本机 SQLite,零云依赖。Viewer 仅 127.0.0.1,密码保护。Fully local: Data in local SQLite, zero cloud dependency. Viewer 127.0.0.1 only, password-protected.
💾
全量写入Full-Write
每次对话自动捕获,语义分片后持久化。Auto-captures every conversation, chunks semantically.
⚡
任务总结与技能进化Tasks & Skills
碎片对话归纳为结构化任务,再提炼为可复用技能并持续升级。Conversations organized into tasks, then distilled into skills that auto-upgrade.
每轮自动:before_agent_start 用用户消息检索 → LLM 过滤相关 → 注入 system 上下文;无结果时提示 agent 自生成 query 调 memory_search。Per turn: before_agent_start searches with user message → LLM filters relevant → inject system context; if no hits, hint agent to call memory_search with self-generated query.
Embedding / Summarizer API 可选,不配自动用本地模型Embedding / Summarizer APIs optional, falls back to local
Step 0:安装 C++ 编译工具(macOS / Linux 推荐)Step 0: Install C++ Build Tools (macOS / Linux recommended)
插件依赖 better-sqlite3 原生模块。macOS / Linux 用户建议先安装编译工具,可大幅提升安装成功率。Windows 用户使用 Node.js LTS 版本时通常有预编译文件,可直接跳到 Step 1。The plugin depends on better-sqlite3, a native C/C++ module. macOS / Linux users should install build tools first. Windows users with Node.js LTS usually have prebuilt binaries and can skip to Step 1.
安装失败?最常见的问题是 better-sqlite3 原生模块编译失败。请确认已执行上方 Step 0,然后手动重建:cd ~/.openclaw/extensions/memos-local-openclaw-plugin && npm rebuild better-sqlite3。更多方案请查看 安装排查指南 或 better-sqlite3 官方文档。Install failed? The most common issue is better-sqlite3 compilation failure. Ensure Step 0 is done, then manually rebuild: cd ~/.openclaw/extensions/memos-local-openclaw-plugin && npm rebuild better-sqlite3. See the troubleshooting guide or official better-sqlite3 docs for more solutions.
升级自动完成依赖安装、旧版清理和原生模块编译,无需手动操作。如果 update 命令不可用,先删除旧目录再重新安装:rm -rf ~/.openclaw/extensions/memos-local-openclaw-plugin && openclaw plugins install @memtensor/memos-local-openclaw-plugin(记忆数据不受影响)。Upgrade automatically handles dependencies, legacy cleanup, and native module compilation. If update is unavailable, delete the old directory first: rm -rf ~/.openclaw/extensions/memos-local-openclaw-plugin && openclaw plugins install @memtensor/memos-local-openclaw-plugin (memory data is stored separately and won't be affected).
配置Configuration
两种方式:编辑 openclaw.json 或通过 Viewer 网页面板在线修改。支持分级模型。Two methods: edit openclaw.json or via Viewer web panel. Tiered models supported.
安装后每次对话自动存入记忆。访问 http://127.0.0.1:18799 使用 Viewer。Every conversation auto-stored. Visit http://127.0.0.1:18799 for Viewer.
🦐 记忆迁移 — 再续前缘🦐 Memory Migration — Reconnect
将 OpenClaw 原生内置的记忆数据(SQLite 存储的对话历史)无缝迁移到 MemOS 的智能记忆系统。你和 AI 共同积累的每一段对话,都值得被记住。Seamlessly migrate OpenClaw's native built-in memory data (SQLite conversation history) to MemOS's intelligent memory system. Every conversation you've built with AI deserves to be remembered.
技能进化:从已完成的任务中提炼可复用技能,生成 SKILL.md 文件并安装到工作区。Skill evolution: Distill reusable skills from completed tasks, generate SKILL.md and install to workspace.
后处理在同一 Agent 内串行执行,不同 Agent 之间可并行(并发度可配置 1–8)。已处理过的会话自动跳过。支持选择只生成任务、只生成技能或两者同时执行。Post-processing runs serially within each agent, with parallel processing across agents (configurable concurrency 1–8). Already processed sessions are auto-skipped. Choose task-only, skill-only, or both.
断点续传Resume & Stop
导入和后处理均支持随时暂停:Both import and post-processing support pause/resume:
点击 停止 按钮后,进度自动保存。Click Stop, progress auto-saved.
刷新页面后自动检测未完成的导入,恢复进度条显示。On page refresh, auto-detect incomplete imports and restore progress display.
再次点击开始即从上次中断处继续,已处理的记忆自动跳过。Click start again to continue from where you left off — processed memories are auto-skipped.
导入和后处理在后台运行,关闭 Viewer 页面不影响执行。Import and post-processing run in the background — closing the Viewer page won't interrupt them.
🦐 来源标识:所有通过迁移导入的记忆都带有 🦐 标识,在 Viewer 的记忆列表中可一眼区分原生导入和对话生成的记忆。🦐 Source Tag: All migrated memories are tagged with 🦐, making them visually distinguishable from conversation-generated memories in the Viewer.
skill_publish 将技能设为公开,其他 Agent 可通过 skill_search 发现并安装。skill_unpublish 设为私有。skill_publish makes a skill public and discoverable via skill_search. skill_unpublish sets it private.skillId (required).
MemOS 原生支持多 Agent 场景。每个 Agent 的记忆和任务通过 owner 字段隔离(格式 agent:{agentId}),检索时自动过滤为当前 Agent + public。MemOS natively supports multi-agent scenarios. Each agent's memories and tasks are isolated via an owner field (agent:{agentId}); retrieval automatically filters to current agent + public.
记忆隔离:Agent A 无法检索 Agent B 的私有记忆Memory Isolation: Agent A cannot retrieve Agent B's private memories
公共记忆:通过 memory_write_public 写入 owner="public" 的记忆,所有 Agent 可检索Public Memory: Use memory_write_public to write owner="public" memories discoverable by all agents
技能共享:通过 skill_publish 将技能设为公开,其他 Agent 可通过 skill_search 发现并安装Skill Sharing: Use skill_publish to make skills public; other agents discover and install via skill_search
Error: Could not locate the bindings file. Tried:
→ .../node_modules/better-sqlite3/build/better_sqlite3.node
→ .../node_modules/better-sqlite3/build/Release/better_sqlite3.node
...
让你的 OpenClaw 越用越聪明Give Your OpenClaw Lasting Intelligence
为 OpenClaw 注入持久记忆与自进化技能 完全本地化 全量可视化管理 分级模型极致省钱Persistent memory and self-evolving skills for OpenClaw agents. 100% local storage, full visualization dashboard, and tiered models for cost efficiency.
把 MemOS 带进你的 OpenClawBring MemOS to your OpenClaw workflow
没有记忆的 Agent,每次都从零开始Without Memory, Every Task Starts from Zero
MemOS 为 OpenClaw 注入持久记忆与自进化技能。MemOS equips OpenClaw with persistent memory and self-evolving skills.
💻
完全本地化Fully Local
记忆、任务、技能全存本机 SQLite,零云依赖。All data stored in local SQLite — zero cloud dependency, complete privacy.
🧠
全量可视化管理Full Visualization
内置管理面板,记忆 / 任务 / 技能完全透明可控。Built-in web dashboard — memories, tasks, and skills fully transparent and controllable.
⚡
任务总结与技能进化Task Summary & Skill Evolution
碎片对话自动归纳为结构化任务,再提炼为可复用技能并持续升级。从「记住」到「学会」,同一个坑不踩两次。Fragmented conversations auto-organized into structured tasks, then distilled into reusable skills that evolve over time. From "remembering" to "mastering" — never repeat the same mistake twice.
💰
分级模型 · 省钱Tiered Models
Embedding 轻量、摘要中等、技能高质量——按需分配,大幅省钱。Lightweight, mid-tier, and high-quality models layered by purpose — maximum performance at minimum cost.
🤝
多智能体协同Multi-Agent Collaboration
记忆隔离 + 公共记忆 + 技能共享。多个 Agent 各有私域记忆,又能共享知识与技能,协同进化。Memory isolation + public memory + skill sharing. Each agent has private memories while sharing knowledge and skills for collective evolution.
🦞
OpenClaw 原生记忆导入Native Memory Import
一键迁移 OpenClaw 内置记忆,智能去重、断点续传、实时进度。你过往的记忆不会丢失,再续前缘。One-click migration from OpenClaw built-in memories. Smart dedup, resume anytime, real-time progress. Your past memories, never lost.
三大引擎,驱动 Agent 协同进化Three Engines That Drive Collaborative Evolution
任务总结与技能自进化Task Summary & Skill Evolution
碎片对话自动归组为结构化任务(目标 → 步骤 → 结果),再由 LLM 评估提炼为可复用技能。遇到相似场景时自动升级——更快、更准、更省 Token。从「能记住」到「会做」,同一个坑不踩两次。任务与技能支持编辑、删除、重试等完整管理。Fragmented conversations are auto-organized into structured tasks (goal → steps → result), then LLM evaluates and distills them into reusable skills. Skills auto-upgrade on similar scenarios — faster, more accurate, lower cost. From "remembering" to "mastering" — never repeat the same mistake. Full CRUD for tasks and skills.
每个 Agent 拥有独立的私域记忆,互不可见。但通过「公共记忆」和「技能共享」机制,Agent 之间能够共享决策、经验与能力。一个 Agent 学会的技能,可以发布为公共技能,其他 Agent 搜索并安装后即可复用。多智能体不再各自为战,而是协同进化、共同进步。Each agent has isolated private memory, invisible to others. But through public memory and skill sharing, agents can share decisions, experiences, and capabilities. Skills learned by one agent can be published for others to discover and install. Multi-agent systems no longer work in silos — they evolve collaboratively, growing together.
OpenAI 兼容 API 即插即用,无配置自动降级本地模型。Any OpenAI-compatible API works out of the box. Automatic fallback to local models when no API key is configured.
OpenAI
Anthropic
Gemini
Bedrock
Cohere
Voyage
Mistral
本地Local
12 个智能工具12 Smart Tools
🧠
auto_recall
每轮自动回忆Auto recall each turn
🔍
memory_search
记忆检索Memory search
📄
memory_get
获取完整记忆Get full memory
📜
memory_timeline
上下文邻居Context neighbors
📢
memory_write_public
写入公共记忆Write public memory
📋
task_summary
任务摘要Task summary
⚡
skill_get
技能指南Skill guide
📦
skill_install
安装技能Install skill
🔎
skill_search
技能发现Skill discovery
🌍
skill_publish
公开技能Publish skill
🔒
skill_unpublish
取消公开Unpublish skill
🌐
memory_viewer
管理面板Dashboard
🦞OpenClaw 原生记忆导入OpenClaw Native Memory Import
再续前缘 — 过往的记忆,不会丢失Reconnect — Your Past Memories, Never Lost
从 OpenClaw 原生 SQLite 和会话记录中无缝迁移,智能去重、自动摘要、技能生成一气呵成。你和 AI 共同积累的每一段对话,都值得被记住。Seamlessly migrate from OpenClaw's native SQLite and session logs. Smart deduplication, auto-summarization, and skill generation — all in one flow. Every conversation you've built with your AI deserves to be preserved.
🚀
一键迁移One-Click Import
自动扫描 OpenClaw 原生记忆文件,一键启动导入,实时显示进度与统计。Automatically scans OpenClaw native memory files. Start import with one click and monitor real-time progress.
🧬
智能去重Smart Dedup
向量相似度 + LLM 判断双重去重,相似内容自动合并,不留冗余。Vector similarity combined with LLM judgment for dual-layer deduplication. Similar content is automatically merged with zero redundancy.
⏸️
断点续传Resume Anytime
支持随时暂停,刷新页面后自动恢复进度。后台持续运行,已处理的自动跳过。Pause anytime and auto-resume on page refresh. Runs in the background, automatically skipping already processed items.
⚡
任务与技能生成Task & Skill Gen
导入后可选生成任务摘要和技能进化,同一 Agent 内串行处理,不同 Agent 之间并行(可配置 1–8 并发度),支持暂停和断点续传。Optionally generate task summaries and evolve skills. Serial within each agent, parallel across agents (configurable 1–8 concurrency), with full pause and resume support.
沉浸体验完整流程Experience the Complete Workflow
从记忆导入到智能检索再到可视化管理,一站式体验 MemOS 的核心能力。From memory import to smart retrieval to visual management — explore MemOS's core capabilities in an interactive demo.
================================================
FILE: apps/openwork-memos-integration/.gitignore
================================================
node_modules/
dist/
out/
.env
.env.local
# OS files
.DS_Store
Thumbs.db
# Lock files
pnpm-lock.yaml
package-lock.json
bun.lock
# Binary assets (fonts, large images, videos)
*.ttf
*.woff
*.woff2
*.mp4
*.webm
public/assets/usecases/
docs/video-thumbnail.png
# Build artifacts
*.tsbuildinfo
================================================
FILE: apps/openwork-memos-integration/CLAUDE.md
================================================
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Openwork is a standalone desktop automation assistant built with Electron. The app hosts a local React UI (bundled via Vite), communicating with the main process through `contextBridge` IPC. The main process spawns the OpenCode CLI (via `node-pty`) to execute user tasks. Users provide their own API key (Anthropic, OpenAI, Google, or xAI) on first launch, stored securely in the OS keychain.
## Common Commands
```bash
pnpm dev # Run desktop app in dev mode (Vite + Electron)
pnpm dev:clean # Dev mode with CLEAN_START=1 (clears stored data)
pnpm build # Build all workspaces
pnpm build:desktop # Build desktop app only
pnpm lint # TypeScript checks
pnpm typecheck # Type validation
pnpm clean # Clean build outputs and node_modules
pnpm -F @accomplish/desktop test:e2e # Playwright E2E tests
pnpm -F @accomplish/desktop test:e2e:ui # E2E with Playwright UI
pnpm -F @accomplish/desktop test:e2e:debug # E2E in debug mode
```
## Architecture
### Monorepo Layout
```
apps/desktop/ # Electron app (main/preload/renderer)
packages/shared/ # Shared TypeScript types
```
### Desktop App Structure (`apps/desktop/src/`)
**Main Process** (`main/`):
- `index.ts` - Electron bootstrap, single-instance enforcement, `accomplish://` protocol handler
- `ipc/handlers.ts` - IPC handlers for task lifecycle, settings, onboarding, API keys
- `opencode/adapter.ts` - OpenCode CLI wrapper using `node-pty`, streams output and handles permissions
- `store/secureStorage.ts` - API key storage via `keytar` (OS keychain)
- `store/appSettings.ts` - App settings via `electron-store` (debug mode, onboarding state)
- `store/taskHistory.ts` - Task history persistence
**Preload** (`preload/index.ts`):
- Exposes `window.accomplish` API via `contextBridge`
- Provides typed IPC methods for task operations, settings, events
**Renderer** (`renderer/`):
- `main.tsx` - React entry with HashRouter
- `App.tsx` - Main routing + onboarding gate
- `pages/` - Home, Execution, History, Settings pages
- `stores/taskStore.ts` - Zustand store for task/UI state
- `lib/accomplish.ts` - Typed wrapper for the IPC API
### IPC Communication Flow
```
Renderer (React)
↓ window.accomplish.* calls
Preload (contextBridge)
↓ ipcRenderer.invoke
Main Process
↓ Native APIs (keytar, node-pty, electron-store)
↑ IPC events
Preload
↑ ipcRenderer.on callbacks
Renderer
```
### Key Dependencies
- `node-pty` - PTY for OpenCode CLI spawning
- `keytar` - Secure API key storage (OS keychain)
- `electron-store` - Local settings/preferences
- `opencode-ai` - Bundled OpenCode CLI (multi-provider: Anthropic, OpenAI, Google, xAI)
## Code Conventions
- TypeScript everywhere (no JS for app logic)
- Use `pnpm -F @accomplish/desktop ...` for desktop-specific commands
- Shared types go in `packages/shared/src/types/`
- Renderer state via Zustand store actions
- IPC handlers in `src/main/ipc/handlers.ts` must match `window.accomplish` API in preload
### Image Assets in Renderer
**IMPORTANT:** Always use ES module imports for images in the renderer, never absolute paths.
```typescript
// CORRECT - Use ES imports
import logoImage from '/assets/logo.png';
// WRONG - Absolute paths break in packaged app
```
**Why:** In development, Vite serves `/assets/...` from the public folder. But in the packaged Electron app, the renderer loads via `file://` protocol, and absolute paths like `/assets/logo.png` resolve to the filesystem root instead of the app bundle. ES imports are processed by Vite to use `import.meta.url`, which works correctly in both environments.
Static assets go in `apps/desktop/public/assets/`.
## Environment Variables
- `CLEAN_START=1` - Clear all stored data on app start
- `E2E_SKIP_AUTH=1` - Skip onboarding flow (for testing)
## Testing
- E2E tests: `pnpm -F @accomplish/desktop test:e2e`
- Tests use Playwright with serial execution (Electron requirement)
- Test config: `apps/desktop/playwright.config.ts`
## Bundled Node.js
The packaged app bundles standalone Node.js v20.18.1 binaries to ensure MCP servers work on machines without Node.js installed.
### Key Files
- `src/main/utils/bundled-node.ts` - Utility to get bundled node/npm/npx paths
- `scripts/download-nodejs.cjs` - Downloads Node.js binaries for all platforms
- `scripts/after-pack.cjs` - Copies correct binary into app bundle during build
### CRITICAL: Spawning npx/node in Main Process
**IMPORTANT:** When spawning `npx` or `node` in the main process, you MUST add the bundled Node.js bin directory to PATH. This is because `npx` uses a `#!/usr/bin/env node` shebang which looks for `node` in PATH.
```typescript
import { spawn } from 'child_process';
import { getNpxPath, getBundledNodePaths } from '../utils/bundled-node';
// Get bundled paths
const npxPath = getNpxPath();
const bundledPaths = getBundledNodePaths();
// Build environment with bundled node in PATH
let spawnEnv: NodeJS.ProcessEnv = { ...process.env };
if (bundledPaths) {
const delimiter = process.platform === 'win32' ? ';' : ':';
spawnEnv.PATH = `${bundledPaths.binDir}${delimiter}${process.env.PATH || ''}`;
}
// Spawn with the modified environment
spawn(npxPath, ['-y', 'some-package@latest'], {
stdio: ['pipe', 'pipe', 'pipe'],
env: spawnEnv,
});
```
**Why:** Without adding `bundledPaths.binDir` to PATH, the spawned process will fail with exit code 127 ("node not found") on machines that don't have Node.js installed system-wide.
### For MCP Server Configs
When generating MCP server configurations, pass `NODE_BIN_PATH` in the environment so spawned servers can add it to their PATH:
```typescript
environment: {
NODE_BIN_PATH: bundledPaths?.binDir || '',
}
```
## Key Behaviors
- Single-instance enforcement - second instance focuses existing window
- API keys stored in OS keychain (macOS Keychain, Windows Credential Vault, Linux Secret Service)
- API key validation via test request to respective provider API
- OpenCode CLI permissions are bridged to UI via IPC `permission:request` / `permission:respond`
- Task output streams through `task:update` and `task:progress` IPC events
================================================
FILE: apps/openwork-memos-integration/CONTRIBUTING.md
================================================
# Contributing to Openwork
Thank you for your interest in contributing to Openwork! This document provides guidelines and instructions for contributing.
## Getting Started
1. Fork the repository
2. Clone your fork: `git clone https://github.com/YOUR_USERNAME/openwork.git`
3. Install dependencies: `pnpm install`
4. Create a branch: `git checkout -b feature/your-feature-name`
## Development
```bash
pnpm dev # Run the desktop app in development mode
pnpm build # Build all workspaces
pnpm typecheck # Run TypeScript checks
pnpm lint # Run linting
```
## Code Style
- TypeScript for all application code
- Follow existing patterns in the codebase
- Use meaningful variable and function names
- Keep functions focused and small
## Pull Request Process
1. Ensure your code builds without errors (`pnpm build`)
2. Run type checking (`pnpm typecheck`)
3. Update documentation if needed
4. Write a clear PR description explaining:
- What the change does
- Why it's needed
- How to test it
## Commit Messages
Use clear, descriptive commit messages:
- `feat: add dark mode support`
- `fix: resolve crash on startup`
- `docs: update README with new instructions`
- `refactor: simplify task queue logic`
## Reporting Issues
When reporting issues, please include:
- OS and version
- Steps to reproduce
- Expected vs actual behavior
- Any error messages or logs
## Security
If you discover a security vulnerability, please see [SECURITY.md](SECURITY.md) for responsible disclosure guidelines.
## License
By contributing, you agree that your contributions will be licensed under the MIT License.
================================================
FILE: apps/openwork-memos-integration/LICENSE
================================================
MIT License
Copyright (c) 2026 Accomplish Inc
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: apps/openwork-memos-integration/README.md
================================================
# Openwork™ - Open Source AI Desktop Agent
Openwork is an open source AI desktop agent that automates file management, document creation, and browser tasks locally on your machine. Bring your own API keys (OpenAI, Anthropic, Google, xAI) or run local models via Ollama.
Runs locally on your machine. Bring your own API keys or local models. MIT licensed.
- Your files stay on your machine
- You decide which folders it can touch
- Nothing gets sent to Openwork (or anyone else)
### 🔑 You bring your own AI
- Use your own API key (OpenAI, Anthropic, etc.)
- Or run with [Ollama](https://ollama.com) (no API key needed)
- No subscription, no upsell
- It's a tool—not a service
### 📖 It's open source
- Every line of code is on GitHub
- MIT licensed
- Change it, fork it, break it, fix it
---
## What it actually does
| | | |
|:--|:--|:--|
| **📁 File Management** | **✍️ Document Writing** | **🔗 Tool Connections** |
| Sort, rename, and move files based on content or rules you give it | Prompt it to write, summarize, or rewrite documents | Works with Notion, Google Drive, Dropbox, and more (through local APIs) |
| | | |
| **⚙️ Custom Skills** | **🛡️ Full Control** | |
| Define repeatable workflows, save them as skills | You approve every action. You can see logs. You can stop it anytime. | |
## Use cases
- Clean up messy folders by project, file type, or date
- Draft, summarize, and rewrite docs, reports, and meeting notes
- Automate browser workflows like research and form entry
- Generate weekly updates from files and notes
- Prepare meeting materials from docs and calendars
## Memory (MemOS)
Openwork can connect to MemOS to provide long-term memory. When a MemOS API key is set, relevant memories are injected into the system prompt and new memories are saved after tasks finish. Learn more in the MemOS docs: https://memos-docs.openmem.net/
## Supported models and providers
- OpenAI
- Anthropic
- Google
- xAI
- Ollama (local models)
## Privacy and local-first
Openwork runs locally on your machine. Your files stay on your device, and you choose which folders it can access.
## System requirements
- macOS (Apple Silicon)
- Windows support coming soon
---
## How to use it
> **Takes 2 minutes to set up.**
| Step | Action | Details |
|:----:|--------|---------|
| **1** | **Install the App** | Download the DMG and drag it into Applications |
| **2** | **Connect Your AI** | Use your own OpenAI or Anthropic API key, or Ollama. No subscriptions. |
| **3** | **Give It Access** | Choose which folders it can see. You stay in control. |
| **4** | **Start Working** | Ask it to summarize a doc, clean a folder, or create a report. You approve everything. |
[**Download for Mac (Apple Silicon)**](https://downloads.openwork.me/downloads/0.2.1/macos/Openwork-0.2.1-mac-arm64.dmg)
---
## Screenshots and Demo
A quick look at Openwork on macOS, plus a short demo video.
## FAQ
**Does Openwork run locally?**
Yes. Openwork runs locally on your machine and you control which folders it can access.
**Do I need an API key?**
You can use your own API keys (OpenAI, Anthropic, Google, xAI) or run local models via Ollama.
**Is Openwork free?**
Yes. Openwork is open source and MIT licensed.
**Which platforms are supported?**
macOS (Apple Silicon) is available now. Windows support is coming soon.
---
## Development
```bash
pnpm install
pnpm dev
```
That's it.
Prerequisites
- Node.js 20+
- pnpm 9+
All Commands
| Command | Description |
|---------|-------------|
| `pnpm dev` | Run desktop app in dev mode |
| `pnpm dev:clean` | Dev mode with clean start |
| `pnpm build` | Build all workspaces |
| `pnpm build:desktop` | Build desktop app only |
| `pnpm lint` | TypeScript checks |
| `pnpm typecheck` | Type validation |
| `pnpm -F @accomplish/desktop test:e2e` | Playwright E2E tests |
Environment Variables
| Variable | Description |
|----------|-------------|
| `CLEAN_START=1` | Clear all stored data on app start |
| `E2E_SKIP_AUTH=1` | Skip onboarding flow (for testing) |
Architecture
```
apps/
desktop/ # Electron app (main + preload + renderer)
packages/
shared/ # Shared TypeScript types
```
The desktop app uses Electron with a React UI bundled via Vite. The main process spawns [OpenCode](https://github.com/sst/opencode) CLI using `node-pty` to execute tasks. API keys are stored securely in the OS keychain.
See [CLAUDE.md](CLAUDE.md) for detailed architecture documentation.
---
## Contributing
Contributions welcome! Feel free to open a PR.
```bash
# Fork → Clone → Branch → Commit → Push → PR
git checkout -b feature/amazing-feature
git commit -m 'Add amazing feature'
git push origin feature/amazing-feature
```
---
**[Openwork website](https://www.openwork.me/)** · **[Openwork blog](https://www.openwork.me/blog/)** · **[Openwork releases](https://github.com/accomplish-ai/openwork/releases)** · **[Issues](https://github.com/accomplish-ai/openwork/issues)** · **[Twitter](https://x.com/openwork_ai)**
MIT License · Built by [Openwork](https://www.openwork.me)
**Keywords:** AI agent, AI desktop agent, desktop automation, file management, document creation, browser automation, local-first, macOS, privacy-first, open source, Electron, computer use, AI assistant, workflow automation, OpenAI, Anthropic, Google, xAI, Claude, GPT-4, Ollama
================================================
FILE: apps/openwork-memos-integration/SECURITY.md
================================================
# Security Policy
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 0.1.x | :white_check_mark: |
## Reporting a Vulnerability
We take security seriously. If you discover a security vulnerability, please report it responsibly.
### How to Report
1. **Do not** open a public GitHub issue for security vulnerabilities
2. Email security concerns to the maintainers (see GitHub profile)
3. Include:
- Description of the vulnerability
- Steps to reproduce
- Potential impact
- Any suggested fixes (optional)
### What to Expect
- Acknowledgment within 48 hours
- Regular updates on progress
- Credit in release notes (if desired)
### Scope
Security issues we're interested in:
- Remote code execution
- Local privilege escalation
- Data exposure
- Authentication/authorization bypasses
- IPC security issues
Out of scope:
- Denial of service
- Social engineering
- Issues requiring physical access
## Security Best Practices
When using Openwork:
- Keep the application updated
- Only grant file permissions when necessary
- Review task outputs before approving sensitive operations
- Use API keys with minimal required permissions
================================================
FILE: apps/openwork-memos-integration/apps/desktop/.eslintrc.json
================================================
{
"root": true,
"env": {
"browser": true,
"es2021": true,
"node": true
},
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": "latest",
"sourceType": "module"
},
"settings": {
"react": {
"version": "detect"
}
},
"plugins": [
"@typescript-eslint",
"react",
"react-hooks"
],
"extends": [
"eslint:recommended",
"plugin:react/recommended",
"plugin:@typescript-eslint/recommended",
"plugin:react-hooks/recommended"
],
"ignorePatterns": [
"dist",
"dist-electron",
"release",
"node_modules"
],
"rules": {
"react/react-in-jsx-scope": "off",
"react/prop-types": "off"
}
}
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/appSettings.integration.test.ts
================================================
/**
* Integration tests for appSettings store
* Tests real electron-store interactions with temporary directories
* @module __tests__/integration/main/appSettings.integration.test
*/
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
// Create a unique temp directory for each test run
let tempDir: string;
let originalCwd: string;
describe('appSettings Integration', () => {
beforeEach(async () => {
// Create a unique temp directory for each test
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'appSettings-test-'));
originalCwd = process.cwd();
// Reset module cache first
vi.resetModules();
// Use doMock (not hoisted) so tempDir is captured with current value
vi.doMock('electron', () => ({
app: {
getPath: (name: string) => {
if (name === 'userData') {
return tempDir;
}
return `/mock/path/${name}`;
},
getVersion: () => '0.1.0',
getName: () => 'Accomplish',
isPackaged: false,
},
}));
});
afterEach(() => {
// Clean up temp directory
if (tempDir && fs.existsSync(tempDir)) {
fs.rmSync(tempDir, { recursive: true, force: true });
}
process.chdir(originalCwd);
});
describe('debugMode', () => {
it('should return false as default value for debugMode', async () => {
// Arrange
const { getDebugMode, clearAppSettings } = await import('@main/store/appSettings');
clearAppSettings(); // Ensure fresh state
// Act
const result = getDebugMode();
// Assert
expect(result).toBe(false);
});
it('should persist debugMode after setting to true', async () => {
// Arrange
const { getDebugMode, setDebugMode } = await import('@main/store/appSettings');
// Act
setDebugMode(true);
const result = getDebugMode();
// Assert
expect(result).toBe(true);
});
it('should persist debugMode after setting to false', async () => {
// Arrange
const { getDebugMode, setDebugMode } = await import('@main/store/appSettings');
// Act - set to true first, then false
setDebugMode(true);
setDebugMode(false);
const result = getDebugMode();
// Assert
expect(result).toBe(false);
});
it('should round-trip debugMode value correctly', async () => {
// Arrange
const { getDebugMode, setDebugMode } = await import('@main/store/appSettings');
// Act
setDebugMode(true);
const afterTrue = getDebugMode();
setDebugMode(false);
const afterFalse = getDebugMode();
setDebugMode(true);
const afterTrueAgain = getDebugMode();
// Assert
expect(afterTrue).toBe(true);
expect(afterFalse).toBe(false);
expect(afterTrueAgain).toBe(true);
});
});
describe('onboardingComplete', () => {
it('should return false as default value for onboardingComplete', async () => {
// Arrange
const { getOnboardingComplete, clearAppSettings } = await import('@main/store/appSettings');
clearAppSettings(); // Ensure fresh state
// Act
const result = getOnboardingComplete();
// Assert
expect(result).toBe(false);
});
it('should persist onboardingComplete after setting to true', async () => {
// Arrange
const { getOnboardingComplete, setOnboardingComplete } = await import('@main/store/appSettings');
// Act
setOnboardingComplete(true);
const result = getOnboardingComplete();
// Assert
expect(result).toBe(true);
});
it('should round-trip onboardingComplete value correctly', async () => {
// Arrange
const { getOnboardingComplete, setOnboardingComplete } = await import('@main/store/appSettings');
// Act
setOnboardingComplete(true);
const afterTrue = getOnboardingComplete();
setOnboardingComplete(false);
const afterFalse = getOnboardingComplete();
// Assert
expect(afterTrue).toBe(true);
expect(afterFalse).toBe(false);
});
});
describe('selectedModel', () => {
it('should return default model on fresh store', async () => {
// Arrange
const { getSelectedModel, clearAppSettings } = await import('@main/store/appSettings');
clearAppSettings(); // Ensure fresh state
// Act
const result = getSelectedModel();
// Assert
expect(result).toEqual({
provider: 'anthropic',
model: 'anthropic/claude-opus-4-5',
});
});
it('should persist selectedModel after setting new value', async () => {
// Arrange
const { getSelectedModel, setSelectedModel } = await import('@main/store/appSettings');
const newModel = { provider: 'openai', model: 'gpt-4' };
// Act
setSelectedModel(newModel);
const result = getSelectedModel();
// Assert
expect(result).toEqual(newModel);
});
it('should round-trip different model values correctly', async () => {
// Arrange
const { getSelectedModel, setSelectedModel } = await import('@main/store/appSettings');
const model1 = { provider: 'anthropic', model: 'claude-3-opus' };
const model2 = { provider: 'google', model: 'gemini-pro' };
const model3 = { provider: 'xai', model: 'grok-4' };
// Act & Assert
setSelectedModel(model1);
expect(getSelectedModel()).toEqual(model1);
setSelectedModel(model2);
expect(getSelectedModel()).toEqual(model2);
setSelectedModel(model3);
expect(getSelectedModel()).toEqual(model3);
});
});
describe('getAppSettings', () => {
it('should return all default settings on fresh store', async () => {
// Arrange
const { getAppSettings, clearAppSettings } = await import('@main/store/appSettings');
clearAppSettings(); // Ensure fresh state
// Act
const result = getAppSettings();
// Assert
expect(result).toEqual({
debugMode: false,
onboardingComplete: false,
ollamaConfig: null,
litellmConfig: null,
selectedModel: {
provider: 'anthropic',
model: 'anthropic/claude-opus-4-5',
},
});
});
it('should return all settings after modifications', async () => {
// Arrange
const { getAppSettings, setDebugMode, setOnboardingComplete, setSelectedModel, clearAppSettings } = await import('@main/store/appSettings');
clearAppSettings(); // Start fresh
const customModel = { provider: 'openai', model: 'gpt-4-turbo' };
// Act
setDebugMode(true);
setOnboardingComplete(true);
setSelectedModel(customModel);
const result = getAppSettings();
// Assert
expect(result).toEqual({
debugMode: true,
onboardingComplete: true,
ollamaConfig: null,
litellmConfig: null,
selectedModel: customModel,
});
});
it('should reflect partial modifications correctly', async () => {
// Arrange
const { getAppSettings, setDebugMode, clearAppSettings } = await import('@main/store/appSettings');
clearAppSettings(); // Start fresh
// Act - only modify debugMode
setDebugMode(true);
const result = getAppSettings();
// Assert
expect(result.debugMode).toBe(true);
expect(result.onboardingComplete).toBe(false);
expect(result.selectedModel).toEqual({
provider: 'anthropic',
model: 'anthropic/claude-opus-4-5',
});
});
});
describe('clearAppSettings', () => {
it('should reset all settings to defaults', async () => {
// Arrange
const {
getAppSettings,
clearAppSettings,
setDebugMode,
setOnboardingComplete,
setSelectedModel
} = await import('@main/store/appSettings');
// Set custom values
setDebugMode(true);
setOnboardingComplete(true);
setSelectedModel({ provider: 'openai', model: 'gpt-4' });
// Act
clearAppSettings();
const result = getAppSettings();
// Assert
expect(result).toEqual({
debugMode: false,
onboardingComplete: false,
ollamaConfig: null,
litellmConfig: null,
selectedModel: {
provider: 'anthropic',
model: 'anthropic/claude-opus-4-5',
},
});
});
it('should reset debugMode to default after clear', async () => {
// Arrange
const { getDebugMode, setDebugMode, clearAppSettings } = await import('@main/store/appSettings');
// Act
setDebugMode(true);
expect(getDebugMode()).toBe(true);
clearAppSettings();
const result = getDebugMode();
// Assert
expect(result).toBe(false);
});
it('should reset onboardingComplete to default after clear', async () => {
// Arrange
const { getOnboardingComplete, setOnboardingComplete, clearAppSettings } = await import('@main/store/appSettings');
// Act
setOnboardingComplete(true);
expect(getOnboardingComplete()).toBe(true);
clearAppSettings();
const result = getOnboardingComplete();
// Assert
expect(result).toBe(false);
});
it('should reset selectedModel to default after clear', async () => {
// Arrange
const { getSelectedModel, setSelectedModel, clearAppSettings } = await import('@main/store/appSettings');
// Act
setSelectedModel({ provider: 'openai', model: 'gpt-4' });
expect(getSelectedModel()).toEqual({ provider: 'openai', model: 'gpt-4' });
clearAppSettings();
const result = getSelectedModel();
// Assert
expect(result).toEqual({
provider: 'anthropic',
model: 'anthropic/claude-opus-4-5',
});
});
it('should allow setting new values after clear', async () => {
// Arrange
const { getDebugMode, setDebugMode, clearAppSettings } = await import('@main/store/appSettings');
// Act
setDebugMode(true);
clearAppSettings();
setDebugMode(true);
const result = getDebugMode();
// Assert
expect(result).toBe(true);
});
});
describe('persistence across module reloads', () => {
it('should persist values to disk and survive module reload', async () => {
// Arrange - first import and set values
const module1 = await import('@main/store/appSettings');
module1.setDebugMode(true);
module1.setOnboardingComplete(true);
module1.setSelectedModel({ provider: 'google', model: 'gemini-ultra' });
// Act - reset modules and reimport
vi.resetModules();
const module2 = await import('@main/store/appSettings');
// Assert - values should be persisted
expect(module2.getDebugMode()).toBe(true);
expect(module2.getOnboardingComplete()).toBe(true);
expect(module2.getSelectedModel()).toEqual({ provider: 'google', model: 'gemini-ultra' });
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/opencode/cli-path.integration.test.ts
================================================
/**
* Integration tests for OpenCode CLI path resolution
*
* Tests the cli-path module which resolves paths to the OpenCode CLI binary
* in both development and packaged app modes.
*
* @module __tests__/integration/main/opencode/cli-path.integration.test
*/
import { describe, it, expect, beforeEach, vi, afterEach } from 'vitest';
import path from 'path';
// Mock electron module before importing the module under test
const mockApp = {
isPackaged: false,
getAppPath: vi.fn(() => '/mock/app/path'),
};
vi.mock('electron', () => ({
app: mockApp,
}));
// Mock fs module
const mockFs = {
existsSync: vi.fn(),
readdirSync: vi.fn(),
readFileSync: vi.fn(),
};
vi.mock('fs', () => ({
default: mockFs,
existsSync: mockFs.existsSync,
readdirSync: mockFs.readdirSync,
readFileSync: mockFs.readFileSync,
}));
// Mock child_process
const mockExecSync = vi.fn();
vi.mock('child_process', () => ({
execSync: mockExecSync,
}));
describe('OpenCode CLI Path Module', () => {
beforeEach(() => {
vi.clearAllMocks();
// Reset module state
vi.resetModules();
// Reset packaged state
mockApp.isPackaged = false;
// Reset HOME environment variable
process.env.HOME = '/Users/testuser';
});
afterEach(() => {
vi.restoreAllMocks();
});
describe('getOpenCodeCliPath()', () => {
describe('Development Mode', () => {
it('should return nvm OpenCode path when available', async () => {
// Arrange
mockApp.isPackaged = false;
const nvmVersionsDir = '/Users/testuser/.nvm/versions/node';
const expectedPath = path.join(nvmVersionsDir, 'v20.10.0', 'bin', 'opencode');
mockFs.existsSync.mockImplementation((p: string) => {
if (p === nvmVersionsDir) return true;
if (p === expectedPath) return true;
return false;
});
mockFs.readdirSync.mockImplementation((p: string) => {
if (p === nvmVersionsDir) return ['v20.10.0'];
return [];
});
// Act
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
const result = getOpenCodeCliPath();
// Assert
expect(result.command).toBe(expectedPath);
expect(result.args).toEqual([]);
});
it('should return global npm OpenCode path when nvm not available', async () => {
// Arrange
mockApp.isPackaged = false;
const globalPath = '/usr/local/bin/opencode';
mockFs.existsSync.mockImplementation((p: string) => {
if (p === globalPath) return true;
return false;
});
mockFs.readdirSync.mockReturnValue([]);
// Act
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
const result = getOpenCodeCliPath();
// Assert
expect(result.command).toBe(globalPath);
expect(result.args).toEqual([]);
});
it('should return Homebrew OpenCode path on Apple Silicon', async () => {
// Arrange
mockApp.isPackaged = false;
const homebrewPath = '/opt/homebrew/bin/opencode';
mockFs.existsSync.mockImplementation((p: string) => {
if (p === homebrewPath) return true;
return false;
});
mockFs.readdirSync.mockReturnValue([]);
// Act
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
const result = getOpenCodeCliPath();
// Assert
expect(result.command).toBe(homebrewPath);
expect(result.args).toEqual([]);
});
it('should return bundled CLI path in node_modules when global not found', async () => {
// Arrange
mockApp.isPackaged = false;
const appPath = '/mock/app/path';
const bundledPath = path.join(appPath, 'node_modules', '.bin', 'opencode');
mockApp.getAppPath.mockReturnValue(appPath);
mockFs.existsSync.mockImplementation((p: string) => {
if (p === bundledPath) return true;
return false;
});
mockFs.readdirSync.mockReturnValue([]);
// Act
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
const result = getOpenCodeCliPath();
// Assert
expect(result.command).toBe(bundledPath);
expect(result.args).toEqual([]);
});
it('should fallback to PATH-based opencode when no paths found', async () => {
// Arrange
mockApp.isPackaged = false;
mockFs.existsSync.mockReturnValue(false);
mockFs.readdirSync.mockReturnValue([]);
// Act
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
const result = getOpenCodeCliPath();
// Assert
expect(result.command).toBe('opencode');
expect(result.args).toEqual([]);
});
});
describe('Packaged Mode', () => {
it('should return unpacked asar path when packaged', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
const expectedPath = path.join(
resourcesPath,
'app.asar.unpacked',
'node_modules',
'opencode-ai',
'bin',
'opencode'
);
mockFs.existsSync.mockImplementation((p: string) => {
if (p === expectedPath) return true;
return false;
});
// Act
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
const result = getOpenCodeCliPath();
// Assert
expect(result.command).toBe(expectedPath);
expect(result.args).toEqual([]);
});
it('should throw error when bundled CLI not found in packaged app', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(false);
// Act & Assert
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
expect(() => getOpenCodeCliPath()).toThrow('OpenCode CLI not found at');
});
});
});
describe('isOpenCodeBundled()', () => {
describe('Development Mode', () => {
it('should return true when nvm OpenCode is available', async () => {
// Arrange
mockApp.isPackaged = false;
const nvmVersionsDir = '/Users/testuser/.nvm/versions/node';
const opencodePath = path.join(nvmVersionsDir, 'v20.10.0', 'bin', 'opencode');
mockFs.existsSync.mockImplementation((p: string) => {
if (p === nvmVersionsDir) return true;
if (p === opencodePath) return true;
return false;
});
mockFs.readdirSync.mockImplementation((p: string) => {
if (p === nvmVersionsDir) return ['v20.10.0'];
return [];
});
// Act
const { isOpenCodeBundled } = await import('@main/opencode/cli-path');
const result = isOpenCodeBundled();
// Assert
expect(result).toBe(true);
});
it('should return true when bundled CLI exists in node_modules', async () => {
// Arrange
mockApp.isPackaged = false;
const appPath = '/mock/app/path';
const bundledPath = path.join(appPath, 'node_modules', '.bin', 'opencode');
mockApp.getAppPath.mockReturnValue(appPath);
mockFs.existsSync.mockImplementation((p: string) => {
if (p === bundledPath) return true;
return false;
});
mockFs.readdirSync.mockReturnValue([]);
// Act
const { isOpenCodeBundled } = await import('@main/opencode/cli-path');
const result = isOpenCodeBundled();
// Assert
expect(result).toBe(true);
});
it('should return true when opencode is available on PATH', async () => {
// Arrange
mockApp.isPackaged = false;
mockFs.existsSync.mockReturnValue(false);
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('/usr/local/bin/opencode');
// Act
const { isOpenCodeBundled } = await import('@main/opencode/cli-path');
const result = isOpenCodeBundled();
// Assert
expect(result).toBe(true);
});
it('should return false when no CLI is found anywhere', async () => {
// Arrange
mockApp.isPackaged = false;
mockFs.existsSync.mockReturnValue(false);
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockImplementation(() => {
throw new Error('Command not found');
});
// Act
const { isOpenCodeBundled } = await import('@main/opencode/cli-path');
const result = isOpenCodeBundled();
// Assert
expect(result).toBe(false);
});
});
describe('Packaged Mode', () => {
it('should return true when bundled CLI exists in unpacked asar', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
const cliPath = path.join(
resourcesPath,
'app.asar.unpacked',
'node_modules',
'opencode-ai',
'bin',
'opencode'
);
mockFs.existsSync.mockImplementation((p: string) => {
if (p === cliPath) return true;
return false;
});
// Act
const { isOpenCodeBundled } = await import('@main/opencode/cli-path');
const result = isOpenCodeBundled();
// Assert
expect(result).toBe(true);
});
it('should return false when bundled CLI missing in unpacked asar', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(false);
// Act
const { isOpenCodeBundled } = await import('@main/opencode/cli-path');
const result = isOpenCodeBundled();
// Assert
expect(result).toBe(false);
});
});
});
describe('getBundledOpenCodeVersion()', () => {
describe('Packaged Mode', () => {
it('should read version from package.json in unpacked asar', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
const packageJsonPath = path.join(
resourcesPath,
'app.asar.unpacked',
'node_modules',
'opencode-ai',
'package.json'
);
mockFs.existsSync.mockImplementation((p: string) => p === packageJsonPath);
mockFs.readFileSync.mockImplementation((p: string) => {
if (p === packageJsonPath) {
return JSON.stringify({ version: '1.2.3' });
}
return '';
});
// Act
const { getBundledOpenCodeVersion } = await import('@main/opencode/cli-path');
const result = getBundledOpenCodeVersion();
// Assert
expect(result).toBe('1.2.3');
});
it('should return null when package.json not found', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(false);
// Act
const { getBundledOpenCodeVersion } = await import('@main/opencode/cli-path');
const result = getBundledOpenCodeVersion();
// Assert
expect(result).toBeNull();
});
});
describe('Development Mode', () => {
it('should execute CLI with --version flag and parse output', async () => {
// Arrange
mockApp.isPackaged = false;
const appPath = '/mock/app/path';
const bundledPath = path.join(appPath, 'node_modules', '.bin', 'opencode');
mockApp.getAppPath.mockReturnValue(appPath);
mockFs.existsSync.mockImplementation((p: string) => {
if (p === bundledPath) return true;
return false;
});
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('opencode 1.5.0\n');
// Act
const { getBundledOpenCodeVersion } = await import('@main/opencode/cli-path');
const result = getBundledOpenCodeVersion();
// Assert
expect(result).toBe('1.5.0');
});
it('should parse version from simple version string', async () => {
// Arrange
mockApp.isPackaged = false;
const appPath = '/mock/app/path';
const bundledPath = path.join(appPath, 'node_modules', '.bin', 'opencode');
mockApp.getAppPath.mockReturnValue(appPath);
mockFs.existsSync.mockImplementation((p: string) => {
if (p === bundledPath) return true;
return false;
});
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('2.0.1');
// Act
const { getBundledOpenCodeVersion } = await import('@main/opencode/cli-path');
const result = getBundledOpenCodeVersion();
// Assert
expect(result).toBe('2.0.1');
});
it('should return null when version command fails', async () => {
// Arrange
mockApp.isPackaged = false;
const appPath = '/mock/app/path';
const bundledPath = path.join(appPath, 'node_modules', '.bin', 'opencode');
mockApp.getAppPath.mockReturnValue(appPath);
mockFs.existsSync.mockImplementation((p: string) => {
if (p === bundledPath) return true;
return false;
});
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockImplementation(() => {
throw new Error('Command failed');
});
// Act
const { getBundledOpenCodeVersion } = await import('@main/opencode/cli-path');
const result = getBundledOpenCodeVersion();
// Assert
expect(result).toBeNull();
});
});
});
describe('NVM Path Scanning', () => {
it('should scan multiple nvm versions and return first found', async () => {
// Arrange
mockApp.isPackaged = false;
const nvmVersionsDir = '/Users/testuser/.nvm/versions/node';
const v18Path = path.join(nvmVersionsDir, 'v18.17.0', 'bin', 'opencode');
const v20Path = path.join(nvmVersionsDir, 'v20.10.0', 'bin', 'opencode');
mockFs.existsSync.mockImplementation((p: string) => {
if (p === nvmVersionsDir) return true;
if (p === v20Path) return true;
if (p === v18Path) return false;
return false;
});
mockFs.readdirSync.mockImplementation((p: string) => {
if (p === nvmVersionsDir) return ['v18.17.0', 'v20.10.0'];
return [];
});
// Act
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
const result = getOpenCodeCliPath();
// Assert
expect(result.command).toBe(v20Path);
});
it('should handle missing nvm directory gracefully', async () => {
// Arrange
mockApp.isPackaged = false;
process.env.HOME = '/Users/testuser';
mockFs.existsSync.mockReturnValue(false);
mockFs.readdirSync.mockReturnValue([]);
// Act
const { getOpenCodeCliPath } = await import('@main/opencode/cli-path');
const result = getOpenCodeCliPath();
// Assert - should fallback to opencode on PATH
expect(result.command).toBe('opencode');
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/opencode/config-generator.integration.test.ts
================================================
/**
* Integration tests for OpenCode config generator
*
* Tests the config-generator module which creates OpenCode configuration files
* with MCP servers, agent definitions, and system prompts.
*
* NOTE: This is a TRUE integration test.
* - Uses REAL filesystem operations with temp directories
* - Only mocks external dependencies (electron APIs)
*
* Mocked external services:
* - electron.app: Native Electron APIs (getPath, getAppPath, isPackaged)
*
* Real implementations used:
* - fs: Real filesystem operations in temp directories
* - path: Real path operations
*
* @module __tests__/integration/main/opencode/config-generator.integration.test
*/
import { describe, it, expect, beforeEach, vi, afterEach } from 'vitest';
import path from 'path';
import fs from 'fs';
import os from 'os';
// Create temp directories for each test
let tempUserDataDir: string;
let tempAppDir: string;
// Mock only the external electron module
const mockApp = {
isPackaged: false,
getAppPath: vi.fn(() => tempAppDir),
getPath: vi.fn((name: string) => {
if (name === 'userData') return tempUserDataDir;
return path.join(tempUserDataDir, name);
}),
};
vi.mock('electron', () => ({
app: mockApp,
}));
// Mock permission-api module (internal but exports constants we need)
vi.mock('@main/permission-api', () => ({
PERMISSION_API_PORT: 9999,
QUESTION_API_PORT: 9227,
}));
describe('OpenCode Config Generator Integration', () => {
let originalEnv: NodeJS.ProcessEnv;
beforeEach(() => {
vi.clearAllMocks();
vi.resetModules();
originalEnv = { ...process.env };
mockApp.isPackaged = false;
// Create real temp directories for each test
tempUserDataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencode-config-test-userData-'));
tempAppDir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencode-config-test-app-'));
// Create skills directory structure in temp app dir
const skillsDir = path.join(tempAppDir, 'skills');
fs.mkdirSync(skillsDir, { recursive: true });
fs.mkdirSync(path.join(skillsDir, 'file-permission', 'src'), { recursive: true });
fs.writeFileSync(path.join(skillsDir, 'file-permission', 'src', 'index.ts'), '// mock file');
// Update mock to use temp directories
mockApp.getAppPath.mockReturnValue(tempAppDir);
mockApp.getPath.mockImplementation((name: string) => {
if (name === 'userData') return tempUserDataDir;
return path.join(tempUserDataDir, name);
});
});
afterEach(() => {
vi.restoreAllMocks();
process.env = originalEnv;
// Clean up temp directories
try {
fs.rmSync(tempUserDataDir, { recursive: true, force: true });
fs.rmSync(tempAppDir, { recursive: true, force: true });
} catch {
// Ignore cleanup errors
}
});
describe('getSkillsPath()', () => {
describe('Development Mode', () => {
it('should return skills path relative to app path in dev mode', async () => {
// Arrange
mockApp.isPackaged = false;
// Act
const { getSkillsPath } = await import('@main/opencode/config-generator');
const result = getSkillsPath();
// Assert
expect(result).toBe(path.join(tempAppDir, 'skills'));
});
});
describe('Packaged Mode', () => {
it('should return skills path in resources folder when packaged', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = path.join(tempAppDir, 'Resources');
fs.mkdirSync(resourcesPath, { recursive: true });
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
// Act
const { getSkillsPath } = await import('@main/opencode/config-generator');
const result = getSkillsPath();
// Assert
expect(result).toBe(path.join(resourcesPath, 'skills'));
});
});
});
describe('generateOpenCodeConfig()', () => {
it('should create config directory if it does not exist', async () => {
// Arrange - config dir does not exist initially
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
await generateOpenCodeConfig();
// Assert - verify directory was created using real fs
const configDir = path.join(tempUserDataDir, 'opencode');
expect(fs.existsSync(configDir)).toBe(true);
});
it('should not recreate directory if it already exists', async () => {
// Arrange - create config dir beforehand
const configDir = path.join(tempUserDataDir, 'opencode');
fs.mkdirSync(configDir, { recursive: true });
const statBefore = fs.statSync(configDir);
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
await generateOpenCodeConfig();
// Assert - directory still exists, no error
expect(fs.existsSync(configDir)).toBe(true);
});
it('should write config file with correct structure', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert - read the real file
expect(fs.existsSync(configPath)).toBe(true);
const configContent = fs.readFileSync(configPath, 'utf-8');
const config = JSON.parse(configContent);
expect(config.$schema).toBe('https://opencode.ai/config.json');
expect(config.default_agent).toBe('accomplish');
expect(config.permission).toBe('allow');
expect(config.enabled_providers).toContain('anthropic');
expect(config.enabled_providers).toContain('openai');
expect(config.enabled_providers).toContain('google');
});
it('should include accomplish agent configuration', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert
const config = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
const agent = config.agent['accomplish'];
expect(agent).toBeDefined();
expect(agent.description).toBe('Browser automation assistant using dev-browser');
expect(agent.mode).toBe('primary');
expect(typeof agent.prompt).toBe('string');
expect(agent.prompt.length).toBeGreaterThan(0);
});
it('should include MCP server configuration for file-permission', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert
const config = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
const filePermission = config.mcp['file-permission'];
expect(filePermission).toBeDefined();
expect(filePermission.type).toBe('local');
expect(filePermission.enabled).toBe(true);
expect(filePermission.command[0]).toBe('npx');
expect(filePermission.command[1]).toBe('tsx');
expect(filePermission.environment.PERMISSION_API_PORT).toBe('9999');
});
it('should inject skills path into system prompt', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert
const config = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
const prompt = config.agent['accomplish'].prompt;
const skillsPath = path.join(tempAppDir, 'skills');
// Prompt should contain the actual skills path, not the template placeholder
expect(prompt).toContain(skillsPath);
expect(prompt).not.toContain('{{SKILLS_PATH}}');
});
it('should set OPENCODE_CONFIG environment variable after generation', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert
expect(process.env.OPENCODE_CONFIG).toBe(configPath);
expect(configPath).toBe(path.join(tempUserDataDir, 'opencode', 'opencode.json'));
});
it('should return the config file path', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const result = await generateOpenCodeConfig();
// Assert
expect(result).toBe(path.join(tempUserDataDir, 'opencode', 'opencode.json'));
expect(fs.existsSync(result)).toBe(true);
});
});
describe('getOpenCodeConfigPath()', () => {
it('should return config path in userData directory', async () => {
// Act
const { getOpenCodeConfigPath } = await import('@main/opencode/config-generator');
const result = getOpenCodeConfigPath();
// Assert
expect(result).toBe(path.join(tempUserDataDir, 'opencode', 'opencode.json'));
});
});
describe('System Prompt Content', () => {
it('should include browser automation guidance', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert
const config = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
const prompt = config.agent['accomplish'].prompt;
expect(prompt).toContain('browser');
expect(prompt.toLowerCase()).toContain('playwright');
});
it('should include file permission rules', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert
const config = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
const prompt = config.agent['accomplish'].prompt;
expect(prompt).toContain('FILE PERMISSION WORKFLOW');
expect(prompt).toContain('request_file_permission');
});
it('should include user communication guidance', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert
const config = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
const prompt = config.agent['accomplish'].prompt;
expect(prompt).toContain('user-communication');
expect(prompt).toContain('AskUserQuestion');
});
});
describe('ACCOMPLISH_AGENT_NAME Export', () => {
it('should export the agent name constant', async () => {
// Act
const { ACCOMPLISH_AGENT_NAME } = await import('@main/opencode/config-generator');
// Assert
expect(ACCOMPLISH_AGENT_NAME).toBe('accomplish');
});
});
describe('Config File Persistence', () => {
it('should overwrite existing config file on regeneration', async () => {
// Arrange - generate config first time
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const firstPath = await generateOpenCodeConfig();
const firstContent = fs.readFileSync(firstPath, 'utf-8');
// Reset modules to re-run generator
vi.resetModules();
// Act - generate again
const { generateOpenCodeConfig: regenerate } = await import('@main/opencode/config-generator');
const secondPath = await regenerate();
const secondContent = fs.readFileSync(secondPath, 'utf-8');
// Assert - same path, same content structure
expect(firstPath).toBe(secondPath);
expect(JSON.parse(firstContent).$schema).toBe(JSON.parse(secondContent).$schema);
});
it('should create valid JSON that can be parsed', async () => {
// Act
const { generateOpenCodeConfig } = await import('@main/opencode/config-generator');
const configPath = await generateOpenCodeConfig();
// Assert - should not throw when parsing
const content = fs.readFileSync(configPath, 'utf-8');
expect(() => JSON.parse(content)).not.toThrow();
// Should be pretty-printed (contains newlines)
expect(content).toContain('\n');
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/permission-api.integration.test.ts
================================================
/**
* Integration tests for Permission API
*
* Tests the REAL exported functions from permission-api module:
* - isFilePermissionRequest() - checks if request ID is a file permission
* - resolvePermission() - resolves a pending permission request
* - initPermissionApi() - initializes the API with window and task getter
* - startPermissionApiServer() - starts the HTTP server
* - PERMISSION_API_PORT - the port constant
*
* These tests mock only electron (external dependency) and test the real
* module behavior.
*/
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
// Mock electron before importing the module
vi.mock('electron', () => ({
BrowserWindow: {
fromWebContents: vi.fn(),
getFocusedWindow: vi.fn(),
getAllWindows: vi.fn(() => []),
},
app: {
isPackaged: false,
getPath: vi.fn(() => '/tmp/test-app'),
},
}));
// Import the REAL module functions after mocking electron
import {
isFilePermissionRequest,
resolvePermission,
initPermissionApi,
startPermissionApiServer,
PERMISSION_API_PORT,
} from '@main/permission-api';
describe('Permission API Integration', () => {
beforeEach(() => {
vi.clearAllMocks();
});
afterEach(() => {
vi.clearAllMocks();
});
describe('isFilePermissionRequest', () => {
it('should return true for IDs starting with filereq_', () => {
expect(isFilePermissionRequest('filereq_123')).toBe(true);
expect(isFilePermissionRequest('filereq_abc_def')).toBe(true);
expect(isFilePermissionRequest('filereq_1234567890_abcdefghi')).toBe(true);
expect(isFilePermissionRequest('filereq_')).toBe(true);
});
it('should return false for IDs not starting with filereq_', () => {
expect(isFilePermissionRequest('req_123')).toBe(false);
expect(isFilePermissionRequest('permission_abc')).toBe(false);
expect(isFilePermissionRequest('file_req_123')).toBe(false);
expect(isFilePermissionRequest('FILEREQ_123')).toBe(false); // case sensitive
expect(isFilePermissionRequest('')).toBe(false);
expect(isFilePermissionRequest('filereq')).toBe(false); // missing underscore
expect(isFilePermissionRequest('_filereq_123')).toBe(false);
});
});
describe('resolvePermission', () => {
it('should return false for non-existent request ID', () => {
// The real function returns false when the request is not in pending
expect(resolvePermission('filereq_nonexistent', true)).toBe(false);
expect(resolvePermission('filereq_notpending', false)).toBe(false);
});
it('should return false when called multiple times with same ID', () => {
const requestId = 'filereq_double_resolve';
// First call returns false (not pending)
expect(resolvePermission(requestId, true)).toBe(false);
// Second call also returns false (still not pending)
expect(resolvePermission(requestId, false)).toBe(false);
});
});
describe('PERMISSION_API_PORT', () => {
it('should be exported with correct value', () => {
expect(PERMISSION_API_PORT).toBe(9226);
});
});
describe('initPermissionApi', () => {
it('should accept window and task getter without throwing', () => {
const mockWindow = {
isDestroyed: () => false,
webContents: {
send: vi.fn(),
isDestroyed: () => false,
},
} as unknown as import('electron').BrowserWindow;
const mockTaskGetter = () => 'task_123';
expect(() => initPermissionApi(mockWindow, mockTaskGetter)).not.toThrow();
});
it('should be a function', () => {
expect(typeof initPermissionApi).toBe('function');
});
});
describe('startPermissionApiServer', () => {
it('should be a function', () => {
expect(typeof startPermissionApiServer).toBe('function');
});
it('should return an HTTP server when called', () => {
const server = startPermissionApiServer();
expect(server).toBeDefined();
// Clean up - close the server
server?.close();
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/secureStorage.integration.test.ts
================================================
/**
* Integration tests for secureStorage module
* Tests real electron-store interactions with encrypted API key storage
* @module __tests__/integration/main/secureStorage.integration.test
*/
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
// Create a unique temp directory for each test run
let tempDir: string;
let originalCwd: string;
// Use a factory function that closes over tempDir
const getTempDir = () => tempDir;
// Mock electron module to control userData path
vi.mock('electron', () => ({
app: {
getPath: (name: string) => {
if (name === 'userData') {
return getTempDir();
}
return `/mock/path/${name}`;
},
getVersion: () => '0.1.0',
getName: () => 'Accomplish',
isPackaged: false,
},
}));
describe('secureStorage Integration', () => {
beforeEach(async () => {
// Create a unique temp directory for each test
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'secureStorage-test-'));
originalCwd = process.cwd();
// Reset module cache to get fresh store instances
vi.resetModules();
});
afterEach(async () => {
// Clear secure storage
try {
const { clearSecureStorage } = await import('@main/store/secureStorage');
clearSecureStorage();
} catch {
// Module may not be loaded
}
// Clean up temp directory
if (tempDir && fs.existsSync(tempDir)) {
fs.rmSync(tempDir, { recursive: true, force: true });
}
process.chdir(originalCwd);
});
describe('storeApiKey and getApiKey', () => {
it('should store and retrieve an API key', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
const testKey = 'sk-test-anthropic-key-12345';
// Act
storeApiKey('anthropic', testKey);
const result = getApiKey('anthropic');
// Assert
expect(result).toBe(testKey);
});
it('should return null for non-existent provider', async () => {
// Arrange
const { getApiKey } = await import('@main/store/secureStorage');
// Act
const result = getApiKey('anthropic');
// Assert
expect(result).toBeNull();
});
it('should encrypt the API key in storage', async () => {
// Arrange
const { storeApiKey } = await import('@main/store/secureStorage');
const testKey = 'sk-test-visible-key';
// Act
storeApiKey('anthropic', testKey);
// Assert - check that the raw file does not contain the key in plain text
const files = fs.readdirSync(tempDir);
const storeFile = files.find(f => f.includes('secure-storage'));
if (storeFile) {
const content = fs.readFileSync(path.join(tempDir, storeFile), 'utf-8');
expect(content).not.toContain(testKey);
}
});
it('should overwrite existing key for same provider', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
const firstKey = 'sk-first-key';
const secondKey = 'sk-second-key';
// Act
storeApiKey('anthropic', firstKey);
storeApiKey('anthropic', secondKey);
const result = getApiKey('anthropic');
// Assert
expect(result).toBe(secondKey);
});
it('should handle special characters in API key', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
const testKey = 'sk-test_key+with/special=chars!@#$%^&*()';
// Act
storeApiKey('anthropic', testKey);
const result = getApiKey('anthropic');
// Assert
expect(result).toBe(testKey);
});
it('should handle very long API keys', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
const testKey = 'sk-' + 'a'.repeat(500);
// Act
storeApiKey('anthropic', testKey);
const result = getApiKey('anthropic');
// Assert
expect(result).toBe(testKey);
});
it('should handle empty string as API key', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
// Act
storeApiKey('anthropic', '');
const result = getApiKey('anthropic');
// Assert
expect(result).toBe('');
});
});
describe('multiple providers', () => {
it('should store API keys for different providers independently', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
// Act
storeApiKey('anthropic', 'anthropic-key-123');
storeApiKey('openai', 'openai-key-456');
storeApiKey('google', 'google-key-789');
storeApiKey('custom', 'custom-key-xyz');
// Assert
expect(getApiKey('anthropic')).toBe('anthropic-key-123');
expect(getApiKey('openai')).toBe('openai-key-456');
expect(getApiKey('google')).toBe('google-key-789');
expect(getApiKey('custom')).toBe('custom-key-xyz');
});
it('should not affect other providers when updating one', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'anthropic-original');
storeApiKey('openai', 'openai-original');
// Act
storeApiKey('anthropic', 'anthropic-updated');
// Assert
expect(getApiKey('anthropic')).toBe('anthropic-updated');
expect(getApiKey('openai')).toBe('openai-original');
});
});
describe('deleteApiKey', () => {
it('should remove only the target provider key', async () => {
// Arrange
const { storeApiKey, getApiKey, deleteApiKey } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'anthropic-key');
storeApiKey('openai', 'openai-key');
// Act
const deleted = deleteApiKey('anthropic');
// Assert
expect(deleted).toBe(true);
expect(getApiKey('anthropic')).toBeNull();
expect(getApiKey('openai')).toBe('openai-key');
});
it('should return false when deleting non-existent key', async () => {
// Arrange
const { deleteApiKey } = await import('@main/store/secureStorage');
// Act
const result = deleteApiKey('anthropic');
// Assert
expect(result).toBe(false);
});
it('should allow re-storing after deletion', async () => {
// Arrange
const { storeApiKey, getApiKey, deleteApiKey } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'original-key');
deleteApiKey('anthropic');
// Act
storeApiKey('anthropic', 'new-key');
const result = getApiKey('anthropic');
// Assert
expect(result).toBe('new-key');
});
});
describe('getAllApiKeys', () => {
it('should return all null for empty store', async () => {
// Arrange
const { getAllApiKeys } = await import('@main/store/secureStorage');
// Act
const result = await getAllApiKeys();
// Assert
expect(result).toEqual({
anthropic: null,
openai: null,
google: null,
xai: null,
deepseek: null,
zai: null,
openrouter: null,
bedrock: null,
litellm: null,
custom: null,
});
});
it('should return all stored API keys', async () => {
// Arrange
const { storeApiKey, getAllApiKeys } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'anthropic-key');
storeApiKey('openai', 'openai-key');
storeApiKey('google', 'google-key');
// Act
const result = await getAllApiKeys();
// Assert
expect(result.anthropic).toBe('anthropic-key');
expect(result.openai).toBe('openai-key');
expect(result.google).toBe('google-key');
expect(result.custom).toBeNull();
});
it('should return partial results when some providers are set', async () => {
// Arrange
const { storeApiKey, getAllApiKeys } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'anthropic-key');
storeApiKey('custom', 'custom-key');
// Act
const result = await getAllApiKeys();
// Assert
expect(result.anthropic).toBe('anthropic-key');
expect(result.openai).toBeNull();
expect(result.google).toBeNull();
expect(result.custom).toBe('custom-key');
});
});
describe('hasAnyApiKey', () => {
it('should return false when no keys are stored', async () => {
// Arrange
const { hasAnyApiKey } = await import('@main/store/secureStorage');
// Act
const result = await hasAnyApiKey();
// Assert
expect(result).toBe(false);
});
it('should return true when at least one key is stored', async () => {
// Arrange
const { storeApiKey, hasAnyApiKey } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'test-key');
// Act
const result = await hasAnyApiKey();
// Assert
expect(result).toBe(true);
});
it('should return true when multiple keys are stored', async () => {
// Arrange
const { storeApiKey, hasAnyApiKey } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'anthropic-key');
storeApiKey('openai', 'openai-key');
// Act
const result = await hasAnyApiKey();
// Assert
expect(result).toBe(true);
});
it('should return false after all keys are deleted', async () => {
// Arrange
const { storeApiKey, deleteApiKey, hasAnyApiKey } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'test-key');
deleteApiKey('anthropic');
// Act
const result = await hasAnyApiKey();
// Assert
expect(result).toBe(false);
});
});
describe('clearSecureStorage', () => {
it('should remove all stored API keys', async () => {
// Arrange
const { storeApiKey, getAllApiKeys, clearSecureStorage } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'anthropic-key');
storeApiKey('openai', 'openai-key');
storeApiKey('google', 'google-key');
// Act
clearSecureStorage();
const result = await getAllApiKeys();
// Assert
expect(result).toEqual({
anthropic: null,
openai: null,
google: null,
xai: null,
deepseek: null,
zai: null,
openrouter: null,
bedrock: null,
litellm: null,
custom: null,
});
});
it('should allow storing new keys after clear', async () => {
// Arrange
const { storeApiKey, getApiKey, clearSecureStorage } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'old-key');
clearSecureStorage();
// Act
storeApiKey('anthropic', 'new-key');
const result = getApiKey('anthropic');
// Assert
expect(result).toBe('new-key');
});
it('should reset salt and derived key', async () => {
// Arrange
const { storeApiKey, getApiKey, clearSecureStorage } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'test-key-1');
// Act
clearSecureStorage();
storeApiKey('anthropic', 'test-key-2');
const result = getApiKey('anthropic');
// Assert - key should be retrievable with new encryption
expect(result).toBe('test-key-2');
});
});
describe('listStoredCredentials', () => {
it('should return empty array when no credentials stored', async () => {
// Arrange
const { listStoredCredentials } = await import('@main/store/secureStorage');
// Act
const result = listStoredCredentials();
// Assert
expect(result).toEqual([]);
});
it('should return all stored credentials with decrypted values', async () => {
// Arrange
const { storeApiKey, listStoredCredentials } = await import('@main/store/secureStorage');
storeApiKey('anthropic', 'anthropic-key-123');
storeApiKey('openai', 'openai-key-456');
// Act
const result = listStoredCredentials();
// Assert
expect(result).toHaveLength(2);
expect(result).toContainEqual({ account: 'apiKey:anthropic', password: 'anthropic-key-123' });
expect(result).toContainEqual({ account: 'apiKey:openai', password: 'openai-key-456' });
});
});
describe('encryption consistency', () => {
it('should decrypt values correctly after module reload', async () => {
// Arrange - store key in first module instance
const module1 = await import('@main/store/secureStorage');
module1.storeApiKey('anthropic', 'persistent-key-123');
// Act - reset modules and reimport
vi.resetModules();
const module2 = await import('@main/store/secureStorage');
const result = module2.getApiKey('anthropic');
// Assert
expect(result).toBe('persistent-key-123');
});
it('should maintain encryption across multiple store/retrieve cycles', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
// Act - multiple cycles
for (let i = 0; i < 5; i++) {
const key = `test-key-cycle-${i}`;
storeApiKey('anthropic', key);
const result = getApiKey('anthropic');
expect(result).toBe(key);
}
});
it('should use unique IV for each encryption', async () => {
// This test verifies that the same plaintext produces different ciphertext
// due to random IV generation by storing the same value twice
// and confirming decryption works for both
const { storeApiKey, getApiKey, clearSecureStorage } = await import('@main/store/secureStorage');
// Store the same plaintext for two different providers
storeApiKey('anthropic', 'same-key-value');
storeApiKey('openai', 'same-key-value');
// Both should decrypt correctly (proving unique IVs didn't break anything)
const anthropicKey = getApiKey('anthropic');
const openaiKey = getApiKey('openai');
expect(anthropicKey).toBe('same-key-value');
expect(openaiKey).toBe('same-key-value');
// If the IVs were the same, we'd have potential security issues,
// but since this is an integration test, we verify the functionality works.
// The encryption implementation uses crypto.randomBytes for IV generation.
});
});
describe('edge cases', () => {
it('should handle unicode characters in API key', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
const unicodeKey = 'sk-test-key-with-unicode-chars';
// Act
storeApiKey('anthropic', unicodeKey);
const result = getApiKey('anthropic');
// Assert
expect(result).toBe(unicodeKey);
});
it('should handle rapid successive stores', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
// Act - rapid stores
for (let i = 0; i < 10; i++) {
storeApiKey('anthropic', `key-${i}`);
}
const result = getApiKey('anthropic');
// Assert - should have the last stored value
expect(result).toBe('key-9');
});
it('should handle concurrent operations on different providers', async () => {
// Arrange
const { storeApiKey, getApiKey } = await import('@main/store/secureStorage');
// Act - interleaved operations
storeApiKey('anthropic', 'a1');
storeApiKey('openai', 'o1');
storeApiKey('anthropic', 'a2');
storeApiKey('google', 'g1');
storeApiKey('openai', 'o2');
// Assert
expect(getApiKey('anthropic')).toBe('a2');
expect(getApiKey('openai')).toBe('o2');
expect(getApiKey('google')).toBe('g1');
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/store/freshInstallCleanup.integration.test.ts
================================================
/**
* Integration tests for Fresh Install Cleanup
*
* Tests the REAL checkAndCleanupFreshInstall function:
* - Returns false in dev mode (app.isPackaged = false)
* - Returns false when bundle mtime cannot be determined
*
* These tests mock external dependencies (electron, fs, store modules)
* and verify the actual module behavior.
*/
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
// Use vi.hoisted() to ensure mock functions are available when vi.mock is hoisted
const {
mockExistsSync,
mockReadFileSync,
mockWriteFileSync,
mockStatSync,
mockMkdirSync,
mockUnlinkSync,
mockGetPath,
mockGetVersion,
mockClearAppSettings,
mockClearTaskHistoryStore,
mockClearSecureStorage,
} = vi.hoisted(() => ({
mockExistsSync: vi.fn(),
mockReadFileSync: vi.fn(),
mockWriteFileSync: vi.fn(),
mockStatSync: vi.fn(),
mockMkdirSync: vi.fn(),
mockUnlinkSync: vi.fn(),
mockGetPath: vi.fn(),
mockGetVersion: vi.fn(),
mockClearAppSettings: vi.fn(),
mockClearTaskHistoryStore: vi.fn(),
mockClearSecureStorage: vi.fn(),
}));
// Mock fs module
vi.mock('fs', () => ({
default: {
existsSync: mockExistsSync,
readFileSync: mockReadFileSync,
writeFileSync: mockWriteFileSync,
statSync: mockStatSync,
mkdirSync: mockMkdirSync,
unlinkSync: mockUnlinkSync,
},
existsSync: mockExistsSync,
readFileSync: mockReadFileSync,
writeFileSync: mockWriteFileSync,
statSync: mockStatSync,
mkdirSync: mockMkdirSync,
unlinkSync: mockUnlinkSync,
}));
// Mock electron app - isPackaged starts as false (dev mode)
vi.mock('electron', () => ({
app: {
isPackaged: false,
getPath: mockGetPath,
getVersion: mockGetVersion,
},
}));
// Mock store modules
vi.mock('@main/store/appSettings', () => ({
clearAppSettings: mockClearAppSettings,
}));
vi.mock('@main/store/taskHistory', () => ({
clearTaskHistoryStore: mockClearTaskHistoryStore,
}));
vi.mock('@main/store/secureStorage', () => ({
clearSecureStorage: mockClearSecureStorage,
}));
// Import the REAL module function after mocking dependencies
import { checkAndCleanupFreshInstall } from '@main/store/freshInstallCleanup';
import { app } from 'electron';
describe('Fresh Install Cleanup Integration', () => {
beforeEach(() => {
vi.clearAllMocks();
// Reset to dev mode by default
(app as unknown as { isPackaged: boolean }).isPackaged = false;
// Setup default path mocks
mockGetPath.mockImplementation((name: string) => {
const paths: Record = {
userData: '/tmp/test-app/userData',
appData: '/tmp/test-app/appData',
exe: '/Applications/Accomplish.app/Contents/MacOS/Accomplish',
};
return paths[name] || '/tmp/test-app';
});
mockGetVersion.mockReturnValue('1.0.0');
});
afterEach(() => {
vi.clearAllMocks();
// Reset to dev mode
(app as unknown as { isPackaged: boolean }).isPackaged = false;
});
describe('checkAndCleanupFreshInstall', () => {
it('should return false in dev mode (app.isPackaged = false)', async () => {
// Arrange - dev mode is the default in beforeEach
expect(app.isPackaged).toBe(false);
// Act - call the REAL function
const result = await checkAndCleanupFreshInstall();
// Assert
expect(result).toBe(false);
// Should not call any cleanup functions in dev mode
expect(mockClearAppSettings).not.toHaveBeenCalled();
expect(mockClearTaskHistoryStore).not.toHaveBeenCalled();
expect(mockClearSecureStorage).not.toHaveBeenCalled();
});
it('should return false when exe path does not contain .app bundle', async () => {
// Arrange - set to packaged mode but with non-.app exe path
(app as unknown as { isPackaged: boolean }).isPackaged = true;
mockGetPath.mockImplementation((name: string) => {
if (name === 'exe') return '/usr/local/bin/accomplish'; // No .app in path
return '/tmp/test-app/userData';
});
// Act
const result = await checkAndCleanupFreshInstall();
// Assert
expect(result).toBe(false);
});
it('should return false when bundle stat fails', async () => {
// Arrange - set to packaged mode with valid .app path but stat fails
(app as unknown as { isPackaged: boolean }).isPackaged = true;
mockStatSync.mockImplementation(() => {
throw new Error('ENOENT: no such file or directory');
});
// Act
const result = await checkAndCleanupFreshInstall();
// Assert
expect(result).toBe(false);
});
it('should return false on first install (no existing data)', async () => {
// Arrange - packaged mode, valid bundle, but no existing data
(app as unknown as { isPackaged: boolean }).isPackaged = true;
const currentMtime = new Date('2024-06-01T00:00:00.000Z');
mockStatSync.mockReturnValue({ mtime: currentMtime });
mockExistsSync.mockReturnValue(false); // No existing marker or data
// Act
const result = await checkAndCleanupFreshInstall();
// Assert - first install creates marker but doesn't cleanup (returns false)
expect(result).toBe(false);
// Should write the marker file
expect(mockWriteFileSync).toHaveBeenCalled();
});
it('should return false when marker matches current bundle', async () => {
// Arrange - packaged mode, marker exists and matches
(app as unknown as { isPackaged: boolean }).isPackaged = true;
const currentMtime = new Date('2024-06-01T00:00:00.000Z');
mockStatSync.mockReturnValue({ mtime: currentMtime });
const existingMarker = {
bundleMtime: currentMtime.toISOString(),
version: '1.0.0',
markerCreated: '2024-06-01T00:00:00.000Z',
};
mockExistsSync.mockImplementation((path: string) => {
return path.includes('.install-marker.json');
});
mockReadFileSync.mockReturnValue(JSON.stringify(existingMarker));
// Act
const result = await checkAndCleanupFreshInstall();
// Assert - no cleanup needed
expect(result).toBe(false);
expect(mockClearAppSettings).not.toHaveBeenCalled();
});
it('should return true and cleanup when bundle mtime differs from marker', async () => {
// Arrange - packaged mode, marker exists but bundle changed
(app as unknown as { isPackaged: boolean }).isPackaged = true;
const currentMtime = new Date('2024-07-01T00:00:00.000Z'); // New version
mockStatSync.mockReturnValue({ mtime: currentMtime });
const existingMarker = {
bundleMtime: '2024-06-01T00:00:00.000Z', // Old version
version: '1.0.0',
markerCreated: '2024-06-01T00:00:00.000Z',
};
mockExistsSync.mockImplementation((path: string) => {
return path.includes('.install-marker.json');
});
mockReadFileSync.mockReturnValue(JSON.stringify(existingMarker));
// Act
const result = await checkAndCleanupFreshInstall();
// Assert - cleanup was performed
expect(result).toBe(true);
expect(mockClearAppSettings).toHaveBeenCalled();
expect(mockClearTaskHistoryStore).toHaveBeenCalled();
expect(mockClearSecureStorage).toHaveBeenCalled();
});
it('should return true and cleanup on reinstall (existing data but no marker)', async () => {
// Arrange - packaged mode, no marker but has existing settings file
(app as unknown as { isPackaged: boolean }).isPackaged = true;
const currentMtime = new Date('2024-06-01T00:00:00.000Z');
mockStatSync.mockReturnValue({ mtime: currentMtime });
// No marker, but app-settings.json exists
mockExistsSync.mockImplementation((path: string) => {
if (path.includes('.install-marker.json')) return false;
if (path.includes('app-settings.json')) return true;
return false;
});
// Act
const result = await checkAndCleanupFreshInstall();
// Assert - cleanup was performed (reinstall scenario)
expect(result).toBe(true);
expect(mockClearAppSettings).toHaveBeenCalled();
expect(mockClearTaskHistoryStore).toHaveBeenCalled();
expect(mockClearSecureStorage).toHaveBeenCalled();
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/taskHistory.integration.test.ts
================================================
/**
* Integration tests for taskHistory store
* Tests real electron-store interactions with task persistence
* @module __tests__/integration/main/taskHistory.integration.test
*/
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import type { Task, TaskMessage } from '@accomplish/shared';
// Create a unique temp directory for each test run
let tempDir: string;
let originalCwd: string;
// Use a factory function that closes over tempDir
const getTempDir = () => tempDir;
// Mock electron module to control userData path
vi.mock('electron', () => ({
app: {
getPath: (name: string) => {
if (name === 'userData') {
return getTempDir();
}
return `/mock/path/${name}`;
},
getVersion: () => '0.1.0',
getName: () => 'Accomplish',
isPackaged: false,
},
}));
// Helper to create a mock task
function createMockTask(id: string, prompt: string = 'Test task'): Task {
return {
id,
prompt,
status: 'pending',
messages: [],
createdAt: new Date().toISOString(),
};
}
// Helper to create a mock message
function createMockMessage(
id: string,
type: 'assistant' | 'user' | 'tool' | 'system' = 'assistant',
content: string = 'Test message'
): TaskMessage {
return {
id,
type,
content,
timestamp: new Date().toISOString(),
};
}
describe('taskHistory Integration', () => {
beforeEach(async () => {
// Create a unique temp directory for each test
tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'taskHistory-test-'));
originalCwd = process.cwd();
// Reset module cache to get fresh electron-store instances
vi.resetModules();
});
afterEach(async () => {
// Flush any pending writes and clear timeouts
try {
const { flushPendingTasks, clearTaskHistoryStore } = await import('@main/store/taskHistory');
flushPendingTasks();
clearTaskHistoryStore();
} catch {
// Module may not be loaded
}
// Clean up temp directory
if (tempDir && fs.existsSync(tempDir)) {
fs.rmSync(tempDir, { recursive: true, force: true });
}
process.chdir(originalCwd);
});
describe('saveTask and getTask', () => {
it('should save and retrieve a task by ID', async () => {
// Arrange
const { saveTask, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
const task = createMockTask('task-1', 'Save and retrieve test');
// Act
saveTask(task);
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result).toBeDefined();
expect(result?.id).toBe('task-1');
expect(result?.prompt).toBe('Save and retrieve test');
expect(result?.status).toBe('pending');
});
it('should return undefined for non-existent task', async () => {
// Arrange
const { getTask } = await import('@main/store/taskHistory');
// Act
const result = getTask('non-existent');
// Assert
expect(result).toBeUndefined();
});
it('should update existing task when saving with same ID', async () => {
// Arrange
const { saveTask, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
const task1 = createMockTask('task-1', 'Original prompt');
const task2 = { ...createMockTask('task-1', 'Updated prompt'), status: 'running' as const };
// Act
saveTask(task1);
flushPendingTasks();
saveTask(task2);
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.prompt).toBe('Updated prompt');
expect(result?.status).toBe('running');
});
it('should preserve task messages when saving', async () => {
// Arrange
const { saveTask, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
const task: Task = {
...createMockTask('task-1'),
messages: [
createMockMessage('msg-1', 'user', 'Hello'),
createMockMessage('msg-2', 'assistant', 'Hi there'),
],
};
// Act
saveTask(task);
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.messages).toHaveLength(2);
expect(result?.messages[0].content).toBe('Hello');
expect(result?.messages[1].content).toBe('Hi there');
});
it('should preserve sessionId when saving', async () => {
// Arrange
const { saveTask, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
const task: Task = {
...createMockTask('task-1'),
sessionId: 'session-abc-123',
};
// Act
saveTask(task);
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.sessionId).toBe('session-abc-123');
});
});
describe('getTasks', () => {
it('should return empty array on fresh store', async () => {
// Arrange
const { getTasks } = await import('@main/store/taskHistory');
// Act
const result = getTasks();
// Assert
expect(result).toEqual([]);
});
it('should return all saved tasks', async () => {
// Arrange
const { saveTask, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1', 'Task 1'));
saveTask(createMockTask('task-2', 'Task 2'));
saveTask(createMockTask('task-3', 'Task 3'));
flushPendingTasks();
// Act
const result = getTasks();
// Assert
expect(result).toHaveLength(3);
});
it('should return tasks in reverse chronological order (newest first)', async () => {
// Arrange
const { saveTask, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1', 'First'));
saveTask(createMockTask('task-2', 'Second'));
saveTask(createMockTask('task-3', 'Third'));
flushPendingTasks();
// Act
const result = getTasks();
// Assert - newest should be first (tasks are unshifted)
expect(result[0].id).toBe('task-3');
expect(result[1].id).toBe('task-2');
expect(result[2].id).toBe('task-1');
});
});
describe('updateTaskStatus', () => {
it('should update task status without affecting other fields', async () => {
// Arrange
const { saveTask, updateTaskStatus, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
const task: Task = {
...createMockTask('task-1', 'Status update test'),
messages: [createMockMessage('msg-1')],
sessionId: 'session-123',
};
saveTask(task);
flushPendingTasks();
// Act
updateTaskStatus('task-1', 'completed');
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.status).toBe('completed');
expect(result?.prompt).toBe('Status update test');
expect(result?.messages).toHaveLength(1);
expect(result?.sessionId).toBe('session-123');
});
it('should set completedAt when provided', async () => {
// Arrange
const { saveTask, updateTaskStatus, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
flushPendingTasks();
const completedAt = new Date().toISOString();
// Act
updateTaskStatus('task-1', 'completed', completedAt);
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.status).toBe('completed');
expect(result?.completedAt).toBe(completedAt);
});
it('should not modify non-existent task', async () => {
// Arrange
const { updateTaskStatus, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
// Act
updateTaskStatus('non-existent', 'completed');
flushPendingTasks();
const result = getTasks();
// Assert
expect(result).toHaveLength(0);
});
it('should transition through various statuses correctly', async () => {
// Arrange
const { saveTask, updateTaskStatus, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
flushPendingTasks();
// Act & Assert
updateTaskStatus('task-1', 'running');
flushPendingTasks();
expect(getTask('task-1')?.status).toBe('running');
updateTaskStatus('task-1', 'waiting_permission');
flushPendingTasks();
expect(getTask('task-1')?.status).toBe('waiting_permission');
updateTaskStatus('task-1', 'running');
flushPendingTasks();
expect(getTask('task-1')?.status).toBe('running');
updateTaskStatus('task-1', 'completed');
flushPendingTasks();
expect(getTask('task-1')?.status).toBe('completed');
});
});
describe('addTaskMessage', () => {
it('should append message to task', async () => {
// Arrange
const { saveTask, addTaskMessage, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
flushPendingTasks();
const message = createMockMessage('msg-1', 'assistant', 'Hello there');
// Act
addTaskMessage('task-1', message);
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.messages).toHaveLength(1);
expect(result?.messages[0].content).toBe('Hello there');
});
it('should append multiple messages in order', async () => {
// Arrange
const { saveTask, addTaskMessage, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
flushPendingTasks();
// Act
addTaskMessage('task-1', createMockMessage('msg-1', 'user', 'First'));
addTaskMessage('task-1', createMockMessage('msg-2', 'assistant', 'Second'));
addTaskMessage('task-1', createMockMessage('msg-3', 'tool', 'Third'));
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.messages).toHaveLength(3);
expect(result?.messages[0].content).toBe('First');
expect(result?.messages[1].content).toBe('Second');
expect(result?.messages[2].content).toBe('Third');
});
it('should not modify non-existent task', async () => {
// Arrange
const { addTaskMessage, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
// Act
addTaskMessage('non-existent', createMockMessage('msg-1'));
flushPendingTasks();
const result = getTasks();
// Assert
expect(result).toHaveLength(0);
});
it('should preserve existing messages when adding new ones', async () => {
// Arrange
const { saveTask, addTaskMessage, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
const task: Task = {
...createMockTask('task-1'),
messages: [createMockMessage('msg-1', 'user', 'Existing')],
};
saveTask(task);
flushPendingTasks();
// Act
addTaskMessage('task-1', createMockMessage('msg-2', 'assistant', 'New'));
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.messages).toHaveLength(2);
expect(result?.messages[0].content).toBe('Existing');
expect(result?.messages[1].content).toBe('New');
});
});
describe('deleteTask', () => {
it('should remove only the target task', async () => {
// Arrange
const { saveTask, deleteTask, getTasks, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1', 'Keep this'));
saveTask(createMockTask('task-2', 'Delete this'));
saveTask(createMockTask('task-3', 'Keep this too'));
flushPendingTasks();
// Act
deleteTask('task-2');
flushPendingTasks();
// Assert
expect(getTasks()).toHaveLength(2);
expect(getTask('task-1')).toBeDefined();
expect(getTask('task-2')).toBeUndefined();
expect(getTask('task-3')).toBeDefined();
});
it('should handle deleting non-existent task gracefully', async () => {
// Arrange
const { saveTask, deleteTask, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
flushPendingTasks();
// Act
deleteTask('non-existent');
flushPendingTasks();
// Assert
expect(getTasks()).toHaveLength(1);
});
it('should allow deleting all tasks one by one', async () => {
// Arrange
const { saveTask, deleteTask, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
saveTask(createMockTask('task-2'));
flushPendingTasks();
// Act
deleteTask('task-1');
deleteTask('task-2');
flushPendingTasks();
// Assert
expect(getTasks()).toHaveLength(0);
});
});
describe('clearHistory', () => {
it('should remove all tasks', async () => {
// Arrange
const { saveTask, clearHistory, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
saveTask(createMockTask('task-2'));
saveTask(createMockTask('task-3'));
flushPendingTasks();
// Act
clearHistory();
flushPendingTasks();
// Assert
expect(getTasks()).toHaveLength(0);
});
it('should allow saving new tasks after clear', async () => {
// Arrange
const { saveTask, clearHistory, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
flushPendingTasks();
clearHistory();
flushPendingTasks();
// Act
saveTask(createMockTask('task-new'));
flushPendingTasks();
// Assert
expect(getTasks()).toHaveLength(1);
expect(getTasks()[0].id).toBe('task-new');
});
});
describe('setMaxHistoryItems', () => {
it('should enforce history limit when saving new tasks', async () => {
// Arrange
const { saveTask, setMaxHistoryItems, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
setMaxHistoryItems(3);
// Act - save more than the limit
saveTask(createMockTask('task-1'));
saveTask(createMockTask('task-2'));
saveTask(createMockTask('task-3'));
saveTask(createMockTask('task-4'));
saveTask(createMockTask('task-5'));
flushPendingTasks();
// Assert - should only keep 3 most recent
const tasks = getTasks();
expect(tasks).toHaveLength(3);
expect(tasks[0].id).toBe('task-5');
expect(tasks[1].id).toBe('task-4');
expect(tasks[2].id).toBe('task-3');
});
it('should trim existing history when limit is reduced', async () => {
// Arrange
const { saveTask, setMaxHistoryItems, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
saveTask(createMockTask('task-2'));
saveTask(createMockTask('task-3'));
saveTask(createMockTask('task-4'));
saveTask(createMockTask('task-5'));
flushPendingTasks();
// Act - reduce limit
setMaxHistoryItems(2);
flushPendingTasks();
// Assert
const tasks = getTasks();
expect(tasks).toHaveLength(2);
expect(tasks[0].id).toBe('task-5');
expect(tasks[1].id).toBe('task-4');
});
it('should not affect history when limit is increased', async () => {
// Arrange
const { saveTask, setMaxHistoryItems, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
setMaxHistoryItems(3);
saveTask(createMockTask('task-1'));
saveTask(createMockTask('task-2'));
saveTask(createMockTask('task-3'));
flushPendingTasks();
// Act
setMaxHistoryItems(10);
flushPendingTasks();
// Assert
expect(getTasks()).toHaveLength(3);
});
});
describe('debounced flush behavior', () => {
it('should batch rapid updates into single write', async () => {
// Arrange
const { saveTask, addTaskMessage, flushPendingTasks, getTask } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
// Act - rapid updates without flush
addTaskMessage('task-1', createMockMessage('msg-1'));
addTaskMessage('task-1', createMockMessage('msg-2'));
addTaskMessage('task-1', createMockMessage('msg-3'));
// Force flush
flushPendingTasks();
// Assert
const task = getTask('task-1');
expect(task?.messages).toHaveLength(3);
});
it('should flush pending tasks when explicitly called', async () => {
// Arrange
const { saveTask, flushPendingTasks, getTasks } = await import('@main/store/taskHistory');
// Act - save without waiting for debounce
saveTask(createMockTask('task-1'));
flushPendingTasks();
// Assert - task should be persisted immediately
const tasks = getTasks();
expect(tasks).toHaveLength(1);
});
it('should handle interleaved saves and reads correctly', async () => {
// Arrange
const { saveTask, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
// Act
saveTask(createMockTask('task-1', 'First'));
const afterFirst = getTask('task-1');
saveTask(createMockTask('task-2', 'Second'));
const afterSecond = getTask('task-2');
flushPendingTasks();
// Assert - both should be readable even before flush
expect(afterFirst?.prompt).toBe('First');
expect(afterSecond?.prompt).toBe('Second');
});
});
describe('updateTaskSessionId', () => {
it('should update session ID for existing task', async () => {
// Arrange
const { saveTask, updateTaskSessionId, getTask, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
flushPendingTasks();
// Act
updateTaskSessionId('task-1', 'new-session-xyz');
flushPendingTasks();
const result = getTask('task-1');
// Assert
expect(result?.sessionId).toBe('new-session-xyz');
});
it('should not modify non-existent task', async () => {
// Arrange
const { updateTaskSessionId, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
// Act
updateTaskSessionId('non-existent', 'session-123');
flushPendingTasks();
// Assert
expect(getTasks()).toHaveLength(0);
});
});
describe('clearTaskHistoryStore', () => {
it('should reset store to defaults', async () => {
// Arrange
const { saveTask, clearTaskHistoryStore, getTasks, flushPendingTasks } = await import('@main/store/taskHistory');
saveTask(createMockTask('task-1'));
saveTask(createMockTask('task-2'));
flushPendingTasks();
// Act
clearTaskHistoryStore();
// Assert
expect(getTasks()).toHaveLength(0);
});
it('should clear pending writes without persisting them', async () => {
// Arrange
const { saveTask, clearTaskHistoryStore, getTasks } = await import('@main/store/taskHistory');
// Act - save without flush, then clear
saveTask(createMockTask('task-1'));
clearTaskHistoryStore();
// Assert - pending task should not be persisted
expect(getTasks()).toHaveLength(0);
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/utils/bundled-node.integration.test.ts
================================================
/**
* Integration tests for Bundled Node.js utilities
*
* Tests the bundled-node module which provides paths to bundled Node.js
* binaries for packaged Electron apps.
*
* @module __tests__/integration/main/utils/bundled-node.integration.test
*/
import { describe, it, expect, beforeEach, vi, afterEach } from 'vitest';
import path from 'path';
// Store original values
const originalPlatform = process.platform;
const originalArch = process.arch;
// Mock electron module
const mockApp = {
isPackaged: false,
};
vi.mock('electron', () => ({
app: mockApp,
}));
// Mock fs module
const mockFs = {
existsSync: vi.fn(),
};
vi.mock('fs', () => ({
default: mockFs,
existsSync: mockFs.existsSync,
}));
describe('Bundled Node.js Utilities', () => {
let getBundledNodePaths: typeof import('@main/utils/bundled-node').getBundledNodePaths;
let isBundledNodeAvailable: typeof import('@main/utils/bundled-node').isBundledNodeAvailable;
let getNodePath: typeof import('@main/utils/bundled-node').getNodePath;
let getNpmPath: typeof import('@main/utils/bundled-node').getNpmPath;
let getNpxPath: typeof import('@main/utils/bundled-node').getNpxPath;
let logBundledNodeInfo: typeof import('@main/utils/bundled-node').logBundledNodeInfo;
beforeEach(async () => {
vi.clearAllMocks();
vi.resetModules();
mockApp.isPackaged = false;
// Re-import module to get fresh state
const module = await import('@main/utils/bundled-node');
getBundledNodePaths = module.getBundledNodePaths;
isBundledNodeAvailable = module.isBundledNodeAvailable;
getNodePath = module.getNodePath;
getNpmPath = module.getNpmPath;
getNpxPath = module.getNpxPath;
logBundledNodeInfo = module.logBundledNodeInfo;
});
afterEach(() => {
vi.restoreAllMocks();
// Restore platform/arch
Object.defineProperty(process, 'platform', { value: originalPlatform });
Object.defineProperty(process, 'arch', { value: originalArch });
});
describe('getBundledNodePaths()', () => {
describe('Development Mode', () => {
it('should return null in development mode', () => {
// Arrange
mockApp.isPackaged = false;
// Act
const result = getBundledNodePaths();
// Assert
expect(result).toBeNull();
});
});
describe('Packaged Mode - macOS (darwin)', () => {
beforeEach(() => {
Object.defineProperty(process, 'platform', { value: 'darwin' });
});
it('should return correct paths for arm64 architecture', async () => {
// Arrange
mockApp.isPackaged = true;
Object.defineProperty(process, 'arch', { value: 'arm64' });
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
// Re-import to pick up new process values
vi.resetModules();
const module = await import('@main/utils/bundled-node');
const paths = module.getBundledNodePaths();
// Assert
expect(paths).not.toBeNull();
expect(paths!.nodeDir).toBe(path.join(resourcesPath, 'nodejs', 'arm64'));
expect(paths!.binDir).toBe(path.join(resourcesPath, 'nodejs', 'arm64', 'bin'));
expect(paths!.nodePath).toBe(path.join(resourcesPath, 'nodejs', 'arm64', 'bin', 'node'));
expect(paths!.npmPath).toBe(path.join(resourcesPath, 'nodejs', 'arm64', 'bin', 'npm'));
expect(paths!.npxPath).toBe(path.join(resourcesPath, 'nodejs', 'arm64', 'bin', 'npx'));
});
it('should return correct paths for x64 architecture', async () => {
// Arrange
mockApp.isPackaged = true;
Object.defineProperty(process, 'arch', { value: 'x64' });
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
// Re-import to pick up new process values
vi.resetModules();
const module = await import('@main/utils/bundled-node');
const paths = module.getBundledNodePaths();
// Assert
expect(paths).not.toBeNull();
expect(paths!.nodeDir).toBe(path.join(resourcesPath, 'nodejs', 'x64'));
expect(paths!.binDir).toBe(path.join(resourcesPath, 'nodejs', 'x64', 'bin'));
});
});
describe('Packaged Mode - Windows (win32)', () => {
it('should return correct paths for Windows', async () => {
// Arrange
mockApp.isPackaged = true;
Object.defineProperty(process, 'platform', { value: 'win32' });
Object.defineProperty(process, 'arch', { value: 'x64' });
const resourcesPath = 'C:\\Program Files\\Accomplish\\resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
// Re-import to pick up new process values
vi.resetModules();
const module = await import('@main/utils/bundled-node');
const paths = module.getBundledNodePaths();
// Assert
expect(paths).not.toBeNull();
expect(paths!.nodeDir).toBe(path.join(resourcesPath, 'nodejs', 'x64'));
// Windows: binDir is same as nodeDir
expect(paths!.binDir).toBe(path.join(resourcesPath, 'nodejs', 'x64'));
expect(paths!.nodePath).toBe(path.join(resourcesPath, 'nodejs', 'x64', 'node.exe'));
expect(paths!.npmPath).toBe(path.join(resourcesPath, 'nodejs', 'x64', 'npm.cmd'));
expect(paths!.npxPath).toBe(path.join(resourcesPath, 'nodejs', 'x64', 'npx.cmd'));
});
});
});
describe('isBundledNodeAvailable()', () => {
it('should return false in development mode', () => {
// Arrange
mockApp.isPackaged = false;
// Act
const result = isBundledNodeAvailable();
// Assert
expect(result).toBe(false);
});
it('should return true when bundled node exists', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(true);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Act
const result = module.isBundledNodeAvailable();
// Assert
expect(result).toBe(true);
expect(mockFs.existsSync).toHaveBeenCalled();
});
it('should return false when bundled node does not exist', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(false);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Act
const result = module.isBundledNodeAvailable();
// Assert
expect(result).toBe(false);
});
});
describe('getNodePath()', () => {
it('should return "node" in development mode', () => {
// Arrange
mockApp.isPackaged = false;
// Act
const result = getNodePath();
// Assert
expect(result).toBe('node');
});
it('should return bundled node path when available', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(true);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Act
const result = module.getNodePath();
// Assert
expect(result).toContain('node');
expect(result).not.toBe('node'); // Should be full path
});
it('should fallback to "node" when bundled not found in packaged app', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(false);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Spy on console.warn
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
// Act
const result = module.getNodePath();
// Assert
expect(result).toBe('node');
expect(warnSpy).toHaveBeenCalledWith(
expect.stringContaining('WARNING: Bundled Node.js not found')
);
warnSpy.mockRestore();
});
});
describe('getNpmPath()', () => {
it('should return "npm" in development mode', () => {
// Arrange
mockApp.isPackaged = false;
// Act
const result = getNpmPath();
// Assert
expect(result).toBe('npm');
});
it('should return bundled npm path when available', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(true);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Act
const result = module.getNpmPath();
// Assert
expect(result).toContain('npm');
expect(result).not.toBe('npm'); // Should be full path
});
it('should fallback to "npm" when bundled not found', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(false);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Suppress console.warn
vi.spyOn(console, 'warn').mockImplementation(() => {});
// Act
const result = module.getNpmPath();
// Assert
expect(result).toBe('npm');
});
});
describe('getNpxPath()', () => {
it('should return "npx" in development mode', () => {
// Arrange
mockApp.isPackaged = false;
// Act
const result = getNpxPath();
// Assert
expect(result).toBe('npx');
});
it('should return bundled npx path when available', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(true);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Act
const result = module.getNpxPath();
// Assert
expect(result).toContain('npx');
expect(result).not.toBe('npx'); // Should be full path
});
it('should fallback to "npx" when bundled not found', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(false);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Suppress console.warn
vi.spyOn(console, 'warn').mockImplementation(() => {});
// Act
const result = module.getNpxPath();
// Assert
expect(result).toBe('npx');
});
});
describe('logBundledNodeInfo()', () => {
it('should log development mode message when not packaged', () => {
// Arrange
mockApp.isPackaged = false;
const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
// Act
logBundledNodeInfo();
// Assert
expect(logSpy).toHaveBeenCalledWith(
expect.stringContaining('Development mode')
);
logSpy.mockRestore();
});
it('should log bundled node configuration when packaged', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
mockFs.existsSync.mockReturnValue(true);
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
const logSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
// Act
module.logBundledNodeInfo();
// Assert
expect(logSpy).toHaveBeenCalledWith(expect.stringContaining('Configuration'));
expect(logSpy).toHaveBeenCalledWith(expect.stringContaining('Platform'));
expect(logSpy).toHaveBeenCalledWith(expect.stringContaining('Architecture'));
expect(logSpy).toHaveBeenCalledWith(expect.stringContaining('Node directory'));
expect(logSpy).toHaveBeenCalledWith(expect.stringContaining('Node path'));
expect(logSpy).toHaveBeenCalledWith(expect.stringContaining('Available'));
logSpy.mockRestore();
});
});
describe('BundledNodePaths Interface', () => {
it('should return all required path properties', async () => {
// Arrange
mockApp.isPackaged = true;
const resourcesPath = '/Applications/Accomplish.app/Contents/Resources';
(process as NodeJS.Process & { resourcesPath: string }).resourcesPath = resourcesPath;
// Re-import
vi.resetModules();
const module = await import('@main/utils/bundled-node');
// Act
const paths = module.getBundledNodePaths();
// Assert
expect(paths).not.toBeNull();
expect(paths).toHaveProperty('nodePath');
expect(paths).toHaveProperty('npmPath');
expect(paths).toHaveProperty('npxPath');
expect(paths).toHaveProperty('binDir');
expect(paths).toHaveProperty('nodeDir');
// All should be strings
expect(typeof paths!.nodePath).toBe('string');
expect(typeof paths!.npmPath).toBe('string');
expect(typeof paths!.npxPath).toBe('string');
expect(typeof paths!.binDir).toBe('string');
expect(typeof paths!.nodeDir).toBe('string');
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/main/utils/system-path.integration.test.ts
================================================
/**
* Integration tests for System PATH utilities
*
* Tests the system-path module which builds extended PATH strings for
* finding Node.js tools in macOS packaged apps.
*
* @module __tests__/integration/main/utils/system-path.integration.test
*/
import { describe, it, expect, beforeEach, vi, afterEach } from 'vitest';
import path from 'path';
// Store original values
const originalPlatform = process.platform;
const originalEnv = { ...process.env };
// Mock fs module
const mockFs = {
existsSync: vi.fn(),
readdirSync: vi.fn(),
statSync: vi.fn(),
accessSync: vi.fn(),
constants: {
X_OK: 1,
},
};
vi.mock('fs', () => ({
default: mockFs,
existsSync: mockFs.existsSync,
readdirSync: mockFs.readdirSync,
statSync: mockFs.statSync,
accessSync: mockFs.accessSync,
constants: mockFs.constants,
}));
// Mock child_process
const mockExecSync = vi.fn();
vi.mock('child_process', () => ({
execSync: mockExecSync,
}));
describe('System PATH Utilities', () => {
let getExtendedNodePath: typeof import('@main/utils/system-path').getExtendedNodePath;
let findCommandInPath: typeof import('@main/utils/system-path').findCommandInPath;
beforeEach(async () => {
vi.clearAllMocks();
vi.resetModules();
// Reset environment
process.env = { ...originalEnv };
process.env.HOME = '/Users/testuser';
// Re-import module to get fresh state
const module = await import('@main/utils/system-path');
getExtendedNodePath = module.getExtendedNodePath;
findCommandInPath = module.findCommandInPath;
});
afterEach(() => {
vi.restoreAllMocks();
Object.defineProperty(process, 'platform', { value: originalPlatform });
process.env = originalEnv;
});
describe('getExtendedNodePath()', () => {
describe('Non-macOS Platforms', () => {
it('should return base PATH unchanged on Linux', async () => {
// Arrange
Object.defineProperty(process, 'platform', { value: 'linux' });
const basePath = '/usr/bin:/usr/local/bin';
// Re-import for platform change
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath(basePath);
// Assert
expect(result).toBe(basePath);
});
it('should return base PATH unchanged on Windows', async () => {
// Arrange
Object.defineProperty(process, 'platform', { value: 'win32' });
const basePath = 'C:\\Windows\\System32';
// Re-import for platform change
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath(basePath);
// Assert
expect(result).toBe(basePath);
});
});
describe('macOS Platform', () => {
beforeEach(() => {
Object.defineProperty(process, 'platform', { value: 'darwin' });
});
it('should include common Node.js paths', async () => {
// Arrange
mockFs.existsSync.mockImplementation((p: string) => {
const existingPaths = [
'/opt/homebrew/bin',
'/usr/local/bin',
];
return existingPaths.includes(p);
});
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('PATH="/usr/bin:/bin"; export PATH;');
// Re-import for platform change
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('/original/path');
// Assert
expect(result).toContain('/opt/homebrew/bin');
expect(result).toContain('/usr/local/bin');
});
it('should include NVM paths when available', async () => {
// Arrange
const nvmPath = '/Users/testuser/.nvm/versions/node/v20.10.0/bin';
mockFs.existsSync.mockImplementation((p: string) => {
if (p === '/Users/testuser/.nvm/versions/node') return true;
if (p === nvmPath) return true;
return false;
});
mockFs.readdirSync.mockImplementation((p: string) => {
if (p === '/Users/testuser/.nvm/versions/node') return ['v20.10.0'];
return [];
});
mockExecSync.mockReturnValue('PATH="/usr/bin"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('');
// Assert
expect(result).toContain(nvmPath);
});
it('should include fnm paths when available', async () => {
// Arrange
const fnmPath = '/Users/testuser/.fnm/node-versions/v20.10.0/installation/bin';
mockFs.existsSync.mockImplementation((p: string) => {
if (p === '/Users/testuser/.fnm/node-versions') return true;
if (p === fnmPath) return true;
return false;
});
mockFs.readdirSync.mockImplementation((p: string) => {
if (p === '/Users/testuser/.fnm/node-versions') return ['v20.10.0'];
return [];
});
mockExecSync.mockReturnValue('PATH="/usr/bin"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('');
// Assert
expect(result).toContain(fnmPath);
});
it('should sort NVM versions with newest first', async () => {
// Arrange
const nvmDir = '/Users/testuser/.nvm/versions/node';
mockFs.existsSync.mockImplementation((p: string) => {
if (p === nvmDir) return true;
if (p.includes('.nvm/versions/node/v')) return true;
return false;
});
mockFs.readdirSync.mockImplementation((p: string) => {
if (p === nvmDir) return ['v18.17.0', 'v20.10.0', 'v16.20.0'];
return [];
});
mockExecSync.mockReturnValue('PATH="/usr/bin"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('');
const pathParts = result.split(':');
// Assert - v20 should come before v18 which should come before v16
const v20Index = pathParts.findIndex((p) => p.includes('v20'));
const v18Index = pathParts.findIndex((p) => p.includes('v18'));
const v16Index = pathParts.findIndex((p) => p.includes('v16'));
expect(v20Index).toBeLessThan(v18Index);
expect(v18Index).toBeLessThan(v16Index);
});
it('should include path_helper output', async () => {
// Arrange
mockFs.existsSync.mockReturnValue(false);
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('PATH="/custom/path:/another/path"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('');
// Assert
expect(result).toContain('/custom/path');
expect(result).toContain('/another/path');
});
it('should handle path_helper failure gracefully', async () => {
// Arrange
mockFs.existsSync.mockReturnValue(false);
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockImplementation(() => {
throw new Error('path_helper failed');
});
const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act - should not throw
const result = module.getExtendedNodePath('/base/path');
// Assert
expect(result).toContain('/base/path');
warnSpy.mockRestore();
});
it('should deduplicate paths', async () => {
// Arrange
mockFs.existsSync.mockImplementation((p: string) => {
return p === '/usr/local/bin';
});
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('PATH="/usr/local/bin:/usr/bin"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('/usr/local/bin');
// Assert - /usr/local/bin should appear only once
const pathParts = result.split(':');
const localBinCount = pathParts.filter((p) => p === '/usr/local/bin').length;
expect(localBinCount).toBe(1);
});
it('should use process.env.PATH as default base', async () => {
// Arrange
process.env.PATH = '/default/env/path';
mockFs.existsSync.mockReturnValue(false);
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('PATH="/usr/bin"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath();
// Assert
expect(result).toContain('/default/env/path');
});
it('should include Volta path when available', async () => {
// Arrange
const voltaPath = '/Users/testuser/.volta/bin';
mockFs.existsSync.mockImplementation((p: string) => {
return p === voltaPath;
});
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('PATH="/usr/bin"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('');
// Assert
expect(result).toContain(voltaPath);
});
it('should include asdf shims path when available', async () => {
// Arrange
const asdfPath = '/Users/testuser/.asdf/shims';
mockFs.existsSync.mockImplementation((p: string) => {
return p === asdfPath;
});
mockFs.readdirSync.mockReturnValue([]);
mockExecSync.mockReturnValue('PATH="/usr/bin"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('');
// Assert
expect(result).toContain(asdfPath);
});
});
});
describe('findCommandInPath()', () => {
it('should find executable command in PATH', () => {
// Arrange
const searchPath = '/usr/bin:/usr/local/bin';
const expectedPath = '/usr/local/bin/node';
mockFs.existsSync.mockImplementation((p: string) => {
return p === expectedPath;
});
mockFs.statSync.mockReturnValue({ isFile: () => true });
mockFs.accessSync.mockImplementation(() => {}); // No throw = executable
// Act
const result = findCommandInPath('node', searchPath);
// Assert
expect(result).toBe(expectedPath);
});
it('should return null when command not found', () => {
// Arrange
const searchPath = '/usr/bin:/usr/local/bin';
mockFs.existsSync.mockReturnValue(false);
// Act
const result = findCommandInPath('nonexistent', searchPath);
// Assert
expect(result).toBeNull();
});
it('should skip non-file entries', () => {
// Arrange
const searchPath = '/usr/bin';
mockFs.existsSync.mockReturnValue(true);
mockFs.statSync.mockReturnValue({ isFile: () => false }); // Directory
// Act
const result = findCommandInPath('node', searchPath);
// Assert
expect(result).toBeNull();
});
it('should skip non-executable files', () => {
// Arrange
const searchPath = '/usr/bin';
mockFs.existsSync.mockReturnValue(true);
mockFs.statSync.mockReturnValue({ isFile: () => true });
mockFs.accessSync.mockImplementation(() => {
throw new Error('Not executable');
});
// Act
const result = findCommandInPath('node', searchPath);
// Assert
expect(result).toBeNull();
});
it('should search directories in order', () => {
// Arrange
const searchPath = '/first/bin:/second/bin';
const firstPath = '/first/bin/node';
const secondPath = '/second/bin/node';
mockFs.existsSync.mockImplementation((p: string) => {
return p === firstPath || p === secondPath;
});
mockFs.statSync.mockReturnValue({ isFile: () => true });
mockFs.accessSync.mockImplementation(() => {});
// Act
const result = findCommandInPath('node', searchPath);
// Assert
expect(result).toBe(firstPath);
});
it('should handle empty path segments', () => {
// Arrange
const searchPath = '/usr/bin::/usr/local/bin';
const expectedPath = '/usr/local/bin/node';
mockFs.existsSync.mockImplementation((p: string) => {
return p === expectedPath;
});
mockFs.statSync.mockReturnValue({ isFile: () => true });
mockFs.accessSync.mockImplementation(() => {});
// Act
const result = findCommandInPath('node', searchPath);
// Assert
expect(result).toBe(expectedPath);
});
it('should handle directory access errors gracefully', () => {
// Arrange
const searchPath = '/nonexistent:/usr/local/bin';
const expectedPath = '/usr/local/bin/node';
mockFs.existsSync.mockImplementation((p: string) => {
if (p.startsWith('/nonexistent')) {
throw new Error('Directory does not exist');
}
return p === expectedPath;
});
mockFs.statSync.mockReturnValue({ isFile: () => true });
mockFs.accessSync.mockImplementation(() => {});
// Act - should not throw
const result = findCommandInPath('node', searchPath);
// Assert
expect(result).toBe(expectedPath);
});
it('should handle statSync errors gracefully', () => {
// Arrange
const searchPath = '/usr/bin:/usr/local/bin';
const expectedPath = '/usr/local/bin/node';
mockFs.existsSync.mockReturnValue(true);
mockFs.statSync.mockImplementation((p: string) => {
if (p === '/usr/bin/node') {
throw new Error('Stat error');
}
return { isFile: () => p === expectedPath };
});
mockFs.accessSync.mockImplementation(() => {});
// Act
const result = findCommandInPath('node', searchPath);
// Assert
expect(result).toBe(expectedPath);
});
});
describe('Path Priority Order', () => {
it('should prioritize version manager paths over system paths', async () => {
// Arrange
Object.defineProperty(process, 'platform', { value: 'darwin' });
const nvmPath = '/Users/testuser/.nvm/versions/node/v20.10.0/bin';
mockFs.existsSync.mockImplementation((p: string) => {
if (p === '/Users/testuser/.nvm/versions/node') return true;
if (p === nvmPath) return true;
if (p === '/opt/homebrew/bin') return true;
if (p === '/usr/local/bin') return true;
return false;
});
mockFs.readdirSync.mockImplementation((p: string) => {
if (p === '/Users/testuser/.nvm/versions/node') return ['v20.10.0'];
return [];
});
mockExecSync.mockReturnValue('PATH="/usr/bin"; export PATH;');
// Re-import
vi.resetModules();
const module = await import('@main/utils/system-path');
// Act
const result = module.getExtendedNodePath('');
const pathParts = result.split(':');
// Assert - NVM should come before Homebrew
const nvmIndex = pathParts.findIndex((p) => p.includes('.nvm'));
const homebrewIndex = pathParts.findIndex((p) => p.includes('homebrew'));
expect(nvmIndex).toBeLessThan(homebrewIndex);
});
});
});
================================================
FILE: apps/openwork-memos-integration/apps/desktop/__tests__/integration/preload/preload.integration.test.ts
================================================
/**
* Integration tests for Preload script
*
* Tests the REAL preload script by:
* 1. Mocking electron APIs (external dependency)
* 2. Importing the real preload module (triggers contextBridge.exposeInMainWorld)
* 3. Verifying the exposed API calls the correct IPC channels
*
* This is a proper integration test - only external dependencies are mocked.
*/
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import pkg from '../../../package.json';
// Create mock functions for electron
const mockExposeInMainWorld = vi.fn();
const mockInvoke = vi.fn(() => Promise.resolve(undefined));
const mockOn = vi.fn();
const mockRemoveListener = vi.fn();
// Mock electron module before importing preload
vi.mock('electron', () => ({
contextBridge: {
exposeInMainWorld: mockExposeInMainWorld,
},
ipcRenderer: {
invoke: mockInvoke,
on: mockOn,
removeListener: mockRemoveListener,
},
}));
// Store captured APIs from exposeInMainWorld calls
let capturedAccomplishAPI: Record = {};
let capturedAccomplishShell: Record = {};
describe('Preload Script Integration', () => {
beforeEach(async () => {
vi.clearAllMocks();
capturedAccomplishAPI = {};
capturedAccomplishShell = {};
// Capture what the real preload exposes
mockExposeInMainWorld.mockImplementation((name: string, api: unknown) => {
if (name === 'accomplish') {
capturedAccomplishAPI = api as Record;
} else if (name === 'accomplishShell') {
capturedAccomplishShell = api as Record;
}
});
// Reset module cache and import the REAL preload module
vi.resetModules();
await import('../../../src/preload/index');
});
afterEach(() => {
vi.clearAllMocks();
});
describe('API Exposure', () => {
it('should expose accomplish API via contextBridge', () => {
expect(mockExposeInMainWorld).toHaveBeenCalledWith('accomplish', expect.any(Object));
expect(capturedAccomplishAPI).toBeDefined();
});
it('should expose accomplishShell info via contextBridge', () => {
expect(mockExposeInMainWorld).toHaveBeenCalledWith('accomplishShell', expect.any(Object));
expect(capturedAccomplishShell).toBeDefined();
});
it('should expose shell info with isElectron=true', () => {
expect(capturedAccomplishShell.isElectron).toBe(true);
});
it('should expose shell info with platform', () => {
expect(capturedAccomplishShell.platform).toBe(process.platform);
});
it('should expose shell info with version matching package.json', () => {
expect(capturedAccomplishShell.version).toBe(pkg.version);
});
});
describe('IPC Method Invocations', () => {
describe('App Info', () => {
it('getVersion should invoke app:version', async () => {
await (capturedAccomplishAPI.getVersion as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('app:version');
});
it('getPlatform should invoke app:platform', async () => {
await (capturedAccomplishAPI.getPlatform as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('app:platform');
});
});
describe('Shell Operations', () => {
it('openExternal should invoke shell:open-external with URL', async () => {
const url = 'https://example.com';
await (capturedAccomplishAPI.openExternal as (url: string) => Promise)(url);
expect(mockInvoke).toHaveBeenCalledWith('shell:open-external', url);
});
});
describe('Task Operations', () => {
it('startTask should invoke task:start with config', async () => {
const config = { description: 'Test task' };
await (capturedAccomplishAPI.startTask as (config: { description: string }) => Promise)(config);
expect(mockInvoke).toHaveBeenCalledWith('task:start', config);
});
it('cancelTask should invoke task:cancel with taskId', async () => {
await (capturedAccomplishAPI.cancelTask as (taskId: string) => Promise)('task_123');
expect(mockInvoke).toHaveBeenCalledWith('task:cancel', 'task_123');
});
it('interruptTask should invoke task:interrupt with taskId', async () => {
await (capturedAccomplishAPI.interruptTask as (taskId: string) => Promise)('task_123');
expect(mockInvoke).toHaveBeenCalledWith('task:interrupt', 'task_123');
});
it('getTask should invoke task:get with taskId', async () => {
await (capturedAccomplishAPI.getTask as (taskId: string) => Promise)('task_123');
expect(mockInvoke).toHaveBeenCalledWith('task:get', 'task_123');
});
it('listTasks should invoke task:list', async () => {
await (capturedAccomplishAPI.listTasks as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('task:list');
});
it('deleteTask should invoke task:delete with taskId', async () => {
await (capturedAccomplishAPI.deleteTask as (taskId: string) => Promise)('task_123');
expect(mockInvoke).toHaveBeenCalledWith('task:delete', 'task_123');
});
it('clearTaskHistory should invoke task:clear-history', async () => {
await (capturedAccomplishAPI.clearTaskHistory as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('task:clear-history');
});
});
describe('Permission Operations', () => {
it('respondToPermission should invoke permission:respond', async () => {
const response = { taskId: 'task_123', allowed: true };
await (capturedAccomplishAPI.respondToPermission as (r: { taskId: string; allowed: boolean }) => Promise)(response);
expect(mockInvoke).toHaveBeenCalledWith('permission:respond', response);
});
});
describe('Session Operations', () => {
it('resumeSession should invoke session:resume', async () => {
await (capturedAccomplishAPI.resumeSession as (s: string, p: string, t?: string) => Promise)('session_123', 'Continue', 'task_456');
expect(mockInvoke).toHaveBeenCalledWith('session:resume', 'session_123', 'Continue', 'task_456');
});
});
describe('Settings Operations', () => {
it('getDebugMode should invoke settings:debug-mode', async () => {
await (capturedAccomplishAPI.getDebugMode as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('settings:debug-mode');
});
it('setDebugMode should invoke settings:set-debug-mode', async () => {
await (capturedAccomplishAPI.setDebugMode as (enabled: boolean) => Promise)(true);
expect(mockInvoke).toHaveBeenCalledWith('settings:set-debug-mode', true);
});
it('getAppSettings should invoke settings:app-settings', async () => {
await (capturedAccomplishAPI.getAppSettings as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('settings:app-settings');
});
});
describe('API Key Operations', () => {
it('hasApiKey should invoke api-key:exists', async () => {
await (capturedAccomplishAPI.hasApiKey as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('api-key:exists');
});
it('setApiKey should invoke api-key:set', async () => {
await (capturedAccomplishAPI.setApiKey as (key: string) => Promise)('sk-test');
expect(mockInvoke).toHaveBeenCalledWith('api-key:set', 'sk-test');
});
it('getApiKey should invoke api-key:get', async () => {
await (capturedAccomplishAPI.getApiKey as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('api-key:get');
});
it('validateApiKey should invoke api-key:validate', async () => {
await (capturedAccomplishAPI.validateApiKey as (key: string) => Promise)('sk-test');
expect(mockInvoke).toHaveBeenCalledWith('api-key:validate', 'sk-test');
});
it('clearApiKey should invoke api-key:clear', async () => {
await (capturedAccomplishAPI.clearApiKey as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('api-key:clear');
});
it('getAllApiKeys should invoke api-keys:all', async () => {
await (capturedAccomplishAPI.getAllApiKeys as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('api-keys:all');
});
it('hasAnyApiKey should invoke api-keys:has-any', async () => {
await (capturedAccomplishAPI.hasAnyApiKey as () => Promise)();
expect(mockInvoke).toHaveBeenCalledWith('api-keys:has-any');
});
});
describe('Onboarding Operations', () => {
it('getOnboardingComplete should invoke onboarding:complete', async () => {
await (capturedAccomplishAPI.getOnboardingComplete as () => Promise