[ { "path": ".agents/skills/cross-repo-testing/SKILL.md", "content": "---\nname: cross-repo-testing\ndescription: This skill should be used when the user asks to \"test a saas cross-repo feature\", \"deploy a feature branch to staging\", \"test SDK against OH Cloud branch\", \"e2e test a cloud workspace feature\", \"test secrets saas inheritance\", or when changes span the SDK and OpenHands enterprise and need end-to-end validation against a staging deployment.\n---\n\n# Cross-Repo Testing: SDK ↔ OpenHands Cloud\n\nHow to end-to-end test features that span `OpenHands/software-agent-sdk` and `OpenHands/OpenHands` (the Cloud backend).\n\n## Repository Map\n\n| Repo | Role | What lives here |\n|------|------|-----------------|\n| [`software-agent-sdk`](https://github.com/OpenHands/software-agent-sdk) | Agent core | `openhands-sdk`, `openhands-workspace`, `openhands-tools` packages. `OpenHandsCloudWorkspace` lives here. |\n| [`OpenHands`](https://github.com/OpenHands/OpenHands) | Cloud backend | FastAPI server (`openhands/app_server/`), sandbox management, auth, enterprise integrations. Deployed as OH Cloud. |\n| [`deploy`](https://github.com/OpenHands/deploy) | Infrastructure | Helm charts + GitHub Actions that build the enterprise Docker image and deploy to staging/production. |\n\n**Data flow:** SDK client → OH Cloud API (`/api/v1/...`) → sandbox agent-server (inside runtime container)\n\n## When You Need This\n\nThere are **two flows** depending on which direction the dependency goes:\n\n| Flow | When | Example |\n|------|------|---------|\n| **A — SDK client → new Cloud API** | The SDK calls an API that doesn't exist yet on production | `workspace.get_llm()` calling `GET /api/v1/users/me?expose_secrets=true` |\n| **B — OH server → new SDK code** | The Cloud server needs unreleased SDK packages or a new agent-server image | Server consumes a new tool, agent behavior, or workspace method from the SDK |\n\nFlow A only requires deploying the server PR. Flow B requires pinning the SDK to an unreleased commit in the server PR **and** using the SDK PR's agent-server image. Both flows may apply simultaneously.\n\n---\n\n## Flow A: SDK Client Tests Against New Cloud API\n\nUse this when the SDK calls an endpoint that only exists on the server PR branch.\n\n### A1. Write and test the server-side changes\n\nIn the `OpenHands` repo, implement the new API endpoint(s). Run unit tests:\n\n```bash\ncd OpenHands\npoetry run pytest tests/unit/app_server/test_.py -v\n```\n\nPush a PR. Wait for the **\"Push Enterprise Image\" (Docker) CI job** to succeed — this builds `ghcr.io/openhands/enterprise-server:sha-`.\n\n### A2. Write the SDK-side changes\n\nIn `software-agent-sdk`, implement the client code (e.g., new methods on `OpenHandsCloudWorkspace`). Run SDK unit tests:\n\n```bash\ncd software-agent-sdk\npip install -e openhands-sdk -e openhands-workspace\npytest tests/ -v\n```\n\nPush a PR. SDK CI is independent — it doesn't need the server changes to pass unit tests.\n\n### A3. Deploy the server PR to staging\n\nSee [Deploying to a Staging Feature Environment](#deploying-to-a-staging-feature-environment) below.\n\n### A4. Run the SDK e2e test against staging\n\nSee [Running E2E Tests Against Staging](#running-e2e-tests-against-staging) below.\n\n---\n\n## Flow B: OH Server Needs Unreleased SDK Code\n\nUse this when the Cloud server depends on SDK changes that haven't been released to PyPI yet. The server's runtime containers run the `agent-server` image built from the SDK repo, so the server PR must be configured to use the SDK PR's image and packages.\n\n### B1. Get the SDK PR merged (or identify the commit)\n\nThe SDK PR must have CI pass so its agent-server Docker image is built. The image is tagged with the **merge-commit SHA** from GitHub Actions — NOT the head-commit SHA shown in the PR.\n\nFind the correct image tag:\n- Check the SDK PR description for an `AGENT_SERVER_IMAGES` section\n- Or check the \"Consolidate Build Information\" CI job for `\"short_sha\": \"\"`\n\n### B2. Pin SDK packages to the commit in the OpenHands PR\n\nIn the `OpenHands` repo PR, update 3 files + regenerate 3 lock files (see the `update-sdk` skill for full details):\n\n**`pyproject.toml`** — pin all 3 SDK packages in **both** `dependencies` and `[tool.poetry.dependencies]`:\n```toml\n# dependencies array (PEP 508)\n\"openhands-sdk @ git+https://github.com/OpenHands/software-agent-sdk.git@#subdirectory=openhands-sdk\",\n\"openhands-agent-server @ git+https://github.com/OpenHands/software-agent-sdk.git@#subdirectory=openhands-agent-server\",\n\"openhands-tools @ git+https://github.com/OpenHands/software-agent-sdk.git@#subdirectory=openhands-tools\",\n\n# [tool.poetry.dependencies]\nopenhands-sdk = { git = \"https://github.com/OpenHands/software-agent-sdk.git\", rev = \"\", subdirectory = \"openhands-sdk\" }\nopenhands-agent-server = { git = \"https://github.com/OpenHands/software-agent-sdk.git\", rev = \"\", subdirectory = \"openhands-agent-server\" }\nopenhands-tools = { git = \"https://github.com/OpenHands/software-agent-sdk.git\", rev = \"\", subdirectory = \"openhands-tools\" }\n```\n\n**`openhands/app_server/sandbox/sandbox_spec_service.py`** — use the SDK's merge-commit SHA:\n```python\nAGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:-python'\n```\n\n**Regenerate lock files:**\n```bash\npoetry lock && uv lock && cd enterprise && poetry lock && cd ..\n```\n\n### B3. Wait for the OpenHands enterprise image to build\n\nPush the pinned changes. The OpenHands CI will build a new enterprise Docker image (`ghcr.io/openhands/enterprise-server:sha-`) that bundles the unreleased SDK. Wait for the \"Push Enterprise Image\" job to succeed.\n\n### B4. Deploy and test\n\nFollow [Deploying to a Staging Feature Environment](#deploying-to-a-staging-feature-environment) using the new OpenHands commit SHA.\n\n### B5. Before merging: remove the pin\n\n**CI guard:** `check-package-versions.yml` blocks merge to `main` if `[tool.poetry.dependencies]` contains `rev` fields. Before the OpenHands PR can merge, the SDK PR must be merged and released to PyPI, then the pin must be replaced with the released version number.\n\n---\n\n## Deploying to a Staging Feature Environment\n\nThe `deploy` repo creates preview environments from OpenHands PRs.\n\n**Option A — GitHub Actions UI (preferred):**\nGo to `OpenHands/deploy` → Actions → \"Create OpenHands preview PR\" → enter the OpenHands PR number. This creates a branch `ohpr--` and opens a deploy PR.\n\n**Option B — Update an existing feature branch:**\n```bash\ncd deploy\ngit checkout ohpr--\n# In .github/workflows/deploy.yaml, update BOTH:\n# OPENHANDS_SHA: \"\"\n# OPENHANDS_RUNTIME_IMAGE_TAG: \"-nikolaik\"\ngit commit -am \"Update OPENHANDS_SHA to \" && git push\n```\n\n**Before updating the SHA**, verify the enterprise Docker image exists:\n```bash\ngh api repos/OpenHands/OpenHands/actions/runs \\\n --jq '.workflow_runs[] | select(.head_sha==\"\") | \"\\(.name): \\(.conclusion)\"' \\\n | grep Docker\n# Must show: \"Docker: success\"\n```\n\nThe deploy CI auto-triggers and creates the environment at:\n```\nhttps://ohpr--.staging.all-hands.dev\n```\n\n**Wait for it to be live:**\n```bash\ncurl -s -o /dev/null -w \"%{http_code}\" https://ohpr--.staging.all-hands.dev/api/v1/health\n# 401 = server is up (auth required). DNS may take 1-2 min on first deploy.\n```\n\n## Running E2E Tests Against Staging\n\n**Critical: Feature deployments have their own Keycloak instance.** API keys from `app.all-hands.dev` or `$OPENHANDS_API_KEY` will NOT work. You need a test API key for the specific feature deployment. The user must provide one.\n\n```python\nfrom openhands.workspace import OpenHandsCloudWorkspace\n\nSTAGING = \"https://ohpr--.staging.all-hands.dev\"\n\nwith OpenHandsCloudWorkspace(\n cloud_api_url=STAGING,\n cloud_api_key=\"\",\n) as workspace:\n # Test the new feature\n llm = workspace.get_llm()\n secrets = workspace.get_secrets()\n print(f\"LLM: {llm.model}, secrets: {list(secrets.keys())}\")\n```\n\nOr run an example script:\n```bash\nOPENHANDS_CLOUD_API_KEY=\"\" \\\nOPENHANDS_CLOUD_API_URL=\"https://ohpr--.staging.all-hands.dev\" \\\npython examples/02_remote_agent_server/10_cloud_workspace_saas_credentials.py\n```\n\n### Recording results\n\nPush test output to the SDK PR's `.pr/logs/` directory:\n```bash\ncd software-agent-sdk\npython test_script.py 2>&1 | tee .pr/logs/.log\ngit add -f .pr/logs/.log .pr/README.md\ngit commit -m \"docs: add e2e test results\" && git push\n```\n\nComment on **both PRs** with pass/fail summary and link to logs.\n\n## Key Gotchas\n\n| Gotcha | Details |\n|--------|---------|\n| **Feature env auth is isolated** | Each `ohpr-*` deployment has its own Keycloak. Production API keys don't work. |\n| **Two SHAs in deploy.yaml** | `OPENHANDS_SHA` and `OPENHANDS_RUNTIME_IMAGE_TAG` must both be updated. The runtime tag is `-nikolaik`. |\n| **Enterprise image must exist** | The Docker CI job on the OpenHands PR must succeed before you can deploy. If it hasn't run, push an empty commit to trigger it. |\n| **DNS propagation** | First deployment of a new branch takes 1-2 min for DNS. Subsequent deploys are instant. |\n| **Merge-commit SHA ≠ head SHA** | SDK CI tags Docker images with GitHub Actions' merge-commit SHA, not the PR head SHA. Check the SDK PR description or CI logs for the correct tag. |\n| **SDK pin blocks merge** | `check-package-versions.yml` prevents merging an OpenHands PR that has `rev` fields in `[tool.poetry.dependencies]`. The SDK must be released to PyPI first. |\n| **Flow A: stock agent-server is fine** | When only the Cloud API changes, `OpenHandsCloudWorkspace` talks to the Cloud server, not the agent-server. No custom image needed. |\n| **Flow B: agent-server image is required** | When the server needs new SDK code inside runtime containers, you must pin to the SDK PR's agent-server image. |\n" }, { "path": ".agents/skills/custom-codereview-guide.md", "content": "---\nname: custom-codereview-guide\ndescription: Repo-specific code review guidelines for OpenHands/software-agent-sdk. Provides SDK-specific review rules in addition to the default code review skill.\ntriggers:\n- /codereview\n---\n\n# OpenHands/software-agent-sdk Code Review Guidelines\n\nYou are an expert code reviewer for the **OpenHands/software-agent-sdk** repository. This skill provides repo-specific review guidelines. Be direct but constructive.\n\n## Review Decisions\n\nYou have permission to **APPROVE** or **COMMENT** on PRs. Do not use REQUEST_CHANGES.\n\n### Review decision policy (eval / benchmark risk)\n\nDo **NOT** submit an **APPROVE** review when the PR changes agent behavior or anything\nthat could plausibly affect benchmark/evaluation performance — **unless** eval evidence\nis already provided (see exception below).\n\nExamples include: prompt templates, tool calling/execution, planning/loop logic,\nmemory/condenser behavior, terminal/stdin/stdout handling, or evaluation harness code.\n\nIf a PR is in this category (or you are uncertain), leave a **COMMENT** review and\nexplicitly flag it for a human maintainer to decide after running lightweight evals.\n\n#### Exception – eval evidence provided\n\nIf the PR description **or** PR comments contain a link to the eval monitor\n(`openhands-eval-monitor.vercel.app`) showing a completed benchmark run **and**\na human maintainer has commented confirming the results (e.g., \"Human review done\",\n\"eval looks good\", or similar), treat the eval-risk requirement as satisfied and\nfollow the normal approval policy. The eval monitor link is authoritative proof of\nbenchmark validation for this repository.\n\n### Default approval policy\n\n**Default to APPROVE**: If your review finds no issues at \"important\" level or higher,\napprove the PR. Minor suggestions or nitpicks alone are not sufficient reason to\nwithhold approval.\n\n**IMPORTANT:** If you determine a PR is worth merging **and it is not in the eval-risk\ncategory above**, you should approve it. Don’t just say a PR is \"worth merging\" or\n\"ready to merge\" without actually submitting an approval. Your words and actions should\nbe consistent.\n\n### When to APPROVE\n\nExamples of straightforward and low-risk PRs you should approve (non-exhaustive):\n\n- **Configuration changes**: Adding models to config files, updating CI/workflow settings\n- **CI/Infrastructure changes**: Changing runner types, fixing workflow paths, updating job configurations\n- **Cosmetic changes**: Typo fixes, formatting, comment improvements, README updates\n- **Documentation-only changes**: Docstring updates, clarifying notes, API documentation improvements\n- **Simple additions**: Adding entries to lists/dictionaries following existing patterns\n- **Test-only changes**: Adding or updating tests without changing production code\n- **Dependency updates**: Version bumps with passing CI, unless the updated package is newer than the repo's 7-day freshness guardrail described in the Security section below\n\n### When NOT to APPROVE - Blocking Issues\n\n**DO NOT APPROVE** PRs that have any of the following issues:\n\n- **Package version bumps in non-release PRs**: If any `pyproject.toml` file has changes to the `version` field (e.g., `version = \"1.12.0\"` → `version = \"1.13.0\"`), and the PR is NOT explicitly a release PR (title/description doesn't indicate it's a release), **DO NOT APPROVE**. Version numbers should only be changed in dedicated release PRs managed by maintainers.\n - Check: Look for changes to `version = \"...\"` in any `*/pyproject.toml` files\n - Exception: PRs with titles like \"release: v1.x.x\" or \"chore: bump version to 1.x.x\" from maintainers\n- **Too-new dependency uploads**: If a dependency bump pulls in a package uploaded within the repo's 7-day freshness window, **DO NOT APPROVE**. See the Security section below for the exact review instructions and the Dependabot / `tool.uv.exclude-newer` caveat.\n\nExamples:\n- A PR adding a new model to `resolve_model_config.py` or `verified_models.py` with corresponding test updates\n- A PR adding documentation notes to docstrings clarifying method behavior (e.g., security considerations, bypass behaviors)\n- A PR changing CI runners or fixing workflow infrastructure issues (e.g., standardizing runner types to fix path inconsistencies)\n\n### When to COMMENT\n\nUse COMMENT when you have feedback or concerns:\n\n- Issues that need attention (bugs, security concerns, missing tests)\n- Suggestions for improvement\n- Questions about design decisions\n- Minor style preferences\n\nIf there are significant issues, leave detailed comments explaining the concerns—but let a human maintainer decide whether to block the PR.\n\n## Security\n\n### Dependency freshness / supply-chain guardrail\n\nThis repository intentionally uses a workspace-wide `uv` resolver guardrail:\n\n- Root `pyproject.toml`: `[tool.uv] exclude-newer = \"7 days\"`\n\n**Important:** Dependabot does **not** currently honor that `uv` guardrail when it opens `uv.lock` update PRs for this repo's workspace setup. A Dependabot PR can therefore bump to a version that was uploaded **less than 7 days ago**, even though a local `uv lock` would normally exclude it.\n\nWhen reviewing dependency update PRs (`uv.lock`, `pyproject.toml`, `requirements*.txt`, etc.), explicitly check for **too-new package uploads**:\n\n1. Check the package upload timestamp on the package index.\n2. For `uv.lock`, use the per-file `upload-time` metadata in the changed package entry.\n3. Treat `upload-time` as the upload time of that specific distribution file to the package index (for example, the wheel uploaded to PyPI) — not the Git tag time or GitHub release time.\n4. Compare that timestamp against the current date and the repo's 7-day freshness window.\n\nIf the updated package was uploaded **within the last 7 days**, treat it as a real security / supply-chain concern:\n\n- Do **NOT** approve the PR.\n- Leave a **COMMENT** review that clearly calls out the package name, version, upload time, and that it is newer than the repo's 7-day guardrail.\n- Explain that this can happen because Dependabot currently ignores `tool.uv.exclude-newer` for this repo's workspace updates.\n- Ask a human maintainer to decide whether to wait until the package ages past the guardrail or to merge intentionally despite the freshness risk.\n\n## Core Principles\n\n1. **Simplicity First**: Question complexity. If something feels overcomplicated, ask \"what's the use case?\" and seek simpler alternatives. Features should solve real problems, not imaginary ones.\n\n2. **Pragmatic Testing**: Test what matters. Avoid duplicate test coverage. Don't test library features (e.g., `BaseModel.model_dump()`). Focus on the specific logic implemented in this codebase.\n\n3. **Type Safety**: Avoid `# type: ignore` - treat it as a last resort. Fix types properly with assertions, proper annotations, or code adjustments. Prefer explicit type checking over `getattr`/`hasattr` guards.\n\n4. **Backward Compatibility**: Evaluate breaking change impact carefully. Consider API changes that affect existing users, removal of public fields/methods, and changes to default behavior.\n\n## What to Check\n\n- **Complexity**: Over-engineered solutions, unnecessary abstractions, complex logic that could be refactored\n- **Testing**: Duplicate test coverage, tests for library features, missing edge case coverage. For code that writes to disk, verify that tests cover the **persistence round-trip** (write → close → reopen → verify), not just in-memory state\n- **Type Safety**: `# type: ignore` usage, missing type annotations, `getattr`/`hasattr` guards, mocking non-existent arguments\n- **Breaking Changes**: API changes affecting users, removed public fields/methods, changed defaults\n- **Code Quality**: Code duplication, missing comments for non-obvious decisions, inline imports (unless necessary for circular deps)\n- **Repository Conventions**: Use `pyright` not `mypy`, put fixtures in `conftest.py`, avoid `sys.path.insert` hacks\n- **Directory Example Entrypoints**: PRs that add or modify folder-based runnable examples under `examples/` should use `main.py` as the entrypoint and add the directory to `_TARGET_DIRECTORIES` in `tests/examples/test_examples.py`; see [Directory-Based Examples](#directory-based-examples)\n- **Event Type Deprecation**: Changes to event types (Pydantic models used in serialization) must handle deprecated fields properly\n- **Thread Safety**: New methods in `LocalConversation` that read or write `self._state` must use `with self._state:` — see the [Concurrency](#concurrency---localconversation-state-lock) section below\n- **Persistence Paths**: Code that computes persistence directories must not double-append the conversation hex — see the [Persistence Paths](#persistence-path-construction) section below\n- **Server-Side Cleanup**: Endpoints that create persistent state (directories, files) must have rollback logic for partial failures — see the [Server Error Handling](#server-side-error-handling) section below\n- **Cross-File Data Flow**: When new code calls existing APIs (constructors, factory methods), trace 1–2 levels into those APIs to verify the caller uses them correctly. Bugs often hide at layer boundaries where the caller's assumptions don't match the callee's behavior\n- **Secret Serialization**: Fields that carry secrets must use `serialize_secret()` from `openhands.sdk.utils.pydantic_secrets`. For `dict[str, str]` secret fields, wrap each value in `SecretStr` and call `serialize_secret` per value. Do not hand-roll redaction logic (e.g. custom sentinels or inline `expose_secrets` checks) in field serializers\n- **Info-Log Payloads**: `logger.info(...)` must not dump objects, dicts, or variable-length lists — see [Logging Hygiene](#logging-hygiene)\n\n## Directory-Based Examples\n\nWhen a PR adds or modifies a runnable example represented by a directory under `examples/`, verify that:\n\n1. The runnable entrypoint is named `main.py`.\n2. Helper modules inside that directory are not accidentally treated as standalone examples.\n3. `tests/examples/test_examples.py` includes the example directory in `_TARGET_DIRECTORIES` when the example should run in the `test-examples` workflow.\n4. The example prints an `EXAMPLE_COST: ...` marker when run by the workflow.\n\nDo not ask for this convention on support scripts that are intentionally named for GitHub workflow consumption (for example reusable automation scripts under `examples/03_github_workflows/`) unless they are presented as a directory-based runnable example.\n\n\n## Event Type Deprecation - Critical Review Checkpoint\n\nWhen reviewing PRs that modify event types (e.g., `TextContent`, `Message`, `Event`, or any Pydantic model used in event serialization), **DO NOT APPROVE** until the following are verified:\n\n### Required for Removing/Deprecating Fields\n\n1. **Model validator present**: If a field is being removed from an event type with `extra=\"forbid\"`, there MUST be a `@model_validator(mode=\"before\")` that uses `handle_deprecated_model_fields()` to remove the deprecated field before validation. Otherwise, old events will fail to load.\n\n2. **Tests for backward compatibility**: The PR MUST include tests that:\n - Load an old event format (with the deprecated field) successfully\n - Load a new event format (without the deprecated field) successfully\n - Verify both can be loaded in sequence (simulating mixed conversations)\n\n3. **Test naming convention**: The version in the test name should be the **LAST version** where a particular event structure exists. For example, if `enable_truncation` was removed in v1.11.1, the test should be named `test_v1_10_0_...` (the last version with that field), not `test_v1_8_0_...` (when it was introduced). This avoids duplicate tests and clearly documents when a field was last present.\n\n**Important**: Deprecated field handlers are **permanent** and should never be removed. They ensure old conversations can always be loaded.\n\n### Example Pattern (Required)\n\n```python\nfrom openhands.sdk.utils.deprecation import handle_deprecated_model_fields\n\nclass MyModel(BaseModel):\n model_config = ConfigDict(extra=\"forbid\")\n\n # Deprecated fields that are silently removed for backward compatibility\n # when loading old events. These are kept permanently.\n _DEPRECATED_FIELDS: ClassVar[tuple[str, ...]] = (\"old_field_name\",)\n\n @model_validator(mode=\"before\")\n @classmethod\n def _handle_deprecated_fields(cls, data: Any) -> Any:\n \"\"\"Remove deprecated fields for backward compatibility with old events.\"\"\"\n return handle_deprecated_model_fields(data, cls._DEPRECATED_FIELDS)\n```\n\n### Why This Matters\n\nProduction systems resume conversations that may contain events serialized with older SDK versions. If the SDK can't load old events, users will see errors like:\n\n```\npydantic_core.ValidationError: Extra inputs are not permitted\n```\n\n**This is a production-breaking change.** Do not approve PRs that modify event types without proper backward compatibility handling and tests.\n\n## SDK Architecture Conventions\n\nThese conventions codify patterns that are easy to violate when adding new features. Each was learned from a real bug.\n\n### Concurrency - LocalConversation State Lock\n\n`LocalConversation` protects mutable state with a FIFOLock accessed via `with self._state:`. **Every** method that reads or writes `self._state.events`, `self._state.stats`, `self._state.agent_state`, `self._state.activated_knowledge_skills`, or any other mutable field on `ConversationState` must hold this lock. There are currently ~13 call sites using this pattern.\n\nWhen reviewing a PR that adds a new method to `LocalConversation`:\n1. Check whether it accesses any `self._state.*` field.\n2. If yes, verify the access is inside a `with self._state:` block.\n3. If not, flag it — the method is unsafe for concurrent use with `run()`.\n\n### Persistence Path Construction\n\n`BaseConversation.get_persistence_dir(base, conversation_id)` returns `str(Path(base) / conversation_id.hex)`. The `LocalConversation.__init__` constructor calls this automatically when `persistence_dir` is provided.\n\n**Rule:** Callers that pass `persistence_dir` to `LocalConversation()` must pass only the **base directory** (e.g., `/data/conversations/`). The constructor appends the conversation hex. Passing a pre-constructed full path (e.g., `/data/conversations/abc123`) causes double-appending: `/data/conversations/abc123/abc123`.\n\nWhen reviewing code that creates a new `LocalConversation` (fork, resume, migration):\n1. Check what value is passed as `persistence_dir`.\n2. Verify it does **not** already include the conversation ID hex.\n\n### Server-Side Error Handling\n\nServer endpoints in `conversation_service.py` that create persistent state (writing directories, files, or calling `fork()` which writes to disk) and then perform follow-up operations (like `_start_event_service`) must handle partial failure.\n\n**Pattern:** If the follow-up operation fails, clean up the already-written persistent state so it doesn't become an orphaned directory that confuses future startups.\n\n```python\n# Good: rollback on failure\nfork_dir = self.conversations_dir / fork_conv_id.hex\ntry:\n fork_event_service = await self._start_event_service(fork_stored)\nexcept Exception:\n safe_rmtree(fork_dir)\n raise\n```\n\nWhen reviewing server endpoints that create conversations or persistent artifacts:\n1. Identify the \"point of no return\" where state is written to disk.\n2. Check that subsequent operations are wrapped in try/except with cleanup.\n3. For client-supplied IDs, verify there's a duplicate check before creating state (return 409 Conflict if taken).\n\n### Logging Hygiene\n\n`logger.info(...)` must not interpolate `model_dump(...)`, `.json()`, `to_dict()`, a list/dict of tool/skill/server names, or arbitrary user-supplied values. Log a count and/or id; move full payloads to `logger.debug(...)`.\n\nWhen reviewing a new or changed `logger.info(...)` call: if any interpolated value is an object, a dict, or a list whose size scales with load (tools, skills, conversations, requests), flag it.\n\n## What NOT to Comment On\n\nDo not leave comments for:\n\n- **Nitpicks**: Minor style preferences, optional improvements, or \"nice-to-haves\" that don't affect correctness or maintainability\n- **Good behavior observed**: Don't comment just to praise code that follows best practices - this adds noise. Simply approve if the code is good.\n- **Suggestions for additional tests on simple changes**: For straightforward PRs (config changes, model additions, etc.), don't suggest adding test coverage unless tests are clearly missing for new logic\n- **Obvious or self-explanatory code**: Don't ask for comments on code that is already clear\n- **`.pr/` directory artifacts**: Files in the `.pr/` directory are temporary PR-specific documents (design notes, analysis, scripts) that are automatically cleaned up when the PR is approved. Do not comment on their presence or suggest removing them.\n\nIf a PR is approvable, just approve it. Don't add \"one small suggestion\" or \"consider doing X\" comments that delay merging without adding real value.\n\n## Communication Style\n\n- Be direct and concise - don't over-explain\n- Use casual, friendly tone (\"lgtm\", \"WDYT?\", emojis are fine 👀)\n- Ask questions to understand use cases before suggesting changes\n- Suggest alternatives, not mandates\n- Approve quickly when code is good (\"LGTM!\")\n- Use GitHub suggestion syntax for code fixes\n" }, { "path": ".agents/skills/debug-test-examples-workflow/SKILL.md", "content": "---\nname: debug-test-examples-workflow\ndescription: Guide for debugging failing example tests in the `test-examples` labeled workflow. Use this skill when investigating CI failures in the run-examples.yml workflow, when example scripts fail to run correctly, when needing to isolate specific test failures, or when analyzing workflow logs and failure patterns.\n---\n\n# Debugging test-examples Workflow\n\n## Overview\n\nThe `run-examples.yml` workflow runs example scripts from `examples/` directory. Triggers:\n- Adding `test-examples` label to a PR\n- Manual workflow dispatch\n- Scheduled nightly runs\n\n## Debugging Steps\n\n### 1. Isolate Failing Tests\n\nModify `tests/examples/test_examples.py` to focus on specific tests:\n\n```python\n_TARGET_DIRECTORIES = (\n # EXAMPLES_ROOT / \"01_standalone_sdk\",\n EXAMPLES_ROOT / \"02_remote_agent_server\", # Keep only failing directory\n)\n```\n\n### 2. Exclude Tests\n\nAdd to `_EXCLUDED_EXAMPLES` with explanation:\n\n```python\n_EXCLUDED_EXAMPLES = {\n # Reason for exclusion\n \"examples/path/to/failing_test.py\",\n}\n```\n\n### 3. Trigger Workflow\n\nToggle the `test-examples` label:\n\n```bash\n# Remove label\ncurl -X DELETE -H \"Authorization: token $GITHUB_TOKEN\" \\\n \"https://api.github.com/repos/OpenHands/software-agent-sdk/issues/${PR_NUMBER}/labels/test-examples\"\n\n# Add label\ncurl -X POST -H \"Authorization: token $GITHUB_TOKEN\" \\\n -H \"Accept: application/vnd.github.v3+json\" \\\n \"https://api.github.com/repos/OpenHands/software-agent-sdk/issues/{PR_NUMBER}/labels\" \\\n -d '{\"labels\":[\"test-examples\"]}'\n```\n\n### 4. Monitor Progress\n\n```bash\n# Check status\ncurl -s -H \"Authorization: token $GITHUB_TOKEN\" \\\n \"https://api.github.com/repos/OpenHands/software-agent-sdk/actions/runs/{RUN_ID}\" | jq '{status, conclusion}'\n\n# Download logs\ncurl -sL -H \"Authorization: token $GITHUB_TOKEN\" \\\n \"https://api.github.com/repos/OpenHands/software-agent-sdk/actions/runs/{RUN_ID}/logs\" -o logs.zip\nunzip logs.zip -d logs\n```\n\n## Common Failure Patterns\n\n| Pattern | Cause | Solution |\n|---------|-------|----------|\n| Port conflicts | Fixed ports (8010, 8011) | Run with `-n 1` or use different ports |\n| Container issues | Docker/Apptainer setup | Check Docker availability, image pulls |\n| LLM failures | Transient API errors | Retry the test |\n| Example bugs | Code errors | Check traceback |\n\n\n## Key Configuration\n\n**Workflow** (`.github/workflows/run-examples.yml`):\n- Runner: `blacksmith-2vcpu-ubuntu-2404`\n- Timeout: 60 minutes\n- Parallelism: `-n 4` (pytest-xdist: 4 parallel workers)\n\n**Tests** (`tests/examples/test_examples.py`):\n- Timeout per example: 600 seconds\n- Target directories: `_TARGET_DIRECTORIES`\n- Excluded examples: `_EXCLUDED_EXAMPLES`\n" }, { "path": ".agents/skills/design-principles.md", "content": "---\nname: design-principles\ndescription: Core architectural design principles of the OpenHands Software Agent SDK. Reference when making architectural decisions, reviewing PRs that change agent/tool/state boundaries, or evaluating whether a proposed change aligns with V1 design goals.\n---\n\n# SDK Design Principles\n\nReference: \n\n## Quick Summary\n\n1. **Optional Isolation over Mandatory Sandboxing**\n Sandboxing is opt-in, not universal. Agent and tool execution runs in a single\n process by default. When isolation is needed, the same stack can be transparently\n containerized.\n\n2. **Stateless by Default, One Source of Truth for State**\n All components — agents, tools, LLMs, configurations — are **immutable Pydantic\n models** validated at construction. The only mutable entity is the conversation\n state. This enables deterministic replay and robust persistence.\n\n3. **Clear Boundaries between Agent and Applications**\n Strict separation between SDK (agent core), tools, workspace, and agent server.\n Applications communicate via APIs, not by embedding the agent.\n\n4. **Composable Components for Extensibility**\n Agents are graphs of interchangeable components — tools, prompts, LLMs, contexts —\n described **declaratively with strong typing**. Developers reconfigure capabilities\n without modifying core code.\n\n## Implications for Development\n\n- Since agents are immutable Pydantic models, their configuration **is** their\n serializable representation. There should be no need to \"reverse-engineer\" agent\n config from runtime instances.\n- Tool implementations (callables) are the only non-serializable part; this is solved\n by `tool_module_qualnames` for remote forwarding.\n- Everything else (system_prompt, model, skills, tool names) is already declarative\n data that can be serialized and forwarded directly.\n- Avoid patterns that create multiple sources of truth for the same configuration\n (e.g., a factory function AND an extracted definition).\n- `model_copy(update=...)` should be used sparingly and through well-defined paths to\n avoid undermining statelessness.\n" }, { "path": ".agents/skills/feature-release-rollout/SKILL.md", "content": "---\nname: feature-release-rollout\ndescription: This skill should be used when the user asks to \"rollout a feature\", \"complete feature release\", \"propagate SDK feature\", \"track feature support\", \"what's missing for feature X\", or mentions checking CLI/GUI/docs/blog support for SDK features. Guides agents through the multi-repository feature release workflow from SDK to docs to marketing.\ntriggers:\n- rollout feature\n- feature release\n- propagate feature\n- feature support\n- complete release\n- docs for feature\n- blog for feature\n- CLI support\n- GUI support\n- what's missing\n---\n\n# Feature Release Rollout\n\nThis skill guides the complete feature release workflow across the OpenHands ecosystem repositories.\n\n## Overview\n\nWhen a feature is implemented in the SDK, it may need propagation through several repositories:\n\n1. **SDK** (`OpenHands/software-agent-sdk`) — Core feature implementation\n2. **CLI** (`OpenHands/OpenHands-CLI`) — Terminal interface support\n3. **GUI** (`OpenHands/OpenHands` frontend directory) — Web interface support\n4. **Docs** (`OpenHands/docs`) — Documentation updates (sdk/ folder)\n5. **Blog** (`OpenHands/growth-utils` blog-post/) — Marketing and announcements\n6. **Video** — Tutorial content (using ElevenLabs + Remotion)\n\n## Workflow\n\n### Phase 1: Feature Discovery\n\nFirst, identify what feature(s) to analyze. The user may specify:\n- A release tag (e.g., `v1.9.0`)\n- A specific feature name\n- A PR or commit reference\n- A comparison between versions\n\n**For release tags:**\n```bash\n# Clone SDK if not present\ngit clone https://github.com/OpenHands/software-agent-sdk.git\n\n# View release notes\ncd software-agent-sdk\ngit log --oneline v1.8.0..v1.9.0 # Changes between versions\ngit show v1.9.0 --stat # What changed in this release\n```\n\n**For specific features:**\nSearch the SDK codebase, examples, and changelog to understand the feature scope.\n\n### Phase 2: Repository Analysis\n\nClone all relevant repositories to analyze current support:\n\n```bash\n# Clone repositories (use GITHUB_TOKEN for authenticated access)\ngit clone https://github.com/OpenHands/software-agent-sdk.git\ngit clone https://github.com/OpenHands/OpenHands-CLI.git\ngit clone https://github.com/OpenHands/OpenHands.git # Frontend in frontend/\ngit clone https://github.com/OpenHands/docs.git\ngit clone https://github.com/OpenHands/growth-utils.git\n```\n\nFor each feature, check support status:\n\n| Repository | Check Location | What to Look For |\n|------------|---------------|------------------|\n| CLI | `openhands_cli/` | Feature flags, commands, TUI widgets |\n| GUI | `OpenHands/frontend/src/` | React components, API integrations |\n| Docs | `docs/sdk/` | Guide pages, API reference, examples |\n| Blog | `growth-utils/blog-post/posts/` | Announcement posts |\n\n### Phase 3: Assess Feature Importance\n\nNot all features warrant full rollout. Evaluate each feature:\n\n**High Impact (full rollout recommended):**\n- New user-facing capabilities\n- Breaking changes or migrations\n- Major performance improvements\n- New integrations or tools\n\n**Medium Impact (docs + selective support):**\n- New API methods or parameters\n- Configuration options\n- Developer experience improvements\n\n**Low Impact (docs only or skip):**\n- Internal refactoring\n- Bug fixes\n- Minor enhancements\n\n**Skip rollout for:**\n- Internal-only changes\n- Test improvements\n- Build/CI changes\n- Documentation typos\n\n### Phase 4: Create Proposal\n\nGenerate a structured proposal for the user:\n\n```markdown\n## Feature Rollout Proposal: [Feature Name]\n\n### Feature Summary\n[Brief description of the feature and its value]\n\n### Current Support Status\n| Component | Status | Notes |\n|-----------|--------|-------|\n| SDK | ✅ Implemented | [version/PR] |\n| CLI | ❌ Missing | [what's needed] |\n| GUI | ⚠️ Partial | [what's implemented vs needed] |\n| Docs | ❌ Missing | [suggested pages] |\n| Blog | ❌ Not started | [whether warranted] |\n| Video | ❌ Not started | [whether warranted] |\n\n### Recommended Actions\n1. **CLI**: [specific implementation needed]\n2. **GUI**: [specific implementation needed]\n3. **Docs**: [pages to create/update]\n4. **Blog**: [recommended or not, with reasoning]\n5. **Video**: [recommended or not, with reasoning]\n\n### Assessment\n- **Overall Priority**: [High/Medium/Low]\n- **Effort Estimate**: [days/hours per component]\n- **Dependencies**: [what must be done first]\n```\n\n### Phase 5: User Confirmation\n\nWait for explicit user approval before proceeding. Ask:\n- Which components to implement\n- Priority ordering\n- Any modifications to the proposal\n\n### Phase 6: Implementation\n\nOnly after user confirmation:\n\n**Create GitHub Issues:**\n```bash\n# Create issue on relevant repo\ngh issue create --repo OpenHands/OpenHands-CLI \\\n --title \"Support [feature] in CLI\" \\\n --body \"## Context\\n[Feature description]\\n\\n## Implementation\\n[Details]\\n\\n## Related\\n- SDK: [link]\\n- Docs: [link]\"\n```\n\n**Implementation order:**\n1. CLI/GUI support (can be parallel)\n2. Documentation (depends on 1)\n3. Blog post (depends on 2)\n4. Video (depends on 3)\n\n## Repository-Specific Guidelines\n\n### CLI (OpenHands/OpenHands-CLI)\n\n- Check `AGENTS.md` for development guidelines\n- Use `uv` for dependency management\n- Run `make lint` and `make test` before commits\n- TUI components in `openhands_cli/tui/`\n- Snapshot tests for UI changes\n\n### GUI (OpenHands/OpenHands frontend)\n\n- Frontend in `frontend/` directory\n- React/TypeScript codebase\n- Run `npm run lint:fix && npm run build` in frontend/\n- Follow TanStack Query patterns for data fetching\n- i18n translations in `frontend/src/i18n/`\n\n### Docs (OpenHands/docs)\n\n- SDK docs in `sdk/` folder\n- Uses Mintlify (`.mdx` files)\n- Code blocks can auto-sync from SDK examples\n- Run `mint broken-links` to validate\n- Follow `openhands/DOC_STYLE_GUIDE.md`\n\n### Blog (OpenHands/growth-utils)\n\n- Posts in `blog-post/posts/YYYYMMDD-title.md`\n- Assets in `blog-post/assets/YYYYMMDD-title/`\n- Frontmatter format:\n ```yaml\n ---\n title: \"Post Title\"\n excerpt: \"Brief description\"\n coverImage: \"/assets/blog/YYYYMMDD-title/cover.png\"\n date: \"YYYY-MM-DDTHH:MM:SS.000Z\"\n authors:\n - name: Author Name\n picture: \"/assets/blog/authors/author.png\"\n ogImage:\n url: \"/assets/blog/YYYYMMDD-title/cover.png\"\n ---\n ```\n\n## Example Feature Analysis\n\n**Feature: Browser Session Recording (SDK v1.8.0)**\n\n1. **SDK**: ✅ Implemented in `openhands.tools.browser`\n2. **CLI**: ❌ No replay/export commands\n3. **GUI**: ❌ No recording viewer component\n4. **Docs**: ✅ Guide at `sdk/guides/browser-session-recording.mdx`\n5. **Blog**: ❌ Could highlight for web scraping users\n6. **Video**: Consider 2-minute demo\n\n**Recommendation**: Medium priority. Docs done, CLI/GUI low urgency (advanced feature), blog post optional.\n\n## Quick Commands\n\n```bash\n# Check SDK feature presence\ngrep -r \"feature_name\" software-agent-sdk/openhands/ --include=\"*.py\"\n\n# Check CLI support\ngrep -r \"feature_name\" OpenHands-CLI/openhands_cli/ --include=\"*.py\"\n\n# Check GUI support\ngrep -r \"featureName\" OpenHands/frontend/src/ --include=\"*.ts\" --include=\"*.tsx\"\n\n# Check docs coverage\ngrep -r \"feature\" docs/sdk/ --include=\"*.mdx\"\n\n# Check blog mentions\ngrep -r \"feature\" growth-utils/blog-post/posts/ --include=\"*.md\"\n```\n\n## Important Notes\n\n- Always get user confirmation before creating issues or starting implementation\n- Consider feature maturity — new features may change before full rollout\n- Cross-reference PRs between repositories in issue descriptions\n- For breaking changes, coordinate release timing across all components\n" }, { "path": ".agents/skills/manage-evals/SKILL.md", "content": "---\nname: manage-evals\ndescription: This skill should be used when the user asks to \"trigger an eval\", \"run evaluation\", \"run swebench\", \"run gaia\", \"run benchmark\", \"compare eval runs\", \"compare evaluation results\", \"check eval regression\", \"compare benchmark results\", \"what changed in the eval\", \"diff eval runs\", or mentions triggering, comparing, or reporting on SWE-bench, GAIA, or other benchmark evaluation results. Provides workflow for triggering evaluations on different benchmarks, finding and comparing runs, and reporting performance differences.\n---\n\n# Managing Evaluations\n\n## Overview\n\nOpenHands evaluations produce results stored on a CDN at `https://results.eval.all-hands.dev/`. Each run is identified by a path: `{benchmark}/{model_slug}/{github_run_id}/`. This skill enables triggering evaluation runs, comparing results between runs, and posting performance reports as GitHub PR comments.\n\n## Quick Start\n\n### Trigger an Evaluation\n\n```bash\npython .agents/skills/manage-evals/scripts/manage_evals.py trigger \\\n --sdk-ref --benchmark swebench --eval-limit 50\n```\n\n### Compare Runs\n\n```bash\npython .agents/skills/manage-evals/scripts/manage_evals.py compare \\\n \"///\" \\\n --auto-baseline\n```\n\n### Compare and Post to PR\n\n```bash\npython .agents/skills/manage-evals/scripts/manage_evals.py compare \\\n \"///\" \\\n --auto-baseline \\\n --post-comment --pr --repo OpenHands/software-agent-sdk\n```\n\n## Triggering Evaluations\n\n### Using the Script\n\n```bash\n# SWE-bench (default) on a PR branch\npython .agents/skills/manage-evals/scripts/manage_evals.py trigger \\\n --sdk-ref my-feature-branch --eval-limit 50\n\n# GAIA benchmark\npython .agents/skills/manage-evals/scripts/manage_evals.py trigger \\\n --sdk-ref main --benchmark gaia --eval-limit 50\n\n# With a specific model\npython .agents/skills/manage-evals/scripts/manage_evals.py trigger \\\n --sdk-ref v1.16.0 --benchmark swebench --model-ids gemini-3-flash --eval-limit 50\n\n# Multiple benchmarks (run the command multiple times)\nfor bench in swebench gaia; do\n python .agents/skills/manage-evals/scripts/manage_evals.py trigger \\\n --sdk-ref main --benchmark \"$bench\" --eval-limit 50 --reason \"Multi-benchmark eval\"\ndone\n```\n\n### Available Benchmarks\n\n| Benchmark | Description |\n|-----------|-------------|\n| `swebench` | SWE-bench (default) — software engineering tasks |\n| `swebenchpro` | SWE-Bench Pro — harder software engineering tasks |\n| `gaia` | GAIA — general AI assistant tasks |\n| `swtbench` | SWT-bench — software testing tasks |\n| `commit0` | Commit0 — commit generation tasks |\n| `swebenchmultimodal` | SWE-bench Multimodal — tasks with images |\n| `terminalbench` | TerminalBench — terminal interaction tasks |\n\n### Trigger Options\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `--sdk-ref` | *(required)* | Branch, tag, or commit SHA to evaluate |\n| `--benchmark` | `swebench` | Benchmark to run |\n| `--eval-limit` | `50` | Number of instances to evaluate |\n| `--model-ids` | *(first in config)* | Comma-separated model IDs from `resolve_model_config.py` |\n| `--tool-preset` | `default` | Tool preset: `default`, `gemini`, `gpt5`, `planning` |\n| `--agent-type` | `default` | Agent type: `default`, `acp-claude`, `acp-codex` |\n| `--instance-ids` | | Specific instance IDs to evaluate (overrides eval-limit) |\n| `--reason` | | Human-readable reason (shown in notifications) |\n| `--benchmarks-branch` | `main` | Branch of the benchmarks repo |\n| `--eval-branch` | `main` | Branch of the evaluation repo |\n\n### Via PR Labels (Alternative)\n\nAdding a label to a PR also triggers evaluations:\n- `run-eval-1` — 1 instance (quick sanity check)\n- `run-eval-50` — 50 instances (standard comparison)\n- `run-eval-200` — 200 instances\n- `run-eval-500` — 500 instances (full benchmark)\n\n## Comparing Evaluation Runs\n\n### Step 1: Find the Current PR's Eval Run\n\nEval runs are triggered by adding labels like `run-eval-50` to a PR. The `all-hands-bot` posts a comment with results when complete.\n\n**Option A — From bot comments on the PR:**\n\n```bash\ngh api repos/OpenHands/software-agent-sdk/issues//comments \\\n --jq '.[] | select(.user.login == \"all-hands-bot\") | .body' \\\n | grep -o 'Evaluation:.*' | head -1\n```\n\nThe evaluation name follows the format `{github_run_id}-{model_slug_short}` (e.g., `23775164157-claude-son`). Extract the `github_run_id` from this.\n\n**Option B — From the \"Evaluation Triggered\" bot comment:**\n\n```bash\ngh api repos/OpenHands/software-agent-sdk/issues//comments \\\n --jq '.[] | select(.body | test(\"Evaluation Triggered\")) | .body'\n```\n\nThis contains the SDK commit SHA. Cross-reference with daily metadata to find the run ID.\n\n**Option C — From daily metadata:**\n\n```bash\ncurl -s \"https://results.eval.all-hands.dev/metadata/$(date -u +%Y-%m-%d).txt\"\n```\n\nEach line is a run path. Match by benchmark and model to find the run.\n\n### Step 2: Identify the Run Path Components\n\nA run path has three components:\n- **benchmark**: `swebench`, `swebenchpro`, `gaia`, `swtbench`, `commit0`, `swebenchmultimodal`, `terminalbench`\n- **model_slug**: Derived from model name with `/:@.` replaced by `-` (e.g., `litellm_proxy-claude-sonnet-4-5-20250929`)\n- **run_id**: The GitHub Actions workflow run ID from the `OpenHands/evaluation` repo\n\n### Step 3: Verify Results Exist\n\n```bash\ncurl -sI \"https://results.eval.all-hands.dev////output.report.json\" | head -1\n```\n\nA `200` status confirms the run completed and results are available.\n\n### Step 4: Find a Baseline for Comparison\n\n**Automatic**: The comparison script's `--auto-baseline` flag scans metadata files backward up to 14 days to find the most recent completed run with the same benchmark and model.\n\n**Manual**: Inspect metadata files or other PR bot comments to identify a specific run:\n\n```bash\n# Check today's runs\ncurl -s \"https://results.eval.all-hands.dev/metadata/$(date -u +%Y-%m-%d).txt\" | grep \"swebench/litellm_proxy-claude\"\n\n# Check yesterday's runs\ncurl -s \"https://results.eval.all-hands.dev/metadata/$(date -u -d yesterday +%Y-%m-%d).txt\" | grep \"swebench/litellm_proxy-claude\"\n```\n\n### Step 5: Run the Comparison\n\n```bash\npython .agents/skills/manage-evals/scripts/manage_evals.py compare \\\n \"swebench/litellm_proxy-claude-sonnet-4-5-20250929/23775164157/\" \\\n --baseline \"swebench/litellm_proxy-claude-sonnet-4-5-20250929/23773892085/\"\n```\n\nOr with auto-baseline and PR comment posting:\n\n```bash\npython .agents/skills/manage-evals/scripts/manage_evals.py compare \\\n \"swebench/litellm_proxy-claude-sonnet-4-5-20250929/23775164157/\" \\\n --auto-baseline \\\n --post-comment --pr 2334 --repo OpenHands/software-agent-sdk\n```\n\n## Available Data Per Run\n\nEach run stores files at `https://results.eval.all-hands.dev/{run_path}/`:\n\n| File | Description |\n|------|-------------|\n| `metadata/params.json` | Run parameters: SDK commit, PR number, model, eval_limit, triggered_by |\n| `output.report.json` | Aggregated results: resolved/submitted/total counts and instance IDs |\n| `cost_report.jsonl` | Per-instance cost data |\n| `results.tar.gz` | Full archive with all outputs |\n\n## Dashboard\n\nThe eval monitor dashboard provides a visual view of runs:\n\n```\nhttps://openhands-eval-monitor.vercel.app/?run={benchmark}/{model_slug}/{run_id}/\n```\n\n## Interpreting Results\n\n- **Success rate** = resolved / min(eval_limit, total_instances)\n- A 50-instance sample has natural variance of ±2-4 resolved instances between runs\n- Focus on **instance-level changes** (gained/lost) to understand regressions vs. noise\n- If the same set of instances is resolved, the difference is likely noise\n\n## Additional Resources\n\n### Reference Files\n- **`references/eval-infrastructure.md`** — Detailed documentation on the evaluation infrastructure, GCS paths, metadata format, and workflow triggers\n\n### Scripts\n- **`scripts/manage_evals.py`** — Standalone comparison script with auto-baseline detection and GitHub comment posting\n" }, { "path": ".agents/skills/manage-evals/references/eval-infrastructure.md", "content": "# Evaluation Infrastructure Reference\n\n## Architecture Overview\n\nThe evaluation pipeline spans three repositories:\n\n1. **OpenHands/software-agent-sdk** — Triggers evaluations via `run-eval.yml` workflow\n2. **OpenHands/evaluation** — Orchestrates the eval job via `eval-job.yml` workflow\n3. **OpenHands/benchmarks** — Contains benchmark runners (inference + evaluation)\n\n## Trigger Flow\n\n### PR Label Trigger\n\n1. A label (`run-eval-1`, `run-eval-50`, `run-eval-200`, `run-eval-500`) is added to a PR\n2. `software-agent-sdk/.github/workflows/run-eval.yml` fires\n3. It resolves model configs from `.github/run-eval/resolve_model_config.py`\n4. Dispatches `eval-job.yml` in `OpenHands/evaluation` with:\n - `sdk_commit`: The PR's head SHA\n - `sdk_workflow_run_id`: The `run-eval.yml` workflow run ID\n - `eval_limit`: Extracted from label name\n - `models_json`: Resolved model configurations\n - `pr_number`: The PR number (for result posting)\n5. Posts an \"Evaluation Triggered\" comment on the PR\n\n### Release Trigger\n\nRuns automatically on `release` events with `eval_limit=50`.\n\n### Manual Trigger\n\nVia `workflow_dispatch` on `run-eval.yml` with explicit parameters.\n\n## Results Storage (GCS)\n\nResults are stored in Google Cloud Storage bucket `openhands-evaluation-results`\nand served via CDN at `https://results.eval.all-hands.dev/`.\n\n### Run Path Format\n\n```\n{benchmark}/{model_slug}/{github_run_id}/\n```\n\n- **benchmark**: `swebench`, `swebenchpro`, `gaia`, `swtbench`, `commit0`, `swebenchmultimodal`, `terminalbench`\n- **model_slug**: Model name with `/:@.` replaced by `-`\n - Example: `litellm_proxy/claude-sonnet-4-5-20250929` → `litellm_proxy-claude-sonnet-4-5-20250929`\n- **github_run_id**: The GitHub Actions run ID from the `OpenHands/evaluation` repo\n\n### Files Per Run\n\n```\n{run_path}/\n├── metadata/\n│ └── params.json # Job parameters (uploaded at job start)\n├── output.report.json # Aggregated evaluation results\n├── cost_report.jsonl # Per-instance cost data\n└── results.tar.gz # Full archive\n```\n\n### params.json Schema\n\n```json\n{\n \"timestamp\": \"2026-03-31T00:54:15Z\",\n \"sdk_commit\": \"42852dc2260a461536acc186cd918ad5a58910dd\",\n \"sdk_workflow_run_id\": \"23775150328\",\n \"eval_limit\": 50,\n \"benchmark\": \"swebench\",\n \"model_name\": \"litellm_proxy/claude-sonnet-4-5-20250929\",\n \"model_id\": \"claude-sonnet-4-5-20250929\",\n \"model_display_name\": \"Claude Sonnet 4.5\",\n \"unique_eval_name\": \"23775164157-claude-son\",\n \"commit\": \"42852dc2260a461536acc186cd918ad5a58910dd\",\n \"pr_number\": \"2334\",\n \"triggered_by\": \"enyst\",\n \"tool_preset\": \"default\",\n \"agent_type\": \"default\",\n \"github_run_id\": \"23775164157\"\n}\n```\n\n### output.report.json Schema\n\n```json\n{\n \"total_instances\": 500,\n \"submitted_instances\": 50,\n \"completed_instances\": 50,\n \"resolved_instances\": 35,\n \"unresolved_instances\": 15,\n \"empty_patch_instances\": 0,\n \"error_instances\": 0,\n \"completed_ids\": [\"instance_id_1\", \"...\"],\n \"resolved_ids\": [\"instance_id_1\", \"...\"],\n \"unresolved_ids\": [\"instance_id_1\", \"...\"],\n \"empty_patch_ids\": [],\n \"error_ids\": []\n}\n```\n\n## Daily Metadata\n\nAll runs registered on a given day are listed in:\n\n```\nhttps://results.eval.all-hands.dev/metadata/YYYY-MM-DD.txt\n```\n\nEach line is a run path. Example:\n\n```\nswebench/litellm_proxy-claude-sonnet-4-5-20250929/23773892085/\nswebench/litellm_proxy-gemini-3-flash-preview/23774756886/\ngaia/litellm_proxy-claude-sonnet-4-5-20250929/23775142614/\n```\n\nMetadata files are updated atomically with generation preconditions and\nhave `Cache-Control: no-cache` set.\n\n## Dashboard\n\nThe eval monitor dashboard at `https://openhands-eval-monitor.vercel.app/`\nprovides a visual view of runs. Construct URLs as:\n\n```\nhttps://openhands-eval-monitor.vercel.app/?run={benchmark}/{model_slug}/{run_id}/\n```\n\n## Bot Comments\n\nWhen an eval completes, `all-hands-bot` posts a comment on the PR (if `pr_number` was provided) with:\n\n- Evaluation name (e.g., `23775164157-claude-son`)\n- Model name\n- Results summary (total, submitted, resolved, unresolved, empty patch, error counts)\n- Success rate\n- Archive link\n\n## Model Slug Computation\n\nThe model slug is derived from the LLM config's `model` field:\n\n```python\nmodel = config[\"model\"] # e.g., \"litellm_proxy/claude-sonnet-4-5-20250929\"\nfor ch in \"/:@.\":\n model = model.replace(ch, \"-\")\n# Result: \"litellm_proxy-claude-sonnet-4-5-20250929\"\n```\n\n## Available Models\n\nModels are defined in `software-agent-sdk/.github/run-eval/resolve_model_config.py`.\nEach model has an `id`, `display_name`, and `llm_config` with the model path and parameters.\n\n## Variance Between Runs\n\nFor 50-instance SWE-bench evaluations:\n- Natural variance is typically ±2-4 resolved instances between identical configurations\n- Focus on instance-level changes (which specific instances gained/lost) to distinguish real regressions from noise\n- If the resolved instance set is identical, the runs are equivalent\n" }, { "path": ".agents/skills/manage-evals/scripts/manage_evals.py", "content": "#!/usr/bin/env python3\n\"\"\"Trigger, compare, and report on OpenHands evaluation runs.\n\nSubcommands:\n trigger Dispatch an evaluation workflow via the GitHub API\n compare Compare two evaluation runs and produce a markdown report\n\nExamples:\n # Trigger a swebench eval on a PR branch\n python manage_evals.py trigger --sdk-ref my-branch --benchmark swebench --eval-limit 50\n\n # Trigger a GAIA eval on a release tag\n python manage_evals.py trigger --sdk-ref v1.16.0 --benchmark gaia --eval-limit 50\n\n # Auto-find baseline and print comparison markdown\n python manage_evals.py compare swebench/litellm_proxy-claude-sonnet-4-5-20250929/23775164157/ --auto-baseline\n\n # Post comparison to PR\n python manage_evals.py compare swebench/.../23775164157/ --auto-baseline \\\\\n --post-comment --pr 2334 --repo OpenHands/software-agent-sdk\n\"\"\" # noqa: E501\n\nfrom __future__ import annotations\n\nimport argparse\nimport json\nimport os\nimport sys\nimport urllib.request\nfrom datetime import UTC, datetime, timedelta\nfrom typing import Any\n\n\nRESULTS_CDN = os.environ.get(\"RESULTS_CDN\", \"https://results.eval.all-hands.dev\")\nDASHBOARD_BASE = \"https://openhands-eval-monitor.vercel.app\"\n\nSDK_REPO = \"OpenHands/software-agent-sdk\"\nBENCHMARKS = [\n \"swebench\",\n \"swebenchpro\",\n \"gaia\",\n \"swtbench\",\n \"commit0\",\n \"swebenchmultimodal\",\n \"terminalbench\",\n]\nTOOL_PRESETS = [\"default\", \"gemini\", \"gpt5\", \"planning\"]\nAGENT_TYPES = [\"default\", \"acp-claude\", \"acp-codex\"]\n\n\ndef fetch_json(url: str) -> dict[str, Any] | None:\n \"\"\"Fetch JSON from a URL, returning None on 404.\"\"\"\n try:\n req = urllib.request.Request(url)\n with urllib.request.urlopen(req, timeout=15) as resp:\n return json.loads(resp.read().decode())\n except urllib.error.HTTPError as e:\n if e.code == 404:\n return None\n raise\n except Exception as e:\n print(f\"Warning: Failed to fetch {url}: {e}\", file=sys.stderr)\n return None\n\n\ndef fetch_text(url: str) -> str | None:\n \"\"\"Fetch text from a URL, returning None on 404.\"\"\"\n try:\n req = urllib.request.Request(url)\n with urllib.request.urlopen(req, timeout=15) as resp:\n return resp.read().decode()\n except urllib.error.HTTPError as e:\n if e.code == 404:\n return None\n raise\n except Exception as e:\n print(f\"Warning: Failed to fetch {url}: {e}\", file=sys.stderr)\n return None\n\n\ndef parse_run_path(path: str) -> tuple[str, str, str]:\n \"\"\"Parse a run path into (benchmark, model_slug, run_id).\n\n Accepts formats:\n swebench/litellm_proxy-claude-sonnet-4-5-20250929/23775164157/\n swebench/litellm_proxy-claude-sonnet-4-5-20250929/23775164157\n \"\"\"\n parts = path.strip(\"/\").split(\"/\")\n if len(parts) != 3:\n raise ValueError(\n f\"Invalid run path: {path!r}. Expected: benchmark/model_slug/run_id\"\n )\n return parts[0], parts[1], parts[2]\n\n\ndef get_report(run_path: str) -> dict[str, Any] | None:\n \"\"\"Fetch output.report.json for a run.\"\"\"\n url = f\"{RESULTS_CDN}/{run_path.strip('/')}/output.report.json\"\n return fetch_json(url)\n\n\ndef get_params(run_path: str) -> dict[str, Any] | None:\n \"\"\"Fetch metadata/params.json for a run.\"\"\"\n url = f\"{RESULTS_CDN}/{run_path.strip('/')}/metadata/params.json\"\n return fetch_json(url)\n\n\ndef get_metadata_for_date(date_str: str) -> list[str]:\n \"\"\"Fetch the metadata listing for a given date (YYYY-MM-DD).\"\"\"\n url = f\"{RESULTS_CDN}/metadata/{date_str}.txt\"\n text = fetch_text(url)\n if not text:\n return []\n return [line.strip() for line in text.strip().split(\"\\n\") if line.strip()]\n\n\ndef find_baseline_run(\n benchmark: str,\n model_slug: str,\n current_run_id: str,\n lookback_days: int = 14,\n current_eval_limit: int | None = None,\n) -> str | None:\n \"\"\"Find the most recent previous run with matching benchmark/model.\n\n Scans metadata files backward from today, looking for a run with the\n same benchmark and model_slug but a different (earlier) run_id.\n Prefers runs with matching eval_limit when available.\n\n Returns the run path or None if no baseline found.\n \"\"\"\n today = datetime.now(UTC).date()\n prefix = f\"{benchmark}/{model_slug}/\"\n\n # Two-pass: first look for matching eval_limit, then any completed run\n candidates: list[tuple[str, dict[str, Any] | None]] = []\n\n for day_offset in range(lookback_days + 1):\n date = today - timedelta(days=day_offset)\n date_str = date.strftime(\"%Y-%m-%d\")\n entries = get_metadata_for_date(date_str)\n\n for entry in reversed(entries):\n if not entry.startswith(prefix):\n continue\n _, _, run_id = parse_run_path(entry)\n if run_id == current_run_id:\n continue\n\n report = get_report(entry)\n if report and report.get(\"submitted_instances\", 0) > 0:\n params = get_params(entry)\n candidates.append((entry, params))\n # Stop after finding enough candidates\n if len(candidates) >= 10:\n break\n if len(candidates) >= 10:\n break\n\n if not candidates:\n return None\n\n # Prefer runs with matching eval_limit\n if current_eval_limit is not None:\n for path, params in candidates:\n if params and params.get(\"eval_limit\") == current_eval_limit:\n return path\n\n # Fall back to most recent completed run\n return candidates[0][0]\n\n\ndef compute_diff(\n current: dict[str, Any],\n baseline: dict[str, Any],\n current_params: dict[str, Any] | None,\n baseline_params: dict[str, Any] | None,\n) -> str:\n \"\"\"Produce a markdown comparison of two eval reports.\"\"\"\n # Extract key metrics\n c_resolved = current.get(\"resolved_instances\", 0)\n b_resolved = baseline.get(\"resolved_instances\", 0)\n c_submitted = current.get(\"submitted_instances\", 0)\n b_submitted = baseline.get(\"submitted_instances\", 0)\n c_total = current.get(\"total_instances\", 0)\n b_total = baseline.get(\"total_instances\", 0)\n c_empty = current.get(\"empty_patch_instances\", 0)\n b_empty = baseline.get(\"empty_patch_instances\", 0)\n c_error = current.get(\"error_instances\", 0)\n b_error = baseline.get(\"error_instances\", 0)\n\n # Eval limit from params\n c_limit = (current_params or {}).get(\"eval_limit\", c_submitted)\n b_limit = (baseline_params or {}).get(\"eval_limit\", b_submitted)\n\n # Denominators for rate calculation\n c_denom = min(c_limit, c_total) if c_total > 0 else c_limit\n b_denom = min(b_limit, b_total) if b_total > 0 else b_limit\n\n c_rate = (c_resolved / c_denom * 100) if c_denom else 0\n b_rate = (b_resolved / b_denom * 100) if b_denom else 0\n rate_delta = c_rate - b_rate\n\n # Instance-level diff\n c_resolved_ids = set(current.get(\"resolved_ids\", []))\n b_resolved_ids = set(baseline.get(\"resolved_ids\", []))\n gained = sorted(c_resolved_ids - b_resolved_ids)\n lost = sorted(b_resolved_ids - c_resolved_ids)\n\n # Delta symbol\n def delta_str(val: float | int) -> str:\n if val > 0:\n return f\"+{val}\"\n return str(val)\n\n # Build markdown\n lines: list[str] = []\n lines.append(\"## 📊 Evaluation Comparison\")\n lines.append(\"\")\n\n # Summary line\n if rate_delta > 0:\n emoji = \"📈\"\n delta_pp = f\"+{rate_delta:.1f}\"\n elif rate_delta < 0:\n emoji = \"📉\"\n delta_pp = f\"{rate_delta:.1f}\"\n else:\n emoji = \"➡️\"\n delta_pp = \"0.0\"\n lines.append(\n f\"{emoji} **Success rate: {c_rate:.1f}% \"\n f\"({delta_pp}pp vs baseline {b_rate:.1f}%)**\"\n )\n lines.append(\"\")\n\n # Metadata\n c_pr = (current_params or {}).get(\"pr_number\")\n b_pr = (baseline_params or {}).get(\"pr_number\")\n c_commit = (current_params or {}).get(\"sdk_commit\", \"unknown\")[:12]\n b_commit = (baseline_params or {}).get(\"sdk_commit\", \"unknown\")[:12]\n c_run_id = (current_params or {}).get(\"github_run_id\", \"\")\n b_run_id = (baseline_params or {}).get(\"github_run_id\", \"\")\n\n lines.append(\"| | Current | Baseline |\")\n lines.append(\"|---|---|---|\")\n if c_run_id or b_run_id:\n lines.append(f\"| **Run ID** | `{c_run_id}` | `{b_run_id}` |\")\n lines.append(f\"| **SDK Commit** | `{c_commit}` | `{b_commit}` |\")\n if c_pr or b_pr:\n c_pr_str = f\"#{c_pr}\" if c_pr else \"—\"\n b_pr_str = f\"#{b_pr}\" if b_pr else \"— (main)\" if not b_pr else f\"#{b_pr}\"\n lines.append(f\"| **PR** | {c_pr_str} | {b_pr_str} |\")\n lines.append(\n f\"| **Resolved** | {c_resolved}/{c_denom} ({c_rate:.1f}%) \"\n f\"| {b_resolved}/{b_denom} ({b_rate:.1f}%) |\"\n )\n lines.append(f\"| **Δ Resolved** | {delta_str(c_resolved - b_resolved)} | — |\")\n lines.append(f\"| **Empty Patches** | {c_empty} | {b_empty} |\")\n lines.append(f\"| **Errors** | {c_error} | {b_error} |\")\n lines.append(\"\")\n\n # Instance-level changes\n if gained or lost:\n lines.append(\"### Instance-Level Changes\")\n lines.append(\"\")\n\n if gained:\n lines.append(\n f\"**✅ Newly resolved ({len(gained)}):** \"\n + \", \".join(f\"`{g}`\" for g in gained[:20])\n )\n if len(gained) > 20:\n lines.append(f\" ... and {len(gained) - 20} more\")\n lines.append(\"\")\n\n if lost:\n lines.append(\n f\"**❌ Regressions ({len(lost)}):** \"\n + \", \".join(f\"`{g}`\" for g in lost[:20])\n )\n if len(lost) > 20:\n lines.append(f\" ... and {len(lost) - 20} more\")\n lines.append(\"\")\n\n if not gained and not lost and c_resolved_ids and b_resolved_ids:\n lines.append(\n \"*Identical set of resolved instances — no regressions or improvements.*\"\n )\n lines.append(\"\")\n\n # Dashboard links\n lines.append(\"### 🔗 Links\")\n lines.append(\"\")\n if c_run_id:\n benchmark = (current_params or {}).get(\"benchmark\", \"swebench\")\n model_slug = (\n (current_params or {})\n .get(\"model_name\", \"\")\n .replace(\"/\", \"-\")\n .replace(\":\", \"-\")\n .replace(\"@\", \"-\")\n .replace(\".\", \"-\")\n )\n c_dash = f\"{DASHBOARD_BASE}/?run={benchmark}/{model_slug}/{c_run_id}/\"\n lines.append(f\"- [Current run dashboard]({c_dash})\")\n if b_run_id:\n benchmark = (baseline_params or {}).get(\"benchmark\", \"swebench\")\n model_slug = (\n (baseline_params or {})\n .get(\"model_name\", \"\")\n .replace(\"/\", \"-\")\n .replace(\":\", \"-\")\n .replace(\"@\", \"-\")\n .replace(\".\", \"-\")\n )\n b_dash = f\"{DASHBOARD_BASE}/?run={benchmark}/{model_slug}/{b_run_id}/\"\n lines.append(f\"- [Baseline run dashboard]({b_dash})\")\n lines.append(\"\")\n\n return \"\\n\".join(lines)\n\n\ndef github_api_request(\n url: str,\n token: str,\n *,\n method: str = \"GET\",\n data: dict[str, Any] | None = None,\n) -> dict[str, Any] | None:\n \"\"\"Make a GitHub API request. Returns parsed JSON or None for 204.\"\"\"\n body = json.dumps(data).encode() if data else None\n req = urllib.request.Request(\n url,\n data=body,\n method=method,\n headers={\n \"Authorization\": f\"token {token}\",\n \"Accept\": \"application/vnd.github+json\",\n \"Content-Type\": \"application/json\",\n },\n )\n with urllib.request.urlopen(req, timeout=30) as resp:\n if resp.status == 204:\n return None\n return json.loads(resp.read().decode())\n\n\ndef post_github_comment(repo: str, pr_number: int, body: str, token: str) -> None:\n \"\"\"Post a comment on a GitHub PR.\"\"\"\n url = f\"https://api.github.com/repos/{repo}/issues/{pr_number}/comments\"\n result = github_api_request(url, token, method=\"POST\", data={\"body\": body})\n if result:\n print(f\"Posted comment: {result.get('html_url', 'unknown')}\", file=sys.stderr)\n\n\ndef trigger_eval(\n token: str,\n *,\n sdk_ref: str,\n benchmark: str = \"swebench\",\n eval_limit: int = 50,\n model_ids: str = \"\",\n reason: str = \"\",\n repo: str = SDK_REPO,\n allow_unreleased: bool = True,\n benchmarks_branch: str = \"main\",\n eval_branch: str = \"main\",\n tool_preset: str = \"default\",\n agent_type: str = \"default\",\n instance_ids: str = \"\",\n) -> None:\n \"\"\"Dispatch an evaluation workflow via the GitHub Actions API.\"\"\"\n inputs: dict[str, str] = {\n \"benchmark\": benchmark,\n \"sdk_ref\": sdk_ref,\n \"eval_limit\": str(eval_limit),\n \"reason\": reason,\n \"benchmarks_branch\": benchmarks_branch,\n \"eval_branch\": eval_branch,\n \"tool_preset\": tool_preset,\n \"agent_type\": agent_type,\n \"allow_unreleased_branches\": str(allow_unreleased).lower(),\n }\n if model_ids:\n inputs[\"model_ids\"] = model_ids\n if instance_ids:\n inputs[\"instance_ids\"] = instance_ids\n\n url = (\n f\"https://api.github.com/repos/{repo}/actions/workflows/run-eval.yml/dispatches\"\n )\n payload = {\"ref\": sdk_ref, \"inputs\": inputs}\n\n print(f\"Dispatching eval workflow on {repo}...\", file=sys.stderr)\n print(f\" benchmark: {benchmark}\", file=sys.stderr)\n print(f\" sdk_ref: {sdk_ref}\", file=sys.stderr)\n print(f\" eval_limit: {eval_limit}\", file=sys.stderr)\n print(f\" model_ids: {model_ids or '(default)'}\", file=sys.stderr)\n print(f\" tool_preset: {tool_preset}\", file=sys.stderr)\n print(f\" agent_type: {agent_type}\", file=sys.stderr)\n if instance_ids:\n print(f\" instance_ids: {instance_ids}\", file=sys.stderr)\n if reason:\n print(f\" reason: {reason}\", file=sys.stderr)\n\n github_api_request(url, token, method=\"POST\", data=payload)\n print(\"✓ Workflow dispatched successfully.\", file=sys.stderr)\n print(\n f\" Monitor at: https://github.com/{repo}/actions/workflows/run-eval.yml\",\n file=sys.stderr,\n )\n\n\ndef _require_token() -> str:\n \"\"\"Return GITHUB_TOKEN or exit with error.\"\"\"\n token = os.environ.get(\"GITHUB_TOKEN\", \"\")\n if not token:\n print(\"ERROR: GITHUB_TOKEN environment variable not set\", file=sys.stderr)\n sys.exit(1)\n return token\n\n\ndef cmd_trigger(args: argparse.Namespace) -> None:\n \"\"\"Handle the 'trigger' subcommand.\"\"\"\n token = _require_token()\n trigger_eval(\n token,\n sdk_ref=args.sdk_ref,\n benchmark=args.benchmark,\n eval_limit=args.eval_limit,\n model_ids=args.model_ids or \"\",\n reason=args.reason or \"\",\n repo=args.repo,\n benchmarks_branch=args.benchmarks_branch,\n eval_branch=args.eval_branch,\n tool_preset=args.tool_preset,\n agent_type=args.agent_type,\n instance_ids=args.instance_ids or \"\",\n )\n\n\ndef cmd_compare(args: argparse.Namespace) -> None:\n \"\"\"Handle the 'compare' subcommand.\"\"\"\n # Validate\n if args.post_comment and (not args.pr or not args.repo):\n print(\"ERROR: --post-comment requires --pr and --repo\", file=sys.stderr)\n sys.exit(1)\n if not args.baseline and not args.auto_baseline:\n print(\"ERROR: Specify --baseline or --auto-baseline\", file=sys.stderr)\n sys.exit(1)\n\n benchmark, model_slug, run_id = parse_run_path(args.current_run_path)\n print(f\"Current run: {benchmark}/{model_slug}/{run_id}\", file=sys.stderr)\n\n # Fetch current run data\n current_report = get_report(args.current_run_path)\n if not current_report:\n print(f\"ERROR: No report found for {args.current_run_path}\", file=sys.stderr)\n sys.exit(1)\n\n current_params = get_params(args.current_run_path)\n\n # Find baseline\n if args.baseline:\n baseline_path = args.baseline\n else:\n current_eval_limit = (\n current_params.get(\"eval_limit\") if current_params else None\n )\n print(\n f\"Searching for baseline (lookback: {args.lookback_days} days, \"\n f\"eval_limit: {current_eval_limit})...\",\n file=sys.stderr,\n )\n baseline_path = find_baseline_run(\n benchmark, model_slug, run_id, args.lookback_days, current_eval_limit\n )\n\n if not baseline_path:\n print(\"No baseline run found. Cannot produce comparison.\", file=sys.stderr)\n sys.exit(1)\n\n print(f\"Baseline run: {baseline_path}\", file=sys.stderr)\n\n baseline_report = get_report(baseline_path)\n if not baseline_report:\n print(f\"ERROR: No report found for baseline {baseline_path}\", file=sys.stderr)\n sys.exit(1)\n\n baseline_params = get_params(baseline_path)\n\n # Generate comparison\n markdown = compute_diff(\n current_report, baseline_report, current_params, baseline_params\n )\n print(markdown)\n\n # Post comment if requested\n if args.post_comment:\n token = _require_token()\n body = (\n markdown\n + \"\\n---\\n\"\n + \"*This comparison was generated by an AI assistant \"\n + \"(OpenHands) on behalf of the user.*\\n\"\n )\n post_github_comment(args.repo, args.pr, body, token)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser(\n description=\"Trigger, compare, and report on OpenHands evaluation runs\",\n )\n subparsers = parser.add_subparsers(dest=\"command\", required=True)\n\n # --- trigger subcommand ---\n p_trigger = subparsers.add_parser(\n \"trigger\",\n help=\"Dispatch an evaluation workflow\",\n description=\"Trigger an eval run via the GitHub Actions workflow_dispatch API.\",\n )\n p_trigger.add_argument(\n \"--sdk-ref\",\n required=True,\n help=\"SDK branch, tag, or commit to evaluate (e.g., main, v1.16.0, my-branch)\",\n )\n p_trigger.add_argument(\n \"--benchmark\",\n default=\"swebench\",\n choices=BENCHMARKS,\n help=\"Benchmark to run (default: swebench)\",\n )\n p_trigger.add_argument(\n \"--eval-limit\",\n type=int,\n default=50,\n help=\"Number of instances to evaluate (default: 50)\",\n )\n p_trigger.add_argument(\n \"--model-ids\",\n default=\"\",\n help=(\n \"Comma-separated model IDs \"\n \"(see .github/run-eval/resolve_model_config.py; default: first model)\"\n ),\n )\n p_trigger.add_argument(\"--reason\", default=\"\", help=\"Human-readable trigger reason\")\n p_trigger.add_argument(\n \"--repo\",\n default=SDK_REPO,\n help=f\"Repository to trigger on (default: {SDK_REPO})\",\n )\n p_trigger.add_argument(\n \"--benchmarks-branch\",\n default=\"main\",\n help=\"Benchmarks repo branch (default: main)\",\n )\n p_trigger.add_argument(\n \"--eval-branch\",\n default=\"main\",\n help=\"Evaluation repo branch (default: main)\",\n )\n p_trigger.add_argument(\n \"--tool-preset\",\n default=\"default\",\n choices=TOOL_PRESETS,\n help=\"Tool preset for file editing (default: default)\",\n )\n p_trigger.add_argument(\n \"--agent-type\",\n default=\"default\",\n choices=AGENT_TYPES,\n help=\"Agent type (default: default)\",\n )\n p_trigger.add_argument(\n \"--instance-ids\",\n default=\"\",\n help=\"Comma-separated instance IDs to evaluate (overrides eval-limit)\",\n )\n\n # --- compare subcommand ---\n p_compare = subparsers.add_parser(\n \"compare\",\n help=\"Compare two evaluation runs\",\n description=\"Fetch results for two eval runs and produce a diff report.\",\n )\n p_compare.add_argument(\n \"current_run_path\",\n help=\"Run path (e.g., swebench/litellm_proxy-claude-.../23775164157/)\",\n )\n p_compare.add_argument(\"--baseline\", help=\"Explicit baseline run path\")\n p_compare.add_argument(\n \"--auto-baseline\",\n action=\"store_true\",\n help=\"Auto-find the most recent previous run as baseline\",\n )\n p_compare.add_argument(\n \"--lookback-days\",\n type=int,\n default=14,\n help=\"Days to search for baseline (default: 14)\",\n )\n p_compare.add_argument(\n \"--post-comment\",\n action=\"store_true\",\n help=\"Post result as a GitHub PR comment\",\n )\n p_compare.add_argument(\"--pr\", type=int, help=\"PR number for commenting\")\n p_compare.add_argument(\"--repo\", help=\"Repository (OWNER/REPO) for commenting\")\n\n args = parser.parse_args()\n\n if args.command == \"trigger\":\n cmd_trigger(args)\n elif args.command == \"compare\":\n cmd_compare(args)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": ".agents/skills/run-eval.md", "content": "---\nname: run-eval\ndescription: Trigger and monitor evaluation runs for benchmarks like SWE-bench, GAIA, and others. Use when running evaluations via GitHub Actions or monitoring eval progress through Datadog and kubectl.\ntriggers:\n- run eval\n- trigger eval\n- evaluation run\n- swebench eval\n---\n\n# Running Evaluations\n\n## Trigger via GitHub API\n\n```bash\ncurl -X POST \\\n -H \"Authorization: token $GITHUB_TOKEN\" \\\n -H \"Accept: application/vnd.github+json\" \\\n \"https://api.github.com/repos/OpenHands/software-agent-sdk/actions/workflows/run-eval.yml/dispatches\" \\\n -d '{\n \"ref\": \"main\",\n \"inputs\": {\n \"benchmark\": \"swebench\",\n \"sdk_ref\": \"main\",\n \"eval_limit\": \"50\",\n \"model_ids\": \"claude-sonnet-4-5-20250929\",\n \"reason\": \"Description of eval run\",\n \"benchmarks_branch\": \"main\"\n }\n }'\n```\n\n**Key parameters:**\n- `benchmark`: `swebench`, `swebenchpro`, `swebenchmultimodal`, `gaia`, `swtbench`, `commit0`, `multiswebench`, `terminalbench`\n- `eval_limit`: Any positive integer (e.g., `1`, `10`, `50`, `200`)\n- `model_ids`: See `.github/run-eval/resolve_model_config.py` for available models\n- `benchmarks_branch`: Use feature branch from the benchmarks repo to test benchmark changes before merging\n\n**Note:** When running a full eval, you must select an `eval_limit` that is greater than or equal to the actual number of instances in the benchmark. If you specify a smaller limit, only that many instances will be evaluated (partial eval).\n\n## Monitoring\n\n**Datadog script** (requires `OpenHands/evaluation` repo; DD_API_KEY, DD_APP_KEY, and DD_SITE environment variables are set):\n```bash\nDD_API_KEY=$DD_API_KEY DD_APP_KEY=$DD_APP_KEY DD_SITE=$DD_SITE \\\n python scripts/analyze_evals.py --job-prefix --time-range 60\n# EVAL_RUN_ID format: typically the workflow run ID from GitHub Actions\n```\n\n**kubectl** (for users with cluster access - the agent does not have kubectl access):\n```bash\nkubectl logs -f job/eval-eval-- -n evaluation-jobs\n```\n\n## Common Errors\n\n| Error | Cause | Fix |\n|-------|-------|-----|\n| `503 Service Unavailable` | Infrastructure overloaded | Ask user to stop some evaluation runs |\n| `429 Too Many Requests` | Rate limiting | Wait or reduce concurrency |\n| `failed after 3 retries` | Instance failures | Check Datadog logs for root cause |\n\n## Limits\n\n- Max 256 parallel runtimes (jobs will queue if this limit is exceeded)\n- Full evals typically take 1-3 hours depending on benchmark size\n" }, { "path": ".agents/skills/sdk-release/SKILL.md", "content": "---\nname: sdk-release\ndescription: >-\n This skill should be used when the user asks to \"release the SDK\",\n \"prepare a release\", \"publish a new version\", \"cut a release\",\n \"do a release\", or mentions the SDK release checklist or release process.\n Guides through the full software-agent-sdk release workflow\n from version bump to PyPI publication, emphasizing human checkpoints.\n---\n\n# SDK Release Guide\n\nThis skill walks through the software-agent-sdk release process step by step.\n\n> **🚨 CRITICAL**: NEVER merge the release PR or create/publish a GitHub\n> release without the human's explicit approval. Release is the last line\n> of human defense. Always present the current status and ask for\n> confirmation before performing any irreversible action.\n\n## Phase 1: Trigger the Prepare-Release Workflow\n\nDetermine the target version (SemVer `X.Y.Z`). Then trigger the\n`prepare-release.yml` workflow, which creates a release branch and PR\nautomatically.\n\n### Via GitHub UI\n\nNavigate to\n,\nclick **Run workflow**, enter the version (e.g. `1.16.0`), and run it.\n\n### Via GitHub API\n\n```bash\ncurl -X POST \\\n -H \"Authorization: token $GITHUB_TOKEN\" \\\n -H \"Accept: application/vnd.github+json\" \\\n \"https://api.github.com/repos/OpenHands/software-agent-sdk/actions/workflows/prepare-release.yml/dispatches\" \\\n -d '{\n \"ref\": \"main\",\n \"inputs\": {\n \"version\": \"1.16.0\"\n }\n }'\n```\n\nThe workflow will:\n1. Validate version format\n2. Create branch `rel-`\n3. Run `make set-package-version version=` across all packages\n4. Update the `sdk_ref` default in the eval workflow\n5. Open a PR titled **\"Release v\\\"** with labels\n `integration-test`, `behavior-test`, and `test-examples`\n\n### ⏸ Checkpoint — Confirm PR Created\n\nVerify the PR exists and the version changes look correct before continuing.\n\n```bash\ngh pr list --repo OpenHands/software-agent-sdk \\\n --head \"rel-\" --json number,title,url\n```\n\n## Phase 2: Address Deprecation Deadlines\n\nThe `deprecation-check` CI job runs on every PR. If the release version\ncrosses any deprecation deadline declared in the codebase, the check will\nfail.\n\nReview the failing check output and either:\n- Remove the deprecated code if the deadline has passed, **or**\n- Extend the deadline with justification.\n\nPush fixes to the release branch. The check must pass before merging.\n\n## Phase 3: Wait for CI — Tests Must Pass\n\nThe release PR triggers three labeled test suites. **All three must pass.**\n\n| Label | Suite | What it covers |\n|-------|-------|----------------|\n| `integration-test` | Integration tests | End-to-end agent scenarios |\n| `behavior-test` | Behavior tests | Agent behavioral guardrails |\n| `test-examples` | Example tests | All runnable examples in `examples/` |\n\nMonitor status:\n\n```bash\ngh pr checks --repo OpenHands/software-agent-sdk\n```\n\n### ⏸ Checkpoint — Human Judgment on Failures\n\nSome test failures may be pre-existing or flaky. Decide with the team\nwhether each failure is:\n- **Blocking** — must fix before release\n- **Known / pre-existing** — acceptable to release with a follow-up issue\n- **Flaky** — re-run the workflow\n\nRe-run failed jobs:\n\n```bash\n# Find the run ID\ngh run list --repo OpenHands/software-agent-sdk \\\n --branch \"rel-\" --limit 5\n\n# Re-run failed jobs\ngh run rerun --repo OpenHands/software-agent-sdk --failed\n```\n\n## Phase 4: Run Evaluation (Optional but Recommended)\n\nTrigger an evaluation run on SWE-bench (or another benchmark) against the\nrelease branch to catch regressions. See the `run-eval` skill for full\ndetails.\n\n```bash\ncurl -X POST \\\n -H \"Authorization: token $GITHUB_TOKEN\" \\\n -H \"Accept: application/vnd.github+json\" \\\n \"https://api.github.com/repos/OpenHands/software-agent-sdk/actions/workflows/run-eval.yml/dispatches\" \\\n -d '{\n \"ref\": \"main\",\n \"inputs\": {\n \"benchmark\": \"swebench\",\n \"sdk_ref\": \"v\",\n \"eval_limit\": \"50\",\n \"reason\": \"Pre-release eval for v\",\n \"allow_unreleased_branches\": \"true\"\n }\n }'\n```\n\n### ⏸ Checkpoint — Evaluate Results\n\nCompare the eval results against the previous release. Significant score\ndrops should block the release.\n\n## Phase 5: Merge the Release PR\n\n> **🚨 STOP — Do NOT merge without explicit human approval.**\n> Present the CI status summary and ask the human to confirm before merging.\n> Merging is effectively irreversible — it automatically triggers the full\n> release pipeline (GitHub release → PyPI publish → downstream version bumps).\n\nOnce the human approves:\n\n```bash\ngh pr merge --repo OpenHands/software-agent-sdk --merge\n```\n\n## Phase 6: Automated Release Pipeline (no action needed)\n\nWhen the release PR is merged, the following happens automatically:\n\n1. **`create-release.yml`** detects the merged `rel-*` branch, creates a\n GitHub release with tag `v` and auto-generated release notes.\n2. **`pypi-release.yml`** triggers on the published release and publishes\n all four packages to PyPI:\n - `openhands-sdk`\n - `openhands-tools`\n - `openhands-workspace`\n - `openhands-agent-server`\n3. **`version-bump-prs.yml`** triggers after successful PyPI publish and\n creates downstream version bump PRs.\n\n### ⏸ Checkpoint — Verify PyPI Publication\n\n```bash\n# Check each package is available (allow a few minutes for indexing)\nfor pkg in openhands-sdk openhands-tools openhands-workspace openhands-agent-server; do\n curl -s -o /dev/null -w \"$pkg: %{http_code}\\n\" \\\n \"https://pypi.org/pypi/$pkg//json\"\ndone\n```\n\nAll should return `200`.\n\n## Phase 7: Post-Release Announcements\n\nAfter the automated pipeline completes, compose a Slack message for the\nhuman to post, including links to the downstream version bump PRs:\n\n```\n🚀 *SDK v published to PyPI!*\n\nVersion bump PRs:\n• |OpenHands>\n• |OpenHands-CLI>\n\nRelease: |v>\n```\n\nSee `references/post-release-checklist.md` for details on reviewing\ndownstream PRs and handling any issues.\n\n## Quick Reference — Full Checklist\n\n- [ ] Trigger `prepare-release.yml` with target version\n- [ ] Verify release PR is created\n- [ ] Fix deprecation deadline failures (if any)\n- [ ] Integration tests pass\n- [ ] Behavior tests pass\n- [ ] Example tests pass\n- [ ] (Optional) Evaluation run shows no regressions\n- [ ] **🚨 Get human approval**, then merge the release PR\n- [ ] _(Automated)_ GitHub release created with auto-generated notes\n- [ ] _(Automated)_ Packages published to PyPI\n- [ ] _(Automated)_ Downstream version bump PRs created\n- [ ] Verify packages appear on PyPI\n- [ ] Send Slack message with downstream version bump PR links\n" }, { "path": ".agents/skills/sdk-release/references/post-release-checklist.md", "content": "# Post-Release Checklist\n\nAfter the GitHub release is published and PyPI packages are available,\nseveral automated and manual follow-up steps occur.\n\n## Automated: Downstream Version Bump PRs\n\nThe `version-bump-prs.yml` workflow runs automatically after `pypi-release`\nsucceeds. It creates PRs in two repositories:\n\n### OpenHands-CLI (`OpenHands/openhands-cli`)\n\n- Branch: `bump-sdk-`\n- Updates `openhands-sdk` and `openhands-tools` via `uv add`\n- Verify the PR passes CLI tests before merging\n\n```bash\ngh pr list --repo OpenHands/openhands-cli \\\n --search \"bump-sdk-\" --json number,title,url\n```\n\n### OpenHands (`All-Hands-AI/OpenHands`)\n\n- Branch: `bump-sdk-`\n- Updates `openhands-sdk`, `openhands-tools`, and `openhands-agent-server`\n in `pyproject.toml`\n- Regenerates `poetry.lock`\n- Updates `AGENT_SERVER_IMAGE` in `sandbox_spec_service.py`\n- Verifies `enterprise/pyproject.toml` does not have explicit SDK pins\n\n```bash\ngh pr list --repo All-Hands-AI/OpenHands \\\n --search \"bump-sdk-\" --json number,title,url\n```\n\n## Manual Review of Downstream PRs\n\nBoth PRs require human review:\n\n1. **Check CI passes** on each downstream PR\n2. **Verify compatibility** — especially if the release includes breaking\n changes or new features that need adoption\n3. **Merge** once satisfied\n\n## Evaluation on OpenHands Index\n\nIf not already done pre-release, trigger a full evaluation run\nagainst the published version:\n\n```bash\ncurl -X POST \\\n -H \"Authorization: token $GITHUB_TOKEN\" \\\n -H \"Accept: application/vnd.github+json\" \\\n \"https://api.github.com/repos/OpenHands/software-agent-sdk/actions/workflows/run-eval.yml/dispatches\" \\\n -d '{\n \"ref\": \"main\",\n \"inputs\": {\n \"benchmark\": \"swebench\",\n \"sdk_ref\": \"v\",\n \"eval_limit\": \"300\",\n \"reason\": \"Post-release eval v\"\n }\n }'\n```\n\n## Documentation Updates\n\nIf the release includes user-facing features, verify documentation is\nupdated in `OpenHands/docs` (SDK docs live under `sdk/`). See the\n`feature-release-rollout` skill for the full downstream propagation\nworkflow.\n\n## Troubleshooting\n\n### PyPI publication failed\n\nRe-run the `pypi-release.yml` workflow manually. It uses `--check-url`\nto skip already-published packages, so partial reruns are safe.\n\n```bash\ngh workflow run pypi-release.yml --repo OpenHands/software-agent-sdk\n```\n\n### Version bump PR has conflicts\n\nThe automated PR may conflict if the downstream repo changed dependency\npins since the workflow ran. Resolve conflicts manually on the bump branch,\nor re-trigger `version-bump-prs.yml` with the version input.\n\n```bash\ngh workflow run version-bump-prs.yml \\\n --repo OpenHands/software-agent-sdk \\\n -f version=\n```\n\n### Downstream tests fail after bump\n\nIf a downstream repo's tests fail on the version bump PR, investigate\nwhether the failure is a breaking change in the SDK release. If so,\neither:\n- Fix the downstream code on the bump branch, or\n- Publish a patch release of the SDK with the fix\n" }, { "path": ".agents/skills/write-behavior-test.md", "content": "---\nname: write-behavior-test\ndescription: Guide for writing behavior tests that verify agents follow system message guidelines and avoid undesirable behaviors. Use when creating integration tests for agent behavior validation.\ntriggers:\n- /write_behavior_test\n---\n\n# Behavior Test Writing Guide\n\nYou are helping to create **behavior tests** for the agent-sdk integration test suite. These tests verify that agents follow system message guidelines and avoid undesirable behaviors.\n\nThe tests are for the agent powered by this SDK, so you may need to refer the codebase for details on how the agent works in order to write effective tests.\n\n## Behavior Tests vs Task Tests\n\n**Task Tests (t*.py)** - REQUIRED tests that verify task completion:\n- Focus: Can the agent successfully complete the task?\n- Example: Fix typos in a file, create a script, implement a feature\n\n**Behavior Tests (b*.py)** - OPTIONAL tests that verify proper behavior:\n- Focus: Does the agent follow best practices and system guidelines?\n- Example: Don't implement when asked for advice, don't over-verify, avoid redundant files\n\n## Key Principles for Writing Behavior Tests\n\n### ✅ DO:\n\n1. **Use Real Repositories**\n - Clone actual GitHub repositories that represent real-world scenarios\n - Pin to a specific historical commit (before a fix/feature was added)\n - Example: `clone_pinned_software_agent_repo(workspace)` helper\n\n2. **Test Realistic Complex, Nuanced Behaviors**\n - Try to make the task as realistic as possible to real HUMAN interactions, from file naming, (somewhat lazy) instruction style, etc\n - Focus on subtle behavioral issues that require judgment\n - Test scenarios where the \"right\" behavior isn't immediately obvious\n - Examples: When to implement vs advise, when to stop testing, whether to add backward compatibility\n\n3. **Clean Up Repository History**\n - Check out to a commit BEFORE the solution exists\n - Reset/remove future commits (see existing tests for examples)\n - Ensures the agent experiences the same context as real users\n\n4. **Use Helper Functions**\n - `find_file_editing_operations(events)` - Find file create/edit operations\n - `find_tool_calls(events, tool_name)` - Find specific tool usage\n - `get_conversation_summary(events)` - Get summary for LLM judge\n - `judge_agent_behavior(...)` - Use LLM to evaluate behavior quality\n\n5. **Leverage LLM Judges**\n - Use `judge_agent_behavior()` for subjective evaluations\n - Provide clear evaluation criteria in the judge prompt\n - Track judge usage costs: `self.add_judge_usage(prompt_tokens, completion_tokens, cost)`\n\n6. **Adaptation of Problem Description to Task**\n - If you find the problem description is not easy to adapt to a behavior test, e.g. it requires complex environment setup like kubernetes, try to come up with a simpler problem description that still captures the essence of the behavior you want to test but is easier to implement in the test framework.\n - Ensure the instructions naturally lead to the behavior you want to evaluate\n\n### ❌ DO NOT:\n\n1. **Avoid Simple Synthetic Tests**\n - Don't create artificial scenarios with minimal setup\n - Don't test behaviors that are too obvious or straightforward\n - Example: Don't create a single-file test with trivial content\n\n2. **Don't Test Basic Functionality**\n - Behavior tests are NOT for testing if the agent can use tools\n - Task tests handle basic capability verification\n - Focus on HOW the agent approaches problems, not IF it can solve them\n\n3. **Don't Overcomplicate Static Assertions**\n - Use assertions for clear-cut checks (e.g., no file edits)\n - Rely on LLM judges for nuanced behavior evaluations\n - Avoid trying to encode subjective judgments purely in code or too much static logic\n\n## Tips for Test Difficulty Calibration\n\n**Make tests challenging but not impossible and too long:**\n\n1. **Context Complexity**: Use real codebases with multiple files and dependencies, either the software-agent-sdk or other popular open-source repos you find suitable\n2. **Ambiguity**: Prefer instructions that could be interpreted multiple ways\n3. **Temptation**: Set up scenarios where the \"easy wrong path\" is tempting\n4. **Realism**: Mirror real user interactions and expectations\n\n**Examples of Good Complexity:**\n- \"How to implement X?\" (tests if agent implements vs advises)\n- \"Update constant Y\" (tests if agent over-verifies with excessive test runs)\n- \"Rename method A to B\" (tests if agent adds unnecessary backward compatibility)\n\n## Example Behavior Test Patterns\n\n1. **Premature Implementation** - Tests if agent implements when asked for advice only\n2. **Over-verification** - Tests if agent runs excessive tests beyond what's needed\n3. **Unnecessary Compatibility** - Tests if agent adds backward compatibility shims when not needed\n4. **Redundant Artifacts** - Tests if agent creates extra files (docs, READMEs) without being asked\n5. **Communication Quality** - Tests if agent provides explanations for actions\n\n## File Naming Convention\n\nName your test file: `b##_descriptive_name.py`\n- `b` prefix indicates behavior test (auto-detected)\n- `##` is a zero-padded number (e.g., 01, 02, 03)\n- Use snake_case for the descriptive name\n\n## Final Checklist\n\nBefore submitting your behavior test, verify:\n\n- [ ] Uses a real repository or complex codebase\n- [ ] Tests a nuanced behavior, not basic functionality\n- [ ] Includes clear and not overly complex verification logic (assertions or LLM judge)\n- [ ] Has a descriptive docstring explaining what behavior is tested\n- [ ] Properly tracks judge usage costs if using LLM evaluation\n- [ ] Follows naming convention: `b##_descriptive_name.py`\n- [ ] Test is realistic and based on actual behavioral issues observed\n\nRemember: The goal is to catch subtle behavioral issues that would appear in real-world usage, serving as regression tests for system message improvements.\n" }, { "path": ".dockerignore", "content": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\nshare/python-wheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n# Note: We keep our custom spec file in version control\n# *.spec\n\n# PyInstaller build directories\nbuild/\ndist/\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.nox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n*.py,cover\n.hypothesis/\n.pytest_cache/\ncover/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\ndb.sqlite3-journal\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\n.pybuilder/\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# IPython\nprofile_default/\nipython_config.py\n\n# pyenv\n# For a library or package, you might want to ignore these files since the code is\n# intended to run in multiple environments; otherwise, check them in:\n# .python-version\n\n# pipenv\n# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.\n# However, in case of collaboration, if having platform-specific dependencies or dependencies\n# having no cross-platform support, pipenv may install dependencies that don't work, or not\n# install all needed dependencies.\n#Pipfile.lock\n\n# poetry\n# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.\n# This is especially recommended for binary packages to ensure reproducibility, and is more\n# commonly ignored for libraries.\n# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control\n# poetry.lock\n\n# pdm\n# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.\n#pdm.lock\n# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it\n# in version control.\n# https://pdm.fming.dev/#use-with-ide\n.pdm.toml\n\n# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm\n__pypackages__/\n\n# Celery stuff\ncelerybeat-schedule\ncelerybeat.pid\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# Pyre type checker\n.pyre/\n\n# pytype static type analyzer\n.pytype/\n\n# Cython debug symbols\ncython_debug/\n\n# PyCharm\n# JetBrains specific template is maintained in a separate JetBrains.gitignore that can\n# be added to the global gitignore or merged into this project gitignore. For a PyCharm\n# project, it is recommended to ignore the entire .idea directory.\n.idea/\n\n# VS Code\n.vscode/\n\n# macOS\n.DS_Store\n.AppleDouble\n.LSOverride\n\n# Windows\nThumbs.db\nehthumbs.db\nDesktop.ini\n$RECYCLE.BIN/\n\n# Linux\n*~\n\n# Temporary files\n*.tmp\n*.temp\n*.swp\n*.swo\n\n# UV specific\n.uv/\n\n# Project specific\n*.log\n.coverage\n.pytest_cache/\n\nworkspace/\n.client\n.docker\n\n\n.git\n.git/**\n\n# VS Code: Ignore all but certain files that specify repo-specific settings.\n# https://stackoverflow.com/questions/32964920/should-i-commit-the-vscode-folder-to-source-control\n.vscode/**/*\n!.vscode/extensions.json\n!.vscode/tasks.json\n\n# VS Code extensions/forks:\n.cursorignore\n.rooignore\n.clineignore\n.windsurfignore\n.cursorrules\n.roorules\n.clinerules\n.windsurfrules\n.cursor/rules\n.roo/rules\n.cline/rules\n.windsurf/rules\n.repomix\nrepomix-output.txt\n\n# misc\n.DS_Store\n.env.local\n.env.development.local\n.env.test.local\n.env.production.local\n\nnpm-debug.log*\nyarn-debug.log*\nyarn-error.log*\n\nlogs\n\n# agent\n.envrc\ncache\n.jinja_cache/\n\n.conversations*\nworkspace/\n\n# Build optimization: exclude files not needed for building agent-server\ntests/\n*.log\n.github/\nscripts/\nexamples/\n.ruff_cache/\n.uv-cache/\nMakefile\ndocs/\n*.md\n!README.md\n.pre-commit-config.yaml\n.python-version\n" }, { "path": ".github/ISSUE_TEMPLATE/bug_template.yml", "content": "---\nname: Bug\ndescription: Report a problem with OpenHands SDK\ntitle: '[Bug]: '\nlabels: [bug]\nbody:\n - type: markdown\n attributes:\n value: |\n ## Thank you for reporting a bug! 🐛\n\n **Please fill out all required fields.** Issues missing critical information (version, installation method, reproduction steps, etc.) will be delayed or closed until complete details are provided.\n\n Clear, detailed reports help us resolve issues faster.\n\n - type: checkboxes\n attributes:\n label: Is there an existing issue for the same bug?\n description: Please search existing issues before creating a new one. If found, react or comment to the duplicate issue instead of making a \n new one. \n options:\n - label: I have searched existing issues and this is not a duplicate.\n required: true\n\n - type: textarea\n id: bug-description\n attributes:\n label: Bug Description\n description: Clearly describe what went wrong. Be specific and concise.\n placeholder: Example - When I use the SDK to create an agent with custom tools, the agent fails to register the tools with a TypeError.\n validations:\n required: true\n\n - type: textarea\n id: expected-behavior\n attributes:\n label: Expected Behavior\n description: What did you expect to happen?\n placeholder: Example - The agent should successfully register custom tools and make them available for use.\n validations:\n required: false\n\n - type: textarea\n id: actual-behavior\n attributes:\n label: Actual Behavior\n description: What actually happened?\n placeholder: \"Example - TypeError: 'NoneType' object is not iterable when calling agent.register_tool()\"\n validations:\n required: false\n\n - type: textarea\n id: reproduction-steps\n attributes:\n label: Steps to Reproduce\n description: Provide clear, step-by-step instructions to reproduce the bug.\n placeholder: |\n 1. Install openhands-sdk using pip\n 2. Import and create an agent instance\n 3. Define a custom tool function\n 4. Call agent.register_tool(custom_tool)\n 5. Error appears\n validations:\n required: false\n\n - type: input\n id: installation\n attributes:\n label: Installation Method\n description: How did you install the OpenHands SDK?\n placeholder: ex. pip install openhands-sdk, uv pip install openhands-sdk, pip install -e ., etc.\n\n - type: input\n id: installation-other\n attributes:\n label: If you selected \"Other\", please specify\n description: Describe your installation method\n placeholder: ex. Poetry, conda, custom setup, etc.\n\n - type: input\n id: sdk-version\n attributes:\n label: SDK Version\n description: What version are you using? Check with `pip show openhands-sdk` or similar for other packages.\n placeholder: ex. 0.1.0, 0.2.0, main branch, commit hash, etc.\n validations:\n required: false\n\n - type: checkboxes\n id: version-confirmation\n attributes:\n label: Version Confirmation\n description: Bugs on older versions may already be fixed. Please upgrade before submitting.\n options:\n - label: I have confirmed this bug exists on the LATEST version of OpenHands SDK\n required: false\n\n - type: input\n id: python-version\n attributes:\n label: Python Version\n description: Which Python version are you using?\n placeholder: ex. 3.10.12, 3.11.5, 3.12.0\n validations:\n required: false\n\n - type: input\n id: model-name\n attributes:\n label: Model Name (if applicable)\n description: Which model(s) are you using?\n placeholder: ex. gpt-4o, claude-3-5-sonnet-20241022, openrouter/deepseek-r1, etc.\n validations:\n required: false\n\n - type: dropdown\n id: os\n attributes:\n label: Operating System\n options:\n - MacOS\n - Linux\n - WSL on Windows\n - Windows\n - Other\n validations:\n required: false\n\n - type: textarea\n id: logs\n attributes:\n label: Logs and Error Messages\n description: |\n **Paste relevant logs, error messages, or stack traces.** Use code blocks (```) for formatting.\n\n Include full stack traces when available.\n placeholder: |\n ```\n Paste error logs here\n ```\n\n - type: textarea\n id: code-sample\n attributes:\n label: Minimal Code Sample\n description: |\n If possible, provide a minimal code sample that reproduces the issue.\n placeholder: |\n ```python\n from openhands.sdk import Agent\n\n # Your minimal reproducible code here\n ```\n\n - type: textarea\n id: additional-context\n attributes:\n label: Screenshots and Additional Context\n description: |\n Add screenshots, environment details, dependency versions, or other context that helps explain the issue.\n\n placeholder: Drag and drop screenshots here, paste links, or add additional context.\n\n - type: markdown\n attributes:\n value: |\n ---\n **Note:** Please help us help you! Well-documented bugs are easier to reproduce and fix. Thank you for your understanding!\n" }, { "path": ".github/ISSUE_TEMPLATE/feature_request.yml", "content": "---\nname: Feature Request or Enhancement\ndescription: Suggest a new feature or improvement for OpenHands SDK\ntitle: '[Feature]: '\nlabels: [enhancement]\nbody:\n - type: markdown\n attributes:\n value: |\n ## Thank you for suggesting a feature! 💡\n\n We encourage you to open the discussion on the feature you need. You are always welcome to implement it, if you wish.\n\n - type: checkboxes\n attributes:\n label: Is there an existing feature request for this?\n description: Please search existing issues and feature requests before creating a new one. If found, react or comment to the duplicate issue\n instead of making a new one. \n options:\n - label: I have searched existing issues and feature requests, and this is not a duplicate.\n required: true\n\n - type: textarea\n id: problem-statement\n attributes:\n label: Problem or Use Case\n description: What problem are you trying to solve? What use case would this feature enable?\n placeholder: |\n Example - As a developer building agents, I need to persist agent state between sessions. Currently, there's no built-in mechanism for saving and loading agent memory, which means agents lose context when the process restarts.\n validations:\n required: true\n\n - type: textarea\n id: proposed-solution\n attributes:\n label: Proposed Solution\n description: Describe your ideal solution. What should this feature do? How should it work?\n placeholder: |\n Example - Add a StateManager class that allows saving and loading agent state to/from disk or database. Provide methods like save_state(), load_state(), and clear_state(). Support multiple backend options (JSON files, SQLite, Redis, etc.).\n validations:\n required: true\n\n - type: textarea\n id: alternatives\n attributes:\n label: Alternatives Considered\n description: Have you considered any alternative solutions or workarounds? What are their limitations?\n placeholder: Example - I tried manually serializing agent state using pickle, but it's not portable across SDK versions and doesn't handle \n complex tool state properly.\n\n - type: dropdown\n id: priority\n attributes:\n label: Priority / Severity\n description: How important is this feature to your workflow?\n options:\n - Critical - Blocking my work, no workaround available\n - High - Significant impact on productivity\n - Medium - Would improve experience\n - Low - Nice to have\n default: 2\n validations:\n required: true\n\n - type: dropdown\n id: scope\n attributes:\n label: Estimated Scope\n description: To the best of your knowledge, how complex do you think this feature would be to implement?\n options:\n - Small - API addition, config option, or minor change\n - Medium - New feature with moderate complexity\n - Large - Significant feature requiring architecture changes\n - Unknown - Not sure about the technical complexity\n default: 3\n\n - type: checkboxes\n id: feature-area\n attributes:\n label: Feature Area\n description: Which part of OpenHands SDK does this feature relate to? If you select \"Other\", please specify the area in the Additional \n Context section below. \n options:\n - label: Agent API / Core functionality\n - label: Tools / Tool system\n - label: Skills / Plugins\n - label: Agent Server\n - label: Workspace management\n - label: Configuration / Settings\n - label: Examples / Templates\n - label: Documentation\n - label: Testing / Development tools\n - label: Performance / Optimization\n - label: Integrations (GitHub, APIs, etc.)\n - label: Other\n\n - type: textarea\n id: technical-details\n attributes:\n label: Technical Implementation Ideas (Optional)\n description: If you have technical expertise, share implementation ideas, API suggestions, or relevant technical details.\n placeholder: |\n Example - Could implement StateManager as an abstract base class with concrete implementations for different backends. Add state_manager parameter to Agent constructor. Use JSON serialization for simple state, MessagePack for better performance.\n\n - type: textarea\n id: additional-context\n attributes:\n label: Additional Context\n description: Add any other context, code examples, API mockups, or references that help illustrate this feature request.\n placeholder: |\n Example code or API design:\n ```python\n from openhands.sdk import Agent, StateManager\n\n agent = Agent(state_manager=StateManager('file://agent_state.json'))\n agent.save_state()\n ```\n" }, { "path": ".github/PULL_REQUEST_TEMPLATE.md", "content": "\n\n\n\n- [ ] A human has tested these changes.\n\n---\n\n## Why\n\n\n\n## Summary\n\n\n-\n\n## Issue Number\n\n\n## How to Test\n\n\n\n## Video/Screenshots\n\n\n\n## Type\n\n- [ ] Bug fix\n- [ ] Feature\n- [ ] Refactor\n- [ ] Breaking change\n- [ ] Docs / chore\n\n## Notes\n\n\n" }, { "path": ".github/dependabot.yml", "content": "---\n# Dependabot configuration for automated dependency updates\n# See: https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file\n#\n# Note: Python (pip) ecosystem is not configured here because Dependabot does not\n# fully support uv workspaces yet. See issue #2510 for tracking.\n\nversion: 2\n\nupdates:\n # GitHub Actions\n - package-ecosystem: github-actions\n directory: /\n schedule:\n interval: weekly\n commit-message:\n prefix: chore(deps)\n" }, { "path": ".github/prompts/update-documentation.md", "content": "# Documentation Update Prompt\n\nYou are a world-class documentation writer tasked with keeping the OpenHands Agent SDK documentation accurate and up-to-date. Your goal is to ensure documentation reflects the current codebase and provides clear, minimal, and actionable guidance.\n\n## Core Objectives\n\n1. **Accuracy**: Ensure all documentation matches the current codebase\n2. **Completeness**: Include all available tools and core components\n3. **Clarity**: Keep examples simple, working, and easy to understand\n4. **Navigation**: Provide source code links for all definitions\n\n## Tasks to Perform\n\n### 1. Codebase Analysis\n\n- Scan `examples/` for available examples\n- Scan `openhands-tools/` for all available runtime tools\n- Check `openhands-sdk/openhands/tool/builtins/` for built-in tools\n- Identify any new tools or removed tools since last update\n\n### 2. Documentation Review\n\nReview these key files for accuracy:\n- `docs/architecture/overview.md` - High-level component interactions and design principles\n- `docs/architecture/tool.md` - Tool system, inheritance, and MCP integration\n- `docs/architecture/agent.md` - Agent architecture and execution flow\n- `docs/architecture/llm.md` - LLM integration and capabilities\n- `docs/architecture/conversation.md` - Conversation interface and persistence\n- `docs/getting-started.mdx` - Make sure we have descriptions of all examples listed out in `examples/`\n- `docs/index.md` - Overview and navigation\n- `README.md` - Root project documentation\n\n### 3. Content Updates Required\n\n#### Architecture Diagrams\n\n- Keep mermaid diagrams SIMPLE and READABLE across all docs/architecture/ files\n- Focus on core components and relationships, not every possible class\n- Include all current runtime tools: TerminalTool, FileEditorTool, TaskTrackerTool, etc.\n- Verify component interactions and inheritance reflect actual codebase structure\n\n#### Tool Documentation\n\nFor each tool, ensure:\n- Accurate usage examples with `.create()` method\n- Working code snippets (test them!)\n- Source code links to GitHub\n- Clear descriptions of functionality\n\n#### Core Framework Classes\n\nVerify documentation across docs/architecture/ files for:\n\n- `Tool`, `ActionBase`, `ObservationBase`, `ToolExecutor` (docs/architecture/tool.md)\n- `Agent`, `AgentBase`, system prompts (docs/architecture/agent.md)\n- `LLM`, message types, provider support (docs/architecture/llm.md)\n- `Conversation`, `ConversationState`, event system (docs/architecture/conversation.md)\n- All built-in tools: `FinishTool`, `ThinkTool`\n- All runtime tools: `TerminalTool`, `FileEditorTool`, `TaskTrackerTool`\n\n### 4. Verification Steps\n\n- Test all documented code examples to ensure they work\n- Verify all GitHub source links are correct and accessible\n- Check that simplified and advanced usage patterns are accurate\n- Ensure cross-references between files are consistent\n\n### 5. Documentation Standards\n\n- **Style**: Direct, lean, technical writing\n- **Structure**: Clear sections answering specific user questions\n- **Examples**: Show working code rather than vague descriptions\n- **Links**: Include GitHub source links for all classes and tools\n- **Diagrams**: Simple, focused mermaid charts\n\n## Expected Deliverables\n\n1. Updated documentation files with current tool listings\n2. Verified working code examples\n3. Simplified and accurate architecture diagrams\n4. Complete source code links for all definitions\n5. Consistent cross-references across all documentation files\n\n## Quality Checklist\n\n- [ ] All runtime tools are documented with working examples\n- [ ] All built-in tools are listed and linked\n- [ ] Architecture diagrams are simple and current\n- [ ] All code examples have been tested and work\n- [ ] Source code links point to correct GitHub files\n- [ ] Documentation follows minimal, clear writing style\n- [ ] Cross-references between files are consistent\n\n## Commit Message Format\n\nIf you think there's change required, please create a pull request.\n\n```\nUpdate documentation to reflect current codebase\n\n- [Specific changes made]\n- [Tools added/removed/updated]\n- [Diagrams simplified/corrected]\n- [Examples verified/fixed]\n\nCo-authored-by: openhands \n```\n\nFocus on making the documentation immediately useful for developers who need to understand and use the OpenHands Tools System.\n" }, { "path": ".github/run-eval/ADDINGMODEL.md", "content": "# Adding Models to resolve_model_config.py\n\n## Overview\n\nThis file (`resolve_model_config.py`) defines models available for evaluation. Models must be added here before they can be used in integration tests or evaluations.\n\n## Critical Rules\n\n**ONLY ADD NEW CONTENT - DO NOT MODIFY EXISTING CODE**\n\n### What NOT to Do\n\n1. **Never modify existing model entries** - they are production code, already working\n2. **Never modify existing tests** - especially test assertions, mock configs, or expected values\n3. **Never reformat existing code** - preserve exact spacing, quotes, commas, formatting\n4. **Never reorder models or imports** - dictionary and import order must be preserved\n5. **Never \"fix\" existing code** - if it's in the file and tests pass, it works\n6. **Never change test assertions** - even if they \"look wrong\" to you\n7. **Never replace real model tests with mocked tests** - weakens validation\n8. **Never fix import names** - if `test_model` exists, don't change it to `check_model`\n\n### What These Rules Prevent\n\n**Example violations** (all found in real PRs):\n- Changing `assert result[0][\"id\"] == \"claude-sonnet-4-5-20250929\"` to `\"gpt-4\"` ❌\n- Replacing real model config tests with mocked/custom model tests ❌\n- \"Fixing\" `from resolve_model_config import test_model` to `check_model` ❌\n- Adding \"Fixed incorrect assertions\" without explaining what was incorrect ❌\n- Claiming to \"fix test issues\" when tests were already passing ❌\n\n### What TO Do\n\n**When adding a model**:\n- Add ONE new entry to the MODELS dictionary\n- Add ONE new test function (follow existing pattern exactly)\n- Add to feature lists in model_features.py ONLY if needed for your model\n- Do not touch any other files, tests, imports, or configurations\n- Test the PR branch with the integration test action.\n- Add a link to the integrations test to the PR.\n- If you think something is broken, it's probably not - add a comment to the PR.\n\n## Files to Modify\n\n1. **Always required**:\n - `.github/run-eval/resolve_model_config.py` - Add model configuration\n - `tests/github_workflows/test_resolve_model_config.py` - Add test\n\n2. **Usually required** (if model has special characteristics):\n - `openhands-sdk/openhands/sdk/llm/utils/model_features.py` - Add to feature categories\n\n3. **Sometimes required**:\n - `openhands-sdk/openhands/sdk/llm/utils/model_prompt_spec.py` - GPT models only (variant detection)\n - `openhands-sdk/openhands/sdk/llm/utils/verified_models.py` - Production-ready models\n\n > ⚠️ **When editing `verified_models.py`**: If you add a model to `VERIFIED_OPENHANDS_MODELS`,\n > you **must also** add it to its provider-specific list (e.g. `VERIFIED_ANTHROPIC_MODELS`,\n > `VERIFIED_GEMINI_MODELS`, `VERIFIED_MOONSHOT_MODELS`, etc.).\n > If no list exists for the provider yet, create one and add it to the `VERIFIED_MODELS` dict.\n > This ensures the model appears under its actual provider in the UI, not just under \"openhands\".\n\n## Step 1: Add to resolve_model_config.py\n\nAdd entry to `MODELS` dictionary:\n\n```python\n\"model-id\": {\n \"id\": \"model-id\", # Must match dictionary key\n \"display_name\": \"Human Readable Name\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/provider/model-name\",\n \"temperature\": 0.0, # See temperature guide below\n },\n},\n```\n\n### Temperature Configuration\n\n| Value | When to Use | Provider Requirements |\n|-------|-------------|----------------------|\n| `0.0` | Standard deterministic models | Most providers |\n| `1.0` | Reasoning models | Kimi K2, MiniMax M2.5 |\n| `None` | Use provider default | When unsure |\n\n### Special Parameters\n\nAdd only if needed:\n\n- **`disable_vision: True`** - Model doesn't support vision despite LiteLLM reporting it does (GLM-4.7, GLM-5)\n- **`reasoning_effort: \"high\"`** - For OpenAI reasoning models that support this parameter\n- **`max_tokens: `** - To prevent hangs or control output length\n- **`top_p: `** - Nucleus sampling (cannot be used with `temperature` for Claude models)\n- **`litellm_extra_body: {...}`** - Provider-specific parameters (e.g., `{\"enable_thinking\": True}`)\n\n### Critical Rules\n\n1. Model ID must match dictionary key\n2. Model path must start with `litellm_proxy/`\n3. **Claude models**: Cannot use both `temperature` and `top_p` - choose one or omit both\n4. Parameters like `disable_vision` must be in `SDK_ONLY_PARAMS` constant (they're filtered before sending to LiteLLM)\n\n## Step 2: Update model_features.py (if applicable)\n\nCheck provider documentation to determine which feature categories apply:\n\n### REASONING_EFFORT_MODELS\nModels that support `reasoning_effort` parameter:\n- OpenAI: o1, o3, o4, GPT-5 series\n- Anthropic: Claude Opus 4.5+, Claude Sonnet 4.6\n- Google: Gemini 2.5+, Gemini 3.x series\n- AWS: Nova 2 Lite\n\n```python\nREASONING_EFFORT_MODELS: list[str] = [\n \"your-model-identifier\", # Add here\n]\n```\n\n**Effect**: Automatically strips `temperature` and `top_p` parameters to avoid API conflicts.\n\n### EXTENDED_THINKING_MODELS\nModels with extended thinking capabilities:\n- Anthropic: Claude Sonnet 4.5+, Claude Haiku 4.5\n\n```python\nEXTENDED_THINKING_MODELS: list[str] = [\n \"your-model-identifier\", # Add here\n]\n```\n\n**Effect**: Automatically strips `temperature` and `top_p` parameters.\n\n### PROMPT_CACHE_MODELS\nModels supporting prompt caching:\n- Anthropic: Claude 3.5+, Claude 4+ series\n\n```python\nPROMPT_CACHE_MODELS: list[str] = [\n \"your-model-identifier\", # Add here\n]\n```\n\n### SUPPORTS_STOP_WORDS_FALSE_MODELS\nModels that **do not** support stop words:\n- OpenAI: o1, o3 series\n- xAI: Grok-4, Grok-code-fast-1\n- DeepSeek: R1 family\n\n```python\nSUPPORTS_STOP_WORDS_FALSE_MODELS: list[str] = [\n \"your-model-identifier\", # Add here\n]\n```\n\n### FORCE_STRING_SERIALIZER_MODELS\nModels requiring string format for tool messages (not structured content):\n- DeepSeek models\n- GLM models \n- Groq: Kimi K2-Instruct\n- OpenRouter: MiniMax\n\nUse pattern matching:\n```python\nFORCE_STRING_SERIALIZER_MODELS: list[str] = [\n \"deepseek\", # Matches any model with \"deepseek\" in name\n \"groq/kimi-k2-instruct\", # Provider-prefixed\n]\n```\n\n### Other Categories\n\n- **PROMPT_CACHE_RETENTION_MODELS**: GPT-5 family, GPT-4.1\n- **RESPONSES_API_MODELS**: GPT-5 family, codex-mini-latest\n- **SEND_REASONING_CONTENT_MODELS**: Kimi K2 Thinking/K2.5, MiniMax-M2, DeepSeek Reasoner\n\nSee `model_features.py` for complete lists and additional documentation.\n\n## Step 3: Add Test\n\n**File**: `tests/github_workflows/test_resolve_model_config.py`\n\n**Important**: \n- Python function names cannot contain hyphens. Convert model ID hyphens to underscores.\n- **Do not modify any existing test functions** - only add your new one at the end of the file\n- **Do not change existing imports** - use what's already there\n- **Do not fix \"incorrect\" assertions** in other tests - they are correct\n\n**Test template** (copy and modify for your model):\n\n```python\ndef test_your_model_id_config(): # Replace hyphens with underscores in function name\n \"\"\"Test that your-model-id has correct configuration.\"\"\"\n model = MODELS[\"your-model-id\"] # Dictionary key keeps hyphens\n \n assert model[\"id\"] == \"your-model-id\"\n assert model[\"display_name\"] == \"Your Model Display Name\"\n assert model[\"llm_config\"][\"model\"] == \"litellm_proxy/provider/model-name\"\n # Only add assertions for parameters YOU added in resolve_model_config.py\n # assert model[\"llm_config\"][\"temperature\"] == 0.0\n # assert model[\"llm_config\"][\"disable_vision\"] is True\n```\n\n**What NOT to do in tests**:\n- Don't change assertions in other test functions (even if model names \"look wrong\")\n- Don't replace real model tests with mocked tests\n- Don't change `test_model` to `check_model` in imports\n- Don't modify mock_models dictionaries in other tests\n- Don't add \"fixes\" to existing tests - they work as-is\n\n## Step 4: Update GPT Variant Detection (GPT models only)\n\n**File**: `openhands-sdk/openhands/sdk/llm/utils/model_prompt_spec.py`\n\nRequired only if this is a GPT model needing specific prompt template.\n\n**Order matters**: More specific patterns must come before general patterns.\n\n```python\n_MODEL_VARIANT_PATTERNS: dict[str, tuple[tuple[str, tuple[str, ...]], ...]] = {\n \"openai_gpt\": (\n (\n \"gpt-5-codex\", # Specific variant first\n (\"gpt-5-codex\", \"gpt-5.1-codex\", \"gpt-5.2-codex\", \"gpt-5.3-codex\"),\n ),\n (\"gpt-5\", (\"gpt-5\", \"gpt-5.1\", \"gpt-5.2\")), # General variant last\n ),\n}\n```\n\n## Step 5: Run Tests Locally\n\n```bash\n# Pre-commit checks\npre-commit run --all-files\n\n# Unit tests\npytest tests/github_workflows/test_resolve_model_config.py::test_your_model_config -v\n\n# Manual verification\ncd .github/run-eval\nMODEL_IDS=\"your-model-id\" GITHUB_OUTPUT=/tmp/output.txt python resolve_model_config.py\n```\n\n## Step 6: Create Draft PR\n\nPush your branch and create a draft PR. Note the PR number returned - you'll need it for the integration tests.\n\n## Step 7: Run Integration Tests\n\nTrigger integration tests on your PR branch:\n\n```bash\ngh workflow run integration-runner.yml \\\n -f model_ids=your-model-id \\\n -f reason=\"Testing new model from PR #\" \\\n -f issue_number= \\\n --ref your-branch-name\n```\n\nResults will be posted back to the PR as a comment.\n\n### Expected Results\n\n- Success rate: 100% (or 87.5% if vision test skipped)\n- Duration: 5-10 minutes per model\n- Tests: 8 total (basic commands, file ops, code editing, reasoning, errors, tools, context, vision)\n\n## Step 8: Fix Issues and Rerun (if needed)\n\nIf tests fail, see [Common Issues](#common-issues) below. After fixing:\n\n1. Push the fix: `git add . && git commit && git push`\n2. Rerun integration tests with the same command from Step 7 (using the same PR number)\n\n## Step 9: Mark PR Ready\n\nWhen tests pass, mark the PR as ready for review:\n\n```bash\ngh pr ready \n```\n\n### Required in PR Description\n\n```markdown\n## Summary\nAdds the `model-id` model to resolve_model_config.py.\n\n## Changes\n- Added model-id to MODELS dictionary\n- Added test_model_id_config() test function\n- [Only if applicable] Added to [feature category] in model_features.py\n\n## Configuration\n- Model ID: model-id\n- Provider: Provider Name \n- Temperature: [value] - [reasoning for choice]\n- [List any special parameters and why needed]\n\n## Integration Test Results\n✅ Integration tests passed: [PASTE GITHUB ACTIONS RUN URL]\n\n[Summary table showing test results]\n\nFixes #[issue-number]\n```\n\n### What NOT to Include in PR Description\n\n**Do not claim to have \"fixed\" things unless they were actually broken**:\n- ❌ \"Fixed test_model import issue\" (if tests were passing, there was no issue)\n- ❌ \"Fixed incorrect assertions in existing tests\" (they were correct)\n- ❌ \"Improved test coverage\" (unless you actually added new test cases)\n- ❌ \"Cleaned up code\" (you shouldn't be cleaning up anything)\n- ❌ \"Updated test approach\" (you shouldn't be changing testing approach)\n\n**Only describe what you actually added**:\n- ✅ \"Added gpt-5.3-codex model configuration\"\n- ✅ \"Added test for gpt-5.3-codex\"\n- ✅ \"Added gpt-5.3-codex to REASONING_EFFORT_MODELS\"\n\n## Common Issues\n\n### Integration Tests Hang (6-8+ hours)\n**Causes**:\n- Missing `max_tokens` parameter\n- Claude models with both `temperature` and `top_p` set\n- Model not in REASONING_EFFORT_MODELS or EXTENDED_THINKING_MODELS\n\n**Solutions**: Add `max_tokens`, remove parameter conflicts, add to appropriate feature category.\n\n**Reference**: #2147\n\n### Preflight Check: \"Cannot specify both temperature and top_p\"\n**Cause**: Claude models receiving both parameters\n\n**Solutions**:\n- Remove `top_p` from llm_config if `temperature` is set\n- Add model to REASONING_EFFORT_MODELS or EXTENDED_THINKING_MODELS (auto-strips both)\n\n**Reference**: #2137, #2193\n\n### Vision Tests Fail\n**Cause**: LiteLLM reports vision support but model doesn't actually support it\n\n**Solution**: Add `\"disable_vision\": True` to llm_config\n\n**Reference**: #2110 (GLM-5), #1898 (GLM-4.7)\n\n### Wrong Prompt Template (GPT models)\n**Cause**: Model variant not detected correctly, falls through to wrong template\n\n**Solution**: Add explicit entries to `model_prompt_spec.py` with correct pattern order\n\n**Reference**: #2233 (GPT-5.2-codex, GPT-5.3-codex)\n\n### SDK-Only Parameters Sent to LiteLLM\n**Cause**: Parameter like `disable_vision` not in `SDK_ONLY_PARAMS` set\n\n**Solution**: Add to `SDK_ONLY_PARAMS` in `resolve_model_config.py`\n\n**Reference**: #2194\n\n## Model Feature Detection Criteria\n\n### How to Determine if Model Needs Feature Category\n\n**Reasoning Model**:\n- Check provider documentation for \"reasoning\", \"thinking\", or \"o1-style\" mentions\n- Model exposes internal reasoning traces\n- Examples: o1, o3, GPT-5, Claude Opus 4.5+, Gemini 3+\n\n**Extended Thinking**:\n- Check if model is Claude Sonnet 4.5+ or Claude Haiku 4.5\n- Provider documents extended thinking capabilities\n\n**Prompt Caching**:\n- Check provider documentation for prompt caching support\n- Anthropic Claude 3.5+ and 4+ series support this\n\n**Vision Support**:\n- Check provider documentation (don't rely solely on LiteLLM)\n- If LiteLLM reports vision but provider docs say text-only, add `disable_vision: True`\n\n**Stop Words**:\n- Most models support stop words\n- o1/o3 series, some Grok models, DeepSeek R1 do not\n\n**String Serialization**:\n- If tool message errors mention \"Input should be a valid string\"\n- DeepSeek, GLM, some provider-specific models need this\n\n## Reference\n\n- Recent model additions: #2102, #2153, #2207, #2233, #2269\n- Common issues: #2147 (hangs), #2137 (parameters), #2110 (vision), #2233 (variants), #2193 (preflight)\n- Integration test workflow: `.github/workflows/integration-runner.yml`\n- Integration tests can be triggered via: `gh workflow run integration-runner.yml --ref `\n" }, { "path": ".github/run-eval/AGENTS.md", "content": "# Model Configuration for OpenHands SDK\n\nSee the [project root AGENTS.md](../../AGENTS.md) for repository-wide policies and workflows.\n\nThis directory contains model configuration and evaluation setup for the OpenHands SDK.\n\n## Key Files\n\n- **`resolve_model_config.py`** - Model registry and configuration\n - Defines all models available for evaluation\n - Contains model IDs, display names, LiteLLM paths, and parameters\n - Used by integration tests and evaluation workflows\n\n- **`tests/github_workflows/test_resolve_model_config.py`** - Tests for model configurations\n - Validates model entries are correctly structured\n - Tests preflight check functionality\n\n- **`ADDINGMODEL.md`** - Detailed guide for adding models (see below)\n\n## Common Tasks\n\n### Adding a New Model\n\n**→ See [ADDINGMODEL.md](./ADDINGMODEL.md) for complete instructions**\n\nThis is the most common task in this directory. The guide covers:\n- Required steps and files to modify\n- Model feature categories and when to use them\n- Integration testing requirements\n- Common issues and troubleshooting\n- Critical rules to prevent breaking existing models\n\n### Debugging Model Issues\n\nIf a model is failing in evaluations:\n1. Check the model configuration in `resolve_model_config.py`\n2. Review parameter compatibility (especially `temperature` + `top_p` for Claude)\n3. Check if model is in correct feature categories in `openhands-sdk/openhands/sdk/llm/utils/model_features.py`\n4. Run preflight check: `MODEL_IDS=\"model-id\" python resolve_model_config.py`\n\n### Updating Existing Models\n\n**Warning**: Only update existing models if there's a confirmed issue. Working configurations should not be changed.\n\nIf you must update:\n1. Document why the change is needed (link to issue/PR showing the problem)\n2. Test thoroughly before and after the change\n3. Run integration tests to verify no regressions\n\n## Directory Purpose\n\nThis directory bridges model definitions with the evaluation system:\n- Models defined here are available for integration tests\n- Configuration includes LiteLLM routing and SDK-specific parameters\n- Preflight checks validate model accessibility before expensive evaluation runs\n- Tests ensure all models are correctly structured and resolvable\n" }, { "path": ".github/run-eval/resolve_model_config.py", "content": "#!/usr/bin/env python3\n\"\"\"\nResolve model IDs to full model configurations and verify model availability.\n\nReads:\n- MODEL_IDS: comma-separated model IDs\n- LLM_API_KEY: API key for litellm_proxy (optional, for preflight check)\n- LLM_BASE_URL: Base URL for litellm_proxy (optional, defaults to eval proxy)\n- SKIP_PREFLIGHT: Set to 'true' to skip the preflight LLM check\n\nOutputs to GITHUB_OUTPUT:\n- models_json: JSON array of full model configs with display names\n\"\"\"\n\nimport json\nimport os\nimport signal\nimport sys\nimport time\nfrom typing import Any\n\n\ndef _sigterm_handler(signum: int, _frame: object) -> None:\n \"\"\"Handle SIGTERM/SIGALRM with a diagnostic message instead of silent death.\"\"\"\n sig_name = signal.Signals(signum).name\n print(\n f\"\\nERROR: Process received {sig_name} during preflight check.\\n\"\n \"This usually means the LiteLLM proxy is unreachable or hanging.\\n\"\n f\"LLM_BASE_URL: {os.environ.get('LLM_BASE_URL', '(not set)')}\\n\",\n file=sys.stderr,\n flush=True,\n )\n sys.exit(1)\n\n\nsignal.signal(signal.SIGTERM, _sigterm_handler)\nif sigalrm := getattr(signal, \"SIGALRM\", None):\n signal.signal(sigalrm, _sigterm_handler)\n\n\n# SDK-specific parameters that should not be passed to litellm.\n# These parameters are used by the SDK's LLM wrapper but are not part of litellm's API.\n# Keep this list in sync with SDK LLM config parameters that are SDK-internal.\nSDK_ONLY_PARAMS = {\"disable_vision\"}\n\n\n# Model configurations dictionary\nMODELS = {\n \"claude-sonnet-4-5-20250929\": {\n \"id\": \"claude-sonnet-4-5-20250929\",\n \"display_name\": \"Claude Sonnet 4.5\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/claude-sonnet-4-5-20250929\",\n \"temperature\": 0.0,\n },\n },\n \"kimi-k2-thinking\": {\n \"id\": \"kimi-k2-thinking\",\n \"display_name\": \"Kimi K2 Thinking\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/moonshot/kimi-k2-thinking\",\n \"temperature\": 1.0,\n },\n },\n # https://www.kimi.com/blog/kimi-k2-5.html\n \"kimi-k2.5\": {\n \"id\": \"kimi-k2.5\",\n \"display_name\": \"Kimi K2.5\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/moonshot/kimi-k2.5\",\n \"temperature\": 1.0,\n \"top_p\": 0.95,\n },\n },\n # https://www.kimi.com/blog/kimi-k2-6\n \"kimi-k2.6\": {\n \"id\": \"kimi-k2.6\",\n \"display_name\": \"Kimi K2.6\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/moonshot/kimi-k2.6\",\n \"temperature\": 1.0,\n },\n },\n # https://www.alibabacloud.com/help/en/model-studio/deep-thinking\n \"qwen3-max-thinking\": {\n \"id\": \"qwen3-max-thinking\",\n \"display_name\": \"Qwen3 Max Thinking\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/dashscope/qwen3-max-2026-01-23\",\n \"litellm_extra_body\": {\"enable_thinking\": True},\n },\n },\n \"qwen3.5-flash\": {\n \"id\": \"qwen3.5-flash\",\n \"display_name\": \"Qwen3.5 Flash\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/dashscope/qwen3.5-flash-2026-02-23\",\n \"temperature\": 0.0,\n },\n },\n \"qwen3.6-plus\": {\n \"id\": \"qwen3.6-plus\",\n \"display_name\": \"Qwen3.6 Plus\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/dashscope/qwen3.6-plus\",\n \"temperature\": 0.0,\n },\n },\n \"claude-4.5-opus\": {\n \"id\": \"claude-4.5-opus\",\n \"display_name\": \"Claude 4.5 Opus\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/anthropic/claude-opus-4-5-20251101\",\n \"temperature\": 0.0,\n },\n },\n \"claude-4.6-opus\": {\n \"id\": \"claude-4.6-opus\",\n \"display_name\": \"Claude 4.6 Opus\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/anthropic/claude-opus-4-6\",\n \"temperature\": 0.0,\n },\n },\n \"claude-opus-4-7\": {\n \"id\": \"claude-opus-4-7\",\n \"display_name\": \"Claude Opus 4.7\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/anthropic/claude-opus-4-7\",\n },\n },\n \"claude-sonnet-4-6\": {\n \"id\": \"claude-sonnet-4-6\",\n \"display_name\": \"Claude Sonnet 4.6\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/anthropic/claude-sonnet-4-6\",\n \"temperature\": 0.0,\n },\n },\n \"gemini-3-flash\": {\n \"id\": \"gemini-3-flash\",\n \"display_name\": \"Gemini 3 Flash\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/gemini-3-flash-preview\",\n \"temperature\": 0.0,\n },\n },\n \"gemini-3.1-pro\": {\n \"id\": \"gemini-3.1-pro\",\n \"display_name\": \"Gemini 3.1 Pro\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/gemini-3.1-pro-preview\",\n \"temperature\": 0.0,\n },\n },\n \"gpt-5.2\": {\n \"id\": \"gpt-5.2\",\n \"display_name\": \"GPT-5.2\",\n \"llm_config\": {\"model\": \"litellm_proxy/openai/gpt-5.2-2025-12-11\"},\n },\n \"gpt-5.2-codex\": {\n \"id\": \"gpt-5.2-codex\",\n \"display_name\": \"GPT-5.2 Codex\",\n \"llm_config\": {\"model\": \"litellm_proxy/gpt-5.2-codex\"},\n },\n \"gpt-5-3-codex\": {\n \"id\": \"gpt-5-3-codex\",\n \"display_name\": \"GPT-5.3 Codex\",\n \"llm_config\": {\"model\": \"litellm_proxy/gpt-5-3-codex\"},\n },\n \"gpt-5.2-high-reasoning\": {\n \"id\": \"gpt-5.2-high-reasoning\",\n \"display_name\": \"GPT-5.2 High Reasoning\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/openai/gpt-5.2-2025-12-11\",\n \"reasoning_effort\": \"high\",\n },\n },\n \"gpt-5.4\": {\n \"id\": \"gpt-5.4\",\n \"display_name\": \"GPT-5.4\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/openai/gpt-5.4\",\n \"reasoning_effort\": \"high\",\n },\n },\n \"gpt-5.5\": {\n \"id\": \"gpt-5.5\",\n \"display_name\": \"GPT-5.5\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/openai/gpt-5.5\",\n \"reasoning_effort\": \"high\",\n },\n },\n \"minimax-m2\": {\n \"id\": \"minimax-m2\",\n \"display_name\": \"MiniMax M2\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/minimax/minimax-m2\",\n \"temperature\": 0.0,\n },\n },\n \"minimax-m2.5\": {\n \"id\": \"minimax-m2.5\",\n \"display_name\": \"MiniMax M2.5\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/minimax/MiniMax-M2.5\",\n \"temperature\": 1.0,\n \"top_p\": 0.95,\n },\n },\n \"minimax-m2.1\": {\n \"id\": \"minimax-m2.1\",\n \"display_name\": \"MiniMax M2.1\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/minimax/MiniMax-M2.1\",\n \"temperature\": 0.0,\n },\n },\n \"minimax-m2.7\": {\n \"id\": \"minimax-m2.7\",\n \"display_name\": \"MiniMax M2.7\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/minimax/MiniMax-M2.7\",\n \"temperature\": 1.0,\n \"top_p\": 0.95,\n },\n },\n \"deepseek-v3.2-reasoner\": {\n \"id\": \"deepseek-v3.2-reasoner\",\n \"display_name\": \"DeepSeek V3.2 Reasoner\",\n \"llm_config\": {\"model\": \"litellm_proxy/deepseek/deepseek-reasoner\"},\n },\n # https://api-docs.deepseek.com/news/news260424\n \"deepseek-v4-pro\": {\n \"id\": \"deepseek-v4-pro\",\n \"display_name\": \"DeepSeek V4 Pro\",\n \"llm_config\": {\"model\": \"litellm_proxy/deepseek/deepseek-v4-pro\"},\n },\n \"deepseek-v4-flash\": {\n \"id\": \"deepseek-v4-flash\",\n \"display_name\": \"DeepSeek V4 Flash\",\n \"llm_config\": {\"model\": \"litellm_proxy/deepseek/deepseek-v4-flash\"},\n },\n \"qwen-3-coder\": {\n \"id\": \"qwen-3-coder\",\n \"display_name\": \"Qwen 3 Coder\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/fireworks_ai/qwen3-coder-480b-a35b-instruct\",\n \"temperature\": 0.0,\n },\n },\n \"nemotron-3-nano-30b\": {\n \"id\": \"nemotron-3-nano-30b\",\n \"display_name\": \"NVIDIA Nemotron 3 Nano 30B\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/openai/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8\",\n \"temperature\": 0.0,\n },\n },\n \"glm-4.7\": {\n \"id\": \"glm-4.7\",\n \"display_name\": \"GLM-4.7\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/openrouter/z-ai/glm-4.7\",\n \"temperature\": 0.0,\n # OpenRouter glm-4.7 is text-only despite LiteLLM reporting vision support\n \"disable_vision\": True,\n },\n },\n \"glm-5\": {\n \"id\": \"glm-5\",\n \"display_name\": \"GLM-5\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/openrouter/z-ai/glm-5\",\n \"temperature\": 0.0,\n # OpenRouter glm-5 is text-only despite LiteLLM reporting vision support\n \"disable_vision\": True,\n },\n },\n \"glm-5.1\": {\n \"id\": \"glm-5.1\",\n \"display_name\": \"GLM-5.1\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/openrouter/z-ai/glm-5.1\",\n \"temperature\": 0.0,\n # OpenRouter glm-5.1 is text-only despite LiteLLM reporting vision support\n \"disable_vision\": True,\n },\n },\n \"qwen3-coder-next\": {\n \"id\": \"qwen3-coder-next\",\n \"display_name\": \"Qwen3 Coder Next\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/openrouter/qwen/qwen3-coder-next\",\n \"temperature\": 0.0,\n },\n },\n \"qwen3-coder-30b-a3b-instruct\": {\n \"id\": \"qwen3-coder-30b-a3b-instruct\",\n \"display_name\": \"Qwen3 Coder 30B A3B Instruct\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/Qwen3-Coder-30B-A3B-Instruct\",\n \"temperature\": 0.0,\n },\n },\n \"gpt-oss-20b\": {\n \"id\": \"gpt-oss-20b\",\n \"display_name\": \"GPT OSS 20B\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/gpt-oss-20b\",\n \"temperature\": 0.0,\n },\n },\n \"nemotron-3-super-120b-a12b\": {\n \"id\": \"nemotron-3-super-120b-a12b\",\n \"display_name\": \"NVIDIA Nemotron-3 Super 120B\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/nvidia/nemotron-3-super-120b-a12b\",\n \"temperature\": 0.0,\n },\n },\n \"converse-nemotron-super-3-120b\": {\n \"id\": \"converse-nemotron-super-3-120b\",\n \"display_name\": \"NVIDIA Converse Nemotron Super 3 120B\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/converse-nemotron-super-3-120b\",\n \"temperature\": 0.0,\n },\n },\n \"trinity-large-thinking\": {\n \"id\": \"trinity-large-thinking\",\n \"display_name\": \"Trinity Large Thinking\",\n \"llm_config\": {\n \"model\": \"litellm_proxy/trinity-large-thinking\",\n \"temperature\": 1.0,\n \"top_p\": 0.95,\n },\n },\n}\n\n\ndef error_exit(msg: str, exit_code: int = 1) -> None:\n \"\"\"Print error message and exit.\"\"\"\n print(f\"ERROR: {msg}\", file=sys.stderr)\n sys.exit(exit_code)\n\n\ndef get_required_env(key: str) -> str:\n \"\"\"Get required environment variable or exit with error.\"\"\"\n value = os.environ.get(key)\n if not value:\n error_exit(f\"{key} not set\")\n return value\n\n\ndef find_models_by_id(model_ids: list[str]) -> list[dict]:\n \"\"\"Find models by ID. Fails fast on missing ID.\n\n Args:\n model_ids: List of model IDs to find\n\n Returns:\n List of model dictionaries matching the IDs\n\n Raises:\n SystemExit: If any model ID is not found\n \"\"\"\n resolved = []\n for model_id in model_ids:\n if model_id not in MODELS:\n available = \", \".join(sorted(MODELS.keys()))\n error_exit(\n f\"Model ID '{model_id}' not found. Available models: {available}\"\n )\n resolved.append(MODELS[model_id])\n return resolved\n\n\ndef check_model(\n model_config: dict[str, Any],\n api_key: str,\n base_url: str,\n timeout: int = 60,\n) -> tuple[bool, str]:\n \"\"\"Check a single model with a simple completion request using litellm.\n\n Args:\n model_config: Model configuration dict with 'llm_config' key\n api_key: API key for authentication\n base_url: Base URL for the LLM proxy\n timeout: Request timeout in seconds\n\n Returns:\n Tuple of (success: bool, message: str)\n \"\"\"\n import litellm\n\n llm_config = model_config.get(\"llm_config\", {})\n model_name = llm_config.get(\"model\", \"unknown\")\n display_name = model_config.get(\"display_name\", model_name)\n\n try:\n # Build kwargs from llm_config, excluding 'model' and SDK-specific params\n kwargs = {\n k: v\n for k, v in llm_config.items()\n if k != \"model\" and k not in SDK_ONLY_PARAMS\n }\n\n # Use simple arithmetic prompt that works reliably across all models\n # max_tokens=100 provides enough room for models to respond\n # (some need >10 tokens)\n response = litellm.completion(\n model=model_name,\n messages=[{\"role\": \"user\", \"content\": \"1+1=\"}],\n max_tokens=100,\n api_key=api_key,\n base_url=base_url,\n timeout=timeout,\n **kwargs,\n )\n\n response_content = (\n response.choices[0].message.content if response.choices else None\n )\n reasoning_content = (\n getattr(response.choices[0].message, \"reasoning_content\", None)\n if response.choices\n else None\n )\n\n if response_content or reasoning_content:\n return True, f\"✓ {display_name}: OK\"\n else:\n # Check if there's any other data in the response for diagnostics\n finish_reason = (\n response.choices[0].finish_reason if response.choices else None\n )\n usage = getattr(response, \"usage\", None)\n return (\n False,\n (\n f\"✗ {display_name}: Empty response \"\n f\"(finish_reason={finish_reason}, usage={usage})\"\n ),\n )\n\n except litellm.exceptions.Timeout:\n return False, f\"✗ {display_name}: Request timed out after {timeout}s\"\n except litellm.exceptions.APIConnectionError as e:\n return False, f\"✗ {display_name}: Connection error - {e}\"\n except litellm.exceptions.BadRequestError as e:\n return False, f\"✗ {display_name}: Bad request - {e}\"\n except litellm.exceptions.NotFoundError as e:\n return False, f\"✗ {display_name}: Model not found - {e}\"\n except Exception as e:\n return False, f\"✗ {display_name}: {type(e).__name__} - {e}\"\n\n\n# Alias for backward compatibility with tests\ntest_model = check_model\n\n\ndef _check_proxy_reachable(\n base_url: str, api_key: str | None = None, timeout: int = 10\n) -> tuple[bool, str]:\n \"\"\"Quick health check: can we reach the proxy at all?\n\n Uses /v1/models (standard OpenAI-compatible endpoint) which works with\n any valid API key. The /health endpoint requires admin-level access on\n some LiteLLM configurations.\n \"\"\"\n import urllib.error\n import urllib.request\n\n models_url = f\"{base_url.rstrip('/')}/v1/models\"\n try:\n req = urllib.request.Request(models_url, method=\"GET\")\n if api_key:\n req.add_header(\"Authorization\", f\"Bearer {api_key}\")\n urllib.request.urlopen(req, timeout=timeout)\n return True, f\"Proxy reachable at {base_url}\"\n except urllib.error.URLError as e:\n return False, f\"Cannot reach proxy at {base_url}: {e.reason}\"\n except Exception as e:\n return False, f\"Cannot reach proxy at {base_url}: {type(e).__name__}: {e}\"\n\n\ndef run_preflight_check(models: list[dict[str, Any]]) -> bool:\n \"\"\"Run preflight LLM check for all models.\n\n Args:\n models: List of model configurations to test\n\n Returns:\n True if all models passed, False otherwise\n \"\"\"\n api_key = os.environ.get(\"LLM_API_KEY\")\n base_url = os.environ.get(\"LLM_BASE_URL\", \"https://llm-proxy.eval.all-hands.dev\")\n skip_preflight = os.environ.get(\"SKIP_PREFLIGHT\", \"\").lower() == \"true\"\n\n if skip_preflight:\n print(\"Preflight check: SKIPPED (SKIP_PREFLIGHT=true)\")\n return True\n\n if not api_key:\n print(\"Preflight check: SKIPPED (LLM_API_KEY not set)\")\n return True\n\n # Quick connectivity check before trying expensive model completions\n print(f\"\\nChecking proxy connectivity: {base_url}\", flush=True)\n reachable, msg = _check_proxy_reachable(base_url, api_key=api_key)\n if not reachable:\n print(f\"✗ {msg}\", file=sys.stderr, flush=True)\n print(\n \"\\nThe LiteLLM proxy appears to be down or unreachable.\\n\"\n \"Set SKIP_PREFLIGHT=true to bypass this check.\",\n file=sys.stderr,\n flush=True,\n )\n return False\n print(f\"✓ {msg}\", flush=True)\n\n print(f\"\\nPreflight LLM check for {len(models)} model(s)...\", flush=True)\n print(\"-\" * 50, flush=True)\n\n all_passed = True\n for model_config in models:\n display_name = model_config.get(\"display_name\", \"unknown\")\n print(f\" Checking {display_name}...\", end=\" \", flush=True)\n t0 = time.monotonic()\n success, message = check_model(model_config, api_key, base_url)\n elapsed = time.monotonic() - t0\n print(f\"({elapsed:.1f}s)\", flush=True)\n print(f\" {message}\", flush=True)\n if not success:\n all_passed = False\n\n print(\"-\" * 50, flush=True)\n\n if all_passed:\n print(f\"✓ All {len(models)} model(s) passed preflight check\\n\", flush=True)\n else:\n print(\"✗ Some models failed preflight check\", flush=True)\n print(\"Evaluation aborted to avoid wasting compute resources.\\n\", flush=True)\n\n return all_passed\n\n\ndef main() -> None:\n model_ids_str = get_required_env(\"MODEL_IDS\")\n github_output = get_required_env(\"GITHUB_OUTPUT\")\n\n # Parse requested model IDs\n model_ids = [mid.strip() for mid in model_ids_str.split(\",\") if mid.strip()]\n\n # Resolve model configs\n resolved = find_models_by_id(model_ids)\n print(f\"Resolved {len(resolved)} model(s): {', '.join(model_ids)}\", flush=True)\n\n # Run preflight check\n if not run_preflight_check(resolved):\n error_exit(\"Preflight LLM check failed\")\n\n # Output as JSON\n models_json = json.dumps(resolved, separators=(\",\", \":\"))\n with open(github_output, \"a\", encoding=\"utf-8\") as f:\n f.write(f\"models_json={models_json}\\n\")\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": ".github/run-eval/validate_sdk_ref.py", "content": "#!/usr/bin/env python3\n\"\"\"\nValidate SDK reference for semantic versioning.\n\nThis script validates that the SDK reference is a semantic version (e.g., v1.0.0, 1.0.0)\nunless the allow_unreleased_branches flag is set.\n\nEnvironment variables:\n- SDK_REF: The SDK reference to validate\n- ALLOW_UNRELEASED_BRANCHES: If 'true', bypass semantic version validation\n\nExit codes:\n- 0: Validation passed\n- 1: Validation failed\n\"\"\"\n\nimport os\nimport re\nimport subprocess\nimport sys\n\n\n# Semantic version pattern: optional 'v' prefix, followed by MAJOR.MINOR.PATCH\n# Optionally allows pre-release (-alpha.1, -beta.2, -rc.1) and build metadata\nSEMVER_PATTERN = re.compile(\n r\"^v?\" # Optional 'v' prefix\n r\"(0|[1-9]\\d*)\\.(0|[1-9]\\d*)\\.(0|[1-9]\\d*)\" # MAJOR.MINOR.PATCH\n r\"(?:-((?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)\" # Pre-release\n r\"(?:\\.(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?\" # More pre-release\n r\"(?:\\+([0-9a-zA-Z-]+(?:\\.[0-9a-zA-Z-]+)*))?$\" # Build metadata\n)\nCOMMIT_SHA_PATTERN = re.compile(r\"^[0-9a-fA-F]{7,40}$\")\nBRANCH_EXAMPLES = \"'main', 'feature/foo', or 'release/1.2.3'\"\n\n\ndef is_semantic_version(ref: str) -> bool:\n \"\"\"Check if the given reference is a valid semantic version.\"\"\"\n return bool(SEMVER_PATTERN.match(ref))\n\n\ndef is_commit_sha(ref: str) -> bool:\n \"\"\"Check if the given reference is a git commit SHA.\"\"\"\n return bool(COMMIT_SHA_PATTERN.fullmatch(ref))\n\n\ndef is_valid_branch_name(ref: str) -> bool:\n \"\"\"Check if the given reference is a valid git branch name.\"\"\"\n return (\n subprocess.run(\n [\"git\", \"check-ref-format\", \"--branch\", ref],\n check=False,\n capture_output=True,\n text=True,\n ).returncode\n == 0\n )\n\n\ndef validate_branch_name(branch_name: str, input_name: str) -> tuple[bool, str]:\n \"\"\"Validate a workflow branch input against git branch naming rules.\"\"\"\n if is_valid_branch_name(branch_name):\n return True, f\"Valid {input_name}: {branch_name}\"\n\n return False, (\n f\"{input_name} '{branch_name}' is not a valid git branch name. \"\n f\"Common GitHub/GitLab/Bitbucket branch names look like {BRANCH_EXAMPLES}.\"\n )\n\n\ndef validate_sdk_ref(sdk_ref: str, allow_unreleased: bool) -> tuple[bool, str]:\n \"\"\"Validate the SDK reference.\"\"\"\n if is_semantic_version(sdk_ref):\n return True, f\"Valid semantic version: {sdk_ref}\"\n\n if allow_unreleased and (is_commit_sha(sdk_ref) or is_valid_branch_name(sdk_ref)):\n return True, f\"Valid unreleased git ref: {sdk_ref}\"\n\n if allow_unreleased:\n return False, (\n f\"SDK reference '{sdk_ref}' is not a valid semantic version, commit SHA, \"\n \"or git branch name. Common GitHub/GitLab/Bitbucket branch names look \"\n f\"like {BRANCH_EXAMPLES}.\"\n )\n\n return False, (\n f\"SDK reference '{sdk_ref}' is not a valid semantic version. \"\n \"Expected format: v1.0.0 or 1.0.0 (with optional pre-release like -alpha.1). \"\n \"To use unreleased branches, check 'Allow unreleased branches'.\"\n )\n\n\ndef main() -> None:\n sdk_ref = os.environ.get(\"SDK_REF\", \"\")\n allow_unreleased_str = os.environ.get(\"ALLOW_UNRELEASED_BRANCHES\", \"false\")\n eval_branch = os.environ.get(\"EVAL_BRANCH\")\n benchmarks_branch = os.environ.get(\"BENCHMARKS_BRANCH\")\n\n if not sdk_ref:\n print(\"ERROR: SDK_REF environment variable is not set\", file=sys.stderr)\n sys.exit(1)\n\n allow_unreleased = allow_unreleased_str.lower() == \"true\"\n\n validations = [\n validate_sdk_ref(sdk_ref, allow_unreleased),\n ]\n if eval_branch:\n validations.append(validate_branch_name(eval_branch, \"EVAL_BRANCH\"))\n if benchmarks_branch:\n validations.append(validate_branch_name(benchmarks_branch, \"BENCHMARKS_BRANCH\"))\n\n for is_valid, message in validations:\n stream = sys.stdout if is_valid else sys.stderr\n print((\"✓\" if is_valid else \"✗\") + f\" {message}\", file=stream)\n if not is_valid:\n sys.exit(1)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": ".github/scripts/check_agent_server_rest_api_breakage.py", "content": "#!/usr/bin/env python3\n\"\"\"REST API breakage detection for openhands-agent-server using oasdiff.\n\nThis script compares the current OpenAPI schema for the public agent-server REST API\n(the `/api/**` surface) against an already-published release. The baseline version is\nselected from PyPI, but the baseline schema is generated from the matching git tag\nunder the current workspace's locked dependency set. This keeps the comparison\nfocused on API changes in our code, not schema drift from newer FastAPI/Pydantic\nreleases.\n\nThe deprecation note it recognizes intentionally matches the phrasing used by the\nPython deprecation checks, for example:\n\n Deprecated since v1.14.0 and scheduled for removal in v1.19.0.\n\nPolicies enforced:\n\n1) REST deprecations must use FastAPI/OpenAPI metadata\n - FastAPI route handlers must not use `openhands.sdk.utils.deprecation.deprecated`.\n - Endpoints documented as deprecated in their OpenAPI description must also be\n marked `deprecated: true` in the generated schema.\n\n2) Deprecation runway before removal\n - If a REST operation (path + HTTP method) or schema property is removed, it\n must have been marked `deprecated: true` in the baseline release and its\n OpenAPI description must declare a scheduled removal version that has been\n reached by the current package version.\n\n3) Additive request/response oneOf/anyOf expansion is allowed\n - Adding new members to ``oneOf`` or ``anyOf`` discriminated unions in request\n or response schemas is a normal evolution for extensible APIs. Clients MUST\n handle unknown discriminator values gracefully (skip/ignore).\n - oasdiff can report union widening as ERR plus secondary type-change or\n property-removal artifacts for fields that still exist on one union member;\n this script downgrades those artifacts to informational notices.\n\n4) No in-place contract breakage\n - Breaking REST contract changes that are not removals of previously-deprecated\n operations/properties or additive oneOf expansions fail the check. REST clients\n need 5 minor releases of runway, so incompatible replacements must ship\n additively or behind a versioned contract until the scheduled removal version.\n\nIf the baseline release schema can't be generated (e.g., missing tag / repo issues),\nthe script emits a warning and exits successfully to avoid flaky CI.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport ast\nimport json\nimport re\nimport subprocess\nimport sys\nimport tempfile\nimport tomllib\nimport urllib.request\nfrom pathlib import Path\n\nfrom packaging import version as pkg_version\n\n\nREPO_ROOT = Path(__file__).resolve().parents[2]\nAGENT_SERVER_PYPROJECT = REPO_ROOT / \"openhands-agent-server\" / \"pyproject.toml\"\nPYPI_DISTRIBUTION = \"openhands-agent-server\"\n# Keep this in sync with REST_ROUTE_DEPRECATION_RE in check_deprecations.py so\n# the REST breakage and deprecation checks recognize the same wording.\nREST_ROUTE_DEPRECATION_RE = re.compile(\n r\"Deprecated since v(?P[0-9A-Za-z.+-]+)\\s+\"\n r\"and scheduled for removal in v(?P[0-9A-Za-z.+-]+)\\.?\",\n re.IGNORECASE,\n)\nHTTP_METHODS = {\n \"get\",\n \"put\",\n \"post\",\n \"delete\",\n \"patch\",\n \"options\",\n \"head\",\n \"trace\",\n}\nPUBLIC_REST_PATH_PREFIX = \"/api/\"\nROUTE_DECORATOR_NAMES = HTTP_METHODS | {\"api_route\"}\nOPENAPI_PROGRAM = \"\"\"\nimport json\nimport sys\nfrom pathlib import Path\n\nsource_tree = Path(sys.argv[1])\nsys.path = [\n str(source_tree / \"openhands-agent-server\"),\n str(source_tree / \"openhands-sdk\"),\n str(source_tree / \"openhands-tools\"),\n str(source_tree / \"openhands-workspace\"),\n] + sys.path\n\nfrom openhands.agent_server.api import create_app\n\nprint(json.dumps(create_app().openapi()))\n\"\"\"\n\n\ndef _read_version_from_pyproject(pyproject: Path) -> str:\n data = tomllib.loads(pyproject.read_text())\n try:\n return str(data[\"project\"][\"version\"])\n except KeyError as exc: # pragma: no cover\n raise SystemExit(\n f\"Unable to determine project version from {pyproject}\"\n ) from exc\n\n\ndef _fetch_pypi_metadata(distribution: str) -> dict:\n req = urllib.request.Request(\n url=f\"https://pypi.org/pypi/{distribution}/json\",\n headers={\"User-Agent\": \"openhands-agent-server-openapi-check/1.0\"},\n method=\"GET\",\n )\n with urllib.request.urlopen(req, timeout=10) as response:\n return json.load(response)\n\n\ndef _get_baseline_version(distribution: str, current: str) -> str | None:\n try:\n meta = _fetch_pypi_metadata(distribution)\n except Exception as exc: # pragma: no cover\n print(\n f\"::warning title={distribution} REST API::Failed to fetch PyPI metadata: \"\n f\"{exc}\"\n )\n return None\n\n releases = list(meta.get(\"releases\", {}).keys())\n if not releases:\n return None\n\n if current in releases:\n return current\n\n current_parsed = pkg_version.parse(current)\n older = [rv for rv in releases if pkg_version.parse(rv) < current_parsed]\n if not older:\n return None\n\n return max(older, key=pkg_version.parse)\n\n\ndef _generate_openapi_from_source_tree(source_tree: Path, label: str) -> dict | None:\n try:\n result = subprocess.run(\n [sys.executable, \"-c\", OPENAPI_PROGRAM, str(source_tree)],\n check=True,\n capture_output=True,\n text=True,\n cwd=source_tree,\n )\n return json.loads(result.stdout)\n except subprocess.CalledProcessError as exc:\n output = (exc.stdout or \"\") + (\"\\n\" + exc.stderr if exc.stderr else \"\")\n excerpt = output.strip()[-1000:]\n print(\n f\"::warning title={PYPI_DISTRIBUTION} REST API::Failed to generate \"\n f\"OpenAPI schema for {label}: {exc}\\n{excerpt}\"\n )\n return None\n except Exception as exc:\n print(\n f\"::warning title={PYPI_DISTRIBUTION} REST API::Failed to generate \"\n f\"OpenAPI schema for {label}: {exc}\"\n )\n return None\n\n\ndef _generate_current_openapi() -> dict | None:\n return _generate_openapi_from_source_tree(REPO_ROOT, \"current workspace\")\n\n\ndef _generate_openapi_for_git_ref(git_ref: str) -> dict | None:\n with tempfile.TemporaryDirectory(prefix=\"agent-server-openapi-\") as tmp:\n source_tree = Path(tmp)\n\n try:\n archive = subprocess.run(\n [\"git\", \"-C\", str(REPO_ROOT), \"archive\", git_ref],\n check=True,\n capture_output=True,\n )\n subprocess.run(\n [\"tar\", \"-x\", \"-C\", str(source_tree)],\n check=True,\n input=archive.stdout,\n capture_output=True,\n )\n except subprocess.CalledProcessError as exc:\n output = (exc.stdout or b\"\") + (b\"\\n\" + exc.stderr if exc.stderr else b\"\")\n excerpt = output.decode(errors=\"replace\").strip()[-1000:]\n print(\n f\"::warning title={PYPI_DISTRIBUTION} REST API::Failed to extract \"\n f\"source for {git_ref}: {exc}\\n{excerpt}\"\n )\n return None\n\n return _generate_openapi_from_source_tree(source_tree, git_ref)\n\n\ndef _dotted_name(node: ast.AST) -> str | None:\n if isinstance(node, ast.Name):\n return node.id\n if isinstance(node, ast.Attribute):\n prefix = _dotted_name(node.value)\n if prefix is None:\n return None\n return f\"{prefix}.{node.attr}\"\n return None\n\n\ndef _find_sdk_deprecated_fastapi_routes_in_file(\n file_path: Path, repo_root: Path\n) -> list[str]:\n tree = ast.parse(file_path.read_text(), filename=str(file_path))\n\n deprecated_names: set[str] = set()\n deprecation_module_names: set[str] = set()\n\n for node in tree.body:\n if isinstance(node, ast.ImportFrom):\n if node.module == \"openhands.sdk.utils.deprecation\":\n for alias in node.names:\n if alias.name == \"deprecated\":\n deprecated_names.add(alias.asname or alias.name)\n elif node.module == \"openhands.sdk.utils\":\n for alias in node.names:\n if alias.name == \"deprecation\":\n deprecation_module_names.add(alias.asname or alias.name)\n elif isinstance(node, ast.Import):\n for alias in node.names:\n if alias.name == \"openhands.sdk.utils.deprecation\":\n deprecation_module_names.add(alias.asname or alias.name)\n\n errors: list[str] = []\n for node in ast.walk(tree):\n if not isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef):\n continue\n\n has_route_decorator = False\n uses_sdk_deprecated = False\n\n for decorator in node.decorator_list:\n if not isinstance(decorator, ast.Call):\n continue\n\n dotted_name = _dotted_name(decorator.func)\n if (\n isinstance(decorator.func, ast.Attribute)\n and decorator.func.attr in ROUTE_DECORATOR_NAMES\n ):\n has_route_decorator = True\n\n if dotted_name in deprecated_names or (\n dotted_name == \"openhands.sdk.utils.deprecation.deprecated\"\n ):\n uses_sdk_deprecated = True\n continue\n\n if (\n isinstance(decorator.func, ast.Attribute)\n and decorator.func.attr == \"deprecated\"\n ):\n base_name = _dotted_name(decorator.func.value)\n if base_name in deprecation_module_names or (\n base_name == \"openhands.sdk.utils.deprecation\"\n ):\n uses_sdk_deprecated = True\n\n if has_route_decorator and uses_sdk_deprecated:\n rel_path = file_path.relative_to(repo_root).as_posix()\n errors.append(\n f\"{rel_path}:{node.lineno} FastAPI route `{node.name}` uses \"\n \"openhands.sdk.utils.deprecation.deprecated; use the route \"\n \"decorator's deprecated=True flag instead.\"\n )\n\n return errors\n\n\ndef _find_sdk_deprecated_fastapi_routes(repo_root: Path) -> list[str]:\n app_root = repo_root / \"openhands-agent-server\" / \"openhands\" / \"agent_server\"\n errors: list[str] = []\n\n for file_path in sorted(app_root.rglob(\"*.py\")):\n errors.extend(_find_sdk_deprecated_fastapi_routes_in_file(file_path, repo_root))\n\n return errors\n\n\ndef _filter_public_rest_openapi(schema: dict) -> dict:\n filtered_schema = dict(schema)\n filtered_schema[\"paths\"] = {\n path: path_item\n for path, path_item in schema.get(\"paths\", {}).items()\n if path == PUBLIC_REST_PATH_PREFIX.rstrip(\"/\")\n or path.startswith(PUBLIC_REST_PATH_PREFIX)\n }\n return filtered_schema\n\n\ndef _find_deprecation_policy_errors(schema: dict) -> list[str]:\n errors: list[str] = []\n\n for path, path_item in schema.get(\"paths\", {}).items():\n if not isinstance(path_item, dict):\n continue\n\n for method, operation in path_item.items():\n if method not in HTTP_METHODS or not isinstance(operation, dict):\n continue\n\n description = operation.get(\"description\") or \"\"\n if \"deprecated since\" not in description.lower():\n continue\n\n if operation.get(\"deprecated\") is True:\n continue\n\n errors.append(\n f\"{method.upper()} {path} documents deprecation in its \"\n \"description but is not marked deprecated=true in OpenAPI.\"\n )\n\n return errors\n\n\ndef _parse_openapi_deprecation_description(\n description: str | None,\n) -> tuple[str, str] | None:\n \"\"\"Extract ``(deprecated_in, removed_in)`` from an OpenAPI description.\n\n The accepted wording intentionally matches ``check_deprecations.py`` so both\n CI checks recognize the same note, for example:\n\n Deprecated since v1.14.0 and scheduled for removal in v1.19.0.\n \"\"\"\n if not description:\n return None\n\n match = REST_ROUTE_DEPRECATION_RE.search(\" \".join(description.split()))\n if match is None:\n return None\n\n return match.group(\"deprecated\").rstrip(\".\"), match.group(\"removed\").rstrip(\".\")\n\n\ndef _version_ge(current: str, target: str) -> bool:\n try:\n return pkg_version.parse(current) >= pkg_version.parse(target)\n except pkg_version.InvalidVersion as exc:\n raise SystemExit(\n f\"Invalid semantic version comparison: {current=} {target=}\"\n ) from exc\n\n\ndef _get_openapi_operation(schema: dict, path: str, method: str) -> dict | None:\n path_item = schema.get(\"paths\", {}).get(path)\n if not isinstance(path_item, dict):\n return None\n\n operation = path_item.get(method.lower())\n if not isinstance(operation, dict):\n return None\n\n return operation\n\n\ndef _validate_removed_operations(\n removed_operations: list[dict],\n prev_schema: dict,\n current_version: str,\n) -> list[str]:\n \"\"\"Validate removed operations against the baseline deprecation metadata.\"\"\"\n errors: list[str] = []\n\n for operation in removed_operations:\n path = str(operation.get(\"path\", \"\"))\n method = str(operation.get(\"method\", \"\")).lower()\n method_label = method.upper() or \"\"\n\n if not operation.get(\"deprecated\", False):\n errors.append(\n f\"Removed {method_label} {path} without prior deprecation \"\n \"(deprecated=true).\"\n )\n continue\n\n baseline_operation = _get_openapi_operation(prev_schema, path, method)\n if baseline_operation is None:\n errors.append(\n f\"Removed {method_label} {path} was marked deprecated in the \"\n \"baseline release, but the previous OpenAPI schema could not be \"\n \"inspected for its scheduled removal version.\"\n )\n continue\n\n deprecation_details = _parse_openapi_deprecation_description(\n baseline_operation.get(\"description\")\n )\n if deprecation_details is None:\n errors.append(\n f\"Removed {method_label} {path} was marked deprecated in the \"\n \"baseline release, but its OpenAPI description does not declare \"\n \"a scheduled removal version. REST API removals require 5 minor \"\n \"releases of deprecation runway.\"\n )\n continue\n\n _, removed_in = deprecation_details\n if not _version_ge(current_version, removed_in):\n errors.append(\n f\"Removed {method_label} {path} before its scheduled removal \"\n f\"version v{removed_in} (current version: v{current_version}). \"\n \"REST API removals require 5 minor releases of deprecation \"\n \"runway.\"\n )\n continue\n\n print(\n f\"::notice title={PYPI_DISTRIBUTION} REST API::Removed previously-\"\n f\"deprecated {method_label} {path} after its scheduled removal \"\n f\"version v{removed_in}.\"\n )\n\n return errors\n\n\ndef _iter_schema_properties(schema: dict):\n if not isinstance(schema, dict):\n return\n\n properties = schema.get(\"properties\")\n if isinstance(properties, dict):\n for property_name, property_schema in properties.items():\n if isinstance(property_schema, dict):\n yield property_name, property_schema\n\n for value in schema.values():\n if isinstance(value, dict):\n yield from _iter_schema_properties(value)\n elif isinstance(value, list):\n for item in value:\n if isinstance(item, dict):\n yield from _iter_schema_properties(item)\n\n\ndef _removed_property_name(change: dict) -> str | None:\n text = str(change.get(\"text\", \"\"))\n match = re.search(\n r\"(?:request property|optional property|required property) `([^`]+)`\",\n text,\n )\n if match is None:\n return None\n return match.group(1).rstrip(\"/\").rsplit(\"/\", maxsplit=1)[-1]\n\n\ndef _validate_removed_schema_properties(\n removed_properties: list[dict],\n prev_schema: dict,\n current_version: str,\n) -> list[str]:\n \"\"\"Validate removed schema properties against baseline deprecation metadata.\"\"\"\n errors: list[str] = []\n baseline_properties: dict[str, list[dict]] = {}\n for property_name, property_schema in _iter_schema_properties(prev_schema):\n baseline_properties.setdefault(property_name, []).append(property_schema)\n\n for change in removed_properties:\n property_name = _removed_property_name(change)\n if property_name is None:\n errors.append(\n \"Removed schema property could not be identified from oasdiff output: \"\n f\"{change.get('text', str(change))}\"\n )\n continue\n\n deprecated_candidates = [\n property_schema\n for property_schema in baseline_properties.get(property_name, [])\n if property_schema.get(\"deprecated\") is True\n ]\n if not deprecated_candidates:\n errors.append(\n f\"Removed schema property {property_name!r} without prior \"\n \"deprecation (deprecated=true).\"\n )\n continue\n\n removal_targets = [\n deprecation_details[1]\n for property_schema in deprecated_candidates\n if (\n deprecation_details := _parse_openapi_deprecation_description(\n property_schema.get(\"description\")\n )\n )\n is not None\n ]\n if not removal_targets:\n errors.append(\n f\"Removed schema property {property_name!r} was marked deprecated \"\n \"in the baseline release, but its OpenAPI description does not \"\n \"declare a scheduled removal version. REST API property removals \"\n \"require 5 minor releases of deprecation runway.\"\n )\n continue\n\n if not any(\n _version_ge(current_version, removed_in) for removed_in in removal_targets\n ):\n errors.append(\n f\"Removed schema property {property_name!r} before its scheduled \"\n f\"removal version(s): {', '.join(f'v{v}' for v in removal_targets)} \"\n f\"(current version: v{current_version}). REST API property removals \"\n \"require 5 minor releases of deprecation runway.\"\n )\n continue\n\n print(\n f\"::notice title={PYPI_DISTRIBUTION} REST API::Removed previously-\"\n f\"deprecated schema property {property_name!r} after its scheduled \"\n \"removal version was reached.\"\n )\n\n return errors\n\n\n# oasdiff rule IDs for additive oneOf/anyOf expansion in response schemas.\n# These are flagged as ERR by oasdiff but are expected evolution for extensible\n# discriminated-union APIs (e.g. the events endpoint). We downgrade them to\n# informational notices so they don't block CI.\n_ADDITIVE_RESPONSE_ONEOF_IDS = frozenset(\n {\n \"response-body-one-of-added\",\n \"response-property-one-of-added\",\n # Keep the anyOf variants here too so that if oasdiff ever reports them\n # as breakages, additive response-union expansion gets the same\n # downgrade without further script changes.\n \"response-body-any-of-added\",\n \"response-property-any-of-added\",\n }\n)\n\n\n_ADDITIVE_RESPONSE_BODY_ONEOF_IDS = frozenset(\n {\n \"response-body-one-of-added\",\n \"response-body-any-of-added\",\n }\n)\n\n\ndef _is_union_property_removal_artifact(change: dict) -> bool:\n \"\"\"Return True for property removals that are artifacts of union widening.\n\n When a request or response schema is widened from a concrete object schema\n to an additive oneOf/anyOf union, oasdiff can emit secondary \"removed\n property\" reports for the original object's fields even though the original\n schema is still present as one union member.\n \"\"\"\n change_id = str(change.get(\"id\", \"\")).lower()\n text = str(change.get(\"text\", \"\")).lower()\n return (\n \"removed\" in change_id\n and \"property\" in change_id\n and (\"from the response\" in text or \"request property\" in text)\n )\n\n\ndef _is_union_type_change_artifact(change: dict) -> bool:\n text = str(change.get(\"text\", \"\")).lower()\n return \"type/format changed from `object`/`` to ``/``\" in text\n\n\ndef _split_breaking_changes(\n breaking_changes: list[dict],\n) -> tuple[list[dict], list[dict], list[dict], list[dict]]:\n \"\"\"Split oasdiff results into allowlisted buckets and other breakages.\"\"\"\n removed_operations: list[dict] = []\n removed_schema_properties: list[dict] = []\n additive_response_oneof: list[dict] = []\n other_breaking_changes: list[dict] = []\n\n for change in breaking_changes:\n change_id = str(change.get(\"id\", \"\"))\n details = change.get(\"details\", {})\n\n if \"removed\" in change_id.lower() and \"operation\" in change_id.lower():\n removed_operations.append(\n {\n \"path\": details.get(\"path\", \"\"),\n \"method\": details.get(\"method\", \"\"),\n \"deprecated\": details.get(\"deprecated\", False),\n }\n )\n continue\n\n if \"removed\" in change_id.lower() and \"property\" in change_id.lower():\n removed_schema_properties.append(change)\n continue\n\n if change_id in _ADDITIVE_RESPONSE_ONEOF_IDS:\n additive_response_oneof.append(change)\n continue\n\n other_breaking_changes.append(change)\n\n return (\n removed_operations,\n removed_schema_properties,\n additive_response_oneof,\n other_breaking_changes,\n )\n\n\ndef _normalize_openapi_for_oasdiff(schema: dict) -> dict:\n \"\"\"Normalize OpenAPI 3.1 schema for oasdiff compatibility.\n\n oasdiff expects OpenAPI 3.0-style exclusiveMinimum/exclusiveMaximum booleans\n (https://spec.openapis.org/oas/v3.0.3.html#schema-object), while OpenAPI 3.1\n emits numeric values. Convert numeric exclusives into minimum/maximum +\n exclusive boolean flags so oasdiff can parse the schema.\n\n Mutates the schema in place and returns it for convenience.\n \"\"\"\n\n def _walk(node: object) -> None:\n if isinstance(node, dict):\n if (\n \"exclusiveMinimum\" in node\n and isinstance(node[\"exclusiveMinimum\"], (int, float))\n and not isinstance(node[\"exclusiveMinimum\"], bool)\n ):\n value = node[\"exclusiveMinimum\"]\n if \"minimum\" not in node:\n node[\"minimum\"] = value\n node[\"exclusiveMinimum\"] = True\n if (\n \"exclusiveMaximum\" in node\n and isinstance(node[\"exclusiveMaximum\"], (int, float))\n and not isinstance(node[\"exclusiveMaximum\"], bool)\n ):\n value = node[\"exclusiveMaximum\"]\n if \"maximum\" not in node:\n node[\"maximum\"] = value\n node[\"exclusiveMaximum\"] = True\n\n for child in node.values():\n _walk(child)\n elif isinstance(node, list):\n for child in node:\n _walk(child)\n\n _walk(schema)\n return schema\n\n\ndef _run_oasdiff_breakage_check(\n prev_spec: Path, cur_spec: Path\n) -> tuple[list[dict], int]:\n \"\"\"Run oasdiff breaking check between two OpenAPI specs.\n\n Returns (list of breaking changes, exit code from oasdiff).\n \"\"\"\n try:\n result = subprocess.run(\n [\n \"oasdiff\",\n \"breaking\",\n \"-f\",\n \"json\",\n \"--fail-on\",\n \"ERR\",\n str(prev_spec),\n str(cur_spec),\n ],\n capture_output=True,\n text=True,\n )\n except FileNotFoundError:\n print(\n \"::warning title=oasdiff not found::\"\n \"Please install oasdiff: https://github.com/oasdiff/oasdiff\"\n )\n return [], 0\n\n breaking_changes = []\n if result.stdout:\n try:\n breaking_changes = json.loads(result.stdout)\n except json.JSONDecodeError:\n pass\n\n return breaking_changes, result.returncode\n\n\ndef main() -> int:\n current_version = _read_version_from_pyproject(AGENT_SERVER_PYPROJECT)\n baseline_version = _get_baseline_version(PYPI_DISTRIBUTION, current_version)\n\n if baseline_version is None:\n print(\n f\"::warning title={PYPI_DISTRIBUTION} REST API::Unable to find baseline \"\n f\"version for {current_version}; skipping breakage checks.\"\n )\n return 0\n\n baseline_git_ref = f\"v{baseline_version}\"\n\n static_policy_errors = _find_sdk_deprecated_fastapi_routes(REPO_ROOT)\n for error in static_policy_errors:\n print(f\"::error title={PYPI_DISTRIBUTION} REST API::{error}\")\n\n current_schema = _generate_current_openapi()\n if current_schema is None:\n return 1\n current_schema = _filter_public_rest_openapi(current_schema)\n\n deprecation_policy_errors = _find_deprecation_policy_errors(current_schema)\n for error in deprecation_policy_errors:\n print(f\"::error title={PYPI_DISTRIBUTION} REST API::{error}\")\n\n prev_schema = _generate_openapi_for_git_ref(baseline_git_ref)\n if prev_schema is None:\n return 0 if not (static_policy_errors or deprecation_policy_errors) else 1\n prev_schema = _filter_public_rest_openapi(prev_schema)\n\n prev_schema = _normalize_openapi_for_oasdiff(prev_schema)\n current_schema = _normalize_openapi_for_oasdiff(current_schema)\n\n with tempfile.TemporaryDirectory(prefix=\"oasdiff-specs-\") as tmp:\n tmp_path = Path(tmp)\n prev_spec_file = tmp_path / \"prev_spec.json\"\n cur_spec_file = tmp_path / \"cur_spec.json\"\n prev_spec_file.write_text(json.dumps(prev_schema, indent=2))\n cur_spec_file.write_text(json.dumps(current_schema, indent=2))\n\n breaking_changes, exit_code = _run_oasdiff_breakage_check(\n prev_spec_file, cur_spec_file\n )\n\n if not breaking_changes:\n if exit_code == 0:\n print(\"No breaking changes detected.\")\n else:\n print(\n f\"oasdiff returned exit code {exit_code} but no breaking changes \"\n \"in JSON format. There may be warnings only.\"\n )\n else:\n (\n removed_operations,\n removed_schema_properties,\n additive_response_oneof,\n other_breaking_changes,\n ) = _split_breaking_changes(breaking_changes)\n response_union_artifacts = [\n change\n for change in removed_schema_properties\n if _is_union_property_removal_artifact(change)\n ]\n removed_schema_properties = [\n change\n for change in removed_schema_properties\n if not _is_union_property_removal_artifact(change)\n ]\n union_type_artifacts = [\n change\n for change in other_breaking_changes\n if _is_union_type_change_artifact(change)\n ]\n other_breaking_changes = [\n change\n for change in other_breaking_changes\n if not _is_union_type_change_artifact(change)\n ]\n\n removal_errors = _validate_removed_operations(\n removed_operations,\n prev_schema,\n current_version,\n )\n property_removal_errors = _validate_removed_schema_properties(\n removed_schema_properties,\n prev_schema,\n current_version,\n )\n\n for error in removal_errors + property_removal_errors:\n print(f\"::error title={PYPI_DISTRIBUTION} REST API::{error}\")\n\n if additive_response_oneof:\n print(\n f\"\\n::notice title={PYPI_DISTRIBUTION} REST API::\"\n \"Additive oneOf/anyOf expansion detected in response schemas. \"\n \"This is expected for extensible discriminated-union APIs and \"\n \"does not break backward compatibility.\"\n )\n for item in additive_response_oneof:\n print(f\" - {item.get('text', str(item))}\")\n if response_union_artifacts:\n print(\n \" - ignored \"\n f\"{len(response_union_artifacts)} request/response-property \"\n \"removal artifact(s) caused by union widening\"\n )\n if union_type_artifacts:\n print(\n \" - ignored \"\n f\"{len(union_type_artifacts)} request/response type-change \"\n \"artifact(s) caused by union widening\"\n )\n\n if other_breaking_changes:\n print(\n \"::error \"\n f\"title={PYPI_DISTRIBUTION} REST API::Detected breaking REST API \"\n \"changes other than removing previously-deprecated operations/\"\n \"properties or additive response oneOf expansions. \"\n \"REST contract changes must preserve compatibility for 5 minor \"\n \"releases; keep the old contract available until its scheduled \"\n \"removal version.\"\n )\n elif (\n response_union_artifacts or union_type_artifacts\n ) and not additive_response_oneof:\n print(\n f\"\\n::notice title={PYPI_DISTRIBUTION} REST API::\"\n f\"Ignored {len(response_union_artifacts)} property-removal and \"\n f\"{len(union_type_artifacts)} type-change artifact(s) reported \"\n \"while widening schemas.\"\n )\n\n print(\"\\nBreaking REST API changes detected compared to baseline release:\")\n for text in breaking_changes:\n print(f\"- {text.get('text', str(text))}\")\n\n if not (removal_errors or property_removal_errors or other_breaking_changes):\n print(\n \"Breaking changes are limited to previously-deprecated operations \"\n \"or properties whose scheduled removal versions have been reached, \"\n \"and/or additive response oneOf expansions.\"\n )\n else:\n return 1\n\n return 1 if (static_policy_errors or deprecation_policy_errors) else 0\n\n\nif __name__ == \"__main__\":\n raise SystemExit(main())\n" }, { "path": ".github/scripts/check_deprecations.py", "content": "#!/usr/bin/env python3\n\"\"\"Static analysis for deprecation deadlines.\n\nThis script scans Python deprecation metadata (`deprecated`, `warn_deprecated`,\n`warn_cleanup`) and agent-server REST routes marked `deprecated=True`. If the\ncurrent project version has reached or passed a feature's removal marker, the\nscript fails with a helpful summary so legacy shims and overdue deprecated REST\nendpoints are cleaned up before release.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport ast\nimport re\nimport sys\nimport tomllib\nfrom collections.abc import Iterable, Iterator, Sequence\nfrom dataclasses import dataclass\nfrom datetime import date\nfrom pathlib import Path\nfrom typing import Literal\n\nfrom packaging import version as pkg_version\n\n\nREST_ROUTE_DEPRECATION_RE = re.compile(\n r\"Deprecated since v(?P[0-9A-Za-z.+-]+)\\s+\"\n r\"and scheduled for removal in v(?P[0-9A-Za-z.+-]+)\\.?\",\n re.IGNORECASE,\n)\nROUTE_DECORATOR_NAMES = {\n \"get\",\n \"put\",\n \"post\",\n \"delete\",\n \"patch\",\n \"options\",\n \"head\",\n \"trace\",\n \"api_route\",\n}\nHTTP_METHODS = ROUTE_DECORATOR_NAMES - {\"api_route\"}\n\nREPO_ROOT = Path(__file__).resolve().parents[2]\n\n\n@dataclass(frozen=True, slots=True)\nclass PackageConfig:\n name: str\n pyproject: Path\n source_roots: tuple[Path, ...]\n\n\nPACKAGES: tuple[PackageConfig, ...] = (\n PackageConfig(\n name=\"openhands-sdk\",\n pyproject=REPO_ROOT / \"openhands-sdk\" / \"pyproject.toml\",\n source_roots=(REPO_ROOT / \"openhands-sdk\" / \"openhands\" / \"sdk\",),\n ),\n PackageConfig(\n name=\"openhands-tools\",\n pyproject=REPO_ROOT / \"openhands-tools\" / \"pyproject.toml\",\n source_roots=(REPO_ROOT / \"openhands-tools\" / \"openhands\" / \"tools\",),\n ),\n PackageConfig(\n name=\"openhands-workspace\",\n pyproject=REPO_ROOT / \"openhands-workspace\" / \"pyproject.toml\",\n source_roots=(REPO_ROOT / \"openhands-workspace\" / \"openhands\" / \"workspace\",),\n ),\n PackageConfig(\n name=\"openhands-agent-server\",\n pyproject=REPO_ROOT / \"openhands-agent-server\" / \"pyproject.toml\",\n source_roots=(\n REPO_ROOT / \"openhands-agent-server\" / \"openhands\" / \"agent_server\",\n ),\n ),\n)\n\n\n@dataclass(slots=True)\nclass DeprecationRecord:\n identifier: str\n removed_in: str | date | None\n deprecated_in: str | None\n path: Path\n line: int\n kind: Literal[\"decorator\", \"warn_call\", \"cleanup_call\", \"rest_route\"]\n package: str\n\n\ndef _load_current_version(pyproject: Path) -> str:\n data = tomllib.loads(pyproject.read_text())\n try:\n return str(data[\"project\"][\"version\"])\n except KeyError as exc: # pragma: no cover - configuration error\n raise SystemExit(\n f\"Unable to determine project version from {pyproject}\"\n ) from exc\n\n\ndef _iter_python_files(root: Path) -> Iterator[Path]:\n for path in root.rglob(\"*.py\"):\n if path.name == \"__init__.py\" and path.parent == root:\n continue\n yield path\n\n\ndef _parse_removed_value(\n node: ast.AST | None,\n *,\n path: Path,\n line: int,\n) -> str | date | None:\n if node is None:\n return None\n\n expression = ast.unparse(node)\n\n if isinstance(node, ast.Constant):\n if isinstance(node.value, str):\n return node.value\n if node.value is None:\n return None\n raise SystemExit(\n f\"Unsupported removed_in literal at {path}:{line}: {expression}\"\n )\n\n if isinstance(node, ast.Call):\n func = node.func\n if isinstance(func, ast.Name) and func.id == \"date\":\n try:\n args = [_safe_int_literal(arg) for arg in node.args]\n kwargs = {\n kw.arg: _safe_int_literal(kw.value)\n for kw in node.keywords\n if kw.arg is not None\n }\n except ValueError as exc:\n raise SystemExit(\n f\"Unsupported removed_in date() arguments at {path}:{line}:\"\n f\" {expression}\"\n ) from exc\n\n if any(kw.arg is None for kw in node.keywords):\n raise SystemExit(\n \"Unsupported removed_in date() call (uses **kwargs) at \"\n f\"{path}:{line}: {expression}\"\n )\n\n try:\n return date(*args, **kwargs)\n except TypeError as exc:\n raise SystemExit(\n f\"Invalid removed_in date() call at {path}:{line}: {expression}\"\n ) from exc\n\n if (\n isinstance(func, ast.Attribute)\n and isinstance(func.value, ast.Name)\n and func.value.id == \"date\"\n and func.attr == \"today\"\n ):\n if node.args or node.keywords:\n raise SystemExit(\n \"date.today() removed_in call must not include arguments at \"\n f\"{path}:{line}: {expression}\"\n )\n return date.today()\n\n raise SystemExit(\n f\"Unsupported removed_in expression at {path}:{line}: {expression}\"\n )\n\n\ndef _parse_deprecated_value(\n node: ast.AST | None,\n *,\n path: Path,\n line: int,\n) -> str | None:\n if node is None:\n return None\n\n expression = ast.unparse(node)\n\n if isinstance(node, ast.Constant):\n if isinstance(node.value, str):\n return node.value\n if node.value is None:\n return None\n\n raise SystemExit(\n f\"Unsupported deprecated_in expression at {path}:{line}: {expression}\"\n )\n\n\ndef _safe_int_literal(node: ast.AST) -> int:\n if not isinstance(node, ast.Constant) or not isinstance(node.value, int):\n raise ValueError(\n f\"Unsupported expression inside literal evaluation: {ast.unparse(node)}\"\n )\n return node.value\n\n\ndef _extract_kw(call: ast.Call, name: str) -> ast.AST | None:\n for kw in call.keywords:\n if kw.arg == name:\n return kw.value\n return None\n\n\ndef _extract_string_literal(node: ast.AST | None) -> str | None:\n if isinstance(node, ast.Constant) and isinstance(node.value, str):\n return node.value\n return None\n\n\ndef _extract_string_sequence(node: ast.AST | None) -> tuple[str, ...] | None:\n if not isinstance(node, (ast.List, ast.Tuple, ast.Set)):\n return None\n\n values: list[str] = []\n for item in node.elts:\n value = _extract_string_literal(item)\n if value is None:\n return None\n values.append(value)\n return tuple(values)\n\n\ndef _extract_route_details(call: ast.Call) -> tuple[tuple[str, str], ...]:\n target = call.func\n if not isinstance(target, ast.Attribute):\n return ()\n\n decorator_name = target.attr\n if decorator_name not in ROUTE_DECORATOR_NAMES:\n return ()\n\n path = _extract_string_literal(call.args[0] if call.args else None)\n if path is None:\n path = _extract_string_literal(_extract_kw(call, \"path\"))\n if path is None:\n return ()\n\n if decorator_name in HTTP_METHODS:\n return ((decorator_name.upper(), path),)\n\n methods = _extract_string_sequence(_extract_kw(call, \"methods\"))\n if methods is None:\n return ((\"GET\", path),)\n\n return tuple(\n (method.upper(), path) for method in methods if method.lower() in HTTP_METHODS\n )\n\n\ndef _parse_rest_route_deprecation_docstring(\n docstring: str | None,\n *,\n path: Path,\n line: int,\n route_identifiers: Sequence[str],\n) -> tuple[str, str]:\n if not docstring:\n raise SystemExit(\n \"Deprecated REST route(s) \"\n f\"{', '.join(route_identifiers)} at {path}:{line} must include a \"\n \"docstring note like 'Deprecated since vX.Y.Z and scheduled for \"\n \"removal in vA.B.C.'\"\n )\n\n match = REST_ROUTE_DEPRECATION_RE.search(\" \".join(docstring.split()))\n if match is None:\n raise SystemExit(\n \"Deprecated REST route(s) \"\n f\"{', '.join(route_identifiers)} at {path}:{line} must include a \"\n \"docstring note like 'Deprecated since vX.Y.Z and scheduled for \"\n \"removal in vA.B.C.'\"\n )\n\n return match.group(\"deprecated\").rstrip(\".\"), match.group(\"removed\").rstrip(\".\")\n\n\ndef _gather_rest_route_deprecations(\n tree: ast.AST, path: Path, *, package: str\n) -> Iterator[DeprecationRecord]:\n for node in ast.walk(tree):\n if not isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef):\n continue\n\n routes: list[tuple[str, str]] = []\n for deco in node.decorator_list:\n if not isinstance(deco, ast.Call):\n continue\n deprecated_value = _extract_kw(deco, \"deprecated\")\n if (\n not isinstance(deprecated_value, ast.Constant)\n or deprecated_value.value is not True\n ):\n continue\n routes.extend(_extract_route_details(deco))\n\n if not routes:\n continue\n\n deprecated_in, removed_in = _parse_rest_route_deprecation_docstring(\n ast.get_docstring(node),\n path=path,\n line=node.lineno,\n route_identifiers=[\n f\"{method} {route_path}\" for method, route_path in routes\n ],\n )\n\n for method, route_path in routes:\n yield DeprecationRecord(\n identifier=f\"{method} {route_path}\",\n removed_in=removed_in,\n deprecated_in=deprecated_in,\n path=path,\n line=node.lineno,\n kind=\"rest_route\",\n package=package,\n )\n\n\ndef _gather_decorators(\n tree: ast.AST, path: Path, *, package: str\n) -> Iterator[DeprecationRecord]:\n for node in ast.walk(tree):\n if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):\n continue\n\n for deco in node.decorator_list:\n call = deco if isinstance(deco, ast.Call) else None\n if call is None:\n continue\n\n target = call.func\n if isinstance(target, ast.Name):\n decorator_name = target.id\n elif isinstance(target, ast.Attribute):\n decorator_name = target.attr\n else:\n continue\n\n if decorator_name != \"deprecated\":\n continue\n\n removed_expr = _extract_kw(call, \"removed_in\")\n deprecated_expr = _extract_kw(call, \"deprecated_in\")\n\n record = DeprecationRecord(\n identifier=_build_identifier(node),\n removed_in=_parse_removed_value(\n removed_expr, path=path, line=node.lineno\n ),\n deprecated_in=_parse_deprecated_value(\n deprecated_expr, path=path, line=node.lineno\n ),\n path=path,\n line=node.lineno,\n kind=\"decorator\",\n package=package,\n )\n yield record\n\n\ndef _gather_warn_calls(\n tree: ast.AST, path: Path, *, package: str\n) -> Iterator[DeprecationRecord]:\n for node in ast.walk(tree):\n if not isinstance(node, ast.Call):\n continue\n\n target = node.func\n if isinstance(target, ast.Name):\n func_name = target.id\n elif isinstance(target, ast.Attribute):\n func_name = target.attr\n else:\n continue\n\n if func_name == \"warn_deprecated\":\n identifier_node = node.args[0] if node.args else None\n if identifier_node is None:\n continue\n identifier = ast.unparse(identifier_node)\n\n removed_expr = _extract_kw(node, \"removed_in\")\n deprecated_expr = _extract_kw(node, \"deprecated_in\")\n\n yield DeprecationRecord(\n identifier=identifier,\n removed_in=_parse_removed_value(\n removed_expr, path=path, line=node.lineno\n ),\n deprecated_in=_parse_deprecated_value(\n deprecated_expr, path=path, line=node.lineno\n ),\n path=path,\n line=node.lineno,\n kind=\"warn_call\",\n package=package,\n )\n elif func_name == \"warn_cleanup\":\n identifier_node = node.args[0] if node.args else None\n if identifier_node is None:\n continue\n identifier = ast.unparse(identifier_node)\n\n cleanup_expr = _extract_kw(node, \"cleanup_by\")\n\n yield DeprecationRecord(\n identifier=identifier,\n removed_in=_parse_removed_value(\n cleanup_expr, path=path, line=node.lineno\n ),\n deprecated_in=None,\n path=path,\n line=node.lineno,\n kind=\"cleanup_call\",\n package=package,\n )\n\n\ndef _build_identifier(node: ast.AST) -> str:\n if isinstance(node, ast.ClassDef):\n return node.name\n if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):\n qual_name = node.name\n if node.decorator_list:\n parent = getattr(node, \"parent\", None)\n if parent and isinstance(parent, ast.ClassDef):\n return f\"{parent.name}.{node.name}\"\n return qual_name\n return \"\"\n\n\ndef _attach_parents(tree: ast.AST) -> None:\n for node in ast.walk(tree):\n for child in ast.iter_child_nodes(node):\n setattr(child, \"parent\", node)\n\n\ndef _collect_records(files: Iterable[Path], *, package: str) -> list[DeprecationRecord]:\n records: list[DeprecationRecord] = []\n for path in files:\n tree = ast.parse(path.read_text())\n _attach_parents(tree)\n records.extend(_gather_decorators(tree, path, package=package))\n records.extend(_gather_warn_calls(tree, path, package=package))\n return records\n\n\ndef _collect_rest_route_records(\n files: Iterable[Path], *, package: str\n) -> list[DeprecationRecord]:\n records: list[DeprecationRecord] = []\n for path in files:\n tree = ast.parse(path.read_text())\n records.extend(_gather_rest_route_deprecations(tree, path, package=package))\n return records\n\n\ndef _version_ge(current: str, target: str) -> bool:\n try:\n return pkg_version.parse(current) >= pkg_version.parse(target)\n except pkg_version.InvalidVersion as exc:\n raise SystemExit(\n f\"Invalid semantic version comparison: {current=} {target=}\"\n ) from exc\n\n\ndef _should_fail(current_version: str, record: DeprecationRecord) -> bool:\n removed = record.removed_in\n if removed is None:\n return False\n if isinstance(removed, date):\n return date.today() >= removed\n try:\n target = str(removed)\n return _version_ge(current_version, target)\n except SystemExit:\n raise\n except Exception as exc: # pragma: no cover - unexpected literal type\n raise SystemExit(\n f\"Unsupported removed_in expression in {record.path}:{record.line}:\"\n f\" {removed!r}\"\n ) from exc\n\n\ndef _format_record(record: DeprecationRecord) -> str:\n location = record.path.relative_to(REPO_ROOT)\n removed = record.removed_in if record.removed_in is not None else \"(none)\"\n\n if record.kind == \"cleanup_call\":\n return (\n f\"- [{record.package}] {record.identifier} ({record.kind})\\n\"\n f\" cleanup by: {removed}\\n\"\n f\" defined at: {location}:{record.line}\"\n )\n\n deprecated = (\n record.deprecated_in if record.deprecated_in is not None else \"(unknown)\"\n )\n return (\n f\"- [{record.package}] {record.identifier} ({record.kind})\\n\"\n f\" deprecated in: {deprecated}\\n\"\n f\" removed in: {removed}\\n\"\n f\" defined at: {location}:{record.line}\"\n )\n\n\ndef main(argv: Sequence[str] | None = None) -> int:\n argv = list(argv or [])\n\n overdue: list[DeprecationRecord] = []\n total_records = 0\n package_summaries: list[tuple[str, str, int]] = []\n\n for package in PACKAGES:\n if not package.pyproject.exists():\n raise SystemExit(\n f\"Unable to locate pyproject.toml for {package.name}: \"\n f\"{package.pyproject}\"\n )\n\n current_version = _load_current_version(package.pyproject)\n\n files: list[Path] = []\n for root in package.source_roots:\n if not root.exists():\n raise SystemExit(\n f\"Source root {root} for package {package.name} does not exist\"\n )\n files.extend(_iter_python_files(root))\n\n records = _collect_records(files, package=package.name)\n if package.name == \"openhands-agent-server\":\n records.extend(_collect_rest_route_records(files, package=package.name))\n\n overdue.extend(r for r in records if _should_fail(current_version, r))\n total_records += len(records)\n package_summaries.append((package.name, current_version, len(records)))\n\n if overdue:\n deprecated_items = [r for r in overdue if r.kind != \"cleanup_call\"]\n cleanup_items = [r for r in overdue if r.kind == \"cleanup_call\"]\n\n if deprecated_items:\n print(\n \"The following deprecated features have passed their removal \"\n \"deadline:\\n\"\n )\n for record in deprecated_items:\n print(_format_record(record))\n print()\n\n if cleanup_items:\n print(\"The following workarounds have passed their cleanup deadline:\\n\")\n for record in cleanup_items:\n print(_format_record(record))\n print()\n\n if deprecated_items:\n print(\n \"Update or remove the listed features before publishing a version that \"\n \"meets or exceeds their removal deadline.\"\n )\n if cleanup_items:\n print(\n \"Remove the listed workarounds before publishing a version that \"\n \"meets or exceeds their cleanup deadline.\"\n )\n return 1\n\n for package_name, version, count in package_summaries:\n print(\n f\"{package_name}: checked {count} deprecation metadata entries against \"\n f\"version {version}.\"\n )\n print(\n f\"Checked {total_records} deprecation metadata entries across \"\n f\"{len(package_summaries)} package(s).\"\n )\n return 0\n\n\nif __name__ == \"__main__\": # pragma: no cover - manual invocation\n sys.exit(main(sys.argv[1:]))\n" }, { "path": ".github/scripts/check_docstrings.py", "content": "#!/usr/bin/env python3\n\"\"\"Validate docstrings conform to MDX-compatible formatting guidelines.\n\nThis script checks that docstrings in the SDK use patterns that render correctly\nin Mintlify MDX documentation. It validates:\n\n1. No REPL-style examples (>>>) - should use fenced code blocks instead\n2. Shell/config examples use fenced code blocks (prevents # becoming headers)\n\nRun with: python scripts/check_docstrings.py\nExit code 0 = all checks pass, 1 = violations found\n\"\"\"\n\nimport ast\nimport sys\nfrom dataclasses import dataclass\nfrom pathlib import Path\n\n\n# Directories to check\nSDK_PATHS = [\n \"openhands-sdk/openhands/sdk\",\n]\n\n# Files/directories to skip\nSKIP_PATTERNS = [\n \"__pycache__\",\n \".pyc\",\n \"test_\",\n \"_test.py\",\n]\n\n# Core public API files to check strictly (these are documented on the website)\n# Other files will be checked but only emit warnings, not failures\nSTRICT_CHECK_FILES = [\n \"agent/agent.py\",\n \"llm/llm.py\",\n \"conversation/conversation.py\",\n \"tool/tool.py\",\n \"workspace/base.py\",\n \"observability/laminar.py\",\n]\n\n\n@dataclass\nclass Violation:\n \"\"\"A docstring formatting violation.\"\"\"\n\n file: Path\n line: int\n name: str\n rule: str\n message: str\n is_strict: bool = False # True if this is in a strictly-checked file\n\n\ndef should_skip(path: Path) -> bool:\n \"\"\"Check if a path should be skipped.\"\"\"\n path_str = str(path)\n return any(pattern in path_str for pattern in SKIP_PATTERNS)\n\n\ndef check_repl_examples(\n docstring: str, name: str, lineno: int, file: Path\n) -> list[Violation]:\n \"\"\"Check for REPL-style examples (>>>).\n\n These should be replaced with fenced code blocks for better MDX rendering.\n \"\"\"\n violations = []\n lines = docstring.split(\"\\n\")\n\n for i, line in enumerate(lines):\n stripped = line.strip()\n if stripped.startswith(\">>>\"):\n violations.append(\n Violation(\n file=file,\n line=lineno + i,\n name=name,\n rule=\"no-repl-examples\",\n message=(\n \"Use fenced code blocks (```python) instead of >>> REPL style. \"\n \"REPL examples don't render well in MDX documentation.\"\n ),\n )\n )\n # Only report once per docstring\n break\n\n return violations\n\n\ndef check_unfenced_shell_config(\n docstring: str, name: str, lineno: int, file: Path\n) -> list[Violation]:\n \"\"\"Check for shell/config examples that aren't in fenced code blocks.\n\n Lines starting with # outside code blocks become markdown headers.\n \"\"\"\n violations = []\n lines = docstring.split(\"\\n\")\n in_code_block = False\n\n for i, line in enumerate(lines):\n stripped = line.strip()\n\n # Track code block state\n if stripped.startswith(\"```\"):\n in_code_block = not in_code_block\n continue\n\n # Skip if inside a code block\n if in_code_block:\n continue\n\n # Check for shell-style comments that look like config\n # Pattern: line starts with # and previous line has = (config pattern)\n if stripped.startswith(\"#\") and not stripped.startswith(\"# \"):\n # This is likely a shell comment without space (less common in prose)\n continue\n\n # Check for unfenced config: KEY=VALUE followed by # comment\n if i > 0:\n prev_line = lines[i - 1].strip() if i > 0 else \"\"\n # If previous line looks like config (VAR=value) and this is a # comment\n if \"=\" in prev_line and prev_line.split(\"=\")[0].isupper():\n if stripped.startswith(\"# \"):\n violations.append(\n Violation(\n file=file,\n line=lineno + i,\n name=name,\n rule=\"fenced-shell-config\",\n message=(\n \"Shell/config examples with # comments should be \"\n \"in ```bash code blocks. Otherwise # becomes a \"\n \"markdown header.\"\n ),\n )\n )\n # Only report once per docstring\n break\n\n return violations\n\n\ndef check_docstring(\n docstring: str, name: str, lineno: int, file: Path\n) -> list[Violation]:\n \"\"\"Run all checks on a docstring.\"\"\"\n if not docstring:\n return []\n\n violations = []\n violations.extend(check_repl_examples(docstring, name, lineno, file))\n violations.extend(check_unfenced_shell_config(docstring, name, lineno, file))\n return violations\n\n\ndef get_docstrings_from_file(file: Path) -> list[tuple[str, str, int]]:\n \"\"\"Extract all docstrings from a Python file.\n\n Returns list of (name, docstring, lineno) tuples.\n \"\"\"\n try:\n source = file.read_text()\n tree = ast.parse(source)\n except (SyntaxError, UnicodeDecodeError) as e:\n print(f\"Warning: Could not parse {file}: {e}\", file=sys.stderr)\n return []\n\n docstrings = []\n\n for node in ast.walk(tree):\n name = None\n lineno = 0\n docstring = None\n\n if isinstance(node, ast.Module):\n docstring = ast.get_docstring(node)\n name = file.stem\n lineno = 1\n elif isinstance(node, ast.ClassDef):\n docstring = ast.get_docstring(node)\n name = node.name\n lineno = node.lineno\n elif isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef):\n docstring = ast.get_docstring(node)\n name = node.name\n lineno = node.lineno\n\n if docstring and name:\n docstrings.append((name, docstring, lineno))\n\n return docstrings\n\n\ndef is_strict_file(file: Path, repo_root: Path) -> bool:\n \"\"\"Check if a file is in the strict check list.\"\"\"\n try:\n rel_path = file.relative_to(repo_root / \"openhands-sdk/openhands/sdk\")\n return any(str(rel_path) == strict for strict in STRICT_CHECK_FILES)\n except ValueError:\n return False\n\n\ndef check_file(file: Path, repo_root: Path) -> list[Violation]:\n \"\"\"Check all docstrings in a file.\"\"\"\n violations = []\n is_strict = is_strict_file(file, repo_root)\n\n for name, docstring, lineno in get_docstrings_from_file(file):\n file_violations = check_docstring(docstring, name, lineno, file)\n for v in file_violations:\n v.is_strict = is_strict\n violations.extend(file_violations)\n\n return violations\n\n\ndef main() -> int:\n \"\"\"Run docstring checks on all SDK files.\"\"\"\n repo_root = Path(__file__).parent.parent.parent\n\n all_violations: list[Violation] = []\n files_checked = 0\n\n for sdk_path in SDK_PATHS:\n path = repo_root / sdk_path\n if not path.exists():\n print(f\"Warning: Path not found: {path}\", file=sys.stderr)\n continue\n\n for py_file in path.rglob(\"*.py\"):\n if should_skip(py_file):\n continue\n\n files_checked += 1\n violations = check_file(py_file, repo_root)\n all_violations.extend(violations)\n\n # Separate strict violations (errors) from warnings\n strict_violations = [v for v in all_violations if v.is_strict]\n warning_violations = [v for v in all_violations if not v.is_strict]\n\n # Report warnings (non-strict files)\n if warning_violations:\n count = len(warning_violations)\n print(f\"\\n⚠️ Found {count} docstring warning(s) in non-core files:\\n\")\n\n by_file: dict[Path, list[Violation]] = {}\n for v in warning_violations:\n by_file.setdefault(v.file, []).append(v)\n\n for file, violations in sorted(by_file.items()):\n rel_path = file.relative_to(repo_root)\n print(f\"📄 {rel_path}\")\n for v in violations:\n print(f\" Line {v.line}: {v.name} ({v.rule})\")\n print()\n\n # Report errors (strict files)\n if strict_violations:\n count = len(strict_violations)\n print(f\"\\n❌ Found {count} docstring error(s) in core API files:\\n\")\n\n by_file: dict[Path, list[Violation]] = {}\n for v in strict_violations:\n by_file.setdefault(v.file, []).append(v)\n\n for file, violations in sorted(by_file.items()):\n rel_path = file.relative_to(repo_root)\n print(f\"📄 {rel_path}\")\n for v in violations:\n print(f\" Line {v.line}: {v.name}\")\n print(f\" Rule: {v.rule}\")\n print(f\" {v.message}\")\n print()\n\n print(\"=\" * 60)\n print(\"To fix these issues:\")\n print(\" 1. Replace >>> examples with ```python code blocks\")\n print(\" 2. Wrap shell/config examples in ```bash code blocks\")\n print(\"=\" * 60)\n return 1\n\n if warning_violations:\n count = len(warning_violations)\n print(f\"✅ Core API files pass. {count} warnings in other files.\")\n else:\n print(f\"✅ All {files_checked} files pass docstring checks\")\n return 0\n\n\nif __name__ == \"__main__\":\n sys.exit(main())\n" }, { "path": ".github/scripts/check_documented_examples.py", "content": "#!/usr/bin/env python3\n\"\"\"\nCheck if all examples in agent-sdk are documented in the docs repository.\n\nThis script:\n1. Scans the docs repository for references to example files\n2. Lists all example Python files in the agent-sdk repository\n3. Compares the two sets to find undocumented examples\n4. Exits with error code 1 if undocumented examples are found\n\"\"\"\n\nimport os\nimport re\nimport sys\nfrom pathlib import Path\n\n\ndef find_documented_examples(docs_path: Path) -> set[str]:\n \"\"\"\n Find all example file references in the docs repository.\n\n Searches for patterns like:\n - examples/01_standalone_sdk/02_custom_tools.py\n - examples/02_remote_agent_server/06_custom_tool/custom_tools/log_data.py\n in MDX files.\n\n Returns:\n Set of normalized example file paths (relative to agent-sdk root)\n \"\"\"\n documented_examples: set[str] = set()\n\n # Pattern to match example file references with arbitrary nesting depth.\n # Matches: examples//.../.py\n pattern = r\"examples/(?:[-\\w]+/)+[-\\w]+\\.py\"\n\n for root, _, files in os.walk(docs_path):\n for file in files:\n if file.endswith(\".mdx\") or file.endswith(\".md\"):\n file_path = Path(root) / file\n try:\n content = file_path.read_text(encoding=\"utf-8\")\n matches = re.findall(pattern, content)\n for match in matches:\n # Normalize the path\n documented_examples.add(match)\n except Exception as e:\n print(f\"Warning: Error reading {file_path}: {e}\")\n continue\n\n return documented_examples\n\n\ndef find_agent_sdk_examples(agent_sdk_path: Path) -> set[str]:\n \"\"\"\n Find all example Python files in the agent-sdk repository.\n\n Excludes examples/03_github_workflows/ since those examples are YAML\n files, not Python files.\n\n Returns:\n Set of example file paths (relative to agent-sdk root)\n \"\"\"\n examples: set[str] = set()\n examples_dir = agent_sdk_path / \"examples\"\n\n if not examples_dir.exists():\n print(f\"Error: Examples directory not found: {examples_dir}\")\n sys.exit(1)\n\n # Find all Python files under examples/\n for root, _, files in os.walk(examples_dir):\n for file in files:\n if file.endswith(\".py\"):\n file_path = Path(root) / file\n # Get relative path from agent-sdk root\n relative_path = file_path.relative_to(agent_sdk_path)\n relative_path_str = str(relative_path)\n\n # Skip GitHub workflow examples (those are YAML files, Python\n # files there are just helpers)\n if relative_path_str.startswith(\"examples/03_github_workflows/\"):\n continue\n\n # Skip LLM-specific tools examples: these are intentionally not\n # enforced by the docs check. See discussion in PR #1486.\n if relative_path_str.startswith(\"examples/04_llm_specific_tools/\"):\n continue\n\n # Skip __init__.py files as they typically don't need documentation\n if file == \"__init__.py\":\n continue\n\n examples.add(relative_path_str)\n\n return examples\n\n\ndef resolve_paths() -> tuple[Path, Path]:\n \"\"\"\n Determine agent-sdk root and docs path.\n\n Priority for docs path:\n 1) DOCS_PATH (env override)\n 2) $GITHUB_WORKSPACE/docs\n 3) agent_sdk_root/'docs'\n 4) agent_sdk_root.parent/'docs'\n\n Returns:\n Tuple of (agent_sdk_root, docs_path)\n \"\"\"\n # agent-sdk repo root (script is at agent-sdk/.github/scripts/...)\n script_file = Path(__file__).resolve()\n agent_sdk_root = script_file.parent.parent.parent\n\n candidates: list[Path] = []\n\n # 1) Explicit env override\n env_override = os.environ.get(\"DOCS_PATH\")\n if env_override:\n candidates.append(Path(env_override).expanduser().resolve())\n\n # 2) Standard GitHub workspace sibling\n gh_ws = os.environ.get(\"GITHUB_WORKSPACE\")\n if gh_ws:\n candidates.append(Path(gh_ws).resolve() / \"docs\")\n\n # 3) Sibling inside the agent-sdk repo root\n candidates.append(agent_sdk_root / \"docs\")\n\n # 4) Parent-of-agent-sdk-root layout\n candidates.append(agent_sdk_root.parent / \"docs\")\n\n print(f\"🔍 Agent SDK root: {agent_sdk_root}\")\n print(\"🔎 Trying docs paths (in order):\")\n for p in candidates:\n print(f\" - {p}\")\n\n for p in candidates:\n if p.exists():\n print(f\"📁 Using docs path: {p}\")\n return agent_sdk_root, p\n\n # If none exist, fail with a helpful message\n print(\"❌ Docs path not found in any of the expected locations.\")\n print(\" Set DOCS_PATH, or checkout the repo to one of the tried paths above.\")\n sys.exit(1)\n\n\ndef main() -> None:\n agent_sdk_root, docs_path = resolve_paths()\n\n print(\"\\n\" + \"=\" * 60)\n print(\"Checking documented examples...\")\n print(\"=\" * 60)\n\n # Find all examples in agent-sdk\n print(\"\\n📋 Scanning agent-sdk examples...\")\n agent_examples = find_agent_sdk_examples(agent_sdk_root)\n print(f\" Found {len(agent_examples)} example file(s)\")\n\n # Find all documented examples in docs\n print(\"\\n📄 Scanning docs repository...\")\n documented_examples = find_documented_examples(docs_path)\n print(f\" Found {len(documented_examples)} documented example(s)\")\n\n # Calculate difference\n undocumented = agent_examples - documented_examples\n\n print(\"\\n\" + \"=\" * 60)\n if undocumented:\n print(f\"❌ Found {len(undocumented)} undocumented example(s):\")\n print(\"=\" * 60)\n for example in sorted(undocumented):\n print(f\" - {example}\")\n print(\"\\n⚠️ Please add documentation for these examples in the docs repo.\")\n print(\"=\" * 60)\n print(\"\\n📚 How to Document Examples:\")\n print(\"=\" * 60)\n print(\"1. Clone the docs repository:\")\n print(\" git clone https://github.com/OpenHands/docs.git\")\n print()\n print(\"2. Create a new .mdx file in sdk/guides/ directory\")\n print(\" (e.g., sdk/guides/my-feature.mdx)\")\n print()\n print(\"3. Add the example code block with this format:\")\n print(' ```python icon=\"python\" expandable examples/path/to/file.py')\n print(\" \")\n print(\" ```\")\n print()\n print(\"4. See the format documentation at:\")\n print(\n \" https://github.com/OpenHands/docs/blob/main/.github/scripts/README.md\"\n )\n print()\n print(\"5. Example documentation files can be found in:\")\n print(\" https://github.com/OpenHands/docs/tree/main/sdk/guides\")\n print()\n print(\"6. After creating the PR in docs repo, reference it in your\")\n print(\" agent-sdk PR description.\")\n print(\"=\" * 60)\n sys.exit(1)\n else:\n print(\"✅ All examples are documented!\")\n print(\"=\" * 60)\n sys.exit(0)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": ".github/scripts/check_duplicate_example_numbers.py", "content": "#!/usr/bin/env python3\n\"\"\"\nCheck for duplicate example numbers in the examples directory.\n\nThis script ensures that within each examples subdirectory, no two files or\nfolders share the same numeric prefix (e.g., two files both starting with \"04_\").\n\nExit codes:\n 0 - No duplicates found\n 1 - Duplicates found\n\"\"\"\n\nimport re\nimport sys\nfrom collections import defaultdict\nfrom pathlib import Path\n\n\ndef find_duplicate_numbers(examples_dir: Path) -> dict[str, list[str]]:\n \"\"\"\n Find duplicate example numbers within each subdirectory.\n\n Returns:\n Dictionary mapping subdirectory paths to lists of duplicate entries.\n Only includes subdirectories that have duplicates.\n \"\"\"\n duplicates: dict[str, list[str]] = {}\n\n # Pattern to extract leading number from filename/dirname\n # e.g., \"04\" from \"04_foo.py\"\n number_pattern = re.compile(r\"^(\\d+)_\")\n\n for subdir in sorted(examples_dir.iterdir()):\n if not subdir.is_dir():\n continue\n\n # Skip hidden directories\n if subdir.name.startswith(\".\"):\n continue\n\n # Group entries by their numeric prefix\n number_to_entries: dict[str, list[str]] = defaultdict(list)\n\n for entry in subdir.iterdir():\n # Skip hidden files/directories\n if entry.name.startswith(\".\"):\n continue\n\n match = number_pattern.match(entry.name)\n if match:\n number = match.group(1)\n number_to_entries[number].append(entry.name)\n\n # Find numbers with multiple entries\n subdir_duplicates = []\n for number, entries in sorted(number_to_entries.items()):\n if len(entries) > 1:\n subdir_duplicates.extend(sorted(entries))\n\n if subdir_duplicates:\n relative_subdir = str(subdir.relative_to(examples_dir.parent))\n duplicates[relative_subdir] = subdir_duplicates\n\n return duplicates\n\n\ndef main() -> None:\n # Find the examples directory relative to this script\n script_file = Path(__file__).resolve()\n repo_root = script_file.parent.parent.parent\n examples_dir = repo_root / \"examples\"\n\n if not examples_dir.exists():\n print(f\"Error: Examples directory not found: {examples_dir}\")\n sys.exit(1)\n\n print(\"=\" * 60)\n print(\"Checking for duplicate example numbers...\")\n print(\"=\" * 60)\n print(f\"\\n📁 Scanning: {examples_dir}\\n\")\n\n duplicates = find_duplicate_numbers(examples_dir)\n\n if duplicates:\n print(\"❌ Found duplicate example numbers:\\n\")\n for subdir, entries in sorted(duplicates.items()):\n print(f\" {subdir}/\")\n for entry in entries:\n print(f\" - {entry}\")\n print()\n\n print(\"=\" * 60)\n print(\"⚠️ Please renumber the examples to remove duplicates.\")\n print(\" Each example should have a unique number within its folder.\")\n print(\"=\" * 60)\n sys.exit(1)\n else:\n print(\"✅ No duplicate example numbers found!\")\n print(\"=\" * 60)\n sys.exit(0)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": ".github/scripts/check_sdk_api_breakage.py", "content": "#!/usr/bin/env python3\n\"\"\"API breakage detection for published OpenHands packages using Griffe.\n\nThis script compares current workspace packages against the most recent PyPI\nrelease (or the matching release if the current version is already published)\nto detect breaking changes in the public API.\n\nIt focuses on the curated public surface:\n- symbols exported via ``__all__``\n- public members removed from classes exported via ``__all__``\n\nIt enforces two policies:\n\n1. **Deprecation runway before removal** – any removed export or removed public\n class member must have been marked deprecated in the *previous* release using\n the canonical deprecation helpers (``@deprecated`` decorator or\n ``warn_deprecated()`` call from ``openhands.sdk.utils.deprecation``), and the\n baseline deprecation metadata must show that the current version has reached a\n scheduled removal target at least **5 minor releases** after\n ``deprecated_in``. For members, the recommended ``warn_deprecated`` feature\n name is qualified (e.g. ``\"LLM.some_method\"``).\n\n2. **MINOR version bump** – any breaking change (removal or structural) requires\n at least a MINOR version bump according to SemVer.\n\nComplementary to the deprecation mechanism:\n- Deprecation (``check_deprecations.py``): enforces cleanup deadlines\n- This script: prevents unannounced removals and enforces SemVer bumps\n\"\"\"\n\nfrom __future__ import annotations\n\nimport ast\nimport json\nimport os\nimport subprocess\nimport sys\nimport tomllib\nimport urllib.request\nfrom collections.abc import Iterable\nfrom dataclasses import dataclass, field\nfrom pathlib import Path\n\nfrom packaging import version as pkg_version\nfrom packaging.requirements import Requirement\n\n\n@dataclass(frozen=True)\nclass PackageConfig:\n \"\"\"Configuration for a single published package.\"\"\"\n\n package: str # dotted module path, e.g. \"openhands.sdk\"\n distribution: str # PyPI distribution name, e.g. \"openhands-sdk\"\n source_dir: str # repo-relative directory, e.g. \"openhands-sdk\"\n\n\n@dataclass(frozen=True, slots=True)\nclass DeprecationMetadata:\n deprecated_in: str | None = None\n removed_in: str | None = None\n\n\n@dataclass(frozen=True, slots=True)\nclass DeprecatedSymbols:\n \"\"\"Deprecated SDK symbols detected in a source tree.\n\n ``top_level`` tracks module-level symbols (exports) like ``LLM``.\n ``qualified`` tracks class members like ``LLM.some_method``.\n ``metadata`` stores the parsed deprecation schedule for each feature.\n \"\"\"\n\n top_level: set[str] = frozenset() # type: ignore[assignment]\n qualified: set[str] = frozenset() # type: ignore[assignment]\n metadata: dict[str, DeprecationMetadata] = field(default_factory=dict)\n\n\nDEPRECATION_RUNWAY_MINOR_RELEASES = 5\n\n\nPACKAGES: tuple[PackageConfig, ...] = (\n PackageConfig(\n package=\"openhands.sdk\",\n distribution=\"openhands-sdk\",\n source_dir=\"openhands-sdk\",\n ),\n PackageConfig(\n package=\"openhands.workspace\",\n distribution=\"openhands-workspace\",\n source_dir=\"openhands-workspace\",\n ),\n PackageConfig(\n package=\"openhands.tools\",\n distribution=\"openhands-tools\",\n source_dir=\"openhands-tools\",\n ),\n)\n\nACP_DEPENDENCY = \"agent-client-protocol\"\nACP_SKIP_ENV = \"ACP_VERSION_CHECK_SKIP\"\nACP_SKIP_TOKEN = \"skip-acp-check\"\nACP_BASE_REF_ENV = \"ACP_VERSION_CHECK_BASE_REF\"\n\n\ndef read_version_from_pyproject(path: str) -> str:\n \"\"\"Read the version string from a pyproject.toml file.\"\"\"\n with open(path, \"rb\") as f:\n data = tomllib.load(f)\n proj = data.get(\"project\", {})\n v = proj.get(\"version\")\n if not v:\n raise SystemExit(f\"Could not read version from {path}\")\n return str(v)\n\n\ndef _read_pyproject(path: str) -> dict:\n with open(path, \"rb\") as f:\n return tomllib.load(f)\n\n\ndef _bool_env(name: str) -> bool:\n value = os.environ.get(name, \"\").strip().lower()\n return value in {\"1\", \"true\", \"yes\", \"on\"}\n\n\ndef _get_dependency_spec(project_data: dict, dependency: str) -> str | None:\n deps = project_data.get(\"project\", {}).get(\"dependencies\", [])\n for dep in deps:\n if dep.startswith(dependency):\n return dep\n return None\n\n\ndef _min_version_from_requirement(req_str: str) -> pkg_version.Version | None:\n try:\n req = Requirement(req_str)\n except Exception as exc:\n print(\n f\"::warning title=ACP version::Unable to parse requirement \"\n f\"'{req_str}': {exc}\"\n )\n return None\n\n lower_bounds: list[pkg_version.Version] = []\n for spec in req.specifier:\n if spec.operator in {\">=\", \">\", \"==\", \"~=\"}:\n try:\n lower_bounds.append(_parse_version(spec.version))\n except Exception as exc:\n print(\n f\"::warning title=ACP version::Unable to parse version \"\n f\"'{spec.version}' from '{req_str}': {exc}\"\n )\n\n if not lower_bounds:\n return None\n\n return max(lower_bounds)\n\n\ndef _git_show_file(ref: str, rel_path: str) -> str | None:\n for candidate in (f\"origin/{ref}\", ref):\n result = subprocess.run(\n [\"git\", \"show\", f\"{candidate}:{rel_path}\"],\n check=False,\n capture_output=True,\n text=True,\n )\n if result.returncode == 0:\n return result.stdout\n return None\n\n\ndef _load_base_pyproject(base_ref: str) -> dict | None:\n rel_path = \"openhands-sdk/pyproject.toml\"\n content = _git_show_file(base_ref, rel_path)\n if content is None:\n print(\n f\"::warning title=ACP version::Unable to read {rel_path} from \"\n f\"{base_ref}; skipping ACP version check\"\n )\n return None\n try:\n return tomllib.loads(content)\n except tomllib.TOMLDecodeError as exc:\n print(\n f\"::warning title=ACP version::Failed to parse {rel_path} from \"\n f\"{base_ref}: {exc}\"\n )\n return None\n\n\ndef _check_acp_version_bump(repo_root: str) -> int:\n if _bool_env(ACP_SKIP_ENV):\n print(\n f\"::notice title=ACP version::Skipping ACP version check because \"\n f\"{ACP_SKIP_ENV} is set (token: [{ACP_SKIP_TOKEN}]).\"\n )\n return 0\n\n base_ref = os.environ.get(ACP_BASE_REF_ENV) or os.environ.get(\"GITHUB_BASE_REF\")\n if not base_ref:\n print(\n \"::warning title=ACP version::No base ref found; skipping ACP version check\"\n )\n return 0\n\n base_data = _load_base_pyproject(base_ref)\n if base_data is None:\n return 0\n\n current_data = _read_pyproject(\n os.path.join(repo_root, \"openhands-sdk\", \"pyproject.toml\")\n )\n old_req = _get_dependency_spec(base_data, ACP_DEPENDENCY)\n new_req = _get_dependency_spec(current_data, ACP_DEPENDENCY)\n\n if not old_req or not new_req:\n print(\n f\"::warning title=ACP version::Unable to locate {ACP_DEPENDENCY} \"\n \"dependency in pyproject.toml; skipping ACP version check\"\n )\n return 0\n\n old_min = _min_version_from_requirement(old_req)\n new_min = _min_version_from_requirement(new_req)\n\n if old_min is None or new_min is None:\n print(\n f\"::warning title=ACP version::Unable to parse {ACP_DEPENDENCY} \"\n \"minimum version; skipping ACP version check\"\n )\n return 0\n\n if new_min <= old_min:\n return 0\n\n if new_min.major != old_min.major or new_min.minor != old_min.minor:\n print(\n \"::error title=ACP version::Detected \"\n f\"{ACP_DEPENDENCY} minor/major version bump \"\n f\"({old_req} -> {new_req}). If intentional, add \"\n f\"[{ACP_SKIP_TOKEN}] to the PR description to bypass.\"\n )\n return 1\n\n return 0\n\n\ndef _parse_version(v: str) -> pkg_version.Version:\n \"\"\"Parse a version string using packaging.\"\"\"\n return pkg_version.parse(v)\n\n\ndef _parse_string_kwarg(call: ast.Call, name: str) -> str | None:\n for kw in call.keywords:\n if kw.arg != name:\n continue\n value = kw.value\n if isinstance(value, ast.Constant) and isinstance(value.value, str):\n return value.value\n return None\n return None\n\n\ndef _minimum_removed_in(deprecated_in: str) -> str:\n parsed = _parse_version(deprecated_in)\n return f\"{parsed.major}.{parsed.minor + DEPRECATION_RUNWAY_MINOR_RELEASES}.0\"\n\n\ndef _deprecation_schedule_errors(\n *,\n feature: str,\n metadata: DeprecationMetadata | None,\n current_version: str,\n) -> list[str]:\n if metadata is None:\n return [\n f\"Removed '{feature}' without prior deprecation. Mark it with \"\n \"@deprecated(...) or warn_deprecated(...), and keep it deprecated for \"\n f\"{DEPRECATION_RUNWAY_MINOR_RELEASES} minor releases before removing.\"\n ]\n\n if metadata.deprecated_in is None:\n return [\n f\"Removed '{feature}' was marked deprecated previously, but its \"\n \"deprecation metadata does not declare deprecated_in. Public API \"\n f\"removals require {DEPRECATION_RUNWAY_MINOR_RELEASES} minor releases \"\n \"of runway.\"\n ]\n\n if metadata.removed_in is None:\n return [\n f\"Removed '{feature}' was marked deprecated previously, but its \"\n \"deprecation metadata does not declare removed_in. Public API removals \"\n f\"require {DEPRECATION_RUNWAY_MINOR_RELEASES} minor releases of runway.\"\n ]\n\n minimum_removed_in = _minimum_removed_in(metadata.deprecated_in)\n if _parse_version(metadata.removed_in) < _parse_version(minimum_removed_in):\n return [\n f\"Removed '{feature}' uses an invalid deprecation schedule: \"\n f\"deprecated_in={metadata.deprecated_in} and \"\n f\"removed_in={metadata.removed_in}. Public API removals require at \"\n f\"least {DEPRECATION_RUNWAY_MINOR_RELEASES} minor releases of runway \"\n f\"(minimum removed_in: {minimum_removed_in}).\"\n ]\n\n if _parse_version(current_version) < _parse_version(metadata.removed_in):\n return [\n f\"Removed '{feature}' before its scheduled removal version \"\n f\"{metadata.removed_in}. Current version is {current_version}. Public \"\n f\"API removals require {DEPRECATION_RUNWAY_MINOR_RELEASES} minor releases \"\n \"of deprecation runway.\"\n ]\n\n return []\n\n\ndef get_pypi_baseline_version(pkg: str, current: str | None) -> str | None:\n \"\"\"Fetch the baseline release version from PyPI.\n\n The baseline is the most recent published release to compare against the\n current workspace. If the current version already exists on PyPI, compare\n against that same release. Otherwise, fall back to the newest release older\n than the current version. If ``current`` is None, use the latest release.\n\n Args:\n pkg: Package name on PyPI (e.g., \"openhands-sdk\")\n current: Current version from the workspace, or None for latest\n\n Returns:\n Baseline version string, or None if not found or on network error\n \"\"\"\n req = urllib.request.Request(\n url=f\"https://pypi.org/pypi/{pkg}/json\",\n headers={\"User-Agent\": \"openhands-sdk-api-check/1.0\"},\n method=\"GET\",\n )\n try:\n with urllib.request.urlopen(req, timeout=10) as r:\n meta = json.load(r)\n except Exception as e:\n print(f\"::warning title={pkg} API::Failed to fetch PyPI metadata: {e}\")\n return None\n\n releases = list(meta.get(\"releases\", {}).keys())\n if not releases:\n return None\n\n def _sort_key(s: str):\n return _parse_version(s)\n\n releases_sorted = sorted(releases, key=_sort_key, reverse=True)\n if current is None:\n return releases_sorted[0]\n\n if current in releases:\n return current\n\n cur_parsed = _parse_version(current)\n older = [rv for rv in releases if _parse_version(rv) < cur_parsed]\n if not older:\n return None\n return sorted(older, key=_sort_key, reverse=True)[0]\n\n\ndef ensure_griffe() -> None:\n \"\"\"Verify griffe is installed, raising an error if not.\"\"\"\n try:\n import griffe # noqa: F401\n except ImportError:\n sys.stderr.write(\n \"ERROR: griffe not installed. Install with: pip install griffe[pypi]\\n\"\n )\n raise SystemExit(1)\n\n\nFIELD_METADATA_KWARGS = frozenset(\n {\n \"deprecated\",\n \"description\",\n \"examples\",\n \"json_schema_extra\",\n \"title\",\n }\n)\n\n\ndef _escape_newlines_in_string_literals(text: str) -> str:\n \"\"\"Escape literal newlines that appear inside quoted string literals.\"\"\"\n chars: list[str] = []\n in_string: str | None = None\n escaped = False\n\n for ch in text:\n if in_string is None:\n chars.append(ch)\n if ch in {\"'\", '\"'}:\n in_string = ch\n continue\n\n if escaped:\n chars.append(ch)\n escaped = False\n continue\n\n if ch == \"\\\\\":\n chars.append(ch)\n escaped = True\n continue\n\n if ch == in_string:\n chars.append(ch)\n in_string = None\n continue\n\n if ch == \"\\n\":\n chars.append(\"\\\\n\")\n continue\n\n chars.append(ch)\n\n return \"\".join(chars)\n\n\ndef _parse_field_call(value: object) -> ast.Call | None:\n \"\"\"Parse a stringified Pydantic ``Field(...)`` value into an AST call.\"\"\"\n try:\n expr = ast.parse(\n _escape_newlines_in_string_literals(str(value)),\n mode=\"eval\",\n ).body\n except SyntaxError:\n return None\n\n if not isinstance(expr, ast.Call):\n return None\n\n func = expr.func\n if isinstance(func, ast.Name):\n func_name = func.id\n elif isinstance(func, ast.Attribute):\n func_name = func.attr\n else:\n return None\n\n if func_name != \"Field\":\n return None\n\n return expr\n\n\ndef _filter_field_metadata_kwargs(call: ast.Call) -> ast.Call:\n \"\"\"Return a copy of a ``Field(...)`` call without metadata-only kwargs.\"\"\"\n return ast.Call(\n func=call.func,\n args=call.args,\n keywords=[kw for kw in call.keywords if kw.arg not in FIELD_METADATA_KWARGS],\n )\n\n\ndef _is_field_metadata_only_change(old_val: object, new_val: object) -> bool:\n \"\"\"Check if the change is only in Field metadata (description, title, etc.).\n\n Field metadata parameters like ``description``, ``title``, ``examples``,\n ``json_schema_extra``, and ``deprecated`` don't affect runtime behavior.\n Changes to these should not be considered breaking API changes.\n\n Returns:\n True if both values are Field() calls and only metadata parameters differ.\n \"\"\"\n old_call = _parse_field_call(old_val)\n new_call = _parse_field_call(new_val)\n if old_call is None or new_call is None:\n return False\n\n return ast.dump(\n _filter_field_metadata_kwargs(old_call),\n include_attributes=False,\n ) == ast.dump(\n _filter_field_metadata_kwargs(new_call),\n include_attributes=False,\n )\n\n\ndef _member_deprecation_metadata(\n cls_obj: object,\n member_name: str,\n deprecated: DeprecatedSymbols,\n) -> DeprecationMetadata | None:\n \"\"\"Return deprecation metadata for a class member, including parent classes.\n\n When a member like ``system_message`` is deprecated on a base class\n (``AgentBase``) but removed from a subclass (``Agent``), griffe reports\n the removal against the subclass name. This helper walks the MRO so that\n ``Agent.system_message`` reuses the base-class deprecation schedule.\n \"\"\"\n cls_name = getattr(cls_obj, \"name\", \"\")\n feature = f\"{cls_name}.{member_name}\"\n if feature in deprecated.qualified:\n return deprecated.metadata.get(feature, DeprecationMetadata())\n if cls_name in deprecated.top_level:\n return deprecated.metadata.get(cls_name, DeprecationMetadata())\n\n for base in getattr(cls_obj, \"resolved_bases\", []):\n base_name = getattr(base, \"name\", None)\n if base_name is None:\n continue\n feature = f\"{base_name}.{member_name}\"\n if feature in deprecated.qualified:\n return deprecated.metadata.get(feature, DeprecationMetadata())\n return None\n\n\ndef _was_deprecated(\n cls_obj: object,\n member_name: str,\n deprecated: DeprecatedSymbols,\n) -> bool:\n return _member_deprecation_metadata(cls_obj, member_name, deprecated) is not None\n\n\ndef _collect_breakages_pairs(\n objs: Iterable[tuple[object, object]],\n *,\n deprecated: DeprecatedSymbols,\n current_version: str,\n title: str,\n) -> tuple[list[object], int]:\n \"\"\"Find breaking changes between pairs of old/new API objects.\n\n Only reports breakages for public API members.\n\n Returns:\n (breakages, removal_policy_errors)\n \"\"\"\n\n import griffe\n from griffe import Alias, AliasResolutionError, BreakageKind, ExplanationStyle, Kind\n\n breakages: list[object] = []\n removal_policy_errors = 0\n\n for old, new in objs:\n try:\n for br in griffe.find_breaking_changes(old, new):\n obj = getattr(br, \"obj\", None)\n if not getattr(obj, \"is_public\", True):\n continue\n\n # Skip ATTRIBUTE_CHANGED_VALUE when it's just Field metadata changes\n # (description, title, examples, etc.) - these don't affect runtime\n if br.kind == BreakageKind.ATTRIBUTE_CHANGED_VALUE:\n old_value = getattr(br, \"old_value\", None)\n new_value = getattr(br, \"new_value\", None)\n if _is_field_metadata_only_change(old_value, new_value):\n print(\n f\"::notice title={title}::Ignoring Field metadata-only \"\n f\"change (non-breaking): {obj.name if obj else 'unknown'}\"\n )\n continue\n\n print(br.explain(style=ExplanationStyle.GITHUB))\n breakages.append(br)\n\n if br.kind != BreakageKind.OBJECT_REMOVED:\n continue\n\n parent = getattr(obj, \"parent\", None)\n if getattr(parent, \"kind\", None) != Kind.CLASS:\n continue\n\n feature = f\"{parent.name}.{obj.name}\"\n errors = _deprecation_schedule_errors(\n feature=feature,\n metadata=_member_deprecation_metadata(parent, obj.name, deprecated),\n current_version=current_version,\n )\n if not errors:\n continue\n\n for error in errors:\n print(f\"::error title={title}::{error}\")\n removal_policy_errors += len(errors)\n except AliasResolutionError as e:\n if isinstance(old, Alias) or isinstance(new, Alias):\n old_target = old.target_path if isinstance(old, Alias) else None\n new_target = new.target_path if isinstance(new, Alias) else None\n if old_target != new_target:\n name = getattr(old, \"name\", None) or getattr(\n new, \"name\", \"\"\n )\n print(\n f\"::warning title={title}::Alias target changed for '{name}': \"\n f\"{old_target!r} -> {new_target!r}\"\n )\n breakages.append(\n {\n \"kind\": \"ALIAS_TARGET_CHANGED\",\n \"name\": name,\n \"old\": old_target,\n \"new\": new_target,\n }\n )\n else:\n print(\n f\"::notice title={title}::Skipping symbol comparison due to \"\n f\"unresolved alias: {e}\"\n )\n except Exception as e:\n print(f\"::warning title={title}::Failed to compute breakages: {e}\")\n\n return breakages, removal_policy_errors\n\n\ndef _extract_exported_names(module) -> set[str]:\n \"\"\"Extract names exported from a module via ``__all__``.\n\n This check is explicitly meant to track the curated public surface. The SDK\n is expected to define ``__all__`` in ``openhands.sdk``; if it's missing or we\n can't statically interpret it, we fail fast rather than silently widening the\n surface area (which would make the check noisy and brittle).\n \"\"\"\n try:\n all_var = module[\"__all__\"]\n except Exception as e:\n raise ValueError(\"Expected __all__ to be defined on the public module\") from e\n\n val = getattr(all_var, \"value\", None)\n elts = getattr(val, \"elements\", None)\n if not elts:\n raise ValueError(\"Unable to statically evaluate __all__\")\n\n names: set[str] = set()\n for el in elts:\n # Griffe represents string literals in __all__ in different ways depending\n # on how the module is loaded / griffe version:\n # - sometimes as plain Python strings (including quotes, e.g. \"'LLM'\")\n # - sometimes as expression nodes with a `.value` attribute\n #\n # We intentionally only support the \"static __all__ of string literals\"\n # case; we just normalize the representation.\n if isinstance(el, str):\n names.add(el.strip(\"\\\"'\"))\n continue\n s = getattr(el, \"value\", None)\n if isinstance(s, str):\n names.add(s)\n\n if not names:\n raise ValueError(\"__all__ resolved to an empty set\")\n\n return names\n\n\ndef _check_version_bump(prev: str, new_version: str, total_breaks: int) -> int:\n \"\"\"Check if version bump policy is satisfied for breaking changes.\n\n Policy: Breaking changes require at least a MINOR version bump.\n\n Returns:\n 0 if policy satisfied, 1 if not\n \"\"\"\n if total_breaks == 0:\n print(\"No breaking changes detected\")\n return 0\n\n parsed_prev = _parse_version(prev)\n parsed_new = _parse_version(new_version)\n\n # MINOR bump required: same major, higher minor OR higher major\n ok = (parsed_new.major > parsed_prev.major) or (\n parsed_new.major == parsed_prev.major and parsed_new.minor > parsed_prev.minor\n )\n\n if not ok:\n print(\n f\"::error title=SemVer::Breaking changes detected ({total_breaks}); \"\n f\"require at least minor version bump from \"\n f\"{parsed_prev.major}.{parsed_prev.minor}.x, but new is {new_version}\"\n )\n return 1\n\n print(\n f\"Breaking changes detected ({total_breaks}) and version bump policy \"\n f\"satisfied ({prev} -> {new_version})\"\n )\n return 0\n\n\ndef _resolve_griffe_object(\n root: object,\n dotted: str,\n root_package: str = \"\",\n) -> object:\n \"\"\"Resolve a dotted path to a griffe object.\"\"\"\n root_path = getattr(root, \"path\", None)\n if root_path == dotted:\n return root\n\n if isinstance(root_path, str) and dotted.startswith(root_path + \".\"):\n dotted = dotted[len(root_path) + 1 :]\n\n try:\n return root[dotted]\n except (KeyError, TypeError) as e:\n print(\n f\"::warning title=SDK API::Unable to resolve {dotted} via \"\n f\"direct lookup; falling back to manual traversal: {e}\"\n )\n\n rel = dotted\n if root_package and dotted.startswith(root_package + \".\"):\n rel = dotted[len(root_package) + 1 :]\n\n obj = root\n for part in rel.split(\".\"):\n try:\n obj = obj[part]\n except (KeyError, TypeError) as e:\n raise KeyError(f\"Unable to resolve {dotted}: failed at {part}\") from e\n return obj\n\n\ndef _load_current(\n griffe_module: object, repo_root: str, cfg: PackageConfig\n) -> object | None:\n try:\n return griffe_module.load(\n cfg.package,\n search_paths=[os.path.join(repo_root, cfg.source_dir)],\n )\n except Exception as e:\n print(\n f\"::error title={cfg.distribution} API::\"\n f\"Failed to load current {cfg.distribution}: {e}\"\n )\n return None\n\n\ndef _load_prev_from_pypi(\n griffe_module: object,\n prev: str,\n cfg: PackageConfig,\n) -> object | None:\n griffe_cache = os.path.expanduser(\"~/.cache/griffe\")\n os.makedirs(griffe_cache, exist_ok=True)\n\n try:\n return griffe_module.load_pypi(\n package=cfg.package,\n distribution=cfg.distribution,\n version_spec=f\"=={prev}\",\n )\n except Exception as e:\n print(\n f\"::error title={cfg.distribution} API::\"\n f\"Failed to load {cfg.distribution}=={prev} from PyPI: {e}\"\n )\n return None\n\n\ndef _find_deprecated_symbols(source_root: Path) -> DeprecatedSymbols:\n \"\"\"Scan source files for symbols marked with the SDK deprecation helpers.\n\n Detects two forms:\n - ``@deprecated(...)`` decorator on a class/function/method\n - ``warn_deprecated('SomeFeature', ...)`` call\n\n Returns:\n DeprecatedSymbols(top_level=..., qualified=..., metadata=...)\n \"\"\"\n\n def _deprecated_metadata(call: ast.Call) -> DeprecationMetadata:\n return DeprecationMetadata(\n deprecated_in=_parse_string_kwarg(call, \"deprecated_in\"),\n removed_in=_parse_string_kwarg(call, \"removed_in\"),\n )\n\n def _is_deprecated_decorator(deco: ast.AST) -> ast.Call | None:\n if not isinstance(deco, ast.Call):\n return None\n target = deco.func\n if isinstance(target, ast.Name) and target.id == \"deprecated\":\n return deco\n if isinstance(target, ast.Attribute) and target.attr == \"deprecated\":\n return deco\n return None\n\n class _Visitor(ast.NodeVisitor):\n def __init__(self) -> None:\n self.class_stack: list[str] = []\n self.top_level: set[str] = set()\n self.qualified: set[str] = set()\n self.metadata: dict[str, DeprecationMetadata] = {}\n\n def visit_ClassDef(self, node: ast.ClassDef) -> None: # noqa: N802\n for deco in node.decorator_list:\n deprecated_call = _is_deprecated_decorator(deco)\n if deprecated_call is None:\n continue\n metadata = _deprecated_metadata(deprecated_call)\n self.top_level.add(node.name)\n self.qualified.add(node.name)\n self.metadata[node.name] = metadata\n break\n\n self.class_stack.append(node.name)\n self.generic_visit(node)\n self.class_stack.pop()\n\n def _visit_function_like(\n self,\n node: ast.FunctionDef | ast.AsyncFunctionDef,\n ) -> None:\n for deco in node.decorator_list:\n deprecated_call = _is_deprecated_decorator(deco)\n if deprecated_call is None:\n continue\n metadata = _deprecated_metadata(deprecated_call)\n if self.class_stack:\n feature = \".\".join([*self.class_stack, node.name])\n self.qualified.add(feature)\n self.metadata[feature] = metadata\n else:\n self.top_level.add(node.name)\n self.qualified.add(node.name)\n self.metadata[node.name] = metadata\n break\n\n self.generic_visit(node)\n\n def visit_FunctionDef(self, node: ast.FunctionDef) -> None: # noqa: N802\n self._visit_function_like(node)\n\n def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> None: # noqa: N802\n self._visit_function_like(node)\n\n def visit_Call(self, node: ast.Call) -> None: # noqa: N802\n target = node.func\n func_name = None\n if isinstance(target, ast.Name):\n func_name = target.id\n elif isinstance(target, ast.Attribute):\n func_name = target.attr\n\n if func_name == \"warn_deprecated\" and node.args:\n feature = _extract_string_literal(node.args[0])\n if feature is not None:\n metadata = _deprecated_metadata(node)\n self.qualified.add(feature)\n top_level_name = feature.split(\".\")[0]\n self.top_level.add(top_level_name)\n self.metadata[feature] = metadata\n self.metadata.setdefault(top_level_name, metadata)\n\n self.generic_visit(node)\n\n top_level: set[str] = set()\n qualified: set[str] = set()\n metadata: dict[str, DeprecationMetadata] = {}\n\n for pyfile in source_root.rglob(\"*.py\"):\n try:\n tree = ast.parse(pyfile.read_text())\n except SyntaxError as e:\n print(\n f\"::warning title=SDK API::Skipping {pyfile}: \"\n f\"failed to parse (SyntaxError: {e})\"\n )\n continue\n\n visitor = _Visitor()\n visitor.visit(tree)\n top_level |= visitor.top_level\n qualified |= visitor.qualified\n metadata.update(visitor.metadata)\n\n return DeprecatedSymbols(\n top_level=top_level, qualified=qualified, metadata=metadata\n )\n\n\ndef _extract_string_literal(node: ast.AST) -> str | None:\n \"\"\"Return the string value if *node* is a simple string literal.\"\"\"\n if isinstance(node, ast.Constant) and isinstance(node.value, str):\n return node.value\n return None\n\n\ndef _get_source_root(griffe_root: object) -> Path | None:\n \"\"\"Derive the package source directory from a griffe module's filepath.\"\"\"\n filepath = getattr(griffe_root, \"filepath\", None)\n if filepath is not None:\n return Path(filepath).parent\n return None\n\n\ndef _compute_breakages(\n old_root,\n new_root,\n cfg: PackageConfig,\n *,\n current_version: str = \"9999.0.0\",\n) -> tuple[int, int]:\n \"\"\"Detect breaking changes between old and new package versions.\n\n Returns:\n ``(total_breaks, removal_policy_errors)`` — *total_breaks* counts all\n structural breakages (for the version-bump policy), while\n *removal_policy_errors* counts public API removals that violate the\n required deprecation runway.\n \"\"\"\n pkg = cfg.package\n title = f\"{cfg.distribution} API\"\n total_breaks = 0\n removal_policy_errors = 0\n\n source_root = _get_source_root(old_root)\n deprecated = (\n _find_deprecated_symbols(source_root) if source_root else DeprecatedSymbols()\n )\n\n try:\n old_mod = _resolve_griffe_object(old_root, pkg, root_package=pkg)\n new_mod = _resolve_griffe_object(new_root, pkg, root_package=pkg)\n except Exception as e:\n raise RuntimeError(f\"Failed to resolve root module '{pkg}'\") from e\n\n new_exports = _extract_exported_names(new_mod)\n try:\n old_exports = _extract_exported_names(old_mod)\n except ValueError as e:\n # The API breakage check relies on a curated public surface defined via\n # __all__. If the baseline release didn't define (or couldn't statically\n # evaluate) __all__, we can't compute meaningful breakages.\n #\n # In this situation, skip rather than failing the entire workflow.\n print(\n f\"::notice title={title}::Skipping breakage check; baseline release \"\n f\"has no statically-evaluable {pkg}.__all__: {e}\"\n )\n return 0, 0\n\n removed = sorted(old_exports - new_exports)\n\n # Check deprecation runway policy (exports)\n for name in removed:\n total_breaks += 1 # every removal is a structural break\n errors = _deprecation_schedule_errors(\n feature=name,\n metadata=(\n deprecated.metadata.get(name, DeprecationMetadata())\n if name in deprecated.top_level\n else None\n ),\n current_version=current_version,\n )\n if not errors:\n print(\n f\"::notice title={title}::Removed previously-deprecated symbol \"\n f\"'{name}' from {pkg}.__all__ after its scheduled removal version\"\n )\n continue\n\n for error in errors:\n print(f\"::error title={title}::{error}\")\n removal_policy_errors += len(errors)\n\n common = sorted(old_exports & new_exports)\n pairs: list[tuple[object, object]] = []\n for name in common:\n try:\n pairs.append((old_mod[name], new_mod[name]))\n except Exception as e:\n print(f\"::warning title={title}::Unable to resolve symbol {name}: {e}\")\n\n breakages, member_policy_errors = _collect_breakages_pairs(\n pairs,\n deprecated=deprecated,\n current_version=current_version,\n title=title,\n )\n total_breaks += len(breakages)\n removal_policy_errors += member_policy_errors\n\n return total_breaks, removal_policy_errors\n\n\ndef _check_package(griffe_module, repo_root: str, cfg: PackageConfig) -> int:\n \"\"\"Run breakage checks for a single package. Returns 0 on success.\"\"\"\n pyproj = os.path.join(repo_root, cfg.source_dir, \"pyproject.toml\")\n new_version = read_version_from_pyproject(pyproj)\n\n title = f\"{cfg.distribution} API\"\n baseline = get_pypi_baseline_version(cfg.distribution, new_version)\n if not baseline:\n print(\n f\"::warning title={title}::No baseline {cfg.distribution} \"\n f\"release found; skipping breakage check\",\n )\n return 0\n\n print(f\"Comparing {cfg.distribution} {new_version} against {baseline}\")\n\n new_root = _load_current(griffe_module, repo_root, cfg)\n if not new_root:\n return 1\n\n old_root = _load_prev_from_pypi(griffe_module, baseline, cfg)\n if not old_root:\n return 1\n\n try:\n total_breaks, removal_policy_errors = _compute_breakages(\n old_root,\n new_root,\n cfg,\n current_version=new_version,\n )\n except Exception as e:\n print(f\"::error title={title}::Failed to compute breakages: {e}\")\n return 1\n\n if removal_policy_errors:\n print(\n f\"::error title={title}::{removal_policy_errors} public API removal \"\n f\"policy violation(s) detected in {cfg.package} — see errors above\"\n )\n\n bump_rc = _check_version_bump(baseline, new_version, total_breaks)\n\n return 1 if (removal_policy_errors or bump_rc) else 0\n\n\ndef main() -> int:\n \"\"\"Main entry point for API breakage detection.\"\"\"\n repo_root = os.getcwd()\n rc = _check_acp_version_bump(repo_root)\n\n ensure_griffe()\n import griffe\n\n for cfg in PACKAGES:\n print(f\"\\n{'=' * 60}\")\n print(f\"Checking {cfg.distribution} ({cfg.package})\")\n print(f\"{'=' * 60}\")\n rc |= _check_package(griffe, repo_root, cfg)\n\n return rc\n\n\nif __name__ == \"__main__\":\n raise SystemExit(main())\n" }, { "path": ".github/scripts/check_version_bumps.py", "content": "\"\"\"Guard package version changes so they only happen in release PRs.\"\"\"\n\nfrom __future__ import annotations\n\nimport os\nimport re\nimport subprocess\nimport sys\nimport tomllib\nfrom dataclasses import dataclass\nfrom pathlib import Path\n\n\nPACKAGE_PYPROJECTS: dict[str, Path] = {\n \"openhands-sdk\": Path(\"openhands-sdk/pyproject.toml\"),\n \"openhands-tools\": Path(\"openhands-tools/pyproject.toml\"),\n \"openhands-workspace\": Path(\"openhands-workspace/pyproject.toml\"),\n \"openhands-agent-server\": Path(\"openhands-agent-server/pyproject.toml\"),\n}\n\n_VERSION_PATTERN = r\"\\d+\\.\\d+\\.\\d+(?:[-+][0-9A-Za-z.]+)?\"\n_RELEASE_TITLE_RE = re.compile(rf\"^Release v(?P{_VERSION_PATTERN})$\")\n_RELEASE_BRANCH_RE = re.compile(rf\"^rel-(?P{_VERSION_PATTERN})$\")\n\n\n@dataclass(frozen=True)\nclass VersionChange:\n package: str\n path: Path\n previous_version: str\n current_version: str\n\n\ndef _read_version_from_pyproject_text(text: str, source: str) -> str:\n data = tomllib.loads(text)\n version = data.get(\"project\", {}).get(\"version\")\n if not isinstance(version, str):\n raise SystemExit(f\"Unable to determine project.version from {source}\")\n return version\n\n\ndef _read_current_version(repo_root: Path, pyproject: Path) -> str:\n return _read_version_from_pyproject_text(\n (repo_root / pyproject).read_text(),\n str(pyproject),\n )\n\n\ndef _read_version_from_git_ref(repo_root: Path, git_ref: str, pyproject: Path) -> str:\n result = subprocess.run(\n [\"git\", \"show\", f\"{git_ref}:{pyproject.as_posix()}\"],\n cwd=repo_root,\n check=False,\n capture_output=True,\n text=True,\n )\n if result.returncode != 0:\n message = result.stderr.strip() or result.stdout.strip() or \"unknown git error\"\n raise SystemExit(\n f\"Unable to read {pyproject} from git ref {git_ref}: {message}\"\n )\n return _read_version_from_pyproject_text(result.stdout, f\"{git_ref}:{pyproject}\")\n\n\ndef _base_ref_candidates(base_ref: str) -> list[str]:\n if base_ref.startswith(\"origin/\"):\n return [base_ref, base_ref.removeprefix(\"origin/\")]\n return [f\"origin/{base_ref}\", base_ref]\n\n\ndef find_version_changes(repo_root: Path, base_ref: str) -> list[VersionChange]:\n changes: list[VersionChange] = []\n candidates = _base_ref_candidates(base_ref)\n\n for package, pyproject in PACKAGE_PYPROJECTS.items():\n current_version = _read_current_version(repo_root, pyproject)\n previous_error: SystemExit | None = None\n previous_version: str | None = None\n\n for candidate in candidates:\n try:\n previous_version = _read_version_from_git_ref(\n repo_root, candidate, pyproject\n )\n break\n except SystemExit as exc:\n previous_error = exc\n\n if previous_version is None:\n assert previous_error is not None\n raise previous_error\n\n if previous_version != current_version:\n changes.append(\n VersionChange(\n package=package,\n path=pyproject,\n previous_version=previous_version,\n current_version=current_version,\n )\n )\n\n return changes\n\n\ndef get_release_pr_version(\n pr_title: str, pr_head_ref: str\n) -> tuple[str | None, list[str]]:\n title_match = _RELEASE_TITLE_RE.fullmatch(pr_title.strip())\n branch_match = _RELEASE_BRANCH_RE.fullmatch(pr_head_ref.strip())\n title_version = title_match.group(\"version\") if title_match else None\n branch_version = branch_match.group(\"version\") if branch_match else None\n\n if title_version and branch_version and title_version != branch_version:\n return None, [\n \"Release PR markers disagree: title requests \"\n f\"v{title_version} but branch is rel-{branch_version}.\"\n ]\n\n return title_version or branch_version, []\n\n\ndef validate_version_changes(\n changes: list[VersionChange],\n pr_title: str,\n pr_head_ref: str,\n) -> list[str]:\n if not changes:\n return []\n\n release_version, errors = get_release_pr_version(pr_title, pr_head_ref)\n if errors:\n return errors\n\n formatted_changes = \", \".join(\n f\"{change.package} ({change.previous_version} -> {change.current_version})\"\n for change in changes\n )\n\n if release_version is None:\n return [\n \"Package version changes are only allowed in release PRs. \"\n f\"Detected changes: {formatted_changes}. \"\n \"Use the Prepare Release workflow so the PR title is 'Release vX.Y.Z' \"\n \"or the branch is 'rel-X.Y.Z'.\"\n ]\n\n mismatched = [\n change for change in changes if change.current_version != release_version\n ]\n if mismatched:\n mismatch_details = \", \".join(\n f\"{change.package} ({change.current_version})\" for change in mismatched\n )\n return [\n f\"Release PR version v{release_version} does not match changed package \"\n f\"versions: {mismatch_details}.\"\n ]\n\n return []\n\n\ndef main() -> int:\n repo_root = Path(__file__).resolve().parents[2]\n base_ref = os.environ.get(\"VERSION_BUMP_BASE_REF\") or os.environ.get(\n \"GITHUB_BASE_REF\"\n )\n if not base_ref:\n print(\"::warning title=Version bump guard::No base ref found; skipping check.\")\n return 0\n\n pr_title = os.environ.get(\"PR_TITLE\", \"\")\n pr_head_ref = os.environ.get(\"PR_HEAD_REF\", \"\")\n\n changes = find_version_changes(repo_root, base_ref)\n errors = validate_version_changes(changes, pr_title, pr_head_ref)\n\n if errors:\n for error in errors:\n print(f\"::error title=Version bump guard::{error}\")\n return 1\n\n if changes:\n changed_packages = \", \".join(change.package for change in changes)\n print(\n \"::notice title=Version bump guard::\"\n f\"Release PR version changes validated for {changed_packages}.\"\n )\n else:\n print(\"::notice title=Version bump guard::No package version changes detected.\")\n\n return 0\n\n\nif __name__ == \"__main__\":\n sys.exit(main())\n" }, { "path": ".github/scripts/update_sdk_ref_default.py", "content": "#!/usr/bin/env python3\n\"\"\"Update the sdk_ref default value in run-eval.yml.\n\nThis script updates the default SDK reference version in the run-eval workflow\nto match a new release version.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport re\nimport sys\nfrom pathlib import Path\n\n\nREPO_ROOT = Path(__file__).resolve().parents[2]\nRUN_EVAL_WORKFLOW = REPO_ROOT / \".github\" / \"workflows\" / \"run-eval.yml\"\n\n# Pattern to match the sdk_ref default line\n# Matches: \"default: vX.Y.Z\" with optional prerelease suffix like -rc1, -beta.1\nSDK_REF_PATTERN = re.compile(\n r\"^(\\s*default:\\s*v)[\\d]+\\.[\\d]+\\.[\\d]+(-[a-zA-Z0-9.]+)?(\\s*)$\"\n)\n\n\ndef update_sdk_ref_default(new_version: str, dry_run: bool = False) -> bool:\n \"\"\"Update the sdk_ref default in run-eval.yml.\n\n Args:\n new_version: The new version (without 'v' prefix, e.g., \"1.12.0\")\n dry_run: If True, print what would change without modifying the file\n\n Returns:\n True if successful, False otherwise\n \"\"\"\n if not RUN_EVAL_WORKFLOW.exists():\n print(f\"❌ File not found: {RUN_EVAL_WORKFLOW}\", file=sys.stderr)\n return False\n\n content = RUN_EVAL_WORKFLOW.read_text()\n lines = content.splitlines(keepends=True)\n\n # Find the sdk_ref input section and its default line\n in_sdk_ref_section = False\n updated = False\n old_version = None\n\n for i, line in enumerate(lines):\n stripped = line.strip()\n\n # Track when we enter the sdk_ref input section\n if stripped == \"sdk_ref:\":\n in_sdk_ref_section = True\n continue\n\n # Track when we exit the sdk_ref section (another input starts)\n if (\n in_sdk_ref_section\n and stripped.endswith(\":\")\n and not stripped.startswith(\"default\")\n ):\n in_sdk_ref_section = False\n\n # Update the default line within the sdk_ref section\n if in_sdk_ref_section:\n match = SDK_REF_PATTERN.match(line)\n if match:\n old_version = line.strip().replace(\"default: \", \"\")\n new_line = f\"{match.group(1)}{new_version}{match.group(3) or ''}\"\n if not line.endswith(\"\\n\") and lines[i].endswith(\"\\n\"):\n new_line += \"\\n\"\n elif line.endswith(\"\\n\"):\n new_line += \"\\n\"\n lines[i] = new_line\n updated = True\n break\n\n if not updated:\n print(\"❌ Could not find sdk_ref default line to update\", file=sys.stderr)\n return False\n\n if dry_run:\n print(f\"Would update sdk_ref default: {old_version} → v{new_version}\")\n return True\n\n # Write the updated content\n RUN_EVAL_WORKFLOW.write_text(\"\".join(lines))\n print(f\"✅ Updated sdk_ref default: {old_version} → v{new_version}\")\n return True\n\n\ndef main() -> int:\n parser = argparse.ArgumentParser(\n description=\"Update the sdk_ref default value in run-eval.yml\"\n )\n parser.add_argument(\n \"version\",\n help=\"New version (without 'v' prefix, e.g., '1.12.0')\",\n )\n parser.add_argument(\n \"--dry-run\",\n action=\"store_true\",\n help=\"Print what would change without modifying the file\",\n )\n args = parser.parse_args()\n\n # Validate version format\n version_pattern = re.compile(r\"^\\d+\\.\\d+\\.\\d+(-[a-zA-Z0-9.]+)?$\")\n if not version_pattern.match(args.version):\n print(\n f\"❌ Invalid version format: {args.version}. \"\n \"Expected: X.Y.Z or X.Y.Z-suffix\",\n file=sys.stderr,\n )\n return 1\n\n success = update_sdk_ref_default(args.version, dry_run=args.dry_run)\n return 0 if success else 1\n\n\nif __name__ == \"__main__\":\n sys.exit(main())\n" }, { "path": ".github/workflows/README-RELEASE.md", "content": "# Release Automation Workflows\n\nThis document describes the automated release workflows for the OpenHands Software Agent SDK.\n\n## Overview\n\nThe release process has been automated with three GitHub Actions workflows:\n\n1. **prepare-release.yml** - Prepares a release PR with version updates\n2. **pypi-release.yml** - Automatically publishes packages to PyPI when a release is created\n3. **release-binaries.yml** - Builds and smoke-tests multi-arch agent-server binaries\n on releases and main pushes; release runs also attach binaries to the release\n\n## How to Create a New Release\n\n### Step 1: Trigger the Prepare Release Workflow\n\n1. Go to the [Actions tab](https://github.com/OpenHands/software-agent-sdk/actions)\n2. Select **\"Prepare Release\"** workflow from the left sidebar\n3. Click **\"Run workflow\"** button\n4. Enter the version number (e.g., `1.2.3`) - must be in format `X.Y.Z`\n5. Click **\"Run workflow\"**\n\nThe workflow will automatically:\n- ✅ Create a new branch named `rel-X.Y.Z`\n- ✅ Update all package versions using `make set-package-version`\n- ✅ Commit the changes\n- ✅ Push the branch\n- ✅ Create a PR with labels `integration-tests` and `test-examples`\n\n### Step 2: Review the PR\n\nThe created PR will include a checklist. Complete the following:\n\n- [ ] Fix any deprecation deadlines if they exist\n- [ ] Verify integration tests pass (triggered by `integration-tests` label)\n- [ ] Verify example checks pass (triggered by `test-examples` label)\n- [ ] Review and approve the PR\n\n### Step 3: Create the GitHub Release\n\n1. Go to [Releases](https://github.com/OpenHands/software-agent-sdk/releases/new)\n2. Click **\"Draft a new release\"**\n3. Configure the release:\n - **Tag**: `vX.Y.Z` (must match the version)\n - **Branch**: `rel-X.Y.Z` (the branch created by the workflow)\n - **Previous tag**: Select the previous release version\n4. Click **\"Generate release notes\"** to auto-generate the changelog\n5. Review and edit the release notes as needed\n6. Click **\"Publish release\"**\n\n### Step 4: PyPI Publication (Automated)\n\nOnce the release is published, the **pypi-release.yml** workflow will automatically:\n- ✅ Build all packages (openhands-sdk, openhands-tools, openhands-workspace, openhands-agent-server)\n- ✅ Publish them to PyPI\n\nYou can monitor the progress in the [Actions tab](https://github.com/OpenHands/software-agent-sdk/actions/workflows/pypi-release.yml).\n\n### Step 4b: Release Binaries + Docker Smoke Test (Automated)\n\nIn parallel with the PyPI workflow, **release-binaries.yml** also fires on `release: published`.\nIt also runs on every push to `main` as ongoing smoke coverage. It:\n\n- ✅ Builds the agent-server PyInstaller binary on a 4-runner matrix\n (linux x86_64/arm64, macOS x86_64/arm64) and smoke-tests each\n- ✅ Generates a combined `SHA256SUMS` and attaches all artifacts to the GitHub\n release as `agent-server---` on release/manual runs\n- ✅ Verifies that the multi-arch Docker manifest\n `ghcr.io/openhands/agent-server:-` published by\n `server.yml` covers both `linux/amd64` and `linux/arm64` for every variant\n (`python`, `java`, `golang`)\n- ✅ Pulls each variant on each architecture with `--platform=linux/`,\n boots the container, and asserts `/health` responds\n\nOn `push` events, `` is the 7-character commit SHA and binaries\nremain as workflow artifacts only. On release/manual runs, `` is the\nrelease version and the binaries are uploaded to the GitHub release.\n\n#### Build time / runner expectations\n\n| Stage | Runtime (typical) | Runners |\n|---|---|---|\n| Binary builds (4-way matrix, parallel) | ~10–15 min on Linux, ~12–18 min on macOS | `ubuntu-24.04`, `ubuntu-24.04-arm`, `macos-13`, `macos-14` |\n| `publish-binaries` (download + checksum + upload) | ~1–2 min | `ubuntu-24.04` |\n| `docker-smoke-test` (6-way matrix, parallel) | Up to 45 min (mostly polling for the docker images) | `ubuntu-24.04` for amd64, `ubuntu-24.04-arm` for arm64 |\n\n#### QEMU / buildx requirements\n\nThe smoke test does **not** require QEMU: each (variant, arch) job runs on a\nrunner whose architecture matches `--platform=linux/`, so containers run\nnatively. We do still set up Docker Buildx so we can call\n`docker buildx imagetools inspect` on the multi-arch manifest list.\n\nThe wait window for the multi-arch manifest is 45 min — long enough to absorb\nthe full `server.yml` matrix runtime (~25–30 min for `build-and-push-image` +\n`merge-manifests`) when this workflow races the corresponding `server.yml` run\nfor a release tag or main-branch push.\n\nIf the matching manifest is already in GHCR, the wait step exits immediately.\n\n### Step 5: Version Bump PRs (Automated)\n\nAfter successful PyPI publication, the workflow will automatically create PRs to update SDK versions in downstream repositories:\n\n- **[OpenHands](https://github.com/All-Hands-AI/OpenHands)** - Updates `openhands-sdk`, `openhands-tools`, and `openhands-agent-server` versions\n- **[OpenHands-CLI](https://github.com/All-Hands-AI/openhands-cli)** - Updates `openhands-sdk` and `openhands-tools` versions\n\nThese PRs will:\n- Be created automatically with branch name `bump-sdk-X.Y.Z`\n- Include links back to the SDK release\n- Need to be reviewed and merged by the respective repository maintainers\n\n### Step 6: Post-Release Tasks\n\n- [ ] Merge the release PR to main\n- [ ] Review and merge the auto-created version bump PRs in OpenHands and OpenHands-CLI\n- [ ] Run evaluation on OpenHands Index (manual step)\n- [ ] Announce the release\n\n## Manual PyPI Release (If Needed)\n\nIf you need to manually trigger the PyPI release workflow:\n\n1. Go to the [Actions tab](https://github.com/OpenHands/software-agent-sdk/actions)\n2. Select **\"Publish all OpenHands packages (uv)\"** workflow\n3. Click **\"Run workflow\"**\n4. Select the branch/tag you want to publish from\n5. Click **\"Run workflow\"**\n\n## Workflow Files\n\n- `.github/workflows/prepare-release.yml` - Automated release preparation\n- `.github/workflows/pypi-release.yml` - PyPI package publication\n- `.github/workflows/release-binaries.yml` - Multi-arch binary publishing and\n docker manifest smoke test on releases and main pushes\n\n## Troubleshooting\n\n### Version Format Error\n\nIf you get a version format error, ensure you're using the format `X.Y.Z` (e.g., `1.2.3`), not `vX.Y.Z`.\n\n### PR Creation Failed\n\nIf the PR creation fails, check:\n- The branch doesn't already exist\n- You have proper permissions\n- The `GITHUB_TOKEN` has sufficient permissions\n\n### PyPI Publication Failed\n\nIf PyPI publication fails:\n- Check that the `PYPI_TOKEN_OPENHANDS` secret is properly configured\n- Verify the version doesn't already exist on PyPI\n- Check the workflow logs for specific error messages\n\n### Release Binaries Failed\n\nIf `release-binaries.yml` fails:\n- **Binary build failure**: re-run the failed matrix job; PyInstaller flakes are\n rare but possible. If it persists, the issue is likely in `agent-server.spec`.\n- **`docker-smoke-test` timed out waiting for the manifest**: `server.yml` did\n not publish multi-arch images for the matching release tag or commit SHA.\n Check that workflow's corresponding run and re-trigger if needed.\n- **`/health` never responded**: open the failing job; the cleanup trap dumps\n the last 100 lines of `docker logs` for the container.\n- Release/manual runs can be re-run against an existing tag via\n `workflow_dispatch` with the `release_tag` input (e.g. `v1.20.1`);\n `gh release upload --clobber` makes this safe.\n\n## Previous Manual Process\n\nFor reference, the previous manual release checklist was:\n\n- [ ] Checkout SDK repo, use `make set-package-version version=x.x.x` to set the version\n- [ ] Push to a branch like `rel-x.x.x` and start a PR\n- [ ] Fix any \"deprecation deadlines\" if they exist\n- [ ] Tag \"integration-tests\" and make sure integration test all pass\n- [ ] Tag \"test-examples\" and make sure example checks all pass\n- [ ] Draft a new release\n- [ ] Use workflow to publish to PyPI on tag `v1.X.X`\n- [ ] Evaluation on OpenHands Index\n\nMost of these steps are now automated!\n" }, { "path": ".github/workflows/agent-server-rest-api-breakage.yml", "content": "---\nname: REST API breakage checks\n\non:\n push:\n branches: [main]\n pull_request:\n branches: [main]\n\njobs:\n agent-server-rest-api:\n name: REST API (OpenAPI)\n runs-on: ubuntu-latest\n permissions:\n contents: read\n pull-requests: write\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with:\n fetch-depth: 0\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n\n - name: Install workspace deps (dev)\n run: uv sync --frozen --group dev\n\n - name: Install oasdiff\n run: |\n curl -L https://raw.githubusercontent.com/oasdiff/oasdiff/main/install.sh | sh -s -- -b /usr/local/bin\n oasdiff --version\n\n - name: Run agent server REST API breakage check\n id: api_breakage\n # Let this step fail so CI is visibly red on breakage.\n # Later reporting steps still run because they use if: always().\n run: |\n uv run --with packaging python .github/scripts/check_agent_server_rest_api_breakage.py 2>&1 | tee api-breakage.log\n exit_code=${PIPESTATUS[0]}\n echo \"exit_code=${exit_code}\" >> \"$GITHUB_OUTPUT\"\n exit \"${exit_code}\"\n\n - name: Write REST API breakage summary\n if: ${{ always() }}\n env:\n EXIT_CODE: ${{ steps.api_breakage.outputs.exit_code }}\n IS_FORK: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository }}\n LOG_PATH: api-breakage.log\n RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}\n run: |\n python3 <<'PY' >> \"$GITHUB_STEP_SUMMARY\"\n import os\n from pathlib import Path\n\n exit_code = int(os.environ.get('EXIT_CODE', '0') or '0')\n is_fork = os.environ.get('IS_FORK', 'false') == 'true'\n run_url = os.environ['RUN_URL']\n status = '✅ **PASSED**' if exit_code == 0 else '❌ **FAILED**'\n\n print(f'## REST API breakage checks (OpenAPI) — {status}')\n print()\n print(f\"**Result:** {status}\")\n if exit_code != 0:\n print()\n print('> ⚠️ Breaking REST API changes or policy violations detected.')\n print()\n\n if is_fork:\n print(\n '_Fork PR detected: sticky PR comment was skipped because '\n 'the GitHub token is read-only for `pull_request` workflows '\n 'from forks._'\n )\n print()\n\n if exit_code != 0:\n try:\n log = Path(os.environ['LOG_PATH']).read_text()\n except Exception as exc:\n log = f'Unable to read log file: {exc}'\n\n excerpt = log[:1000].replace('```', '``\\\\`')\n print('
Log excerpt (first 1000 characters)')\n print()\n print('```text')\n print(excerpt)\n print('```')\n print()\n print('
')\n print()\n\n print(f'[Action log]({run_url})')\n PY\n\n - name: Post REST API breakage report to PR\n if: ${{ always() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }}\n uses: actions/github-script@v9\n env:\n EXIT_CODE: ${{ steps.api_breakage.outputs.exit_code }}\n LOG_PATH: api-breakage.log\n with:\n script: |\n const fs = require('fs');\n\n const marker = '';\n const exitCode = Number(process.env.EXIT_CODE || '0');\n const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;\n const status = exitCode === 0 ? '✅ **PASSED**' : '❌ **FAILED**';\n\n let body = `${marker}\\n## REST API breakage checks (OpenAPI) — ${status}\\n\\n**Result:** ${status}\\n`;\n\n if (exitCode !== 0) {\n body += `\\n> ⚠️ Breaking REST API changes or policy violations detected.\\n`;\n let log = '';\n try {\n log = fs.readFileSync(process.env.LOG_PATH, 'utf8');\n } catch (e) {\n log = `Unable to read log file: ${e}`;\n }\n\n const excerpt = log.slice(0, 1000).replace(/```/g, '``\\\\`');\n body += `\\n
Log excerpt (first 1000 characters)\\n\\n\\`\\`\\`text\\n${excerpt}\\n\\`\\`\\`\\n\\n
\\n`;\n }\n\n body += `\\n[Action log](${runUrl})\\n`;\n\n const { owner, repo } = context.repo;\n const issue_number = context.issue.number;\n const { data: comments } = await github.rest.issues.listComments({\n owner,\n repo,\n issue_number,\n per_page: 100,\n });\n\n const existing = comments.find((c) => c.body && c.body.includes(marker));\n if (existing) {\n await github.rest.issues.updateComment({\n owner,\n repo,\n comment_id: existing.id,\n body,\n });\n } else {\n await github.rest.issues.createComment({\n owner,\n repo,\n issue_number,\n body,\n });\n }\n" }, { "path": ".github/workflows/api-breakage.yml", "content": "---\nname: Python API breakage checks\n\non:\n push:\n branches: [main]\n pull_request:\n branches: [main]\n\njobs:\n sdk-api:\n name: Python API\n runs-on: ubuntu-latest\n permissions:\n contents: read\n pull-requests: write\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with:\n fetch-depth: 0\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n - name: Install workspace deps (dev)\n run: uv sync --frozen --group dev\n - name: Run Python API breakage check\n id: api_breakage\n # Let this step fail so CI is visibly red on breakage.\n # Later reporting steps still run because they use if: always().\n env:\n ACP_VERSION_CHECK_BASE_REF: ${{ github.event_name == 'pull_request' && github.base_ref || github.event.before }}\n ACP_VERSION_CHECK_SKIP: ${{ github.event_name == 'pull_request' && contains(github.event.pull_request.body || '', 'skip-acp-check') \n }}\n run: |\n uv run python .github/scripts/check_sdk_api_breakage.py 2>&1 | tee api-breakage.log\n exit_code=${PIPESTATUS[0]}\n echo \"exit_code=${exit_code}\" >> \"$GITHUB_OUTPUT\"\n exit \"${exit_code}\"\n - name: Write API breakage summary\n if: ${{ always() }}\n env:\n EXIT_CODE: ${{ steps.api_breakage.outputs.exit_code }}\n IS_FORK: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository }}\n LOG_PATH: api-breakage.log\n RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}\n run: |\n python3 <<'PY' >> \"$GITHUB_STEP_SUMMARY\"\n import os\n from pathlib import Path\n\n exit_code = int(os.environ.get('EXIT_CODE', '0') or '0')\n is_fork = os.environ.get('IS_FORK', 'false') == 'true'\n run_url = os.environ['RUN_URL']\n status = '✅ **PASSED**' if exit_code == 0 else '❌ **FAILED**'\n\n print(f'## Python API breakage checks — {status}')\n print()\n print(f\"**Result:** {status}\")\n if exit_code != 0:\n print()\n print('> ⚠️ Breaking API changes or policy violations detected.')\n print()\n\n if is_fork:\n print(\n '_Fork PR detected: sticky PR comment was skipped because '\n 'the GitHub token is read-only for `pull_request` workflows '\n 'from forks._'\n )\n print()\n\n if exit_code != 0:\n try:\n log = Path(os.environ['LOG_PATH']).read_text()\n except Exception as exc:\n log = f'Unable to read log file: {exc}'\n\n excerpt = log[:1000].replace('```', '``\\\\`')\n print('
Log excerpt (first 1000 characters)')\n print()\n print('```text')\n print(excerpt)\n print('```')\n print()\n print('
')\n print()\n\n print(f'[Action log]({run_url})')\n PY\n\n - name: Post API breakage report to PR\n if: ${{ always() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository }}\n uses: actions/github-script@v9\n env:\n EXIT_CODE: ${{ steps.api_breakage.outputs.exit_code }}\n LOG_PATH: api-breakage.log\n with:\n script: |\n const fs = require('fs');\n\n const marker = '';\n const exitCode = Number(process.env.EXIT_CODE || '0');\n const runUrl = `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;\n const status = exitCode === 0 ? '✅ **PASSED**' : '❌ **FAILED**';\n\n let body = `${marker}\\n## Python API breakage checks — ${status}\\n\\n**Result:** ${status}\\n`;\n\n if (exitCode !== 0) {\n body += `\\n> ⚠️ Breaking API changes or policy violations detected.\\n`;\n let log = '';\n try {\n log = fs.readFileSync(process.env.LOG_PATH, 'utf8');\n } catch (e) {\n log = `Unable to read log file: ${e}`;\n }\n\n const excerpt = log.slice(0, 1000).replace(/```/g, '``\\\\`');\n body += `\\n
Log excerpt (first 1000 characters)\\n\\n\\`\\`\\`text\\n${excerpt}\\n\\`\\`\\`\\n\\n
\\n`;\n }\n\n body += `\\n[Action log](${runUrl})\\n`;\n\n const { owner, repo } = context.repo;\n const issue_number = context.issue.number;\n const { data: comments } = await github.rest.issues.listComments({\n owner,\n repo,\n issue_number,\n per_page: 100,\n });\n\n const existing = comments.find((c) => c.body && c.body.includes(marker));\n if (existing) {\n await github.rest.issues.updateComment({\n owner,\n repo,\n comment_id: existing.id,\n body,\n });\n } else {\n await github.rest.issues.createComment({\n owner,\n repo,\n issue_number,\n body,\n });\n }\n" }, { "path": ".github/workflows/api-compliance-runner.yml", "content": "---\nname: API Compliance Tests\n\non:\n pull_request:\n types: [labeled]\n workflow_dispatch:\n inputs:\n reason:\n description: Reason for running compliance tests\n required: true\n patterns:\n description: Comma-separated patterns to test (empty = all)\n required: false\n models:\n description: Comma-separated model IDs (empty = all defaults)\n required: false\n\nenv:\n # Default models to test (matches DEFAULT_MODELS in run_compliance.py)\n DEFAULT_MODELS: claude-sonnet-4-5,gpt-5.2,gemini-3.1-pro\n\njobs:\n run-compliance-tests:\n # Only run on api-compliance-test label or workflow_dispatch\n if: |\n github.event_name == 'workflow_dispatch' ||\n (github.event_name == 'pull_request' && github.event.label.name == 'api-compliance-test')\n runs-on: ubuntu-latest\n permissions:\n contents: read\n pull-requests: write\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n with:\n repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}\n ref: ${{ github.event.pull_request.head.sha || github.ref }}\n persist-credentials: false\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n version: latest\n python-version: '3.13'\n\n - name: Install dependencies\n run: uv sync --dev\n\n - name: Determine test parameters\n id: params\n run: |\n # Use input values or defaults\n if [ \"${{ github.event_name }}\" = \"workflow_dispatch\" ]; then\n PATTERNS=\"${{ github.event.inputs.patterns }}\"\n MODELS=\"${{ github.event.inputs.models }}\"\n else\n PATTERNS=\"\"\n MODELS=\"\"\n fi\n\n # Build command args\n ARGS=\"\"\n if [ -n \"$PATTERNS\" ]; then\n ARGS=\"$ARGS --patterns $PATTERNS\"\n fi\n if [ -n \"$MODELS\" ]; then\n ARGS=\"$ARGS --models $MODELS\"\n else\n ARGS=\"$ARGS --models $DEFAULT_MODELS\"\n fi\n\n echo \"args=$ARGS\" >> $GITHUB_OUTPUT\n\n - name: Run API compliance tests\n id: compliance\n env:\n LLM_API_KEY: ${{ secrets.LLM_API_KEY_EVAL }}\n LLM_BASE_URL: https://llm-proxy.eval.all-hands.dev\n GITHUB_RUN_ID: ${{ github.run_id }}\n run: |\n uv run python tests/integration/api_compliance/run_compliance.py \\\n ${{ steps.params.outputs.args }} \\\n --output-dir compliance-results/\n continue-on-error: true # Tests may \"fail\" but that's expected\n\n - name: Upload results\n uses: actions/upload-artifact@v7\n with:\n name: compliance-results\n path: compliance-results/\n retention-days: 30\n\n - name: Post results to PR\n if: github.event_name == 'pull_request'\n uses: actions/github-script@v9\n with:\n script: |\n const fs = require('fs');\n const path = require('path');\n\n // Find the report directory\n const resultsDir = 'compliance-results';\n const dirs = fs.readdirSync(resultsDir);\n if (dirs.length === 0) {\n console.log('No results found');\n return;\n }\n\n const latestDir = path.join(resultsDir, dirs[0]);\n const reportPath = path.join(latestDir, 'compliance_report.md');\n\n if (!fs.existsSync(reportPath)) {\n console.log('Report not found at', reportPath);\n return;\n }\n\n let report = fs.readFileSync(reportPath, 'utf8');\n\n // Truncate if too long\n if (report.length > 60000) {\n report = report.substring(0, 60000) + '\\n\\n... (truncated)';\n }\n\n await github.rest.issues.createComment({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: context.payload.pull_request.number,\n body: report\n });\n" }, { "path": ".github/workflows/assign-reviews.yml", "content": "---\n# To set this up:\n# 1. Change the name below to something relevant to your task\n# 2. Modify the \"env\" section below with your prompt\n# 3. Add your LLM_API_KEY to the repository secrets\n# 4. Commit this file to your repository\n# 5. Trigger the workflow manually or set up a schedule\nname: Assign Reviews\n\non:\n # Manual trigger\n workflow_dispatch:\n # Scheduled trigger (disabled by default, uncomment and customize as needed)\n schedule:\n # Run at 12 PM UTC every day\n - cron: 0 12 * * *\n\npermissions:\n contents: write\n pull-requests: write\n issues: write\n\njobs:\n run-task:\n # Only run scheduled jobs in the main repository, not in forks\n if: github.repository == 'OpenHands/software-agent-sdk' || github.event_name == 'workflow_dispatch'\n runs-on: ubuntu-24.04\n env:\n # Configuration (modify these values as needed)\n AGENT_SCRIPT_URL: https://raw.githubusercontent.com/OpenHands/agent-sdk/main/examples/03_github_workflows/01_basic_action/agent_script.py\n # Provide either PROMPT_LOCATION (URL/file) OR PROMPT_STRING (direct text), not both\n # Option 1: Use a URL or file path for the prompt\n PROMPT_LOCATION: ''\n # PROMPT_LOCATION: 'https://example.com/prompts/maintenance.txt'\n # Option 2: Use direct text for the prompt\n PROMPT_STRING: >\n Use GITHUB_TOKEN and the github API to organize open pull requests and issues in the repo.\n Read the sections below in order, and perform each in order. Do NOT take action\n on the same issue or PR twice.\n\n # Issues with needs-info - Check for OP Response\n\n Find all open issues that have the \"needs-info\" label. For each issue:\n 1. Identify the original poster (issue author)\n 2. Check if there are any comments from the original poster AFTER the \"needs-info\" label was added\n 3. To determine when the label was added, use: GET /repos/{owner}/{repo}/issues/{issue_number}/timeline\n and look for \"labeled\" events with the label \"needs-info\"\n 4. If the original poster has commented after the label was added:\n - Remove the \"needs-info\" label\n - Add the \"needs-triage\" label\n # Issues with needs-triage\n\n Find all open issues that have the \"needs-triage\" label. For each issue that has been in this state for more than 2 days:\n 1. First, check if the issue has already been triaged by verifying it does NOT have:\n - The \"enhancement\" label\n - Any \"priority\" label (priority:low, priority:medium, priority:high, etc.)\n 2. If the issue has already been triaged (has enhancement or priority label), remove the \"needs-triage\" label\n 3. For issues that have NOT been triaged yet:\n - Read the issue description and comments\n - Check if it is a bug report, feature request, or question and add the appropriate label\n - If it is a bug report and it does not have a priority label\n * Read the MAINTAINERS file in the repository root to get the list of maintainers\n * Extract all usernames from lines starting with \"- @\" and join them with spaces, each prefixed with @\n (e.g., if the file contains \"- @user1\" and \"- @user2\", format as \"@user1 @user2\")\n * Tag ALL maintainers with: \"[Automatic Post]: This issue has been waiting for triage. , could you\n please take a look and add the appropriate priority label when you have a chance?\"\n (Replace with the formatted list from the previous step)\n\n # Need Reviewer Action\n\n Find all open PRs where:\n 1. The PR is waiting for review (there are no open review comments or change requests)\n 2. The PR is in a \"clean\" state (CI passing, no merge conflicts)\n 3. The PR is not marked as draft (draft: false)\n 4. The PR has had no activity (comments, commits, reviews) for more than 3 days.\n\n In this case, send a message to the reviewers:\n [Automatic Post]: This PR seems to be currently waiting for review.\n {reviewer_names}, could you please take a look when you have a chance?\n\n # Need Author Action\n\n Find all open PRs where the most recent change or comment was made on the pull\n request more than 5 days ago (use 14 days if the PR is marked as draft).\n\n And send a message to the author:\n\n [Automatic Post]: It has been a while since there was any activity on this PR.\n {author}, are you still working on it? If so, please go ahead, if not then\n please request review, close it, or request that someone else follow up.\n\n # Need Reviewers\n\n Find all open pull requests that TRULY have NO reviewers assigned. To do this correctly:\n\n 1. Use the GitHub API to fetch PR details: GET /repos/{owner}/{repo}/pulls/{pull_number}\n 2. Check the \"requested_reviewers\" and \"requested_teams\" arrays\n 3. ALSO check for submitted reviews: GET /repos/{owner}/{repo}/pulls/{pull_number}/reviews\n 4. A PR needs reviewers ONLY if ALL of these are true:\n - The \"requested_reviewers\" array is empty (no pending review requests)\n - The \"requested_teams\" array is empty (no pending team review requests)\n - The reviews array is empty (no reviews have been submitted yet)\n 5. IMPORTANT: If ANY of these has entries, SKIP this PR - it already has or had reviewers!\n\n Example API responses showing a PR that DOES NOT need reviewers (skip this):\n\n Case 1 - Has requested reviewers:\n GET /pulls/{number}: {\"requested_reviewers\": [{\"login\": \"someuser\"}], \"requested_teams\": []}\n\n Case 2 - Has submitted reviews (even if requested_reviewers is empty):\n GET /pulls/{number}: {\"requested_reviewers\": [], \"requested_teams\": []}\n GET /pulls/{number}/reviews: [{\"user\": {\"login\": \"someuser\"}, \"state\": \"COMMENTED\"}]\n\n Example API response showing a PR that DOES need reviewers (process this):\n GET /pulls/{number}: {\"requested_reviewers\": [], \"requested_teams\": []}\n GET /pulls/{number}/reviews: []\n\n Additional criteria for PRs that need reviewers:\n 1. Are not marked as draft (draft: false)\n 2. Were created more than 1 day ago\n 3. CI is passing and there are no merge conflicts\n\n For each PR that truly has NO reviewers:\n 1) Read git blame for changed files to identify recent, active contributors.\n 2) From those blame-derived candidates, ONLY consider maintainers who are repository collaborators with write access or higher. Verify that\n with the GitHub API before requesting review:\n - Preferred: GET /repos/{owner}/{repo}/collaborators (no permission filter). Filter client-side using either:\n role_name in [\"write\", \"maintain\", \"admin\"] OR permissions.push || permissions.admin. Note: paginate if > 30 collaborators.\n - Alternative: GET /repos/{owner}/{repo}/collaborators/{username}/permission and accept if permission in {push, maintain, admin}.\n 3) If one or more blame-derived maintainers qualify, request review from exactly one of them. Prefer the maintainer with the lowest current\n review load. Add this message:\n\n [Automatic Post]: I have assigned {reviewer} as a reviewer based on git blame information.\n Thanks in advance for the help!\n\n 4) If no blame-derived maintainer qualifies, read the MAINTAINERS file in the repository root. Parse usernames from lines starting with\n \"- @username\" and treat that file as the canonical list of active maintainers.\n 5) From that MAINTAINERS list, keep only users who still have write access or higher via the GitHub API, exclude the PR author, and request\n review from exactly one of them, again preferring the maintainer with the lowest current review load. Add this message:\n\n [Automatic Post]: I have assigned {reviewer} as a reviewer based on the repository MAINTAINERS file.\n Thanks in advance for the help!\n\n 6) If neither path yields a qualified maintainer, do not request review from anyone and do not fall back to a broader collaborator pool.\n\n LLM_MODEL: litellm_proxy/claude-sonnet-4-5-20250929\n LLM_BASE_URL: https://llm-proxy.app.all-hands.dev\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: '3.13'\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n\n - name: Install OpenHands dependencies\n run: |\n # Install OpenHands SDK and tools from git repository\n uv pip install --system \"openhands-sdk @ git+https://github.com/OpenHands/agent-sdk.git@main#subdirectory=openhands-sdk\"\n uv pip install --system \"openhands-tools @ git+https://github.com/OpenHands/agent-sdk.git@main#subdirectory=openhands-tools\"\n\n - name: Check required configuration\n env:\n LLM_API_KEY: ${{ secrets.LLM_API_KEY }}\n run: |\n if [ -z \"$LLM_API_KEY\" ]; then\n echo \"Error: LLM_API_KEY secret is not set.\"\n exit 1\n fi\n\n # Check that exactly one of PROMPT_LOCATION or PROMPT_STRING is set\n if [ -n \"$PROMPT_LOCATION\" ] && [ -n \"$PROMPT_STRING\" ]; then\n echo \"Error: Both PROMPT_LOCATION and PROMPT_STRING are set.\"\n echo \"Please provide only one in the env section of the workflow file.\"\n exit 1\n fi\n\n if [ -z \"$PROMPT_LOCATION\" ] && [ -z \"$PROMPT_STRING\" ]; then\n echo \"Error: Neither PROMPT_LOCATION nor PROMPT_STRING is set.\"\n echo \"Please set one in the env section of the workflow file.\"\n exit 1\n fi\n\n if [ -n \"$PROMPT_LOCATION\" ]; then\n echo \"Prompt location: $PROMPT_LOCATION\"\n else\n echo \"Using inline PROMPT_STRING (${#PROMPT_STRING} characters)\"\n fi\n echo \"LLM model: $LLM_MODEL\"\n if [ -n \"$LLM_BASE_URL\" ]; then\n echo \"LLM base URL: $LLM_BASE_URL\"\n fi\n\n - name: Run task\n env:\n LLM_API_KEY: ${{ secrets.LLM_API_KEY }}\n GITHUB_TOKEN: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}\n PYTHONPATH: ''\n run: |\n echo \"Running agent script: $AGENT_SCRIPT_URL\"\n\n # Download script if it's a URL\n if [[ \"$AGENT_SCRIPT_URL\" =~ ^https?:// ]]; then\n echo \"Downloading agent script from URL...\"\n curl -sSL \"$AGENT_SCRIPT_URL\" -o /tmp/agent_script.py\n AGENT_SCRIPT_PATH=\"/tmp/agent_script.py\"\n else\n AGENT_SCRIPT_PATH=\"$AGENT_SCRIPT_URL\"\n fi\n\n # Run with appropriate prompt argument\n if [ -n \"$PROMPT_LOCATION\" ]; then\n echo \"Using prompt from: $PROMPT_LOCATION\"\n uv run python \"$AGENT_SCRIPT_PATH\" \"$PROMPT_LOCATION\"\n else\n echo \"Using PROMPT_STRING (${#PROMPT_STRING} characters)\"\n uv run python \"$AGENT_SCRIPT_PATH\"\n fi\n\n - name: Upload logs as artifact\n uses: actions/upload-artifact@v7\n if: always()\n with:\n name: openhands-task-logs\n path: |\n *.log\n output/\n retention-days: 7\n" }, { "path": ".github/workflows/auto-label-issues.yml", "content": "---\nname: Auto-label New Issues\n\non:\n issues:\n types: [opened]\n\npermissions:\n issues: write\n\njobs:\n add-triage-label:\n runs-on: ubuntu-latest\n steps:\n - name: Add needs-triage label\n uses: actions/github-script@v9\n with:\n github-token: ${{ secrets.GITHUB_TOKEN }}\n script: |\n // Get the issue details\n const issue = context.payload.issue;\n const labels = issue.labels.map(label => label.name);\n\n // Check if issue has already been triaged\n const hasEnhancement = labels.includes('enhancement');\n const hasPriority = labels.some(label => label.startsWith('priority'));\n\n // Only add needs-triage if not already triaged\n if (!hasEnhancement && !hasPriority) {\n await github.rest.issues.addLabels({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: context.issue.number,\n labels: ['needs-triage']\n });\n }\n" }, { "path": ".github/workflows/cancel-eval.yml", "content": "---\nname: Cancel Eval\n\nrun-name: Cancel Eval (${{ inputs.run_id }})\n\non:\n workflow_dispatch:\n inputs:\n run_id:\n description: Workflow run ID to cancel\n required: true\n type: string\n reason:\n description: Reason for cancellation\n required: false\n type: string\n\nenv:\n EVAL_REPO: OpenHands/evaluation\n EVAL_WORKFLOW: kill-eval-job.yml\n\npermissions:\n contents: read\n\njobs:\n cancel-eval:\n runs-on: ubuntu-latest\n steps:\n - name: Cancel evaluation job\n env:\n DISPATCH_TOKEN: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_EVAL_DISPATCH }}\n RUN_ID: ${{ github.event.inputs.run_id }}\n REASON: ${{ github.event.inputs.reason }}\n run: |-\n set -euo pipefail\n\n if [ -z \"$DISPATCH_TOKEN\" ]; then\n echo \"Missing dispatch token\" >&2\n exit 1\n fi\n\n echo \"Canceling evaluation workflow run: $RUN_ID\"\n\n # Dispatch kill workflow in evaluation repo\n PAYLOAD=$(jq -n \\\n --arg ref \"main\" \\\n --arg run_id \"$RUN_ID\" \\\n --arg reason \"$REASON\" \\\n '{ref: $ref, inputs: {run_id: $run_id, reason: $reason}}')\n\n RESPONSE=$(curl -sS -o /tmp/dispatch.out -w \"%{http_code}\" -X POST \\\n -H \"Authorization: token $DISPATCH_TOKEN\" \\\n -H \"Accept: application/vnd.github+json\" \\\n -d \"$PAYLOAD\" \\\n \"https://api.github.com/repos/${EVAL_REPO}/actions/workflows/${EVAL_WORKFLOW}/dispatches\")\n\n if [ \"$RESPONSE\" != \"204\" ]; then\n echo \"Dispatch failed (status $RESPONSE):\" >&2\n cat /tmp/dispatch.out >&2\n exit 1\n fi\n\n echo \"Cancellation dispatched successfully for run: $RUN_ID\"\n" }, { "path": ".github/workflows/check-docstrings.yml", "content": "---\n# .github/workflows/check-docstrings.yml\nname: Check Docstrings\n\non:\n push:\n branches: [main]\n pull_request:\n branches: ['**']\n\njobs:\n check-docstrings:\n runs-on: ubuntu-24.04\n\n steps:\n - name: Checkout code\n uses: actions/checkout@v6\n\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: '3.13'\n\n - name: Check docstring formatting\n run: python .github/scripts/check_docstrings.py\n" }, { "path": ".github/workflows/check-documented-examples.yml", "content": "---\nname: '[Optional] Docs example'\n\non:\n pull_request:\n branches:\n - '**'\n paths:\n - examples/**/*.py\n - '!examples/03_github_workflows/**'\n - '!examples/04_llm_specific_tools/**'\n - .github/workflows/check-documented-examples.yml\n - .github/scripts/check_documented_examples.py\n workflow_dispatch:\n\npermissions:\n contents: read\n pull-requests: read\n\njobs:\n check-examples:\n runs-on: ubuntu-latest\n steps:\n - name: Checkout agent-sdk repository\n uses: actions/checkout@v6\n with:\n fetch-depth: 0\n\n - name: Checkout docs repository (try feature branch)\n uses: actions/checkout@v6\n continue-on-error: true\n id: checkout-feature\n with:\n repository: OpenHands/docs\n path: docs\n fetch-depth: 0\n ref: ${{ github.head_ref || github.ref_name }}\n\n - name: Checkout docs repository (fallback to main)\n if: steps.checkout-feature.outcome == 'failure'\n uses: actions/checkout@v6\n with:\n repository: OpenHands/docs\n path: docs\n fetch-depth: 0\n ref: main\n\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: '3.13'\n\n - name: Check documented examples\n env:\n DOCS_PATH: ${{ github.workspace }}/docs\n shell: bash\n run: |\n set -euo pipefail\n python .github/scripts/check_documented_examples.py\n" }, { "path": ".github/workflows/check-duplicate-examples.yml", "content": "---\nname: Check duplicate example numbers\n\non:\n pull_request:\n branches:\n - '**'\n paths:\n - examples/**\n - .github/workflows/check-duplicate-examples.yml\n - .github/scripts/check_duplicate_example_numbers.py\n push:\n branches:\n - main\n paths:\n - examples/**\n workflow_dispatch:\n\npermissions:\n contents: read\n\njobs:\n check-duplicates:\n runs-on: ubuntu-latest\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: '3.13'\n\n - name: Check for duplicate example numbers\n run: python .github/scripts/check_duplicate_example_numbers.py\n" }, { "path": ".github/workflows/condenser-runner.yml", "content": "---\nname: Run Condenser Tests\n\non:\n # Use pull_request_target to access secrets even on fork PRs\n # This is safe because we only run when the 'condenser-test' label is added by a maintainer\n pull_request_target:\n types:\n - labeled\n workflow_dispatch:\n inputs:\n reason:\n description: Reason for manual trigger\n required: true\n default: ''\n\nenv:\n N_PROCESSES: 2 # Fewer parallel processes for condenser tests (only 2 LLMs)\n\njobs:\n post-initial-comment:\n if: >\n github.event_name == 'pull_request_target' &&\n github.event.label.name == 'condenser-test'\n runs-on: ubuntu-latest\n permissions:\n pull-requests: write\n steps:\n - name: Comment on PR\n uses: KeisukeYamashita/create-comment@v1\n with:\n unique: false\n comment: |\n Hi! I started running the condenser tests on your PR. You will receive a comment with the results shortly.\n\n Note: These are non-blocking tests that validate condenser functionality across different LLMs.\n\n run-condenser-tests:\n # Security: Only run when condenser-test label is present or via workflow_dispatch\n # This prevents automatic execution on fork PRs without maintainer approval\n if: |\n always() && (\n (\n github.event_name == 'pull_request_target' &&\n github.event.label.name == 'condenser-test'\n ) ||\n github.event_name == 'workflow_dispatch'\n )\n runs-on: ubuntu-22.04\n permissions:\n contents: read\n id-token: write\n pull-requests: write\n strategy:\n matrix:\n python-version: ['3.13']\n job-config:\n # Only run against 2 LLMs for condenser tests:\n # - Claude Opus 4.5 (primary - supports thinking blocks)\n # - GPT-5.1 Codex Max (secondary - cross-LLM validation)\n - name: Claude Opus 4.5\n run-suffix: opus_condenser_run\n llm-config:\n model: litellm_proxy/anthropic/claude-opus-4-5-20251101\n extended_thinking: true\n - name: GPT-5.1 Codex Max\n run-suffix: gpt51_condenser_run\n llm-config:\n model: litellm_proxy/gpt-5.1-codex-max\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n with:\n # For pull_request_target: checkout fork PR code (requires explicit repository)\n # For other events: fallback to current repository and ref\n repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}\n ref: ${{ github.event.pull_request.head.sha || github.ref }}\n # Security: Don't persist credentials to prevent untrusted PR code from using them\n persist-credentials: false\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n version: latest\n python-version: ${{ matrix.python-version }}\n\n - name: Install Python dependencies using uv\n run: |\n uv sync --dev\n uv pip install pytest\n\n - name: Run condenser test evaluation for ${{ matrix.job-config.name }}\n env:\n LLM_CONFIG: ${{ toJson(matrix.job-config.llm-config) }}\n LLM_API_KEY: ${{ secrets.LLM_API_KEY }}\n LLM_BASE_URL: https://llm-proxy.app.all-hands.dev\n run: |\n set -eo pipefail\n\n AGENT_SDK_VERSION=$(git rev-parse --short HEAD)\n EVAL_NOTE=\"${AGENT_SDK_VERSION}_${{ matrix.job-config.run-suffix }}\"\n\n echo \"Running condenser tests only (c*.py pattern)\"\n\n uv run python tests/integration/run_infer.py \\\n --llm-config \"$LLM_CONFIG\" \\\n --num-workers $N_PROCESSES \\\n --eval-note \"$EVAL_NOTE\" \\\n --test-type condenser\n\n # get condenser tests JSON results\n RESULTS_FILE=$(find tests/integration/outputs/*${{ matrix.job-config.run-suffix }}* -name \"results.json\" -type f | head -n 1)\n echo \"RESULTS_FILE: $RESULTS_FILE\"\n if [ -f \"$RESULTS_FILE\" ]; then\n echo \"JSON_RESULTS_FILE=$RESULTS_FILE\" >> $GITHUB_ENV\n else\n echo \"JSON_RESULTS_FILE=\" >> $GITHUB_ENV\n fi\n\n - name: Wait a little bit\n run: sleep 10\n\n - name: Create archive of evaluation outputs\n run: |\n TIMESTAMP=$(date +'%y-%m-%d-%H-%M')\n cd tests/integration/outputs # Change to the outputs directory\n tar -czvf ../../../condenser_tests_${{ matrix.job-config.run-suffix }}_${TIMESTAMP}.tar.gz *${{ matrix.job-config.run-suffix }}* # Include result directories for this model\n\n - name: Upload evaluation results as artifact\n uses: actions/upload-artifact@v7\n id: upload_results_artifact\n with:\n name: condenser-test-outputs-${{ matrix.job-config.run-suffix }}-${{ github.run_id }}-${{ github.run_attempt }}\n path: condenser_tests_${{ matrix.job-config.run-suffix }}_*.tar.gz\n\n - name: Save test results for consolidation\n run: |\n # Copy the structured JSON results file for consolidation\n mkdir -p test_results_summary\n\n if [ -n \"${{ env.JSON_RESULTS_FILE }}\" ] && [ -f \"${{ env.JSON_RESULTS_FILE }}\" ]; then\n # Copy the JSON results file directly\n cp \"${{ env.JSON_RESULTS_FILE }}\" \"test_results_summary/${{ matrix.job-config.run-suffix }}_results.json\"\n echo \"✓ Copied JSON results file for consolidation\"\n else\n echo \"✗ No JSON results file found\"\n exit 1\n fi\n\n - name: Upload test results summary\n uses: actions/upload-artifact@v7\n with:\n name: test-results-${{ matrix.job-config.run-suffix }}\n path: test_results_summary/${{ matrix.job-config.run-suffix }}_results.json\n\n consolidate-results:\n needs: run-condenser-tests\n if: |\n always() && (\n (\n github.event_name == 'pull_request_target' &&\n github.event.label.name == 'condenser-test'\n ) ||\n github.event_name == 'workflow_dispatch'\n )\n runs-on: ubuntu-24.04\n permissions:\n contents: read\n pull-requests: write\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n with:\n # When using pull_request_target, explicitly checkout the PR branch\n # This ensures we use the scripts from the actual PR code\n ref: ${{ github.event.pull_request.head.sha || github.ref }}\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n version: latest\n python-version: '3.13'\n\n - name: Install Python dependencies using uv\n run: |\n uv sync --dev\n\n - name: Download all test results\n uses: actions/download-artifact@v8\n with:\n pattern: test-results-*\n merge-multiple: true\n path: all_results\n\n - name: Download all condenser test artifacts\n uses: actions/download-artifact@v8\n with:\n pattern: condenser-test-outputs-*\n path: artifacts\n\n - name: Consolidate test results\n env:\n EVENT_NAME: ${{ github.event_name }}\n PR_NUMBER: ${{ github.event.pull_request.number }}\n MANUAL_REASON: ${{ github.event.inputs.reason }}\n COMMIT_SHA: ${{ github.sha }}\n PYTHONPATH: ${{ github.workspace }}\n GITHUB_SERVER_URL: ${{ github.server_url }}\n GITHUB_REPOSITORY: ${{ github.repository }}\n GITHUB_RUN_ID: ${{ github.run_id }}\n run: |\n uv run python tests/integration/utils/consolidate_json_results.py \\\n --results-dir all_results \\\n --artifacts-dir artifacts \\\n --output-file consolidated_results.json\n\n echo \"Consolidated results generated successfully\"\n\n uv run python tests/integration/utils/generate_markdown_report.py \\\n --input-file consolidated_results.json \\\n --output-file consolidated_report.md\n\n - name: Upload consolidated report\n uses: actions/upload-artifact@v7\n with:\n name: consolidated-condenser-report\n path: consolidated_report.md\n\n - name: Create consolidated PR comment\n if: github.event_name == 'pull_request_target'\n run: |\n # Add header to clarify these are non-blocking tests\n echo \"## Condenser Test Results (Non-Blocking)\" > final_report.md\n echo \"\" >> final_report.md\n echo \"> These tests validate condenser functionality and do not block PR merges.\" >> final_report.md\n echo \"\" >> final_report.md\n cat consolidated_report.md >> final_report.md\n\n # Sanitize @OpenHands mentions to prevent self-mention loops\n COMMENT_BODY=$(uv run python -c \"from openhands.sdk.utils.github import sanitize_openhands_mentions; import sys; print(sanitize_openhands_mentions(sys.stdin.read()), end='')\" < final_report.md)\n # Use GitHub CLI to create comment with explicit PR number\n echo \"$COMMENT_BODY\" | gh pr comment ${{ github.event.pull_request.number }} --body-file -\n env:\n GH_TOKEN: ${{ github.token }}\n" }, { "path": ".github/workflows/create-release.yml", "content": "---\nname: Create GitHub Release\n\n# Automatically create a GitHub release when a release PR is merged into main.\n# This bridges the gap between merging the release PR and the pypi-release\n# workflow (which triggers on release published).\n\non:\n pull_request:\n types: [closed]\n branches: [main]\n\njobs:\n create-release:\n # Only run when a release PR is merged (not just closed)\n if: >\n github.event.pull_request.merged == true &&\n startsWith(github.event.pull_request.head.ref, 'rel-')\n runs-on: ubuntu-24.04\n permissions:\n actions: write\n contents: write\n steps:\n - name: Extract version from branch name\n id: version\n run: |\n BRANCH=\"${{ github.event.pull_request.head.ref }}\"\n VERSION=\"${BRANCH#rel-}\"\n\n if ! [[ \"$VERSION\" =~ ^[0-9]+\\.[0-9]+\\.[0-9]+$ ]]; then\n echo \"❌ Could not extract valid version from branch: $BRANCH\"\n exit 1\n fi\n\n echo \"version=$VERSION\" >> \"$GITHUB_OUTPUT\"\n echo \"📦 Version: $VERSION\"\n\n - name: Check release does not already exist\n id: check\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n VERSION: ${{ steps.version.outputs.version }}\n run: |\n if gh release view \"v${VERSION}\" --repo \"${{ github.repository }}\" > /dev/null 2>&1; then\n echo \"⚠️ Release v${VERSION} already exists, skipping\"\n echo \"exists=true\" >> \"$GITHUB_OUTPUT\"\n else\n echo \"exists=false\" >> \"$GITHUB_OUTPUT\"\n fi\n\n - name: Find previous release tag\n if: steps.check.outputs.exists == 'false'\n id: prev_tag\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n run: |\n PREV_TAG=$(gh release list --repo \"${{ github.repository }}\" \\\n --exclude-drafts --exclude-pre-releases --limit 1 \\\n --json tagName --jq '.[0].tagName')\n echo \"prev_tag=${PREV_TAG}\" >> \"$GITHUB_OUTPUT\"\n echo \"📌 Previous release tag: ${PREV_TAG:-}\"\n\n - name: Create GitHub Release\n if: steps.check.outputs.exists == 'false'\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n VERSION: ${{ steps.version.outputs.version }}\n PREV_TAG: ${{ steps.prev_tag.outputs.prev_tag }}\n run: |\n NOTES_FLAG=()\n if [ -n \"$PREV_TAG\" ]; then\n NOTES_FLAG=(--notes-start-tag \"$PREV_TAG\")\n fi\n\n gh release create \"v${VERSION}\" \\\n --repo \"${{ github.repository }}\" \\\n --target \"${{ github.event.pull_request.merge_commit_sha }}\" \\\n --title \"v${VERSION}\" \\\n --generate-notes \\\n \"${NOTES_FLAG[@]}\"\n\n echo \"✅ Release v${VERSION} created!\"\n echo \"🔗 https://github.com/${{ github.repository }}/releases/tag/v${VERSION}\"\n\n - name: Dispatch PyPI release workflow\n if: steps.check.outputs.exists == 'false'\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n VERSION: ${{ steps.version.outputs.version }}\n run: |\n gh workflow run pypi-release.yml \\\n --repo \"${{ github.repository }}\" \\\n --ref \"v${VERSION}\"\n\n echo \"🚀 Dispatched pypi-release.yml for v${VERSION}\"\n\n - name: Dispatch Agent Server image build\n # server.yml builds versioned Docker images (e.g. 1.21.0-python) when\n # triggered on a tag ref. Tags created by GITHUB_TOKEN don't trigger\n # workflow runs automatically, so we dispatch it explicitly here.\n if: steps.check.outputs.exists == 'false'\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n VERSION: ${{ steps.version.outputs.version }}\n run: |\n gh workflow run server.yml \\\n --repo \"${{ github.repository }}\" \\\n --ref \"v${VERSION}\"\n\n echo \"🐳 Dispatched server.yml image build for v${VERSION}\"\n\n - name: Dispatch release binaries workflow\n # Same GITHUB_TOKEN limitation applies to release-binaries.yml\n # which triggers on release:published events.\n if: steps.check.outputs.exists == 'false'\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n VERSION: ${{ steps.version.outputs.version }}\n run: |\n gh workflow run release-binaries.yml \\\n --repo \"${{ github.repository }}\" \\\n --ref \"v${VERSION}\" \\\n -f release_tag=\"v${VERSION}\"\n\n echo \"📦 Dispatched release-binaries.yml for v${VERSION}\"\n\n - name: Summary\n env:\n VERSION: ${{ steps.version.outputs.version }}\n run: |\n echo \"## ✅ Release v${VERSION} Created\" >> \"$GITHUB_STEP_SUMMARY\"\n echo \"\" >> \"$GITHUB_STEP_SUMMARY\"\n echo \"- **Tag**: v${VERSION}\" >> \"$GITHUB_STEP_SUMMARY\"\n echo \"- **Release**: https://github.com/${{ github.repository }}/releases/tag/v${VERSION}\" >> \"$GITHUB_STEP_SUMMARY\"\n echo \"\" >> \"$GITHUB_STEP_SUMMARY\"\n echo \"The \\`pypi-release.yml\\` workflow was dispatched to publish packages to PyPI.\" >> \"$GITHUB_STEP_SUMMARY\"\n echo \"The \\`server.yml\\` workflow was dispatched to build versioned Docker images.\" >> \"$GITHUB_STEP_SUMMARY\"\n echo \"The \\`release-binaries.yml\\` workflow was dispatched to build and attach release binaries.\" >> \"$GITHUB_STEP_SUMMARY\"\n" }, { "path": ".github/workflows/deploy-docs.yml", "content": "---\nname: Dispatch to docs repo\n\non:\n push:\n branches:\n - main\n paths:\n - openhands-agent-server/**\n workflow_dispatch:\njobs:\n dispatch:\n runs-on: ubuntu-24.04\n permissions:\n contents: write\n steps:\n - name: Trigger docs repo sync\n uses: peter-evans/repository-dispatch@v4\n with:\n token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}\n repository: OpenHands/docs\n event-type: update\n client-payload: '{\"ref\": \"${{ github.ref }}\", \"sha\": \"${{ github.sha }}\"}'\n" }, { "path": ".github/workflows/deprecation-check.yml", "content": "---\nname: Deprecation deadlines\n\non:\n push:\n branches: [main]\n pull_request:\n branches: ['**']\n\njobs:\n check:\n runs-on: ubuntu-24.04\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n python-version: '3.13'\n\n - name: Verify deprecation removals\n run: uv run --with packaging python .github/scripts/check_deprecations.py\n" }, { "path": ".github/workflows/integration-runner.yml", "content": "---\nname: Run Integration Tests\nrun-name: >-\n Run Integration Tests ${{ inputs.reason || github.event.label.name || 'scheduled' }}\n\non:\n # Use pull_request_target to access secrets even on fork PRs\n # This is safe because we only run when the 'integration-test' label is added by a maintainer\n pull_request_target:\n types:\n - labeled\n workflow_dispatch:\n inputs:\n reason:\n description: Reason for manual trigger\n required: true\n default: ''\n test_type:\n description: Select which tests to run (all, integration, behavior)\n required: false\n default: all\n model_ids:\n description: >-\n Comma-separated model IDs to test (from resolve_model_config.py).\n Example: claude-sonnet-4-6,glm-4.7. Defaults to a standard set.\n required: false\n default: ''\n type: string\n issue_number:\n description: Issue or PR number to post results to (optional)\n required: false\n default: ''\n type: string\n tool_preset:\n description: >-\n Tool preset for file editing (default, gemini, gpt5, planning).\n 'default' uses FileEditorTool, 'gemini' uses read_file/write_file/edit/list_directory,\n 'gpt5' uses apply_patch tool.\n required: false\n default: default\n type: choice\n options:\n - default\n - gemini\n - gpt5\n - planning\n schedule:\n - cron: 30 22 * * * # Runs at 10:30pm UTC every day\n\nenv:\n N_PROCESSES: 4 # Global configuration for number of parallel processes for evaluation\n # Default models for scheduled/label-triggered runs (subset of models from resolve_model_config.py)\n DEFAULT_MODEL_IDS: claude-sonnet-4-6,deepseek-v4-flash,kimi-k2.6,gemini-3.1-pro\n\njobs:\n setup-matrix:\n runs-on: ubuntu-latest\n outputs:\n matrix: ${{ steps.resolve-models.outputs.matrix }}\n issue_number: ${{ steps.resolve-issue.outputs.issue_number }}\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n with:\n repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }}\n ref: ${{ github.event.pull_request.head.sha || github.ref }}\n persist-credentials: false\n\n - name: Set up Python\n uses: actions/setup-python@v5\n with:\n python-version: '3.13'\n\n - name: Resolve model configurations\n id: resolve-models\n env:\n MODEL_IDS_INPUT: ${{ github.event.inputs.model_ids || '' }}\n DEFAULT_MODEL_IDS: ${{ env.DEFAULT_MODEL_IDS }}\n run: |\n # Use input model_ids if provided, otherwise use defaults\n if [ -z \"$MODEL_IDS_INPUT\" ]; then\n MODEL_IDS=\"$DEFAULT_MODEL_IDS\"\n echo \"No model_ids specified, using defaults: $MODEL_IDS\"\n else\n MODEL_IDS=\"$MODEL_IDS_INPUT\"\n echo \"Using specified model_ids: $MODEL_IDS\"\n fi\n\n # Resolve model configs using resolve_model_config.py\n # Transform output to matrix format for integration tests\n MATRIX=$(python3 << EOF\n import json\n import sys\n sys.path.insert(0, '.github/run-eval')\n from resolve_model_config import MODELS\n\n model_ids = \"$MODEL_IDS\".split(\",\")\n model_ids = [m.strip() for m in model_ids if m.strip()]\n\n matrix = []\n for model_id in model_ids:\n if model_id not in MODELS:\n available = \", \".join(sorted(MODELS.keys()))\n print(f\"Error: Model ID '{model_id}' not found. Available: {available}\", file=sys.stderr)\n sys.exit(1)\n model = MODELS[model_id]\n # Create run-suffix from model id (replace special chars with underscore)\n run_suffix = model_id.replace(\"-\", \"_\").replace(\".\", \"_\") + \"_run\"\n matrix.append({\n \"id\": model_id,\n \"name\": model[\"display_name\"],\n \"run-suffix\": run_suffix,\n \"llm-config\": model[\"llm_config\"]\n })\n\n print(json.dumps(matrix))\n EOF\n )\n\n if [ $? -ne 0 ]; then\n echo \"Failed to resolve model configurations\" >&2\n exit 1\n fi\n\n echo \"matrix=$MATRIX\" >> \"$GITHUB_OUTPUT\"\n echo \"Resolved models: $(echo \"$MATRIX\" | jq -r '.[].name' | paste -sd', ' -)\"\n\n - name: Resolve issue number\n id: resolve-issue\n env:\n ISSUE_NUMBER_INPUT: ${{ github.event.inputs.issue_number || '' }}\n PR_NUMBER: ${{ github.event.pull_request.number }}\n run: |\n # Priority: explicit input > PR number from label trigger\n if [ -n \"$ISSUE_NUMBER_INPUT\" ]; then\n echo \"issue_number=$ISSUE_NUMBER_INPUT\" >> \"$GITHUB_OUTPUT\"\n elif [ -n \"$PR_NUMBER\" ]; then\n echo \"issue_number=$PR_NUMBER\" >> \"$GITHUB_OUTPUT\"\n else\n echo \"issue_number=\" >> \"$GITHUB_OUTPUT\"\n fi\n\n # Post initial comment for label triggers (no dependencies - runs immediately)\n post-label-comment:\n if: >\n github.event_name == 'pull_request_target' && (\n github.event.label.name == 'integration-test' ||\n github.event.label.name == 'behavior-test'\n )\n runs-on: ubuntu-latest\n permissions:\n pull-requests: write\n steps:\n - name: Comment on PR (integration tests via label)\n if: github.event.label.name == 'integration-test'\n uses: KeisukeYamashita/create-comment@v1\n with:\n unique: false\n comment: |\n Hi! I started running the integration tests on your PR. You will receive a comment with the results shortly.\n - name: Comment on PR (behavior tests via label)\n if: github.event.label.name == 'behavior-test'\n uses: KeisukeYamashita/create-comment@v1\n with:\n unique: false\n comment: |\n Hi! I started running the behavior tests on your PR. You will receive a comment with the results shortly.\n\n # Post initial comment for workflow_dispatch (depends on setup-matrix for issue_number resolution)\n post-dispatch-comment:\n needs: setup-matrix\n if: github.event_name == 'workflow_dispatch' && github.event.inputs.issue_number != ''\n runs-on: ubuntu-latest\n permissions:\n issues: write\n steps:\n - name: Comment on issue/PR (workflow_dispatch)\n env:\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n ISSUE_NUMBER: ${{ github.event.inputs.issue_number }}\n MODEL_IDS: ${{ github.event.inputs.model_ids || 'all models' }}\n TEST_TYPE: ${{ github.event.inputs.test_type || 'all' }}\n REASON: ${{ github.event.inputs.reason }}\n run: |\n # Sanitize @OpenHands mentions to prevent self-mention loops\n SANITIZED_REASON=$(echo \"$REASON\" | sed 's/@OpenHands/@\\u200BOpenHands/g; s/@openhands/@\\u200Bopenhands/g')\n SANITIZED_MODEL_IDS=$(echo \"$MODEL_IDS\" | sed 's/@OpenHands/@\\u200BOpenHands/g; s/@openhands/@\\u200Bopenhands/g')\n COMMENT_BODY=$(cat <> \"$GITHUB_ENV\"\n\n - name: Run integration test evaluation for ${{ matrix.job-config['name'] }}\n env:\n LLM_CONFIG: ${{ toJson(matrix.job-config['llm-config']) }}\n LLM_API_KEY: ${{ secrets.LLM_API_KEY_EVAL }}\n LLM_BASE_URL: https://llm-proxy.eval.all-hands.dev\n TOOL_PRESET: ${{ github.event.inputs.tool_preset || 'default' }}\n run: |\n set -eo pipefail\n\n AGENT_SDK_VERSION=$(git rev-parse --short HEAD)\n EVAL_NOTE=\"${AGENT_SDK_VERSION}_${{ matrix.job-config['run-suffix'] }}\"\n\n echo \"Invoking test runner with TEST_TYPE_ARGS='$TEST_TYPE_ARGS' TOOL_PRESET='$TOOL_PRESET'\"\n\n uv run python tests/integration/run_infer.py \\\n --llm-config \"$LLM_CONFIG\" \\\n --num-workers $N_PROCESSES \\\n --eval-note \"$EVAL_NOTE\" \\\n --tool-preset \"$TOOL_PRESET\" \\\n $TEST_TYPE_ARGS\n\n # get integration tests JSON results\n RESULTS_FILE=$(find tests/integration/outputs/*${{ matrix.job-config['run-suffix'] }}* -name \"results.json\" -type f | head -n 1)\n echo \"RESULTS_FILE: $RESULTS_FILE\"\n if [ -f \"$RESULTS_FILE\" ]; then\n echo \"JSON_RESULTS_FILE=$RESULTS_FILE\" >> $GITHUB_ENV\n else\n echo \"JSON_RESULTS_FILE=\" >> $GITHUB_ENV\n fi\n\n - name: Wait a little bit\n run: sleep 10\n\n\n\n\n\n - name: Create archive of evaluation outputs\n run: |\n TIMESTAMP=$(date +'%y-%m-%d-%H-%M')\n cd tests/integration/outputs # Change to the outputs directory\n tar -czvf ../../../integration_tests_${{ matrix.job-config['run-suffix'] }}_${TIMESTAMP}.tar.gz *${{ matrix.job-config['run-suffix'] }}* # Include result directories for this model\n\n - name: Upload evaluation results as artifact\n uses: actions/upload-artifact@v7\n id: upload_results_artifact\n with:\n name: integration-test-outputs-${{ matrix.job-config['run-suffix'] }}-${{ github.run_id }}-${{ github.run_attempt }}\n path: integration_tests_${{ matrix.job-config['run-suffix'] }}_*.tar.gz\n\n - name: Save test results for consolidation\n run: |\n # Copy the structured JSON results file for consolidation\n mkdir -p test_results_summary\n\n if [ -n \"${{ env.JSON_RESULTS_FILE }}\" ] && [ -f \"${{ env.JSON_RESULTS_FILE }}\" ]; then\n # Copy the JSON results file directly\n cp \"${{ env.JSON_RESULTS_FILE }}\" \"test_results_summary/${{ matrix.job-config['run-suffix'] }}_results.json\"\n echo \"✓ Copied JSON results file for consolidation\"\n else\n echo \"✗ No JSON results file found\"\n exit 1\n fi\n\n - name: Upload test results summary\n uses: actions/upload-artifact@v7\n with:\n name: test-results-${{ matrix.job-config['run-suffix'] }}\n path: test_results_summary/${{ matrix.job-config['run-suffix'] }}_results.json\n\n consolidate-results:\n needs: [setup-matrix, run-integration-tests]\n if: |\n always() && (\n (\n github.event_name == 'pull_request_target' && (\n github.event.label.name == 'integration-test' ||\n github.event.label.name == 'behavior-test'\n )\n ) ||\n github.event_name == 'workflow_dispatch' ||\n (github.event_name == 'schedule' && github.repository == 'OpenHands/software-agent-sdk')\n )\n runs-on: ubuntu-24.04\n permissions:\n contents: read\n pull-requests: write\n issues: write\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n with:\n # When using pull_request_target, explicitly checkout the PR branch\n # This ensures we use the scripts from the actual PR code\n ref: ${{ github.event.pull_request.head.sha || github.ref }}\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n version: latest\n python-version: '3.13'\n\n - name: Install Python dependencies using uv\n run: |\n uv sync --dev\n\n - name: Download all test results\n uses: actions/download-artifact@v8\n with:\n pattern: test-results-*\n merge-multiple: true\n path: all_results\n\n - name: Download all integration test artifacts\n uses: actions/download-artifact@v8\n with:\n pattern: integration-test-outputs-*\n path: artifacts\n\n - name: Consolidate test results\n env:\n EVENT_NAME: ${{ github.event_name }}\n PR_NUMBER: ${{ github.event.pull_request.number }}\n MANUAL_REASON: ${{ github.event.inputs.reason }}\n COMMIT_SHA: ${{ github.sha }}\n PYTHONPATH: ${{ github.workspace }}\n GITHUB_SERVER_URL: ${{ github.server_url }}\n GITHUB_REPOSITORY: ${{ github.repository }}\n GITHUB_RUN_ID: ${{ github.run_id }}\n run: |\n uv run python tests/integration/utils/consolidate_json_results.py \\\n --results-dir all_results \\\n --artifacts-dir artifacts \\\n --output-file consolidated_results.json\n\n echo \"Consolidated results generated successfully\"\n\n uv run python tests/integration/utils/generate_markdown_report.py \\\n --input-file consolidated_results.json \\\n --output-file consolidated_report.md\n\n - name: Upload consolidated report\n uses: actions/upload-artifact@v7\n with:\n name: consolidated-report\n path: consolidated_report.md\n\n - name: Create consolidated PR comment\n if: github.event_name == 'pull_request_target'\n run: |\n # Sanitize @OpenHands mentions to prevent self-mention loops\n COMMENT_BODY=$(uv run python -c \"from openhands.sdk.utils.github import sanitize_openhands_mentions; import sys; print(sanitize_openhands_mentions(sys.stdin.read()), end='')\" < consolidated_report.md)\n # Use GitHub CLI to create comment with explicit PR number\n echo \"$COMMENT_BODY\" | gh pr comment ${{ github.event.pull_request.number }} --body-file -\n env:\n GH_TOKEN: ${{ github.token }}\n\n - name: Comment on specified issue/PR (workflow_dispatch)\n if: github.event_name == 'workflow_dispatch' && needs.setup-matrix.outputs.issue_number != ''\n env:\n GH_TOKEN: ${{ github.token }}\n ISSUE_NUMBER: ${{ needs.setup-matrix.outputs.issue_number }}\n run: |\n # Sanitize @OpenHands mentions to prevent self-mention loops\n COMMENT_BODY=$(uv run python -c \"from openhands.sdk.utils.github import sanitize_openhands_mentions; import sys; print(sanitize_openhands_mentions(sys.stdin.read()), end='')\" < consolidated_report.md)\n # Use GitHub CLI to create comment on the specified issue/PR\n echo \"$COMMENT_BODY\" | gh issue comment \"$ISSUE_NUMBER\" --body-file -\n\n - name: Read consolidated report for tracker issue\n if: github.event_name == 'schedule'\n id: read_report\n run: |\n # Read and sanitize the report, then set as output\n REPORT_CONTENT=$(uv run python -c \"from openhands.sdk.utils.github import sanitize_openhands_mentions; import sys; print(sanitize_openhands_mentions(sys.stdin.read()), end='')\" < consolidated_report.md)\n echo \"report<> $GITHUB_OUTPUT\n echo \"$REPORT_CONTENT\" >> $GITHUB_OUTPUT\n echo \"EOF\" >> $GITHUB_OUTPUT\n\n - name: Comment with results on tracker issue\n if: github.event_name == 'schedule'\n uses: KeisukeYamashita/create-comment@v1\n with:\n number: 2078\n unique: false\n comment: |\n **Trigger:** Nightly Scheduled Run\n **Commit:** ${{ github.sha }}\n\n ${{ steps.read_report.outputs.report }}\n\n" }, { "path": ".github/workflows/issue-duplicate-checker.yml", "content": "---\nname: Issue Duplicate Check via OpenHands Cloud\n\non:\n issues:\n types: [opened]\n schedule:\n - cron: 0 9 * * *\n workflow_dispatch:\n inputs:\n mode:\n description: Which workflow path to run\n required: true\n type: choice\n options:\n - smoke-clone\n - issue-check\n - auto-close\n default: smoke-clone\n issue_number:\n description: Existing issue number to analyze when mode is issue-check\n required: false\n type: number\n close_after_days:\n description: Days to wait before auto-closing duplicate candidates in auto-close mode\n required: false\n type: number\n default: 3\n\n\npermissions:\n contents: read\n issues: write\n\njobs:\n smoke-clone:\n if: github.event_name == 'workflow_dispatch' && inputs.mode == 'smoke-clone'\n runs-on: ubuntu-latest\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n\n - name: Clone software-agent-sdk\n run: |\n git clone --depth 1 \"https://github.com/${{ github.repository }}.git\" /tmp/software-agent-sdk\n echo \"software-agent-sdk HEAD: $(git -C /tmp/software-agent-sdk rev-parse --short HEAD)\"\n\n - name: Summarize smoke test\n run: |\n {\n echo \"## Smoke clone completed\"\n echo\n echo \"- software-agent-sdk cloned to /tmp/software-agent-sdk\"\n } >> \"$GITHUB_STEP_SUMMARY\"\n\n issue-duplicate-check:\n if: |\n github.event_name == 'issues' ||\n (github.event_name == 'workflow_dispatch' && inputs.mode == 'issue-check' && inputs.issue_number != null)\n runs-on: ubuntu-latest\n timeout-minutes: 35\n concurrency:\n group: issue-duplicate-check-${{ github.repository }}-${{ github.event.issue.number || inputs.issue_number }}\n cancel-in-progress: false\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: '3.13'\n\n - name: Validate duplicate check inputs\n env:\n OPENHANDS_API_KEY: ${{ secrets.OPENHANDS_API_KEY }}\n ISSUE_NUMBER: ${{ github.event.issue.number || inputs.issue_number }}\n run: |\n if [ -z \"$OPENHANDS_API_KEY\" ]; then\n echo \"Error: OPENHANDS_API_KEY secret is required\"\n exit 1\n fi\n if [ -z \"$ISSUE_NUMBER\" ]; then\n echo \"Error: ISSUE_NUMBER is required\"\n exit 1\n fi\n\n - name: Run OpenHands duplicate check conversation\n id: run_check\n env:\n OPENHANDS_API_KEY: ${{ secrets.OPENHANDS_API_KEY }}\n GITHUB_TOKEN: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC || github.token }}\n ISSUE_NUMBER: ${{ github.event.issue.number || inputs.issue_number }}\n OUTPUT_PATH: ${{ runner.temp }}/issue-duplicate-check-result.json\n run: |\n python scripts/issue_duplicate_check_openhands.py \\\n --repository \"${{ github.repository }}\" \\\n --issue-number \"$ISSUE_NUMBER\" \\\n --output \"$OUTPUT_PATH\"\n test -f \"$OUTPUT_PATH\" || {\n echo \"Error: Output file not created\"\n exit 1\n }\n echo \"result_path=$OUTPUT_PATH\" >> \"$GITHUB_OUTPUT\"\n\n - name: Parse duplicate check result\n id: parsed_result\n env:\n RESULT_PATH: ${{ steps.run_check.outputs.result_path }}\n run: |\n python - <<'PY'\n import json\n import os\n import sys\n from pathlib import Path\n\n try:\n result = json.loads(Path(os.environ['RESULT_PATH']).read_text())\n except (FileNotFoundError, json.JSONDecodeError) as exc:\n print(\n f\"Error: Failed to read duplicate check result: {exc}\",\n file=sys.stderr,\n )\n raise SystemExit(1) from exc\n output_path = Path(os.environ['GITHUB_OUTPUT'])\n summary_path = Path(os.environ['GITHUB_STEP_SUMMARY'])\n\n def write_multiline(name: str, value: str) -> None:\n delimiter = f\"EOF_{os.urandom(8).hex()}\"\n with output_path.open('a', encoding='utf-8') as fh:\n fh.write(f\"{name}<<{delimiter}\\n{value}\\n{delimiter}\\n\")\n\n canonical_issue_number = result.get('canonical_issue_number')\n with output_path.open('a', encoding='utf-8') as fh:\n fh.write(f\"should_comment={'true' if result.get('should_comment') else 'false'}\\n\")\n fh.write(f\"is_duplicate={'true' if result.get('is_duplicate') else 'false'}\\n\")\n fh.write(\n f\"auto_close_candidate={'true' if result.get('auto_close_candidate') else 'false'}\\n\"\n )\n fh.write(f\"confidence={result.get('confidence', '')}\\n\")\n fh.write(f\"classification={result.get('classification', '')}\\n\")\n fh.write(\n f\"canonical_issue_number={canonical_issue_number if canonical_issue_number is not None else ''}\\n\"\n )\n fh.write(f\"conversation_url={result.get('conversation_url', '')}\\n\")\n fh.write(f\"app_conversation_id={result.get('app_conversation_id', '')}\\n\")\n\n write_multiline('summary', str(result.get('summary', '')).strip())\n write_multiline(\n 'candidate_issues_json',\n json.dumps(result.get('candidate_issues', []), ensure_ascii=False),\n )\n\n candidate_lines = []\n for candidate in result.get('candidate_issues', []):\n candidate_lines.append(\n f\"- #{candidate.get('number')}: {candidate.get('title')} ({candidate.get('url')}) — {candidate.get('similarity_reason', '')}\"\n )\n\n summary_path.write_text(\n \"\\n\".join(\n [\n \"## Duplicate check result\",\n \"\",\n f\"- Repository: {result.get('repository')}\",\n f\"- Issue: #{result.get('issue_number')}\",\n f\"- Should comment: {result.get('should_comment')}\",\n f\"- Exact duplicate: {result.get('is_duplicate')}\",\n f\"- Auto-close candidate: {result.get('auto_close_candidate')}\",\n f\"- Classification: {result.get('classification')}\",\n f\"- Confidence: {result.get('confidence')}\",\n f\"- Canonical issue: {canonical_issue_number}\",\n f\"- Conversation: {result.get('conversation_url')}\",\n \"\",\n \"### Summary\",\n result.get('summary', ''),\n \"\",\n \"### Candidate issues\",\n *(candidate_lines or [\"- None\"]),\n ]\n )\n + \"\\n\",\n encoding='utf-8',\n )\n PY\n\n - name: Post duplicate overlap notice\n if: steps.parsed_result.outputs.should_comment == 'true'\n uses: actions/github-script@v9\n env:\n ISSUE_NUMBER: ${{ github.event.issue.number || inputs.issue_number }}\n SUMMARY: ${{ steps.parsed_result.outputs.summary }}\n CANDIDATE_ISSUES_JSON: ${{ steps.parsed_result.outputs.candidate_issues_json }}\n CLASSIFICATION: ${{ steps.parsed_result.outputs.classification }}\n AUTO_CLOSE_CANDIDATE: ${{ steps.parsed_result.outputs.auto_close_candidate }}\n CANONICAL_ISSUE_NUMBER: ${{ steps.parsed_result.outputs.canonical_issue_number }}\n CLOSE_AFTER_DAYS: ${{ inputs.close_after_days || '3' }}\n with:\n github-token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC || github.token }}\n script: |\n const issueNumber = Number(process.env.ISSUE_NUMBER);\n const summary = (process.env.SUMMARY || '').trim();\n const classification = process.env.CLASSIFICATION || 'no-match';\n const autoClose = process.env.AUTO_CLOSE_CANDIDATE === 'true';\n const closeAfterDays = process.env.CLOSE_AFTER_DAYS || '3';\n let candidates = [];\n try {\n candidates = JSON.parse(process.env.CANDIDATE_ISSUES_JSON || '[]');\n } catch (error) {\n core.setFailed(`Invalid candidate JSON: ${error.message}`);\n return;\n }\n if (!Array.isArray(candidates)) {\n core.setFailed('CANDIDATE_ISSUES_JSON is not an array');\n return;\n }\n if (candidates.length === 0) {\n core.setFailed(`No candidate issues were returned for issue #${issueNumber}.`);\n return;\n }\n const canonicalIssueRaw = process.env.CANONICAL_ISSUE_NUMBER || candidates[0].number;\n const canonicalIssueNumber = canonicalIssueRaw ? Number(canonicalIssueRaw) : Number.NaN;\n const candidateLabel = 'duplicate-candidate';\n\n function parseDuplicateCheckMarker(body) {\n if (!body) {\n return null;\n }\n const match = body.match(//);\n if (!match) {\n return null;\n }\n return {\n canonicalIssueNumber: Number(match[1]),\n autoClose: match[2] === 'true',\n };\n }\n\n async function ensureCanonicalIssueIsOpenIssue() {\n let canonicalIssue;\n try {\n ({ data: canonicalIssue } = await github.rest.issues.get({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: canonicalIssueNumber,\n }));\n } catch (error) {\n if (error.status === 404) {\n core.setFailed(`Canonical issue #${canonicalIssueNumber} does not exist.`);\n return false;\n }\n throw error;\n }\n if (canonicalIssue.pull_request) {\n core.setFailed(`Canonical issue #${canonicalIssueNumber} is a pull request, not an issue.`);\n return false;\n }\n if (canonicalIssue.state !== 'open' || canonicalIssue.locked) {\n core.setFailed(`Canonical issue #${canonicalIssueNumber} must be an open, unlocked issue.`);\n return false;\n }\n return true;\n }\n\n async function ensureCandidateLabelOnIssue() {\n try {\n await github.rest.issues.getLabel({\n owner: context.repo.owner,\n repo: context.repo.repo,\n name: candidateLabel,\n });\n } catch (error) {\n if (error.status !== 404) {\n throw error;\n }\n await github.rest.issues.createLabel({\n owner: context.repo.owner,\n repo: context.repo.repo,\n name: candidateLabel,\n color: 'C5DEF5',\n description: 'Potential duplicate awaiting auto-close or maintainer review',\n });\n }\n\n const { data: issue } = await github.rest.issues.get({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: issueNumber,\n });\n const labelNames = (issue.labels || []).map((label) => (\n typeof label === 'string' ? label : label.name\n ));\n if (!labelNames.includes(candidateLabel)) {\n await github.rest.issues.addLabels({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: issueNumber,\n labels: [candidateLabel],\n });\n }\n }\n\n async function removeCandidateLabelFromIssue() {\n try {\n await github.rest.issues.removeLabel({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: issueNumber,\n name: candidateLabel,\n });\n } catch (error) {\n if (error.status !== 404) {\n throw error;\n }\n }\n }\n\n if (!Number.isInteger(canonicalIssueNumber) || canonicalIssueNumber <= 0) {\n core.setFailed(`No canonical issue number was returned for issue #${issueNumber}.`);\n return;\n }\n \n if (!(await ensureCanonicalIssueIsOpenIssue())) {\n return;\n }\n\n const marker = ``;\n const header = candidates.length === 1\n ? 'Found 1 possible duplicate issue:'\n : `Found ${candidates.length} possible duplicate issues:`;\n const candidateLines = candidates.map((candidate, index) => (\n `${index + 1}. [#${candidate.number}](${candidate.url}) — ${candidate.title}`\n ));\n\n const sections = [];\n if (summary) {\n sections.push(summary, '');\n }\n sections.push(header, '', ...candidateLines);\n\n if (classification === 'overlapping-scope') {\n sections.push(\n '',\n 'These may not be exact duplicates, but the scope appears to overlap enough that keeping discussion in one place may be more useful.'\n );\n }\n\n if (autoClose) {\n sections.push(\n '',\n `This issue will be automatically closed as a duplicate in ${closeAfterDays} days.`,\n '',\n '- If your issue is a duplicate, please close it and 👍 the existing issue instead',\n '- To prevent auto-closure, add a comment or 👎 this comment'\n );\n }\n\n sections.push(\n '',\n marker,\n '_This comment was created by an AI assistant (OpenHands) on behalf of the repository maintainer._'\n );\n const body = sections.join('\\n').trim();\n\n const MAX_COMMENT_PAGES = 50;\n let allComments = [];\n let page = 1;\n while (page <= MAX_COMMENT_PAGES) {\n const { data: comments } = await github.rest.issues.listComments({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: issueNumber,\n per_page: 100,\n page,\n });\n if (!comments || comments.length === 0) {\n break;\n }\n allComments = allComments.concat(comments);\n if (comments.length < 100) {\n break;\n }\n page += 1;\n }\n if (page > MAX_COMMENT_PAGES) {\n core.setFailed(\n `Stopped loading comments for issue #${issueNumber} after ${MAX_COMMENT_PAGES} pages.`\n );\n return;\n }\n\n const existing = allComments.find((comment) => comment.body && comment.body.includes('';\n const body = `${marker}\n ✅ **PR Artifacts Cleaned Up**\n\n The \\`.pr/\\` directory has been automatically removed.\n `;\n\n const { data: comments } = await github.rest.issues.listComments({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: context.issue.number,\n });\n\n const existing = comments.find(c => c.body.includes(marker));\n if (existing) {\n await github.rest.issues.updateComment({\n owner: context.repo.owner,\n repo: context.repo.repo,\n comment_id: existing.id,\n body: body,\n });\n }\n\n # Warn if .pr/ directory exists (will be auto-removed on approval)\n check-pr-artifacts:\n if: github.event_name == 'pull_request'\n runs-on: ubuntu-latest\n permissions:\n contents: read\n pull-requests: write\n steps:\n - uses: actions/checkout@v6\n\n - name: Check for .pr/ directory\n id: check\n run: |\n if [ -d \".pr\" ]; then\n echo \"exists=true\" >> $GITHUB_OUTPUT\n echo \"::warning::.pr/ directory exists and will be automatically removed when the PR is approved. For fork PRs, manual removal is required before merging.\"\n else\n echo \"exists=false\" >> $GITHUB_OUTPUT\n fi\n\n - name: Post or update PR comment\n if: steps.check.outputs.exists == 'true'\n uses: actions/github-script@v9\n with:\n script: |\n const marker = '';\n const body = `${marker}\n 📁 **PR Artifacts Notice**\n\n This PR contains a \\`.pr/\\` directory with PR-specific documents. This directory will be **automatically removed** when the PR is approved.\n\n > For fork PRs: Manual removal is required before merging.\n `;\n\n const { data: comments } = await github.rest.issues.listComments({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: context.issue.number,\n });\n\n const existing = comments.find(c => c.body.includes(marker));\n if (!existing) {\n await github.rest.issues.createComment({\n owner: context.repo.owner,\n repo: context.repo.repo,\n issue_number: context.issue.number,\n body: body,\n });\n }\n" }, { "path": ".github/workflows/pr-review-by-openhands.yml", "content": "---\nname: PR Review by OpenHands\n\non:\n # Use pull_request for same-repo PRs so workflow changes can self-verify in PRs.\n pull_request:\n types: [opened, ready_for_review, labeled, review_requested]\n # Use pull_request_target for fork PRs.\n # The bot token used here is intentionally scoped to PR review operations,\n # so the remaining blast radius is bounded even though PR content is untrusted.\n pull_request_target:\n types: [opened, ready_for_review, labeled, review_requested]\n\npermissions:\n contents: read\n pull-requests: write\n issues: write\n\njobs:\n pr-review:\n # Run on same-repo PRs via pull_request and on fork PRs via pull_request_target.\n # Trigger when one of the following conditions is met:\n # 1. A new non-draft PR is opened by a non-first-time contributor, OR\n # 2. A draft PR is converted to ready for review by a non-first-time contributor, OR\n # 3. The 'review-this' label is added, OR\n # 4. openhands-agent or all-hands-bot is requested as a reviewer\n # Note: FIRST_TIME_CONTRIBUTOR and NONE PRs require manual trigger via label/reviewer request.\n if: |\n (\n (\n github.event_name == 'pull_request' &&\n github.event.pull_request.head.repo.full_name == github.repository\n ) ||\n (\n github.event_name == 'pull_request_target' &&\n github.event.pull_request.head.repo.full_name != github.repository\n )\n ) &&\n (\n (github.event.action == 'opened' && github.event.pull_request.draft == false && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||\n (github.event.action == 'ready_for_review' && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||\n (github.event.action == 'labeled' && github.event.label.name == 'review-this') ||\n (\n github.event.action == 'review_requested' &&\n (\n github.event.requested_reviewer.login == 'openhands-agent' ||\n github.event.requested_reviewer.login == 'all-hands-bot'\n )\n )\n )\n concurrency:\n group: pr-review-${{ github.event.pull_request.number }}\n cancel-in-progress: true\n runs-on: ubuntu-24.04\n steps:\n - name: Run PR Review\n uses: OpenHands/extensions/plugins/pr-review@main\n with:\n llm-model: litellm_proxy/claude-sonnet-4-5-20250929\n llm-base-url: https://llm-proxy.app.all-hands.dev\n # Enable experimental sub-agent delegation for file-level reviews\n use-sub-agents: 'true'\n llm-api-key: ${{ secrets.LLM_API_KEY }}\n github-token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC || github.token }}\n lmnr-api-key: ${{ secrets.LMNR_SKILLS_API_KEY }}\n" }, { "path": ".github/workflows/pr-review-evaluation.yml", "content": "---\nname: PR Review Evaluation\n\n# This workflow evaluates how well PR review comments were addressed.\n# It runs when a PR is closed to assess review effectiveness.\n#\n# Security note: pull_request_target is safe here because:\n# 1. Only triggers on PR close (not on code changes)\n# 2. Does not checkout PR code - only downloads artifacts from trusted workflow runs\n# 3. Runs evaluation scripts from the extensions repo, not from the PR\n\non:\n pull_request_target:\n types: [closed]\n\npermissions:\n contents: read\n pull-requests: read\n\njobs:\n evaluate:\n runs-on: ubuntu-24.04\n env:\n PR_NUMBER: ${{ github.event.pull_request.number }}\n REPO_NAME: ${{ github.repository }}\n PR_MERGED: ${{ github.event.pull_request.merged }}\n\n steps:\n - name: Download review trace artifact\n id: download-trace\n uses: dawidd6/action-download-artifact@v21\n continue-on-error: true\n with:\n workflow: pr-review-by-openhands.yml\n name: pr-review-trace-${{ github.event.pull_request.number }}\n path: trace-info\n search_artifacts: true\n if_no_artifact_found: warn\n\n - name: Check if trace file exists\n id: check-trace\n run: |\n if [ -f \"trace-info/laminar_trace_info.json\" ]; then\n echo \"trace_exists=true\" >> $GITHUB_OUTPUT\n echo \"Found trace file for PR #$PR_NUMBER\"\n else\n echo \"trace_exists=false\" >> $GITHUB_OUTPUT\n echo \"No trace file found for PR #$PR_NUMBER - skipping evaluation\"\n fi\n\n # Always checkout main branch for security - cannot test script changes in PRs\n - name: Checkout extensions repository\n if: steps.check-trace.outputs.trace_exists == 'true'\n uses: actions/checkout@v6\n with:\n repository: OpenHands/extensions\n path: extensions\n\n - name: Set up Python\n if: steps.check-trace.outputs.trace_exists == 'true'\n uses: actions/setup-python@v6\n with:\n python-version: '3.12'\n\n - name: Install dependencies\n if: steps.check-trace.outputs.trace_exists == 'true'\n run: pip install lmnr\n\n - name: Run evaluation\n if: steps.check-trace.outputs.trace_exists == 'true'\n env:\n # Script expects LMNR_PROJECT_API_KEY; org secret is named LMNR_SKILLS_API_KEY\n LMNR_PROJECT_API_KEY: ${{ secrets.LMNR_SKILLS_API_KEY }}\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n run: |\n python extensions/plugins/pr-review/scripts/evaluate_review.py \\\n --trace-file trace-info/laminar_trace_info.json\n\n - name: Upload evaluation logs\n uses: actions/upload-artifact@v7\n if: always() && steps.check-trace.outputs.trace_exists == 'true'\n with:\n name: pr-review-evaluation-${{ github.event.pull_request.number }}\n path: '*.log'\n retention-days: 30\n" }, { "path": ".github/workflows/precommit.yml", "content": "---\n# .github/workflows/precommit.yml\nname: Pre-commit checks\n\non:\n push:\n branches: [main]\n pull_request:\n branches: ['**']\n\njobs:\n pre-commit:\n runs-on: ubuntu-24.04\n\n steps:\n - name: Checkout code\n uses: actions/checkout@v6\n\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: '3.13'\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n\n - name: Install dependencies\n run: uv sync --frozen --group dev\n\n - name: Run pre-commit (all files)\n run: uv run pre-commit run --all-files --show-diff-on-failure\n" }, { "path": ".github/workflows/prepare-release.yml", "content": "---\nname: Prepare Release\n\non:\n workflow_dispatch:\n inputs:\n version:\n description: Release version (e.g., 1.2.3)\n required: true\n type: string\n\njobs:\n prepare-release:\n runs-on: ubuntu-24.04\n steps:\n - name: Validate version format\n run: |\n if ! [[ \"${{ inputs.version }}\" =~ ^[0-9]+\\.[0-9]+\\.[0-9]+$ ]]; then\n echo \"❌ Invalid version format. Expected: X.Y.Z (e.g., 1.2.3)\"\n exit 1\n fi\n echo \"✅ Version format is valid: ${{ inputs.version }}\"\n\n - name: Checkout repository\n uses: actions/checkout@v6\n with:\n token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n version: latest\n python-version: '3.13'\n\n - name: Configure Git\n run: |\n git config user.name \"github-actions[bot]\"\n git config user.email \"github-actions[bot]@users.noreply.github.com\"\n\n - name: Create release branch\n run: |\n BRANCH_NAME=\"rel-${{ inputs.version }}\"\n echo \"Creating branch: $BRANCH_NAME\"\n git checkout -b \"$BRANCH_NAME\"\n echo \"BRANCH_NAME=$BRANCH_NAME\" >> $GITHUB_ENV\n\n - name: Set package version\n run: |\n echo \"🔧 Setting version to ${{ inputs.version }}\"\n make set-package-version version=${{ inputs.version }}\n\n - name: Update sdk_ref default in run-eval workflow\n run: python3 .github/scripts/update_sdk_ref_default.py \"${{ inputs.version }}\"\n\n - name: Commit version changes\n run: |\n git add .\n if git diff --staged --quiet; then\n echo \"No changes to commit\"\n else\n git commit -m \"Release v${{ inputs.version }}\" -m \"Co-authored-by: openhands \"\n echo \"✅ Changes committed\"\n fi\n\n - name: Push release branch\n run: |\n git push -u origin \"${{ env.BRANCH_NAME }}\"\n echo \"✅ Branch pushed: ${{ env.BRANCH_NAME }}\"\n\n - name: Create Pull Request\n env:\n GH_TOKEN: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}\n run: |\n cat > pr_body.txt << 'EOF'\n ## Release v${{ inputs.version }}\n\n This PR prepares the release for version **${{ inputs.version }}**.\n\n ### Release Checklist\n - [x] Version set to ${{ inputs.version }}\n - [ ] Fix any deprecation deadlines if they exist\n - [ ] Integration tests pass (tagged with `integration-test`)\n - [ ] Behavior tests pass (tagged with `behavior-test`)\n - [ ] Example tests pass (tagged with `test-examples`)\n - [ ] Evaluation on OpenHands Index\n\n ### What happens on merge\n When this PR is merged, the `create-release.yml` workflow will automatically:\n 1. Create a GitHub release with tag `v${{ inputs.version }}` and auto-generated notes\n 2. Trigger `pypi-release.yml` to publish all packages to PyPI\n 3. Trigger `version-bump-prs.yml` to create downstream version bump PRs\n EOF\n\n gh pr create \\\n --title \"Release v${{ inputs.version }}\" \\\n --body-file pr_body.txt \\\n --base main \\\n --head \"${{ env.BRANCH_NAME }}\" \\\n --label \"integration-test\" \\\n --label \"behavior-test\" \\\n --label \"test-examples\"\n\n rm pr_body.txt\n echo \"✅ Pull request created successfully!\"\n\n # Get PR URL and display it\n PR_URL=$(gh pr view \"${{ env.BRANCH_NAME }}\" --json url --jq '.url')\n echo \"🔗 PR URL: $PR_URL\"\n echo \"PR_URL=$PR_URL\" >> $GITHUB_ENV\n\n - name: Summary\n run: |\n echo \"## ✅ Release Preparation Complete!\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"- **Version**: ${{ inputs.version }}\" >> $GITHUB_STEP_SUMMARY\n echo \"- **Branch**: ${{ env.BRANCH_NAME }}\" >> $GITHUB_STEP_SUMMARY\n echo \"- **PR URL**: ${{ env.PR_URL }}\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"### Next Steps:\" >> $GITHUB_STEP_SUMMARY\n echo \"1. Review the PR and address any deprecation deadlines\" >> $GITHUB_STEP_SUMMARY\n echo \"2. Wait for integration, behavior, and example tests to pass\" >> $GITHUB_STEP_SUMMARY\n echo \"3. Merge the PR — a GitHub release and PyPI publish will happen automatically\" >> $GITHUB_STEP_SUMMARY\n" }, { "path": ".github/workflows/pypi-release.yml", "content": "---\nname: Publish all OpenHands packages (uv)\n\non:\n # Run manually\n workflow_dispatch:\n # Run automatically when a release is published\n release:\n types: [published]\n\njobs:\n publish:\n # Skip PyPI publishing for pre-releases (e.g., release candidates).\n # Pre-releases can still be created on GitHub for testing without\n # pushing packages to PyPI. Manual workflow_dispatch always runs.\n if: >\n github.event_name == 'workflow_dispatch' ||\n !github.event.release.prerelease\n runs-on: ubuntu-24.04\n permissions:\n actions: write\n contents: read\n outputs:\n version: ${{ steps.extract_version.outputs.version }}\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n\n - name: Extract version from release tag\n id: extract_version\n run: |\n # Get version from release tag (e.g., v1.2.3 -> 1.2.3)\n if [[ \"${{ github.event_name }}\" == \"release\" ]]; then\n VERSION=\"${{ github.event.release.tag_name }}\"\n VERSION=\"${VERSION#v}\" # Remove 'v' prefix if present\n else\n # For manual dispatch, extract from pyproject.toml\n VERSION=$(grep -m1 '^version = ' openhands-sdk/pyproject.toml | cut -d'\"' -f2)\n fi\n echo \"version=$VERSION\" >> $GITHUB_OUTPUT\n echo \"📦 Version: $VERSION\"\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n version: latest\n python-version: '3.13'\n\n - name: Build and publish all packages\n env:\n UV_PUBLISH_TOKEN: ${{ secrets.PYPI_TOKEN_OPENHANDS }}\n run: |\n set -euo pipefail\n\n if [ -z \"${UV_PUBLISH_TOKEN:-}\" ]; then\n echo \"❌ Missing secret PYPI_TOKEN_OPENHANDS\"\n exit 1\n fi\n\n PACKAGES=(\n openhands-sdk\n openhands-tools\n openhands-workspace\n openhands-agent-server\n )\n\n echo \"🚀 Building and publishing all packages...\"\n for PKG in \"${PACKAGES[@]}\"; do\n echo \"===== $PKG =====\"\n uv build --package \"$PKG\"\n done\n\n # Use --check-url to skip files that already exist on PyPI\n # This allows re-running the workflow after partial failures\n uv publish --token \"$UV_PUBLISH_TOKEN\" --check-url https://pypi.org/simple/\n echo \"✅ All packages built and published successfully!\"\n echo \"\"\n echo \"📋 Note: Version bump PRs will be created by the 'Create Version Bump PRs' workflow\"\n echo \" which is dispatched after this publish succeeds.\"\n\n - name: Dispatch version bump workflow\n env:\n GH_TOKEN: ${{ github.token }}\n VERSION: ${{ steps.extract_version.outputs.version }}\n run: |\n gh workflow run version-bump-prs.yml \\\n --repo \"${{ github.repository }}\" \\\n -f \"version=${VERSION}\"\n\n echo \"🚀 Dispatched version-bump-prs.yml for v${VERSION}\"\n" }, { "path": ".github/workflows/qa-changes-by-openhands.yml", "content": "---\n# Automated QA validation of PR changes using OpenHands.\n#\n# Unlike pr-review (which reads diffs and posts code-review comments),\n# this workflow actually runs the code — setting up the environment,\n# executing tests, exercising changed behavior, and posting a structured\n# QA report as a PR comment.\nname: QA Changes by OpenHands\n\non:\n pull_request:\n types: [opened, ready_for_review, labeled, review_requested]\n\npermissions:\n contents: read\n pull-requests: write\n issues: write\n\njobs:\n qa-changes:\n # Only run for same-repo PRs (secrets aren't available for forks).\n # Trigger conditions mirror pr-review, but use the 'qa-this' label\n # and openhands-agent reviewer request.\n if: |\n github.event.pull_request.head.repo.full_name == github.repository && (\n (github.event.action == 'opened' && github.event.pull_request.draft == false && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||\n (github.event.action == 'ready_for_review' && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||\n github.event.label.name == 'qa-this' ||\n github.event.requested_reviewer.login == 'openhands-agent' ||\n github.event.requested_reviewer.login == 'all-hands-bot'\n )\n concurrency:\n group: qa-changes-${{ github.event.pull_request.number }}\n cancel-in-progress: true\n runs-on: ubuntu-24.04\n timeout-minutes: 30\n steps:\n - name: Run QA Changes\n uses: OpenHands/extensions/plugins/qa-changes@main\n with:\n llm-model: litellm_proxy/claude-sonnet-4-5-20250929\n llm-base-url: https://llm-proxy.app.all-hands.dev\n max-budget: '10.0'\n timeout-minutes: '30'\n max-iterations: '500'\n llm-api-key: ${{ secrets.LLM_API_KEY }}\n github-token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}\n lmnr-api-key: ${{ secrets.LMNR_SKILLS_API_KEY }}\n" }, { "path": ".github/workflows/qa-changes-evaluation.yml", "content": "---\nname: QA Changes Evaluation\n\n# This workflow evaluates how well QA validation performed.\n# It runs when a PR is closed to assess QA effectiveness.\n#\n# Security note: pull_request_target is safe here because this workflow\n# never checks out or executes PR code. It only:\n# 1. Downloads artifacts produced by a trusted workflow run\n# 2. Runs evaluation scripts from the extensions repo (main/pinned branch)\n\non:\n pull_request_target:\n types: [closed]\n\npermissions:\n contents: read\n pull-requests: read\n\njobs:\n evaluate:\n runs-on: ubuntu-24.04\n env:\n PR_NUMBER: ${{ github.event.pull_request.number }}\n REPO_NAME: ${{ github.repository }}\n PR_MERGED: ${{ github.event.pull_request.merged }}\n\n steps:\n - name: Download QA trace artifact\n id: download-trace\n uses: dawidd6/action-download-artifact@v21\n continue-on-error: true\n with:\n workflow: qa-changes-by-openhands.yml\n name: qa-changes-trace-${{ github.event.pull_request.number }}\n path: trace-info\n search_artifacts: true\n if_no_artifact_found: warn\n\n - name: Check if trace file exists\n id: check-trace\n run: |\n if [ -f \"trace-info/laminar_trace_info.json\" ]; then\n echo \"trace_exists=true\" >> $GITHUB_OUTPUT\n echo \"Found trace file for PR #$PR_NUMBER\"\n else\n echo \"trace_exists=false\" >> $GITHUB_OUTPUT\n echo \"No trace file found for PR #$PR_NUMBER - skipping evaluation\"\n fi\n\n - name: Checkout extensions repository\n if: steps.check-trace.outputs.trace_exists == 'true'\n uses: actions/checkout@v6\n with:\n repository: OpenHands/extensions\n path: extensions\n\n - name: Set up Python\n if: steps.check-trace.outputs.trace_exists == 'true'\n uses: actions/setup-python@v6\n with:\n python-version: '3.12'\n\n - name: Install dependencies\n if: steps.check-trace.outputs.trace_exists == 'true'\n run: pip install lmnr\n\n - name: Run evaluation\n if: steps.check-trace.outputs.trace_exists == 'true'\n env:\n # Script expects LMNR_PROJECT_API_KEY; org secret is named LMNR_SKILLS_API_KEY\n LMNR_PROJECT_API_KEY: ${{ secrets.LMNR_SKILLS_API_KEY }}\n GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n run: |\n python extensions/plugins/qa-changes/scripts/evaluate_qa_changes.py \\\n --trace-file trace-info/laminar_trace_info.json\n\n - name: Upload evaluation logs\n uses: actions/upload-artifact@v7\n if: always() && steps.check-trace.outputs.trace_exists == 'true'\n with:\n name: qa-changes-evaluation-${{ github.event.pull_request.number }}\n path: '*.log'\n retention-days: 30\n" }, { "path": ".github/workflows/release-binaries.yml", "content": "---\nname: Publish agent-server release artifacts\n\n# On release published or push to main:\n# 1. Build the agent-server PyInstaller binary on Linux + macOS for both\n# x86_64 and arm64, smoke-test it, and upload workflow artifacts.\n# 2. On release events/manual runs, attach those binaries plus a combined\n# SHA256SUMS file to the GitHub release.\n# 3. Smoke-test the multi-arch Docker images pushed by `server.yml`,\n# verifying that every published variant has a manifest covering both\n# linux/amd64 and linux/arm64 and that the container actually boots\n# and answers /health on each architecture.\n\non:\n push:\n branches: [main]\n release:\n types: [published]\n workflow_dispatch:\n inputs:\n release_tag:\n description: Existing release tag (e.g. v1.20.1)\n required: true\n type: string\n\npermissions:\n contents: write\n packages: read\n\njobs:\n resolve-tag:\n name: Resolve artifact and image tag\n runs-on: ubuntu-24.04\n outputs:\n tag: ${{ steps.resolve.outputs.tag }}\n version: ${{ steps.resolve.outputs.version }}\n image_tag: ${{ steps.resolve.outputs.image_tag }}\n steps:\n - id: resolve\n shell: bash\n run: |\n set -euo pipefail\n if [[ \"${{ github.event_name }}\" == \"release\" ]]; then\n TAG=\"${{ github.event.release.tag_name }}\"\n VERSION=\"${TAG#v}\"\n elif [[ \"${{ github.event_name }}\" == \"workflow_dispatch\" ]]; then\n TAG=\"${{ inputs.release_tag }}\"\n VERSION=\"${TAG#v}\"\n elif [[ \"${{ github.event_name }}\" == \"push\" ]]; then\n TAG=\"\"\n VERSION=\"${GITHUB_SHA::7}\"\n else\n echo \"ERROR: unsupported event '${{ github.event_name }}'\"\n exit 1\n fi\n\n if [[ -n \"$TAG\" ]] && ! [[ \"$VERSION\" =~ ^[0-9]+\\.[0-9]+\\.[0-9]+([a-zA-Z0-9.+-]*)?$ ]]; then\n echo \"ERROR: unexpected version '$VERSION' (from tag '$TAG')\"\n exit 1\n fi\n\n echo \"tag=$TAG\" >> \"$GITHUB_OUTPUT\"\n echo \"version=$VERSION\" >> \"$GITHUB_OUTPUT\"\n echo \"image_tag=$VERSION\" >> \"$GITHUB_OUTPUT\"\n echo \"📦 Tag: ${TAG:-} Image tag: $VERSION\"\n\n build-binary:\n name: Build (${{ matrix.os_label }}-${{ matrix.arch }})\n needs: resolve-tag\n runs-on: ${{ matrix.runner }}\n strategy:\n fail-fast: false\n matrix:\n include:\n - runner: ubuntu-24.04\n os_label: linux\n arch: x86_64\n - runner: ubuntu-24.04-arm\n os_label: linux\n arch: arm64\n - runner: macos-13\n os_label: macos\n arch: x86_64\n - runner: macos-14\n os_label: macos\n arch: arm64\n - runner: windows-2022\n os_label: windows\n arch: x86_64\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n version: latest\n python-version: '3.13'\n\n - name: Install dependencies\n run: uv sync --dev\n\n - name: Build binary (Unix)\n if: runner.os != 'Windows'\n run: make build-server\n\n - name: Build binary (Windows)\n if: runner.os == 'Windows'\n shell: bash\n run: uv run pyinstaller openhands-agent-server/openhands/agent_server/agent-server.spec\n\n - name: Smoke-test binary\n shell: bash\n run: |\n set -euo pipefail\n\n if [[ \"${RUNNER_OS:-}\" == \"Windows\" ]]; then\n BIN=./dist/openhands-agent-server.exe\n else\n BIN=./dist/openhands-agent-server\n fi\n\n \"$BIN\" --help\n\n echo \"Testing server startup and template loading...\"\n \"$BIN\" --port 8002 > server_test.log 2>&1 &\n SERVER_PID=$!\n\n cleanup() {\n kill \"$SERVER_PID\" 2>/dev/null || true\n wait \"$SERVER_PID\" 2>/dev/null || true\n if [ -f server_test.log ]; then\n echo \"----- server_test.log (tail) -----\"\n tail -100 server_test.log || true\n rm -f server_test.log\n fi\n }\n trap cleanup EXIT\n\n # Poll /health for up to 90s; fail if it never comes up.\n for i in $(seq 1 30); do\n if grep -q \"system_prompt.j2.*not found\" server_test.log 2>/dev/null; then\n echo \"ERROR: Template files not found in binary!\"\n exit 1\n fi\n if ! kill -0 \"$SERVER_PID\" 2>/dev/null; then\n echo \"ERROR: Server process exited before /health responded\"\n exit 1\n fi\n if curl -f -s http://localhost:8002/health >/dev/null 2>&1; then\n echo \"✓ /health responded after ${i} attempt(s)\"\n echo \"✓ Binary smoke test passed\"\n exit 0\n fi\n sleep 3\n done\n\n echo \"ERROR: /health never responded within 90s\"\n exit 1\n\n - name: Stage release asset\n shell: bash\n env:\n ASSET: agent-server-${{ needs.resolve-tag.outputs.version }}-${{ matrix.os_label }}-${{ matrix.arch }}\n run: |\n set -euo pipefail\n mkdir -p release-assets\n if [[ \"${RUNNER_OS:-}\" == \"Windows\" ]]; then\n cp dist/openhands-agent-server.exe \"release-assets/${ASSET}.exe\"\n else\n cp dist/openhands-agent-server \"release-assets/${ASSET}\"\n fi\n ls -la release-assets/\n\n - name: Upload binary as workflow artifact\n uses: actions/upload-artifact@v7\n with:\n name: binary-${{ matrix.os_label }}-${{ matrix.arch }}\n path: release-assets/agent-server-*\n retention-days: 7\n if-no-files-found: error\n\n publish-binaries:\n name: Publish binaries + SHA256SUMS\n needs: [resolve-tag, build-binary]\n if: github.event_name != 'push'\n runs-on: ubuntu-24.04\n steps:\n - name: Download binary artifacts\n uses: actions/download-artifact@v8\n with:\n pattern: binary-*\n merge-multiple: true\n path: release-assets\n\n - name: Generate combined SHA256SUMS\n shell: bash\n run: |\n set -euo pipefail\n cd release-assets\n ls -la\n shasum -a 256 agent-server-* | sort > SHA256SUMS\n cat SHA256SUMS\n\n - name: Attach binaries + SHA256SUMS to release\n env:\n GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n TAG: ${{ needs.resolve-tag.outputs.tag }}\n shell: bash\n run: |\n set -euo pipefail\n cd release-assets\n gh release upload \"$TAG\" \\\n agent-server-* \\\n SHA256SUMS \\\n --clobber \\\n --repo \"${{ github.repository }}\"\n\n docker-smoke-test:\n name: Docker (${{ matrix.variant }}-${{ matrix.arch }})\n needs: resolve-tag\n runs-on: ${{ matrix.runner }}\n strategy:\n fail-fast: false\n matrix:\n include:\n - variant: python\n arch: amd64\n runner: ubuntu-24.04\n - variant: python\n arch: arm64\n runner: ubuntu-24.04-arm\n - variant: java\n arch: amd64\n runner: ubuntu-24.04\n - variant: java\n arch: arm64\n runner: ubuntu-24.04-arm\n - variant: golang\n arch: amd64\n runner: ubuntu-24.04\n - variant: golang\n arch: arm64\n runner: ubuntu-24.04-arm\n env:\n IMAGE: ghcr.io/openhands/agent-server\n IMAGE_TAG: ${{ needs.resolve-tag.outputs.image_tag }}\n VARIANT: ${{ matrix.variant }}\n ARCH: ${{ matrix.arch }}\n steps:\n - name: Set up Docker Buildx\n uses: docker/setup-buildx-action@v4\n\n - name: Log in to GHCR\n uses: docker/login-action@v4\n with:\n registry: ghcr.io\n username: ${{ github.actor }}\n password: ${{ secrets.GITHUB_TOKEN }}\n\n - name: Wait for multi-arch manifest\n shell: bash\n run: |\n set -euo pipefail\n TAG_FQN=\"${IMAGE}:${IMAGE_TAG}-${VARIANT}\"\n DEADLINE=$(( $(date +%s) + 2700 )) # 45 minutes\n while ! docker buildx imagetools inspect \"$TAG_FQN\" >/dev/null 2>&1; do\n if [ \"$(date +%s)\" -ge \"$DEADLINE\" ]; then\n echo \"ERROR: timed out waiting for $TAG_FQN\"\n exit 1\n fi\n echo \"Waiting for $TAG_FQN ...\"\n sleep 30\n done\n echo \"✓ Manifest available: $TAG_FQN\"\n\n - name: Verify manifest covers linux/amd64 + linux/arm64\n shell: bash\n run: |\n set -euo pipefail\n TAG_FQN=\"${IMAGE}:${IMAGE_TAG}-${VARIANT}\"\n PLATFORMS=$(docker buildx imagetools inspect \"$TAG_FQN\" --raw \\\n | jq -r '.manifests[]?.platform | \"\\(.os)/\\(.architecture)\"' \\\n | sort -u)\n echo \"Platforms in $TAG_FQN:\"\n echo \"$PLATFORMS\"\n for required in linux/amd64 linux/arm64; do\n if ! echo \"$PLATFORMS\" | grep -qx \"$required\"; then\n echo \"ERROR: $required missing from $TAG_FQN manifest\"\n exit 1\n fi\n done\n echo \"✓ Both linux/amd64 and linux/arm64 are present\"\n\n - name: Pull and run on linux/${{ matrix.arch }}\n shell: bash\n run: |\n set -euo pipefail\n TAG_FQN=\"${IMAGE}:${IMAGE_TAG}-${VARIANT}\"\n CONTAINER=\"agent-server-smoke-${VARIANT}-${ARCH}\"\n\n echo \"Pulling $TAG_FQN for linux/${ARCH} ...\"\n docker pull --platform=\"linux/${ARCH}\" \"$TAG_FQN\"\n\n echo \"Starting container ...\"\n docker run --platform=\"linux/${ARCH}\" -d --rm \\\n --name \"$CONTAINER\" \\\n -p 8000:8000 \\\n \"$TAG_FQN\"\n\n cleanup() {\n docker logs \"$CONTAINER\" 2>&1 | tail -100 || true\n docker rm -f \"$CONTAINER\" >/dev/null 2>&1 || true\n }\n trap cleanup EXIT\n\n for i in $(seq 1 40); do\n if curl -f -s http://localhost:8000/health >/dev/null 2>&1; then\n echo \"✓ /health responded for $TAG_FQN on linux/${ARCH}\"\n exit 0\n fi\n sleep 3\n done\n\n echo \"ERROR: /health never responded for $TAG_FQN on linux/${ARCH}\"\n exit 1\n" }, { "path": ".github/workflows/remove-duplicate-candidate-label.yml", "content": "---\nname: Remove duplicate candidate label on activity\n\non:\n issue_comment:\n types: [created]\n\npermissions:\n issues: write\n\nconcurrency:\n group: remove-duplicate-${{ github.repository }}-${{ github.event.issue.number }}\n cancel-in-progress: false\n\njobs:\n remove-duplicate-candidate:\n if: |\n github.event.issue.state == 'open' &&\n github.event.issue.pull_request == null &&\n contains(github.event.issue.labels.*.name, 'duplicate-candidate') &&\n github.event.comment.user.type != 'Bot' &&\n !startsWith(github.event.comment.body || '', '\n ---\n **Agent Server images for this PR**\n\n • **GHCR package:** ${GHCR_URL}\n\n **Variants & Base Images**\n | Variant | Architectures | Base Image | Docs / Tags |\n |---|---|---|---|\n ${VARIANTS_TABLE}\n\n **Pull (multi-arch manifest)**\n \\`\\`\\`bash\n # Each variant is a multi-arch manifest supporting both amd64 and arm64\n docker pull ${IMAGE}:${SHORT_SHA}-python\n \\`\\`\\`\n\n **Run**\n \\`\\`\\`bash\n docker run -it --rm \\\\\n -p 8000:8000 \\\\\n --name agent-server-${SHORT_SHA}-python \\\\\n ${IMAGE}:${SHORT_SHA}-python\n \\`\\`\\`\n\n **All tags pushed for this build**\n \\`\\`\\`\n ${ALL_TAGS}\n \\`\\`\\`\n\n **About Multi-Architecture Support**\n - Each variant tag (e.g., \\`${SHORT_SHA}-python\\`) is a **multi-arch manifest** supporting both **amd64** and **arm64**\n - Docker automatically pulls the correct architecture for your platform\n - Individual architecture tags (e.g., \\`${SHORT_SHA}-python-amd64\\`) are also available if needed\n \n EOF\n )\n\n # Set output for the next step\n {\n echo 'pr_content<> $GITHUB_OUTPUT\n\n - name: Update PR description with docker image details\n uses: nefrob/pr-description@v1.2.0\n with:\n content: ${{ steps.generate_description.outputs.pr_content }}\n regex: .*?\n regexFlags: s\n token: ${{ secrets.GITHUB_TOKEN }}\n" }, { "path": ".github/workflows/stale.yml", "content": "---\n# Workflow that marks issues and PRs with no activity for 30 days with \"Stale\" and closes them after 7 more days of no activity\nname: Close stale issues\n\n# Runs every day at 01:30\non:\n schedule:\n - cron: 30 1 * * *\n\npermissions:\n issues: write\n pull-requests: write\n\njobs:\n stale:\n # Only run scheduled jobs in the main repository, not in forks\n if: github.repository == 'OpenHands/software-agent-sdk'\n runs-on: ubuntu-22.04\n steps:\n - uses: actions/stale@v10\n with:\n repo-token: ${{ secrets.GITHUB_TOKEN }}\n stale-issue-message: This issue is stale because it has been open for 40 days with no activity. Remove the stale label or leave a \n comment, otherwise it will be closed in 10 days.\n stale-pr-message: This PR is stale because it has been open for 40 days with no activity. Remove the stale label or leave a comment,\n otherwise it will be closed in 10 days.\n days-before-stale: 40\n exempt-issue-labels: roadmap,backlog\n close-issue-message: This issue was automatically closed due to 50 days of inactivity. We do this to help keep the issues somewhat \n manageable and focus on active issues.\n close-pr-message: This PR was closed because it had no activity for 50 days. If you feel this was closed in error, and you would \n like to continue the PR, please resubmit or let us know.\n days-before-close: 10\n operations-per-run: 150\n" }, { "path": ".github/workflows/tests.yml", "content": "---\nname: Run tests\n\non:\n push:\n branches: [main]\n pull_request:\n branches: ['**']\n\npermissions:\n contents: write\n pull-requests: write\n\njobs:\n test-directory-guard:\n name: Test directory allowlist\n runs-on: ubuntu-latest\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n\n - name: Verify test directories\n run: |\n # Allowed top-level directories under tests/\n # Each must have a corresponding CI job or workflow that runs them.\n # tests.yml: sdk, tools, workspace, agent_server, cross\n # run-examples.yml: examples\n # integration-runner.yml: integration\n # (data-only): fixtures\n ALLOWED=\"sdk tools workspace agent_server cross examples integration fixtures\"\n\n violations=\"\"\n for entry in tests/*/; do\n dir_name=\"$(basename \"$entry\")\"\n # skip __pycache__ and hidden dirs\n [[ \"$dir_name\" == __* || \"$dir_name\" == .* ]] && continue\n if ! echo \"$ALLOWED\" | grep -qw \"$dir_name\"; then\n violations=\"$violations tests/$dir_name/\\n\"\n fi\n done\n\n # Also reject top-level test files (they won't be picked up by any job)\n for f in tests/test_*.py; do\n [ -f \"$f\" ] && violations=\"$violations $f\\n\"\n done\n\n # Detect test files hiding inside source packages instead of tests/\n # Excludes */testing/* dirs (testing utilities, not runnable tests)\n stray=$(find openhands-sdk openhands-tools openhands-workspace openhands-agent-server \\\n \\( -name 'test_*.py' -o -name '*_test.py' \\) \\\n -not -path '*/testing/*' \\\n 2>/dev/null || true)\n for f in $stray; do\n violations=\"$violations $f (stray test outside tests/)\\n\"\n done\n\n if [ -n \"$violations\" ]; then\n echo \"ERROR: Found test paths outside the allowed directories.\"\n echo \"The following will NOT be run by any CI job:\"\n echo \"\"\n printf \"$violations\"\n echo \"\"\n echo \"Allowed directories: $ALLOWED\"\n echo \"Move tests into one of the allowed directories so CI can run them.\"\n exit 1\n fi\n echo \"✓ All test directories are in the allowlist\"\n\n sdk-tests:\n runs-on: blacksmith-2vcpu-ubuntu-2404\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with: {fetch-depth: 0}\n\n - name: Detect sdk changes\n id: changed\n uses: tj-actions/changed-files@v47\n with:\n files: |\n openhands-sdk/**\n tests/sdk/**\n pyproject.toml\n uv.lock\n .github/workflows/tests.yml\n\n - name: Install uv\n if: steps.changed.outputs.any_changed == 'true'\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n python-version: '3.13'\n\n - name: Install deps\n if: steps.changed.outputs.any_changed == 'true'\n run: uv sync --frozen --group dev\n\n - name: Check for openhands.tools imports in sdk tests\n if: steps.changed.outputs.any_changed == 'true'\n run: |\n echo \"Checking for openhands.tools imports in tests/sdk...\"\n if grep -r \"from openhands\\.tools\" tests/sdk/ || grep -r \"import openhands\\.tools\" tests/sdk/; then\n echo \"ERROR: Found openhands.tools imports in tests/sdk/\"\n echo \"SDK tests should only import from openhands.sdk\"\n echo \"Please move tests that use openhands.tools to tests/cross/\"\n exit 1\n fi\n echo \"✓ No openhands.tools imports found in tests/sdk/\"\n\n - name: Run sdk tests with coverage\n if: steps.changed.outputs.any_changed == 'true'\n run: |\n # Clean up any existing coverage file\n rm -f .coverage\n # Use pytest-xdist (-n auto) for parallel execution with proper\n # coverage collection. --forked prevents coverage from child processes.\n CI=true uv run python -m pytest -vvs \\\n -n auto \\\n --cov=openhands-sdk \\\n --cov-report=term-missing \\\n --cov-fail-under=0 \\\n --cov-config=pyproject.toml \\\n tests/sdk\n # Rename coverage file for upload\n if [ -f .coverage ]; then\n mv .coverage coverage-sdk.dat\n echo \"SDK coverage file prepared for upload\"\n fi\n\n - name: Upload sdk coverage\n if: steps.changed.outputs.any_changed == 'true' && always()\n uses: actions/upload-artifact@v7\n with:\n name: coverage-sdk\n path: coverage-sdk.dat\n if-no-files-found: warn\n\n tools-tests:\n runs-on: blacksmith-2vcpu-ubuntu-2404\n timeout-minutes: 15\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with: {fetch-depth: 0}\n\n - name: Detect tools changes\n id: changed\n uses: tj-actions/changed-files@v47\n with:\n files: |\n openhands-tools/**\n tests/tools/**\n pyproject.toml\n uv.lock\n .github/workflows/tests.yml\n\n - name: Install uv\n if: steps.changed.outputs.any_changed == 'true'\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n python-version: '3.13'\n\n - name: Install deps\n if: steps.changed.outputs.any_changed == 'true'\n run: uv sync --frozen --group dev\n\n - name: Run tools tests with coverage\n if: steps.changed.outputs.any_changed == 'true'\n run: |\n # Clean up any existing coverage file\n rm -f .coverage\n # Use --forked for tools tests due to terminal test conflicts\n # when running in parallel (shared /tmp paths, subprocess management)\n CI=true uv run python -m pytest -vvs \\\n --forked \\\n --cov=openhands-tools \\\n --cov-report=term-missing \\\n --cov-fail-under=0 \\\n --cov-config=pyproject.toml \\\n tests/tools\n # Rename coverage file for upload\n if [ -f .coverage ]; then\n mv .coverage coverage-tools.dat\n echo \"Tools coverage file prepared for upload\"\n fi\n\n - name: Upload tools coverage\n if: steps.changed.outputs.any_changed == 'true' && always()\n uses: actions/upload-artifact@v7\n with:\n name: coverage-tools\n path: coverage-tools.dat\n if-no-files-found: warn\n\n windows-tests:\n runs-on: windows-latest\n timeout-minutes: 30\n env:\n PYTHONUTF8: '1'\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with: {fetch-depth: 0}\n\n - name: Detect Windows-relevant changes\n id: changed\n uses: tj-actions/changed-files@v47\n with:\n files: |\n openhands-tools/**\n tests/tools/**\n pyproject.toml\n uv.lock\n .github/workflows/tests.yml\n\n - name: Install uv\n if: steps.changed.outputs.any_changed == 'true'\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n python-version: '3.13'\n\n - name: Install deps\n if: steps.changed.outputs.any_changed == 'true'\n run: uv sync --frozen --group dev\n\n - name: Install Chromium\n if: steps.changed.outputs.any_changed == 'true'\n run: uvx playwright install chromium\n\n - name: Run Windows test suite\n if: steps.changed.outputs.any_changed == 'true'\n run: |\n if (Test-Path .coverage) {\n Remove-Item .coverage -Force\n }\n $env:CI = 'true'\n # Keep the initial Windows pass non-blocking on coverage while\n # OS-specific gaps tracked in #2989 are still open.\n # Browser/file-editor e2e and terminal shell assumptions remain\n # tracked in #2986 and #2988.\n uv run python -m pytest -vvs `\n --cov=openhands-tools `\n --cov-report=term-missing `\n --cov-fail-under=0 `\n --cov-config=pyproject.toml `\n tests/tools `\n --ignore=tests/tools/browser_use/test_browser_executor_e2e.py `\n --ignore=tests/tools/file_editor/test_memory_usage.py `\n --ignore=tests/tools/terminal/test_conversation_cleanup.py `\n --ignore=tests/tools/terminal/test_session_factory.py `\n --ignore=tests/tools/terminal/test_shell_path_configuration.py `\n --ignore=tests/tools/terminal/test_shutdown_handling.py `\n --ignore=tests/tools/terminal/test_terminal_session.py `\n --ignore=tests/tools/terminal/test_terminal_tool_auto_detection.py\n if (Test-Path .coverage) {\n Move-Item .coverage coverage-windows.dat\n Write-Host 'Windows coverage file prepared for upload'\n }\n\n - name: Upload Windows coverage\n if: steps.changed.outputs.any_changed == 'true' && always()\n uses: actions/upload-artifact@v7\n with:\n name: coverage-windows\n path: coverage-windows.dat\n if-no-files-found: warn\n\n\n agent-server-tests:\n runs-on: blacksmith-2vcpu-ubuntu-2404\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with: {fetch-depth: 0}\n\n - name: Detect Agent Server changes\n id: changed\n uses: tj-actions/changed-files@v47\n with:\n files: |\n openhands-agent-server/**\n tests/agent_server/**\n pyproject.toml\n uv.lock\n .github/workflows/tests.yml\n\n - name: Install uv\n if: steps.changed.outputs.any_changed == 'true'\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n python-version: '3.13'\n\n - name: Install deps\n if: steps.changed.outputs.any_changed == 'true'\n run: uv sync --frozen --group dev\n\n - name: Run Agent Server tests with coverage\n if: steps.changed.outputs.any_changed == 'true'\n run: |\n # Clean up any existing coverage file\n rm -f .coverage\n # Use pytest-xdist (-n auto) for parallel execution with proper\n # coverage collection. --forked prevents coverage from child processes.\n CI=true uv run python -m pytest -vvs \\\n -n auto \\\n --cov=openhands-agent-server \\\n --cov-report=term-missing \\\n --cov-fail-under=0 \\\n --cov-config=pyproject.toml \\\n tests/agent_server\n # Rename coverage file for upload\n if [ -f .coverage ]; then\n mv .coverage coverage-agent-server.dat\n echo \"Agent Server coverage file prepared for upload\"\n fi\n\n - name: Upload Agent Server coverage\n if: steps.changed.outputs.any_changed == 'true' && always()\n uses: actions/upload-artifact@v7\n with:\n name: coverage-agent-server\n path: coverage-agent-server.dat\n if-no-files-found: warn\n\n workspace-tests:\n runs-on: blacksmith-2vcpu-ubuntu-2404\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with: {fetch-depth: 0}\n\n - name: Detect workspace changes\n id: changed\n uses: tj-actions/changed-files@v47\n with:\n files: |\n openhands-workspace/**\n tests/workspace/**\n pyproject.toml\n uv.lock\n .github/workflows/tests.yml\n\n - name: Install uv\n if: steps.changed.outputs.any_changed == 'true'\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n python-version: '3.13'\n\n - name: Install deps\n if: steps.changed.outputs.any_changed == 'true'\n run: uv sync --frozen --group dev\n\n - name: Run workspace tests with coverage\n if: steps.changed.outputs.any_changed == 'true'\n run: |\n # Clean up any existing coverage file\n rm -f .coverage\n CI=true uv run python -m pytest -vvs \\\n -n auto \\\n --cov=openhands-workspace \\\n --cov-report=term-missing \\\n --cov-fail-under=0 \\\n --cov-config=pyproject.toml \\\n tests/workspace\n # Rename coverage file for upload\n if [ -f .coverage ]; then\n mv .coverage coverage-workspace.dat\n echo \"Workspace coverage file prepared for upload\"\n fi\n\n - name: Upload workspace coverage\n if: steps.changed.outputs.any_changed == 'true' && always()\n uses: actions/upload-artifact@v7\n with:\n name: coverage-workspace\n path: coverage-workspace.dat\n if-no-files-found: warn\n\n cross-tests:\n runs-on: blacksmith-2vcpu-ubuntu-2404\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with: {fetch-depth: 0}\n\n - name: Detect cross changes\n id: changed\n uses: tj-actions/changed-files@v47\n with:\n files: |\n tests/**\n openhands/**\n pyproject.toml\n uv.lock\n .github/workflows/tests.yml\n\n - name: Install uv\n if: steps.changed.outputs.any_changed == 'true'\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n python-version: '3.13'\n\n - name: Install deps\n if: steps.changed.outputs.any_changed == 'true'\n run: uv sync --frozen --group dev\n\n - name: Run cross tests with coverage\n if: steps.changed.outputs.any_changed == 'true'\n run: |\n # Clean up any existing coverage file\n rm -f .coverage\n CI=true uv run python -m pytest -vvs \\\n --basetemp=\"${{ runner.temp }}/pytest\" \\\n -o tmp_path_retention=none \\\n -o tmp_path_retention_count=0 \\\n --cov=openhands \\\n --cov-report=term-missing \\\n --cov-fail-under=0 \\\n --cov-config=pyproject.toml \\\n tests/cross\n # Rename coverage file for upload\n if [ -f .coverage ]; then\n mv .coverage coverage-cross.dat\n echo \"Cross coverage file prepared for upload\"\n fi\n\n - name: Upload cross coverage\n if: steps.changed.outputs.any_changed == 'true' && always()\n uses: actions/upload-artifact@v7\n with:\n name: coverage-cross\n path: coverage-cross.dat\n if-no-files-found: warn\n\n coverage-report:\n runs-on: blacksmith-2vcpu-ubuntu-2404\n needs: [sdk-tests, tools-tests, agent-server-tests, workspace-tests, cross-tests]\n if: always() && github.event_name == 'pull_request'\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n python-version: '3.13'\n\n - name: Install deps (for coverage CLI)\n run: uv sync --frozen --group dev\n\n - name: Download coverage artifacts\n uses: actions/download-artifact@v8\n with:\n path: ./cov\n continue-on-error: true\n\n - name: Combine coverage data\n run: |\n shopt -s nullglob\n # For some reason, the github action won't properly upload the original\n # .converage* files\n # Convert uploaded .dat files back to .coverage format for coverage tool\n for dat_file in cov/**/coverage-*.dat; do\n if [[ \"$dat_file\" == *coverage-sdk.dat ]]; then\n cp \"$dat_file\" .coverage.sdk\n elif [[ \"$dat_file\" == *coverage-tools.dat ]]; then\n cp \"$dat_file\" .coverage.tools \n elif [[ \"$dat_file\" == *coverage-agent-server.dat ]]; then\n cp \"$dat_file\" .coverage.agent-server\n elif [[ \"$dat_file\" == *coverage-workspace.dat ]]; then\n cp \"$dat_file\" .coverage.workspace\n elif [[ \"$dat_file\" == *coverage-cross.dat ]]; then\n cp \"$dat_file\" .coverage.cross\n fi\n done\n\n # Check if we have any coverage files\n coverage_files=(.coverage.*)\n if [ ${#coverage_files[@]} -eq 0 ]; then\n echo \"No coverage files found; skipping combined report.\"\n exit 0\n fi\n\n echo \"Found ${#coverage_files[@]} coverage files\"\n uv run coverage combine\n uv run coverage xml -i -o coverage.xml\n uv run coverage report -m\n\n - name: Pytest coverage PR comment\n if: always()\n continue-on-error: true\n uses: MishaKav/pytest-coverage-comment@v1\n with:\n github-token: ${{ secrets.GITHUB_TOKEN }}\n pytest-xml-coverage-path: coverage.xml\n title: Coverage Report\n create-new-comment: false\n hide-report: false\n xml-skip-covered: true\n report-only-changed-files: true\n remove-links-to-files: true\n remove-links-to-lines: true\n" }, { "path": ".github/workflows/todo-management.yml", "content": "---\n# Automated TODO Management Workflow\n#\n# This workflow automatically scans for TODO(openhands) comments and creates\n# pull requests to implement them using the OpenHands agent.\n#\n# Setup:\n# 1. Add LLM_API_KEY to repository secrets\n# 2. Ensure GITHUB_TOKEN has appropriate permissions\n# 3. Make sure Github Actions are allowed to create and review PRs\n# 4. Commit this file to .github/workflows/ in your repository\n# 5. Configure the schedule or trigger manually\n\nname: Automated TODO Management\n\non:\n # Manual trigger\n workflow_dispatch:\n inputs:\n max_todos:\n description: Maximum number of TODOs to process in this run\n required: false\n default: '3'\n type: string\n todo_identifier:\n description: TODO identifier to search for (e.g., TODO(openhands))\n required: false\n default: TODO(openhands)\n type: string\n\n # Trigger when 'automatic-todo' label is added to a PR\n pull_request:\n types: [labeled]\n\n # Scheduled trigger (disabled by default, uncomment and customize as needed)\n # schedule:\n # # Run every Monday at 9 AM UTC\n # - cron: \"0 9 * * 1\"\n\npermissions:\n contents: write\n pull-requests: write\n issues: write\n\njobs:\n scan-todos:\n runs-on: ubuntu-24.04\n # Only run if triggered manually or if 'automatic-todo' label was added\n if: >\n github.event_name == 'workflow_dispatch' ||\n (github.event_name == 'pull_request' &&\n github.event.label.name == 'automatic-todo')\n outputs:\n todos: ${{ steps.scan.outputs.todos }}\n todo-count: ${{ steps.scan.outputs.todo-count }}\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n with:\n fetch-depth: 0 # Full history for better context\n\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: '3.13'\n\n - name: Copy TODO scanner\n run: |\n cp examples/03_github_workflows/03_todo_management/scanner.py /tmp/scanner.py\n chmod +x /tmp/scanner.py\n\n - name: Scan for TODOs\n id: scan\n run: |\n echo \"Scanning for TODO comments...\"\n\n # Run the scanner and capture output\n TODO_IDENTIFIER=\"${{ github.event.inputs.todo_identifier || 'TODO(openhands)' }}\"\n python /tmp/scanner.py . --identifier \"$TODO_IDENTIFIER\" > todos.json\n\n # Count TODOs\n TODO_COUNT=$(python -c \\\n \"import json; data=json.load(open('todos.json')); print(len(data))\")\n echo \"Found $TODO_COUNT $TODO_IDENTIFIER items\"\n\n # Limit the number of TODOs to process\n MAX_TODOS=\"${{ github.event.inputs.max_todos || '3' }}\"\n if [ \"$TODO_COUNT\" -gt \"$MAX_TODOS\" ]; then\n echo \"Limiting to first $MAX_TODOS TODOs\"\n python -c \"\n import json\n data = json.load(open('todos.json'))\n limited = data[:$MAX_TODOS]\n json.dump(limited, open('todos.json', 'w'), indent=2)\n \"\n TODO_COUNT=$MAX_TODOS\n fi\n\n # Set outputs\n echo \"todos=$(cat todos.json | jq -c .)\" >> $GITHUB_OUTPUT\n echo \"todo-count=$TODO_COUNT\" >> $GITHUB_OUTPUT\n\n # Display found TODOs\n echo \"## 📋 Found TODOs\" >> $GITHUB_STEP_SUMMARY\n if [ \"$TODO_COUNT\" -eq 0 ]; then\n echo \"No TODO(openhands) comments found.\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"Found $TODO_COUNT TODO(openhands) items:\" \\\n >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n python -c \"\n import json\n data = json.load(open('todos.json'))\n for i, todo in enumerate(data, 1):\n print(f'{i}. **{todo[\\\"file\\\"]}:{todo[\\\"line\\\"]}** - ' +\n f'{todo[\\\"description\\\"]}')\n \" >> $GITHUB_STEP_SUMMARY\n fi\n\n process-todos:\n needs: scan-todos\n if: needs.scan-todos.outputs.todo-count > 0\n runs-on: ubuntu-24.04\n strategy:\n matrix:\n todo: ${{ fromJson(needs.scan-todos.outputs.todos) }}\n max-parallel: 1 # Process one TODO at a time to avoid conflicts\n steps:\n - name: Checkout repository\n uses: actions/checkout@v6\n with:\n fetch-depth: 0\n token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}\n\n - name: Switch to feature branch with TODO management files\n run: |\n git checkout openhands/todo-management-example\n git pull origin openhands/todo-management-example\n\n - name: Set up Python\n uses: actions/setup-python@v6\n with:\n python-version: '3.13'\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n enable-cache: true\n\n - name: Install OpenHands dependencies\n run: |\n # Install OpenHands SDK and tools from git repository\n uv pip install --system \"openhands-sdk @ git+https://github.com/OpenHands/agent-sdk.git@main#subdirectory=openhands-sdk\"\n uv pip install --system \"openhands-tools @ git+https://github.com/OpenHands/agent-sdk.git@main#subdirectory=openhands-tools\"\n\n - name: Copy agent files\n run: |\n cp examples/03_github_workflows/03_todo_management/agent_script.py agent.py\n cp examples/03_github_workflows/03_todo_management/prompt.py prompt.py\n chmod +x agent.py\n\n - name: Configure Git\n run: |\n git config --global user.name \"openhands-bot\"\n git config --global user.email \\\n \"openhands-bot@users.noreply.github.com\"\n\n - name: Process TODO\n env:\n LLM_MODEL: litellm_proxy/claude-sonnet-4-5-20250929\n LLM_BASE_URL: https://llm-proxy.app.all-hands.dev\n LLM_API_KEY: ${{ secrets.LLM_API_KEY }}\n GITHUB_TOKEN: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}\n GITHUB_REPOSITORY: ${{ github.repository }}\n TODO_FILE: ${{ matrix.todo.file }}\n TODO_LINE: ${{ matrix.todo.line }}\n TODO_DESCRIPTION: ${{ matrix.todo.description }}\n PYTHONPATH: ''\n run: |\n echo \"Processing TODO: $TODO_DESCRIPTION\"\n echo \"File: $TODO_FILE:$TODO_LINE\"\n\n # Create a unique branch name for this TODO\n BRANCH_NAME=\"todo/$(echo \"$TODO_DESCRIPTION\" | \\\n sed 's/[^a-zA-Z0-9]/-/g' | \\\n sed 's/--*/-/g' | \\\n sed 's/^-\\|-$//g' | \\\n tr '[:upper:]' '[:lower:]' | \\\n cut -c1-50)\"\n echo \"Branch name: $BRANCH_NAME\"\n\n # Create and switch to new branch (force create if exists)\n git checkout -B \"$BRANCH_NAME\"\n\n # Run the agent to process the TODO\n # Stay in repository directory for git operations\n\n # Create JSON payload for the agent\n TODO_JSON=$(cat <&1 | tee agent_output.log\n AGENT_EXIT_CODE=$?\n set -e\n\n echo \"Agent exit code: $AGENT_EXIT_CODE\"\n echo \"Agent output log:\"\n cat agent_output.log\n\n # Show files in working directory\n echo \"Files in working directory:\"\n ls -la\n\n # If agent failed, show more details\n if [ $AGENT_EXIT_CODE -ne 0 ]; then\n echo \"Agent failed with exit code $AGENT_EXIT_CODE\"\n echo \"Last 50 lines of agent output:\"\n tail -50 agent_output.log\n exit $AGENT_EXIT_CODE\n fi\n\n # Check if any changes were made\n cd \"$GITHUB_WORKSPACE\"\n if git diff --quiet; then\n echo \"No changes made by agent, skipping PR creation\"\n exit 0\n fi\n\n # Commit changes\n git add -A\n git commit -m \"Implement TODO: $TODO_DESCRIPTION\n\n Automatically implemented by OpenHands agent.\n\n Co-authored-by: openhands \"\n\n # Push branch\n git push origin \"$BRANCH_NAME\"\n\n # Create pull request\n PR_TITLE=\"Implement TODO: $TODO_DESCRIPTION\"\n PR_BODY=\"## 🤖 Automated TODO Implementation\n\n This PR automatically implements the following TODO:\n\n **File:** \\`$TODO_FILE:$TODO_LINE\\`\n **Description:** $TODO_DESCRIPTION\n\n ### Implementation\n The OpenHands agent has analyzed the TODO and implemented the\n requested functionality.\n\n ### Review Notes\n - Please review the implementation for correctness\n - Test the changes in your development environment\n - The original TODO comment will be updated with this PR URL\n once merged\n\n ---\n *This PR was created automatically by the TODO Management workflow.*\"\n\n # Create PR using GitHub CLI or API\n curl -X POST \\\n -H \"Authorization: token $GITHUB_TOKEN\" \\\n -H \"Accept: application/vnd.github.v3+json\" \\\n \"https://api.github.com/repos/${{ github.repository }}/pulls\" \\\n -d \"{\n \\\"title\\\": \\\"$PR_TITLE\\\",\n \\\"body\\\": \\\"$PR_BODY\\\",\n \\\"head\\\": \\\"$BRANCH_NAME\\\",\n \\\"base\\\": \\\"${{ github.ref_name }}\\\"\n }\"\n\n summary:\n needs: [scan-todos, process-todos]\n if: always()\n runs-on: ubuntu-24.04\n steps:\n - name: Generate Summary\n run: |\n echo \"# 🤖 TODO Management Summary\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n\n TODO_COUNT=\"${{ needs.scan-todos.outputs.todo-count || '0' }}\"\n echo \"**TODOs Found:** $TODO_COUNT\" >> $GITHUB_STEP_SUMMARY\n\n if [ \"$TODO_COUNT\" -gt 0 ]; then\n echo \"**Processing Status:** ✅ Completed\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"Check the pull requests created for each TODO\" \\\n \"implementation.\" >> $GITHUB_STEP_SUMMARY\n else\n echo \"**Status:** ℹ️ No TODOs found to process\" \\\n >> $GITHUB_STEP_SUMMARY\n fi\n\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"---\" >> $GITHUB_STEP_SUMMARY\n echo \"*Workflow completed at $(date)*\" >> $GITHUB_STEP_SUMMARY\n" }, { "path": ".github/workflows/version-bump-guard.yml", "content": "---\nname: Version bump guard\n\non:\n pull_request:\n branches: [main]\n\njobs:\n version-bump-guard:\n name: Check package versions\n runs-on: ubuntu-latest\n permissions:\n contents: read\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n with:\n fetch-depth: 0\n\n - name: Validate package version changes\n env:\n VERSION_BUMP_BASE_REF: ${{ github.base_ref }}\n PR_TITLE: ${{ github.event.pull_request.title }}\n PR_HEAD_REF: ${{ github.event.pull_request.head.ref }}\n run: python3 .github/scripts/check_version_bumps.py\n" }, { "path": ".github/workflows/version-bump-prs.yml", "content": "---\nname: Create Version Bump PRs\n\non:\n # Dispatched by pypi-release.yml after a successful publish.\n # Also supports manual reruns for a specific version.\n workflow_dispatch:\n inputs:\n version:\n description: Version to bump to (e.g., 1.11.3)\n required: true\n type: string\n\njobs:\n create-version-bump-prs:\n runs-on: ubuntu-24.04\n env:\n GH_TOKEN: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}\n SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}\n steps:\n - name: Checkout\n uses: actions/checkout@v6\n\n - name: Get version from release or input\n id: get_version\n run: |\n VERSION=\"${{ github.event.inputs.version }}\"\n echo \"version=$VERSION\" >> $GITHUB_OUTPUT\n echo \"📦 Version: $VERSION\"\n\n - name: Validate version\n env:\n VERSION: ${{ steps.get_version.outputs.version }}\n run: |\n if [ -z \"$VERSION\" ]; then\n echo \"❌ Version is empty\"\n exit 1\n fi\n echo \"📦 Creating version bump PRs for version: $VERSION\"\n\n - name: Install uv\n uses: astral-sh/setup-uv@v7\n with:\n version: latest\n python-version: '3.12'\n enable-cache: false\n\n - name: Wait for packages to be available on PyPI\n env:\n VERSION: ${{ steps.get_version.outputs.version }}\n run: |\n set -euo pipefail\n\n PACKAGES=(\n openhands-sdk\n openhands-tools\n openhands-workspace\n openhands-agent-server\n )\n\n MAX_ATTEMPTS=60\n SLEEP_SECONDS=20\n\n echo \"⏳ Waiting for packages to be available on PyPI...\"\n\n # Use uv pip compile --dry-run to verify packages are resolvable\n # via the Simple API (the same index uv add uses).\n # The JSON API propagates faster than the Simple API, so a curl\n # check alone is insufficient.\n # Keep this isolated from the SDK repo's exclude-newer guardrail:\n # this workflow intentionally consumes just-published packages.\n for PKG in \"${PACKAGES[@]}\"; do\n echo \"Checking $PKG==$VERSION...\"\n ATTEMPT=1\n while [ $ATTEMPT -le $MAX_ATTEMPTS ]; do\n if uv pip compile --no-config --no-cache --python-version 3.12 - <<< \"$PKG==$VERSION\" > /dev/null 2>&1; then\n echo \"✅ $PKG==$VERSION is resolvable on PyPI\"\n break\n fi\n\n echo \" Attempt $ATTEMPT/$MAX_ATTEMPTS: $PKG==$VERSION not yet resolvable, waiting ${SLEEP_SECONDS}s...\"\n sleep $SLEEP_SECONDS\n ATTEMPT=$((ATTEMPT + 1))\n done\n\n if [ $ATTEMPT -gt $MAX_ATTEMPTS ]; then\n echo \"❌ Timeout waiting for $PKG==$VERSION to be resolvable on PyPI\"\n exit 1\n fi\n done\n\n echo \"✅ All packages are resolvable on PyPI!\"\n\n # OpenHands-CLI step runs first since it's simpler and less error-prone\n - name: Create PR for OpenHands-CLI repo\n env:\n VERSION: ${{ steps.get_version.outputs.version }}\n run: |\n set -euo pipefail\n\n REPO=\"OpenHands/openhands-cli\"\n BRANCH=\"bump-sdk-$VERSION\"\n\n echo \"🔄 Creating PR for $REPO...\"\n\n # Clone the repo\n git clone \"https://x-access-token:${GH_TOKEN}@github.com/${REPO}.git\" openhands-cli-repo\n cd openhands-cli-repo\n\n # Configure git\n git config user.name \"github-actions[bot]\"\n git config user.email \"github-actions[bot]@users.noreply.github.com\"\n\n # Check if branch already exists on remote\n if git ls-remote --heads origin \"$BRANCH\" | grep -q \"$BRANCH\"; then\n echo \"⚠️ Branch $BRANCH already exists, checking out existing branch\"\n git fetch origin \"$BRANCH\"\n git checkout \"$BRANCH\"\n else\n # Create branch\n git checkout -b \"$BRANCH\"\n fi\n\n # OpenHands-CLI currently requires Python 3.12, so resolve with that interpreter.\n # The target repo uses exclude-newer-package to exempt openhands-sdk/tools\n # from its 7-day freshness guardrail, so no UV_EXCLUDE_NEWER override\n # is needed — doing so would actually break the per-package exemptions.\n # We use --no-cache to avoid stale index data from just-published packages.\n uv add --python 3.12 --no-cache \\\n \"openhands-sdk==$VERSION\" \\\n \"openhands-tools==$VERSION\"\n\n # Check if there are changes\n if git diff --quiet; then\n echo \"⚠️ No changes detected in $REPO - versions may already be up to date\"\n exit 0\n fi\n\n # Commit and push\n git add pyproject.toml uv.lock\n git commit -m \"Bump openhands-sdk, openhands-tools to $VERSION\" \\\n -m \"Automated version bump after PyPI release.\" \\\n -m \"Co-authored-by: openhands \"\n git push -u origin \"$BRANCH\"\n\n # Check if PR already exists\n EXISTING_PR=$(gh pr list --repo \"$REPO\" --head \"$BRANCH\" --json number --jq '.[0].number')\n if [ -n \"$EXISTING_PR\" ]; then\n echo \"✅ PR #$EXISTING_PR already exists for $REPO\"\n else\n # Create PR\n gh pr create \\\n --repo \"$REPO\" \\\n --title \"Bump SDK packages to v$VERSION\" \\\n --body \"## Automated Version Bump\n\n This PR updates the following packages to version **$VERSION**:\n - \\`openhands-sdk\\`\n - \\`openhands-tools\\`\n\n **Triggered by:** Release of [software-agent-sdk v$VERSION](https://github.com/OpenHands/software-agent-sdk/releases/tag/v$VERSION)\n\n ---\n _This PR was automatically created by the version-bump-prs workflow._\" \\\n --base main \\\n --head \"$BRANCH\"\n\n echo \"✅ PR created for $REPO\"\n fi\n\n - name: Create PR for OpenHands repo\n env:\n VERSION: ${{ steps.get_version.outputs.version }}\n run: |\n set -euo pipefail\n\n REPO=\"OpenHands/OpenHands\"\n BRANCH=\"bump-sdk-$VERSION\"\n\n echo \"🔄 Creating PR for $REPO...\"\n\n # Clone the repo\n git clone \"https://x-access-token:${GH_TOKEN}@github.com/${REPO}.git\" openhands-repo\n cd openhands-repo\n\n # Configure git\n git config user.name \"github-actions[bot]\"\n git config user.email \"github-actions[bot]@users.noreply.github.com\"\n\n # Check if branch already exists on remote\n if git ls-remote --heads origin \"$BRANCH\" | grep -q \"$BRANCH\"; then\n echo \"⚠️ Branch $BRANCH already exists, checking out existing branch\"\n git fetch origin \"$BRANCH\"\n git checkout \"$BRANCH\"\n else\n # Create branch\n git checkout -b \"$BRANCH\"\n fi\n\n # Match the base branch's lockfile generator so reruns can\n # repair any existing bump branch that used a newer Poetry.\n POETRY_VERSION=$(git show origin/main:poetry.lock | sed -n -E 's/^# This file is automatically @generated by Poetry ([^ ]+) and should not be changed by hand\\.$/\\1/p')\n if [ -z \"$POETRY_VERSION\" ]; then\n echo \"❌ Could not determine Poetry version from poetry.lock\"\n exit 1\n fi\n echo \"📦 Installing Poetry $POETRY_VERSION from poetry.lock...\"\n pipx install \"poetry==$POETRY_VERSION\"\n poetry --version\n\n # 1. Update versions in pyproject.toml and poetry.lock using poetry (root)\n # The --lock flag updates both pyproject.toml AND poetry.lock\n # Note: enterprise/pyproject.toml gets these dependencies transitively via openhands-ai\n echo \"📝 Updating root pyproject.toml and poetry.lock...\"\n\n # Verify enterprise/pyproject.toml does NOT have SDK packages explicitly listed\n # If they exist there, they will become stale since we only update root pyproject.toml\n if [ -f \"enterprise/pyproject.toml\" ]; then\n echo \"🔍 Verifying enterprise/pyproject.toml doesn't have explicit SDK packages...\"\n SDK_PACKAGES=(\"openhands-sdk\" \"openhands-tools\" \"openhands-agent-server\")\n for pkg in \"${SDK_PACKAGES[@]}\"; do\n # Match package name as a TOML key (with optional leading whitespace) followed by =\n # This catches both 'openhands-sdk = \"1.2.3\"' and 'openhands-sdk=\"1.2.3\"'\n if grep -qE \"^[[:space:]]*${pkg}[[:space:]]*=\" enterprise/pyproject.toml; then\n echo \"❌ ERROR: enterprise/pyproject.toml contains explicit reference to '$pkg'\"\n echo \" These packages should come transitively via openhands-ai dependency.\"\n echo \" Please remove '$pkg' from enterprise/pyproject.toml to avoid version drift.\"\n exit 1\n fi\n done\n echo \"✅ enterprise/pyproject.toml does not have explicit SDK packages\"\n fi\n\n # 1. Update versions in pyproject.toml using sed for exact pinning\n # Note: We use sed instead of `poetry add --lock` because Poetry normalizes\n # version constraints (e.g., \"==1.13.1\" becomes \"1.13\") which causes\n # inconsistencies between [tool.poetry.dependencies] and [project].dependencies\n echo \"📝 Updating pyproject.toml with exact version pins...\"\n\n PYPROJECT_FMT_CONFIG=\"dev_config/python/.pre-commit-config.yaml\"\n if [ ! -f \"$PYPROJECT_FMT_CONFIG\" ]; then\n echo \"❌ pyproject-fmt config not found at expected path\"\n exit 1\n fi\n if ! grep -q \"args: \\\\[--keep-full-version\\\\]\" \"$PYPROJECT_FMT_CONFIG\"; then\n sed -i '/^[[:space:]]*- id: pyproject-fmt[[:space:]]*$/a\\ args: [--keep-full-version]' \"$PYPROJECT_FMT_CONFIG\"\n echo \"✅ Configured pyproject-fmt to preserve full versions\"\n fi\n\n # Update [tool.poetry.dependencies] section\n # Matches: openhands-sdk = \"1.13\" or openhands-sdk = \"1.13.0\"\n sed -i -E 's/^(openhands-sdk = )\"[^\"]*\"/\\1\"=='\"$VERSION\"'\"/' pyproject.toml\n sed -i -E 's/^(openhands-tools = )\"[^\"]*\"/\\1\"=='\"$VERSION\"'\"/' pyproject.toml\n sed -i -E 's/^(openhands-agent-server = )\"[^\"]*\"/\\1\"=='\"$VERSION\"'\"/' pyproject.toml\n\n # Update [project].dependencies section (PEP 621 format)\n # Matches: \"openhands-sdk==1.13.1\", or \"openhands-sdk==1.13\",\n sed -i -E 's/\"openhands-sdk==[^\"]*\"/\"openhands-sdk=='\"$VERSION\"'\"/' pyproject.toml\n sed -i -E 's/\"openhands-tools==[^\"]*\"/\"openhands-tools=='\"$VERSION\"'\"/' pyproject.toml\n sed -i -E 's/\"openhands-agent-server==[^\"]*\"/\"openhands-agent-server=='\"$VERSION\"'\"/' pyproject.toml\n\n # Update mypy additional_dependencies pins so type-checking uses the same SDK version\n sed -i -E 's/\"openhands-sdk==[^\"]*\"/\"openhands-sdk=='\"$VERSION\"'\"/' \"$PYPROJECT_FMT_CONFIG\"\n sed -i -E 's/\"openhands-tools==[^\"]*\"/\"openhands-tools=='\"$VERSION\"'\"/' \"$PYPROJECT_FMT_CONFIG\"\n\n echo \"✅ Updated pyproject.toml\"\n\n # 2. Regenerate poetry.lock with the new versions\n # Note: In Poetry 2.x, the default behavior is to not update packages already\n # in the lock file (the --no-update flag was removed in Poetry 2.x)\n echo \"📝 Regenerating poetry.lock...\"\n poetry lock\n\n # 2b. Regenerate enterprise/poetry.lock so its transitive SDK pins\n # match the root. enterprise/pyproject.toml depends on the root via\n # `openhands-ai = { path = \"../\", develop = true }`, but it keeps its\n # OWN poetry.lock that pins openhands-sdk/tools/agent-server. Without\n # this step the enterprise lockfile drifts behind (see PR #14409 that\n # had to be opened manually after PR #14350 missed it).\n # --no-cache invalidates the stale build of the path-installed\n # openhands-ai package; without it Poetry leaves the entries pinned\n # at the previous version.\n if [ -f \"enterprise/poetry.lock\" ] && [ -f \"enterprise/pyproject.toml\" ]; then\n echo \"📝 Regenerating enterprise/poetry.lock...\"\n (cd enterprise && poetry lock --no-cache)\n echo \"✅ Updated enterprise/poetry.lock\"\n fi\n\n echo \"📝 Regenerating uv.lock...\"\n # --no-config bypasses ~/.config/uv/uv.toml where setup-uv writes its\n # 7-day freshness guardrail. Unlike --exclude-newer=, it does not\n # bake a timestamp into uv.lock's [options] section (which would create\n # noise in every future bump PR).\n uv lock --no-cache --no-config\n echo \"✅ Updated uv.lock\"\n\n # 3. Update the version in sandbox_spec_service.py\n echo \"🔧 Updating AGENT_SERVER_IMAGE...\"\n SANDBOX_SPEC_FILE=\"openhands/app_server/sandbox/sandbox_spec_service.py\"\n if [ -f \"$SANDBOX_SPEC_FILE\" ]; then\n # Update the AGENT_SERVER_IMAGE line with the new hash\n sed -i \"s|AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:[^']*'|AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:${VERSION}-python'|\" \"$SANDBOX_SPEC_FILE\"\n echo \"✅ Updated AGENT_SERVER_IMAGE to: ghcr.io/openhands/agent-server:${VERSION}-python\"\n else\n echo \"❌ sandbox_spec_service.py not found at expected path\"\n exit 1\n fi\n\n # 4. Run pre-commit to fix formatting with the target repo's config.\n echo \"🔧 Running pre-commit to fix formatting...\"\n pip install pre-commit\n pre-commit run --files pyproject.toml \"$PYPROJECT_FMT_CONFIG\" --config ./dev_config/python/.pre-commit-config.yaml || true\n\n # Check if there are changes\n if git diff --quiet; then\n echo \"⚠️ No changes detected in $REPO - versions may already be up to date\"\n exit 0\n fi\n\n # Commit and push\n git add pyproject.toml poetry.lock uv.lock \"$SANDBOX_SPEC_FILE\" \"$PYPROJECT_FMT_CONFIG\"\n if [ -f \"enterprise/poetry.lock\" ]; then\n git add enterprise/poetry.lock\n fi\n git commit -m \"Bump openhands-sdk, openhands-tools, openhands-agent-server to $VERSION\" \\\n -m \"Automated version bump after PyPI release.\" \\\n -m \"\" \\\n -m \"Changes:\" \\\n -m \"- Updated SDK packages to v$VERSION with exact pins in pyproject.toml\" \\\n -m \"- Regenerated poetry.lock\" \\\n -m \"- Regenerated enterprise/poetry.lock to keep transitive SDK pins aligned\" \\\n -m \"- Regenerated uv.lock\" \\\n -m \"- Updated AGENT_SERVER_IMAGE to ${VERSION}\" \\\n -m \"- Updated mypy additional_dependencies pins in pre-commit config\" \\\n -m \"\" \\\n -m \"Co-authored-by: openhands \"\n git push -u origin \"$BRANCH\"\n\n # Check if PR already exists\n EXISTING_PR=$(gh pr list --repo \"$REPO\" --head \"$BRANCH\" --json number --jq '.[0].number')\n if [ -n \"$EXISTING_PR\" ]; then\n echo \"✅ PR #$EXISTING_PR already exists for $REPO\"\n else\n # Create PR\n gh pr create \\\n --repo \"$REPO\" \\\n --title \"Bump SDK packages to v$VERSION\" \\\n --body \"## Automated Version Bump\n\n This PR updates the following packages to version **$VERSION**:\n - \\`openhands-sdk\\`\n - \\`openhands-tools\\`\n - \\`openhands-agent-server\\`\n\n ### Changes\n - Updated SDK packages in \\`pyproject.toml\\` with exact pins\n - Regenerated \\`poetry.lock\\` with the target repo's Poetry version\n - Regenerated \\`enterprise/poetry.lock\\` so its transitive SDK pins match the root\n - Regenerated \\`uv.lock\\` to match the updated SDK versions\n - Updated \\`AGENT_SERVER_IMAGE\\` to \\`${VERSION}\\` in \\`sandbox_spec_service.py\\`\n - Updated mypy \\`additional_dependencies\\` pins in \\`.pre-commit-config.yaml\\`\n\n **Triggered by:** Release of [software-agent-sdk v$VERSION](https://github.com/OpenHands/software-agent-sdk/releases/tag/v$VERSION)\n\n ---\n _This PR was automatically created by the version-bump-prs workflow._\" \\\n --base main \\\n --head \"$BRANCH\"\n\n echo \"✅ PR created for $REPO\"\n fi\n\n - name: Summary\n env:\n VERSION: ${{ steps.get_version.outputs.version }}\n run: |\n echo \"## ✅ Version Bump PRs Created\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"PRs have been created to bump SDK packages to version **$VERSION**:\" >> $GITHUB_STEP_SUMMARY\n echo \"\" >> $GITHUB_STEP_SUMMARY\n echo \"- [OpenHands](https://github.com/OpenHands/OpenHands/pulls?q=is%3Apr+bump-sdk-$VERSION)\" >> $GITHUB_STEP_SUMMARY\n echo \"- [OpenHands-CLI](https://github.com/OpenHands/openhands-cli/pulls?q=is%3Apr+bump-sdk-$VERSION)\" >> $GITHUB_STEP_SUMMARY\n\n - name: Notify Slack\n if: env.SLACK_BOT_TOKEN != ''\n uses: slackapi/slack-github-action@v3.0.3\n with:\n method: chat.postMessage\n token: ${{ env.SLACK_BOT_TOKEN }}\n payload: |\n channel: C08E1SYKEM9\n text: \"🚀 *SDK v${{ steps.get_version.outputs.version }} published to PyPI!*\\n\\nVersion bump PRs created:\\n• \\n• \\n\\n\"\n" }, { "path": ".gitignore", "content": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\nshare/python-wheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\nrequirements.txt\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.nox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n*.py,cover\n.hypothesis/\n.pytest_cache/\ncover/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\ndb.sqlite3\ndb.sqlite3-journal\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\n.pybuilder/\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# IPython\nprofile_default/\nipython_config.py\n\n# pyenv\n# For a library or package, you might want to ignore these files since the code is\n# intended to run in multiple environments; otherwise, check them in:\n\n# pipenv\n# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.\n# However, in case of collaboration, if having platform-specific dependencies or dependencies\n# having no cross-platform support, pipenv may install dependencies that don't work, or not\n# install all needed dependencies.\n#Pipfile.lock\n\n# poetry\n# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.\n# This is especially recommended for binary packages to ensure reproducibility, and is more\n# commonly ignored for libraries.\n# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control\n# poetry.lock\n\n# pdm\n# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.\n#pdm.lock\n# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it\n# in version control.\n# https://pdm.fming.dev/#use-with-ide\n.pdm.toml\n\n# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm\n__pypackages__/\n\n# Celery stuff\ncelerybeat-schedule\ncelerybeat.pid\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\n.env.bak\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n.dmypy.json\ndmypy.json\n\n# Pyre type checker\n.pyre/\n\n# pytype static type analyzer\n.pytype/\n\n# Cython debug symbols\ncython_debug/\n\n# PyCharm\n# JetBrains specific template is maintained in a separate JetBrains.gitignore that can\n# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore\n# and can be added to the global gitignore or merged into this file. For a more nuclear\n# option (not recommended) you can uncomment the following to ignore the entire idea folder.\n.idea/\n\n# VS Code: Ignore all but certain files that specify repo-specific settings.\n# https://stackoverflow.com/questions/32964920/should-i-commit-the-vscode-folder-to-source-control\n.vscode/**/*\n!.vscode/extensions.json\n!.vscode/tasks.json\n\n# VS Code extensions/forks:\n.cursorignore\n.rooignore\n.clineignore\n.windsurfignore\n.cursorrules\n.roorules\n.clinerules\n.windsurfrules\n.cursor/rules\n.roo/rules\n.cline/rules\n.windsurf/rules\n.repomix\nrepomix-output.txt\n\n# misc\n.DS_Store\n.env.local\n.env.development.local\n.env.test.local\n.env.production.local\n\nnpm-debug.log*\nyarn-debug.log*\nyarn-error.log*\n\nlogs\n\n# agent\n.envrc\ncache\n.jinja_cache/\n\n.conversations*\n/workspace/\nopenapi.json\n.client/\n\n# Local workspace files\n.beads/*.db\n.worktrees/\nagent-sdk.workspace.code-workspace\n\n# Integration test outputs\ntests/integration/outputs/\ntests/integration/api_compliance/outputs/\n\n# Agent-generated temp\n.agent_tmp/\n" }, { "path": ".openhands/hooks/on_stop.sh", "content": "#!/bin/bash\n# Stop hook: runs pre-commit, pytest, and checks CI status before allowing agent to finish\n#\n# This hook runs when the agent attempts to stop/finish.\n# It can BLOCK the stop by:\n# - Exiting with code 2 (blocked)\n# - Outputting JSON: {\"decision\": \"deny\", \"additionalContext\": \"feedback message\"}\n#\n# Environment variables available:\n# OPENHANDS_PROJECT_DIR - Project directory\n# OPENHANDS_SESSION_ID - Session ID\n# GITHUB_TOKEN - GitHub API token (if available)\n\nset -o pipefail\n\nPROJECT_DIR=\"${OPENHANDS_PROJECT_DIR:-$(pwd)}\"\ncd \"$PROJECT_DIR\" || exit 1\n\n# Collect all issues to report back to the agent\nISSUES=\"\"\nBLOCK_STOP=false\n\nlog_issue() {\n ISSUES=\"${ISSUES}${1}\\n\"\n BLOCK_STOP=true\n}\n\n>&2 echo \"=== Stop Hook ===\"\n>&2 echo \"Project directory: $PROJECT_DIR\"\n>&2 echo \"\"\n\n# --------------------------\n# Step 1: Run pre-commit on all files\n# --------------------------\n>&2 echo \"=== Running pre-commit run --all-files ===\"\nif command -v uv &> /dev/null; then\n PRECOMMIT_OUTPUT=$(uv run pre-commit run --all-files 2>&1)\n PRECOMMIT_EXIT=$?\nelse\n PRECOMMIT_OUTPUT=$(pre-commit run --all-files 2>&1)\n PRECOMMIT_EXIT=$?\nfi\n\n>&2 echo \"$PRECOMMIT_OUTPUT\"\n\nif [ $PRECOMMIT_EXIT -ne 0 ]; then\n >&2 echo \"⚠️ pre-commit found issues (exit code: $PRECOMMIT_EXIT)\"\n log_issue \"## Pre-commit Failed\\n\\nPre-commit checks failed. Please fix the following issues:\\n\\n\\`\\`\\`\\n${PRECOMMIT_OUTPUT}\\n\\`\\`\\`\"\nelse\n >&2 echo \"✓ pre-commit passed\"\nfi\n>&2 echo \"\"\n\n# --------------------------\n# Step 2: Detect changed files and run appropriate tests\n# --------------------------\n>&2 echo \"=== Detecting changed files and running appropriate tests ===\"\n\n# Get changed files from git (staged, unstaged, and untracked)\nCHANGED_FILES=$(git status --porcelain 2>/dev/null | awk '{print $NF}')\n\nif [ -n \"$CHANGED_FILES\" ]; then\n >&2 echo \"Changed files:\"\n >&2 echo \"$CHANGED_FILES\" | head -20\n >&2 echo \"\"\n\n # Map changed files to test directories\n PROJECTS_TO_TEST=\"\"\n\n add_project() {\n local project=\"$1\"\n if [[ ! \"$PROJECTS_TO_TEST\" =~ \"$project\" ]]; then\n PROJECTS_TO_TEST=\"$PROJECTS_TO_TEST $project\"\n fi\n }\n\n while IFS= read -r file; do\n case \"$file\" in\n openhands-sdk/*) add_project \"tests/sdk\" ;;\n openhands-tools/*) add_project \"tests/tools\" ;;\n openhands-workspace/*) add_project \"tests/workspace\" ;;\n openhands-agent-server/*) add_project \"tests/agent_server\" ;;\n tests/sdk/*) add_project \"tests/sdk\" ;;\n tests/tools/*) add_project \"tests/tools\" ;;\n tests/workspace/*) add_project \"tests/workspace\" ;;\n tests/agent_server/*) add_project \"tests/agent_server\" ;;\n tests/cross/*) add_project \"tests/cross\" ;;\n tests/examples/*) add_project \"tests/examples\" ;;\n tests/github_workflows/*) add_project \"tests/github_workflows\" ;;\n examples/*) add_project \"tests/examples\" ;;\n scripts/*) add_project \"tests/cross\" ;;\n pyproject.toml|uv.lock) add_project \"tests/cross\" ;;\n esac\n done <<< \"$CHANGED_FILES\"\n\n PROJECTS_TO_TEST=$(echo \"$PROJECTS_TO_TEST\" | xargs)\n\n if [ -n \"$PROJECTS_TO_TEST\" ]; then\n >&2 echo \"Running tests for: $PROJECTS_TO_TEST\"\n >&2 echo \"\"\n\n for project in $PROJECTS_TO_TEST; do\n if [ -d \"$project\" ]; then\n >&2 echo \"=== Testing $project ===\"\n if command -v uv &> /dev/null; then\n PYTEST_OUTPUT=$(uv run pytest \"$project\" -v --tb=short -x 2>&1)\n PYTEST_EXIT=$?\n else\n PYTEST_OUTPUT=$(pytest \"$project\" -v --tb=short -x 2>&1)\n PYTEST_EXIT=$?\n fi\n >&2 echo \"$PYTEST_OUTPUT\"\n\n if [ $PYTEST_EXIT -ne 0 ]; then\n >&2 echo \"⚠️ pytest failed for $project\"\n log_issue \"## Pytest Failed for $project\\n\\nTests failed. Please fix the following:\\n\\n\\`\\`\\`\\n${PYTEST_OUTPUT}\\n\\`\\`\\`\"\n fi\n >&2 echo \"\"\n fi\n done\n else\n >&2 echo \"No tests to run for changed files\"\n fi\nelse\n >&2 echo \"No changed files detected, skipping local tests\"\nfi\n>&2 echo \"\"\n\n# --------------------------\n# Step 3: Check if there's a pushed commit and wait for CI\n# --------------------------\n>&2 echo \"=== Checking GitHub CI status ===\"\n\n# Check if we're in a git repo with a GitHub remote\nGITHUB_REMOTE=$(git remote -v 2>/dev/null | grep -E \"(github\\.com.*push)\" | head -1)\nif [ -z \"$GITHUB_REMOTE\" ]; then\n >&2 echo \"No GitHub remote found, skipping CI check\"\nelse\n # Extract owner/repo from remote URL\n # Handle both HTTPS and SSH formats\n REPO_INFO=$(echo \"$GITHUB_REMOTE\" | sed -E 's|.*github\\.com[:/]([^/]+)/([^/.]+)(\\.git)?.*|\\1/\\2|')\n \n if [ -z \"$REPO_INFO\" ]; then\n >&2 echo \"Could not parse GitHub repository info\"\n else\n >&2 echo \"Repository: $REPO_INFO\"\n \n # Get current branch\n CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD 2>/dev/null)\n >&2 echo \"Current branch: $CURRENT_BRANCH\"\n \n # Get the latest commit SHA\n LOCAL_SHA=$(git rev-parse HEAD 2>/dev/null)\n >&2 echo \"Local commit: ${LOCAL_SHA:0:8}\"\n \n # Check if this commit has been pushed\n REMOTE_SHA=$(git ls-remote origin \"$CURRENT_BRANCH\" 2>/dev/null | awk '{print $1}')\n \n if [ -z \"$REMOTE_SHA\" ]; then\n >&2 echo \"Branch not pushed to remote, skipping CI check\"\n elif [ \"$LOCAL_SHA\" != \"$REMOTE_SHA\" ]; then\n >&2 echo \"Local commit differs from remote (remote: ${REMOTE_SHA:0:8}), skipping CI check\"\n else\n >&2 echo \"Commit has been pushed, checking CI status...\"\n \n # Check if GITHUB_TOKEN is available\n if [ -z \"$GITHUB_TOKEN\" ]; then\n >&2 echo \"GITHUB_TOKEN not set, cannot check CI status\"\n else\n # Use gh CLI if available, otherwise fall back to API\n if command -v gh &> /dev/null; then\n >&2 echo \"Using gh CLI to check CI status...\"\n \n # Get check runs for this commit\n CI_STATUS=$(gh api \"repos/$REPO_INFO/commits/$LOCAL_SHA/check-runs\" \\\n --jq '.check_runs | map({name: .name, status: .status, conclusion: .conclusion})' 2>&1)\n \n if [ $? -ne 0 ]; then\n >&2 echo \"Failed to get CI status: $CI_STATUS\"\n else\n # Parse the status\n TOTAL_CHECKS=$(echo \"$CI_STATUS\" | jq 'length')\n \n if [ \"$TOTAL_CHECKS\" -eq 0 ]; then\n >&2 echo \"No CI checks found for this commit\"\n else\n >&2 echo \"Found $TOTAL_CHECKS CI check(s)\"\n \n # Check for in-progress runs\n IN_PROGRESS=$(echo \"$CI_STATUS\" | jq '[.[] | select(.status != \"completed\")] | length')\n FAILED=$(echo \"$CI_STATUS\" | jq '[.[] | select(.conclusion == \"failure\" or .conclusion == \"timed_out\" or .conclusion == \"cancelled\")] | length')\n \n if [ \"$IN_PROGRESS\" -gt 0 ]; then\n >&2 echo \"⏳ $IN_PROGRESS check(s) still in progress\"\n \n # Wait for CI to complete (with timeout)\n MAX_WAIT=300 # 5 minutes\n WAIT_INTERVAL=15\n TOTAL_WAITED=0\n \n while [ \"$IN_PROGRESS\" -gt 0 ] && [ \"$TOTAL_WAITED\" -lt \"$MAX_WAIT\" ]; do\n >&2 echo \"Waiting for CI... (${TOTAL_WAITED}s / ${MAX_WAIT}s max)\"\n sleep $WAIT_INTERVAL\n TOTAL_WAITED=$((TOTAL_WAITED + WAIT_INTERVAL))\n \n CI_STATUS=$(gh api \"repos/$REPO_INFO/commits/$LOCAL_SHA/check-runs\" \\\n --jq '.check_runs | map({name: .name, status: .status, conclusion: .conclusion})' 2>&1)\n IN_PROGRESS=$(echo \"$CI_STATUS\" | jq '[.[] | select(.status != \"completed\")] | length')\n done\n \n if [ \"$IN_PROGRESS\" -gt 0 ]; then\n >&2 echo \"⚠️ CI still running after ${MAX_WAIT}s timeout\"\n log_issue \"## CI Still Running\\n\\nCI checks are still in progress after waiting ${MAX_WAIT} seconds. Please wait for CI to complete before finishing.\"\n fi\n fi\n \n # Re-check for failures after waiting\n FAILED=$(echo \"$CI_STATUS\" | jq '[.[] | select(.conclusion == \"failure\" or .conclusion == \"timed_out\" or .conclusion == \"cancelled\")] | length')\n \n if [ \"$FAILED\" -gt 0 ]; then\n >&2 echo \"❌ $FAILED check(s) failed!\"\n \n # Get details of failed checks\n FAILED_DETAILS=$(echo \"$CI_STATUS\" | jq -r '.[] | select(.conclusion == \"failure\" or .conclusion == \"timed_out\" or .conclusion == \"cancelled\") | \"- \\(.name): \\(.conclusion)\"')\n >&2 echo \"$FAILED_DETAILS\"\n \n # Try to get failure logs\n FAILED_NAMES=$(echo \"$CI_STATUS\" | jq -r '.[] | select(.conclusion == \"failure\") | .name')\n \n FAILURE_MSG=\"## CI Failed\\n\\nThe following CI checks failed:\\n\\n${FAILED_DETAILS}\\n\"\n \n # Try to get the workflow run logs for more context\n WORKFLOW_RUNS=$(gh api \"repos/$REPO_INFO/actions/runs?head_sha=$LOCAL_SHA\" \\\n --jq '.workflow_runs[] | select(.conclusion == \"failure\") | {id: .id, name: .name}' 2>/dev/null)\n \n if [ -n \"$WORKFLOW_RUNS\" ]; then\n FAILURE_MSG=\"${FAILURE_MSG}\\nYou can view the full logs at: https://github.com/$REPO_INFO/actions\\n\"\n \n # Try to get job logs\n FIRST_RUN_ID=$(echo \"$WORKFLOW_RUNS\" | jq -r '.id' | head -1)\n if [ -n \"$FIRST_RUN_ID\" ]; then\n JOBS_OUTPUT=$(gh api \"repos/$REPO_INFO/actions/runs/$FIRST_RUN_ID/jobs\" \\\n --jq '.jobs[] | select(.conclusion == \"failure\") | \"### \\(.name)\\nConclusion: \\(.conclusion)\\nSteps:\\n\" + (.steps | map(\"- \\(.name): \\(.conclusion)\") | join(\"\\n\"))' 2>/dev/null | head -100)\n if [ -n \"$JOBS_OUTPUT\" ]; then\n FAILURE_MSG=\"${FAILURE_MSG}\\n### Failed Job Details:\\n\\`\\`\\`\\n${JOBS_OUTPUT}\\n\\`\\`\\`\"\n fi\n fi\n fi\n \n log_issue \"$FAILURE_MSG\"\n else\n >&2 echo \"✓ All CI checks passed!\"\n fi\n fi\n fi\n else\n # Fallback to curl\n >&2 echo \"gh CLI not available, using API directly...\"\n CI_RESPONSE=$(curl -s -H \"Authorization: token $GITHUB_TOKEN\" \\\n -H \"Accept: application/vnd.github.v3+json\" \\\n \"https://api.github.com/repos/$REPO_INFO/commits/$LOCAL_SHA/check-runs\" 2>&1)\n \n TOTAL_CHECKS=$(echo \"$CI_RESPONSE\" | jq '.total_count // 0')\n \n if [ \"$TOTAL_CHECKS\" -gt 0 ]; then\n IN_PROGRESS=$(echo \"$CI_RESPONSE\" | jq '[.check_runs[] | select(.status != \"completed\")] | length')\n FAILED=$(echo \"$CI_RESPONSE\" | jq '[.check_runs[] | select(.conclusion == \"failure\")] | length')\n \n if [ \"$IN_PROGRESS\" -gt 0 ]; then\n >&2 echo \"⏳ CI checks still in progress\"\n log_issue \"## CI In Progress\\n\\nCI checks are still running. Please wait for CI to complete.\"\n elif [ \"$FAILED\" -gt 0 ]; then\n FAILED_NAMES=$(echo \"$CI_RESPONSE\" | jq -r '.check_runs[] | select(.conclusion == \"failure\") | .name')\n >&2 echo \"❌ CI failed: $FAILED_NAMES\"\n log_issue \"## CI Failed\\n\\nThe following CI checks failed:\\n${FAILED_NAMES}\\n\\nPlease fix the issues and try again.\"\n else\n >&2 echo \"✓ All CI checks passed!\"\n fi\n else\n >&2 echo \"No CI checks found\"\n fi\n fi\n fi\n fi\n fi\nfi\n>&2 echo \"\"\n\n# --------------------------\n# Final decision\n# --------------------------\nif [ \"$BLOCK_STOP\" = true ]; then\n >&2 echo \"=== BLOCKING STOP: Issues found ===\"\n # Output JSON to provide feedback to the agent\n # Escape the issues for JSON\n ESCAPED_ISSUES=$(echo -e \"$ISSUES\" | jq -Rs .)\n echo \"{\\\"decision\\\": \\\"deny\\\", \\\"reason\\\": \\\"Checks failed\\\", \\\"additionalContext\\\": $ESCAPED_ISSUES}\"\n exit 2\nfi\n\n>&2 echo \"=== All checks passed, allowing stop ===\"\necho '{\"decision\": \"allow\"}'\nexit 0\n" }, { "path": ".openhands/hooks.json", "content": "{\n \"stop\": [\n {\n \"matcher\": \"*\",\n \"hooks\": [\n {\n \"type\": \"command\",\n \"command\": \".openhands/hooks/on_stop.sh\",\n \"timeout\": 600\n }\n ]\n }\n ]\n}\n" }, { "path": ".openhands/setup.sh", "content": "#!/bin/bash\n\nif ! command -v uv &> /dev/null; then\n echo \"uv is not installed. Installing...\"\n curl -LsSf https://astral.sh/uv/install.sh | sh\nelse\n echo \"uv is already installed.\"\n uv self update # always update to the latest version\nfi\n\nmake build\n" }, { "path": ".pre-commit-config.yaml", "content": "---\nrepos:\n - repo: https://github.com/jumanjihouse/pre-commit-hook-yamlfmt\n rev: 0.2.1 # or other specific tag\n hooks:\n - id: yamlfmt\n - repo: local\n hooks:\n - id: ruff-format\n name: Ruff format\n entry: uv\n args: [run, ruff, format]\n language: system\n types: [python]\n pass_filenames: true\n always_run: false\n - id: ruff-check\n name: Ruff lint\n entry: uv\n args: [run, ruff, check, --fix, --exit-non-zero-on-fix]\n language: system\n types: [python]\n pass_filenames: true\n always_run: false\n - id: pycodestyle\n name: PEP8 style check (pycodestyle)\n entry: uv\n args: [run, pycodestyle, --max-line-length=88, '--ignore=E203,E501,W503,E704']\n language: system\n types: [python]\n pass_filenames: true\n always_run: false\n - id: pyright\n name: Type check with pyright\n entry: uv\n args: [run, pyright]\n language: system\n types: [python]\n pass_filenames: true\n always_run: false\n - id: check-import-rules\n name: Check import dependency rules\n entry: uv\n args: [run, python, scripts/check_import_rules.py]\n language: system\n types: [python]\n pass_filenames: true\n always_run: false\n - id: check-tool-registration\n name: Check Tool subclass registration\n entry: uv\n args: [run, python, scripts/check_tool_registration.py]\n language: system\n types: [python]\n pass_filenames: true\n always_run: false\n" }, { "path": ".python-version", "content": "3.13\n" }, { "path": "AGENTS.md", "content": "\nYou are a collaborative software engineering partner with a strong focus on code quality and simplicity. Your approach is inspired by proven engineering principles from successful open-source projects, emphasizing pragmatic solutions and maintainable code.\n\n# Core Engineering Principles\n\n1. **Simplicity and Clarity**\n\"The best solutions often come from looking at problems from a different angle, where special cases disappear and become normal cases.\"\n • Prefer solutions that eliminate edge cases rather than adding conditional checks\n • Good design patterns emerge from experience and careful consideration\n • Simple, clear code is easier to maintain and debug\n\n2. **Backward Compatibility**\n\"Stability is a feature, not a constraint.\"\n • Changes should not break existing functionality\n • Consider the impact on users and existing integrations\n • Compatibility enables trust and adoption\n\n3. **Pragmatic Problem-Solving**\n\"Focus on solving real problems with practical solutions.\"\n • Address actual user needs rather than theoretical edge cases\n • Prefer proven, straightforward approaches over complex abstractions\n • Code should serve real-world requirements\n\n4. **Maintainable Architecture**\n\"Keep functions focused and code readable.\"\n • Functions should be short and have a single responsibility\n • Avoid deep nesting - consider refactoring when indentation gets complex\n • Clear naming and structure reduce cognitive load\n\n# Collaborative Approach\n\n## Communication Style\n • **Constructive**: Focus on helping improve code and solutions\n • **Collaborative**: Work together as partners toward better outcomes\n • **Clear**: Provide specific, actionable feedback\n • **Respectful**: Maintain a supportive tone while being technically rigorous\n\n## Problem Analysis Process\n\n### 1. Understanding Requirements\nWhen reviewing a requirement, confirm understanding by restating it clearly:\n> \"Based on your description, I understand you need: [clear restatement of the requirement]. Is this correct?\"\n\n### 2. Collaborative Problem Decomposition\n\n#### Data Structure Analysis\n\"Well-designed data structures often lead to simpler code.\"\n • What are the core data elements and their relationships?\n • How does data flow through the system?\n • Are there opportunities to simplify data handling?\n\n#### Complexity Assessment\n\"Let's look for ways to simplify this.\"\n • What's the essential functionality we need to implement?\n • Which parts of the current approach add unnecessary complexity?\n • How can we make this more straightforward?\n\n#### Compatibility Review\n\"Let's make sure this doesn't break existing functionality.\"\n • What existing features might be affected?\n • How can we implement this change safely?\n • What migration path do users need?\n\n#### Practical Validation\n\"Let's focus on the real-world use case.\"\n • Does this solve an actual problem users face?\n • Is the complexity justified by the benefit?\n • What's the simplest approach that meets the need?\n\n## 3. Constructive Feedback Format\n\nAfter analysis, provide feedback in this format:\n\n**Assessment**: [Clear evaluation of the approach]\n\n**Key Observations**:\n- Data Structure: [insights about data organization]\n- Complexity: [areas where we can simplify]\n- Compatibility: [potential impact on existing code]\n\n**Suggested Approach**:\nIf the solution looks good:\n1. Start with the simplest data structure that works\n2. Eliminate special cases where possible\n3. Implement clearly and directly\n4. Ensure backward compatibility\n\nIf there are concerns:\n\"I think we might be able to simplify this. The core issue seems to be [specific problem]. What if we tried [alternative approach]?\"\n\n## 4. Code Review Approach\nWhen reviewing code, provide constructive feedback:\n\n**Overall Assessment**: [Helpful evaluation]\n\n**Specific Suggestions**:\n- [Concrete improvements with explanations]\n- [Alternative approaches to consider]\n- [Ways to reduce complexity]\n\n**Next Steps**: [Clear action items]\n\n\n## Repository Memory\n- Programmatic settings live in `openhands-sdk/openhands/sdk/settings/`. Treat `AgentSettings` and `export_settings_schema()` as the canonical structured settings surface in the SDK, and keep that schema focused on neutral config semantics rather than client-specific presentation details.\n- `SettingsFieldSchema` intentionally does not export a `required` flag. If a consumer needs nullability semantics, inspect the underlying Python typing rather than inferring from SDK defaults.\n- `AgentSettings.tools` is part of the exported settings schema so the schema stays aligned with the settings payload that round-trips through `AgentSettings` and drives `create_agent()`.\n- `AgentSettings.mcp_config` now uses FastMCP's typed `MCPConfig` at runtime. When serializing settings back to plain data (e.g. `model_dump()` or `create_agent()`), keep the output compact with `exclude_none=True, exclude_defaults=True` so callers still see the familiar `.mcp.json`-style dict shape.\n- Persisted SDK settings should use the direct `model_dump()` shape with a top-level `schema_version`; avoid adding wrapped payload formats or legacy migration shims in `openhands/sdk/settings/model.py`.\n- Because persisted settings are not in production yet, prefer removing temporary compatibility fields and serializers outright instead of carrying legacy settings shims in the SDK.\n- Do not expose settings schema versions as public `CURRENT_PERSISTED_VERSION` class constants on `AgentSettings` or `ConversationSettings`; keep versioning internal to the `schema_version` field/defaults and private module constants.\n- `ConversationSettings` owns the conversation-scoped confirmation controls directly (`confirmation_mode`, `security_analyzer`); keep those fields top-level on the model and grouped into the exported `verification` section via schema metadata rather than nested helper models, and prefer the direct settings-model constructor `create_request(...)` over separate request-wrapper helpers.\n- Anthropic malformed tool-use/tool-result history errors (for example, missing or duplicated ``tool_result`` blocks) are intentionally mapped to a dedicated `LLMMalformedConversationHistoryError` and caught separately in `Agent.step()`, so recovery can still use condensation while logs preserve that this was malformed history rather than a true context-window overflow.\n- AgentSkills progressive disclosure goes through `AgentContext.get_system_message_suffix()` into ``, and `openhands.sdk.context.skills.to_prompt()` truncates each prompt description to 1024 characters because the AgentSkills specification caps `description` at 1-1024 characters.\n- Workspace-wide uv resolver guardrails belong in the repository root `[tool.uv]` table. When `exclude-newer` is configured there, `uv lock` persists it into the root `uv.lock` `[options]` section as both an absolute cutoff and `exclude-newer-span`, and `uv sync --frozen` continues to use that locked workspace state.\n- `pr-review-by-openhands` delegates to `OpenHands/extensions/plugins/pr-review@main`. Repo-specific reviewer instructions live in `.agents/skills/custom-codereview-guide.md`, and because task-trigger matching is substring-based, that `/codereview` skill is also auto-injected for the workflow's `/codereview-roasted` prompt.\n- Directory-based runnable examples under `examples/` should expose their entrypoint as `main.py`, and `tests/examples/test_examples.py` should explicitly list the example directory in `_TARGET_DIRECTORIES` so the non-recursive example workflow collects it without accidentally running helper modules.\n- The duplicate-issue automation scripts should validate `owner/repo` arguments before interpolating GitHub API paths, handle per-issue auto-close failures without aborting the whole batch, and keep `app_conversation_id` paths unquoted because OpenHands conversation IDs are already canonicalized for those endpoints.\n- `agent-server` now defaults `TMUX_TMPDIR` to a per-process directory under the system temp dir (`openhands-agent-server-`) when the environment variable is unset. This isolates tmux sockets/cleanup across concurrent server instances while still respecting an explicit `TMUX_TMPDIR` override.\n- Conversation worktrees for git-backed local workspaces live under `/tmp/conversation-worktrees//`, and if the original workspace points at a subdirectory inside the repo, the active workspace should preserve that relative path inside the worktree.\n\n- Agent-server Docker publish tags are defined centrally in `openhands-agent-server/openhands/agent_server/docker/build.py`; keep `server.yml` manifest publication derived from the emitted per-arch tags so SHA/branch/git-tag aliases stay in sync, while preserving the legacy `latest-` alias used by workspace defaults.\n- The published agent-server Docker images in `.github/workflows/server.yml` must pass `OPENHANDS_BUILD_GIT_SHA` and `OPENHANDS_BUILD_GIT_REF` as explicit `docker/build-push-action` build args; the workflow only uses `docker/build.py` for context/tag generation, so those runtime env vars are otherwise left at the Dockerfile `unknown` defaults.\n- The PyInstaller agent-server binary should copy OpenHands distribution metadata (`openhands-agent-server`, `openhands-sdk`, `openhands-tools`, `openhands-workspace`) in `agent-server.spec`, otherwise `/server_info` version lookups via `importlib.metadata` can fall back to `unknown` inside published binary images.\n\n\n- Auto-title generation should not re-read `ConversationState.events` from a background task triggered by a freshly received `MessageEvent`; extract message text synchronously from the incoming event and then reuse shared title helpers (`extract_message_text`, `generate_title_from_message`) to avoid persistence-order races.\n- `RemoteConversation.generate_title()` now reconciles remote events and reuses the shared local `generate_conversation_title(...)` helper instead of calling the removed deprecated agent-server `/generate_title` REST route, so explicit remote title generation still works without a transport-only compatibility endpoint.\n\n\n- Remote workspace git operations should call `/api/git/changes` and `/api/git/diff` via the `path` query parameter with slash-normalized strings; building those URLs with `pathlib.Path` leaks host-platform separators and breaks Windows paths. The grep tool now prefers `rg`, then system `grep`, then Python; both the real grep executor and the SDK's terminal-command compatibility fallback should keep that order. For grep parity, the Python fallback should hide dotfiles by default but still let explicit `include` globs surface files like `.env`, matching ripgrep. For glob parity, any symlink-preservation regression test should force the Python fallback path, because ripgrep availability changes whether the fallback implementation runs at all.\n- Keep path helpers split by purpose: `is_absolute_path_source()` is for cross-platform source/wire syntax detection, while local filesystem writes/validation (for example, the file editor) should use host-native absolute-path semantics so POSIX does not silently accept Windows drive paths as creatable files.\n- Tool availability filtering belongs in `openhands-sdk/openhands/sdk/tool/registry.py` via `list_usable_tools()`, which preserves registration order and defaults tools to usable unless they expose an `is_usable()` callable. Environment-specific checks like Chromium detection should live on the concrete tool class (`BrowserToolSet.is_usable()`), while agent-server surfaces such as `/server_info` should consume the registry helper rather than re-implement per-tool filtering.\n- Pydantic secret field helpers live in `openhands-sdk/openhands/sdk/utils/pydantic_secrets.py`. `serialize_secret()` handles serialization (cipher / `expose_secrets` / default Pydantic masking); `validate_secret()` handles deserialization (cipher decryption, redacted/empty → `None`); `is_redacted_secret()` checks for the sentinel; `REDACTED_SECRET_VALUE` is the canonical sentinel string. For `dict[str, str]` fields whose values are all secrets, wrap each value in `SecretStr` and call `serialize_secret` per value (see `LookupSecret._serialize_secrets` and `ACPAgent._serialize_acp_env`). Do not hand-roll redaction logic in field serializers.\n\n- `LookupSecret` normalizes hostless URLs against `OH_INTERNAL_SERVER_URL` (set by `openhands-agent-server.__main__` from the bound host/port, rewriting wildcard binds to loopback) and otherwise falls back to `http://127.0.0.1:8000`, so relative secret URLs can safely target the current agent-server instance.\n\n\n\n\n## Package-specific guidance\nWhen reviewing or modifying code, read the closest AGENTS file for the\npackage(s) containing the changed files. If a PR spans multiple packages,\nconsult each relevant package-level AGENTS.md.\n\n- SDK: [openhands-sdk/openhands/sdk/AGENTS.md](openhands-sdk/openhands/sdk/AGENTS.md)\n- Subagents: [openhands-sdk/openhands/sdk/subagent/AGENTS.md](openhands-sdk/openhands/sdk/subagent/AGENTS.md)\n- Tools: [openhands-tools/openhands/tools/AGENTS.md](openhands-tools/openhands/tools/AGENTS.md)\n- Workspace: [openhands-workspace/openhands/workspace/AGENTS.md](openhands-workspace/openhands/workspace/AGENTS.md)\n- Agent server: [openhands-agent-server/AGENTS.md](openhands-agent-server/AGENTS.md)\n- Eval config: [.github/run-eval/AGENTS.md](.github/run-eval/AGENTS.md)\n\n## API compatibility pointers\n\n- For SDK Python API deprecation/removal policy, read\n [openhands-sdk/openhands/sdk/AGENTS.md](openhands-sdk/openhands/sdk/AGENTS.md).\n Public API removals require deprecation metadata with a removal target at\n least **5 minor releases** after `deprecated_in`, and breaking SDK API\n changes require at least a **MINOR** SemVer bump.\n- The SDK API breakage checker should treat metadata-only changes to\n Pydantic `Field(...)` declarations as non-breaking, including adding,\n removing, or editing `description`, `title`, `examples`,\n `json_schema_extra`, and `deprecated` kwargs.\n- The SDK API breakage checker compares stringified `Field(...)` values by\n parsing them as Python expressions after escaping literal newlines inside\n quoted strings; this avoids false positives on multiline descriptions that\n include embedded quotes like `'security_policy.j2'`.\n- For public REST APIs, read\n [openhands-agent-server/AGENTS.md](openhands-agent-server/AGENTS.md).\n REST contract breaks need a deprecation notice and a runway of\n **5 minor releases** before removing the old contract or making an\n incompatible replacement mandatory.\n\n\n- Make sure you `make build` to configure the dependencies first\n- We use pre-commit hooks `.pre-commit-config.yaml` that includes:\n - type check through pyright\n - linting and formatter with `uv ruff`\n- NEVER USE `mypy`!\n- Do NOT commit ALL the file, just commit the relevant file you've changed!\n- In every commit message, you should add \"Co-authored-by: openhands \"\n- You can run pytest with `uv run pytest`\n\n# Instruction for fixing \"E501 Line too long\"\n\n- If it is just code, you can modify it so it spans multiple lines.\n- If it is a single-line string, you can break it into a multi-line string by doing \"ABC\" -> (\"A\"\\n\"B\"\\n\"C\")\n- If it is a long multi-line string (e.g., docstring), you should just add type ignore AFTER the ending \"\"\". You should NEVER ADD IT INSIDE the docstring.\n\n\n\n\n\n# PR-Specific Documents\n\nWhen working on a PR that requires design documents, scripts meant for development-only, or other temporary artifacts that should NOT be merged to main, store them in a `.pr/` directory at the repository root.\n\n## Usage\n\n```bash\n# Create the directory if it doesn't exist\nmkdir -p .pr\n\n# Add your PR-specific documents\n.pr/\n├── design.md # Design decisions and architecture notes\n├── analysis.md # Investigation or debugging notes\n└── notes.md # Any other PR-specific content\n```\n\n## How It Works\n\n1. **Notification**: When `.pr/` exists, a single comment is posted to the PR conversation alerting reviewers\n2. **Auto-cleanup**: When the PR is approved, the `.pr/` directory is automatically removed via commit\n3. **Fork PRs**: Auto-cleanup cannot push to forks, so manual removal is required before merging\n\n## Important Notes\n\n- Do NOT put anything in `.pr/` that needs to be preserved\n- The `.pr/` check passes (green ✅) during development - it only posts a notification, not a blocking error\n- For fork PRs: You must manually remove `.pr/` before the PR can be merged\n\n## When to Use\n\n- Complex refactoring that benefits from written design rationale\n- Debugging sessions where you want to document your investigation\n- Feature implementations that need temporary planning docs\n- Temporary script that are intended to show reviewers that the feature works\n- Any analysis that helps reviewers understand the PR but isn't needed long-term\n\n\n\n- Critically evaluate each review comment before acting on it. Not all feedback is worth implementing:\n - Does it fix a real bug or improve clarity significantly?\n - Does it align with the project's engineering principles (simplicity, maintainability)?\n - Is the suggested change proportional to the benefit, or does it add unnecessary complexity?\n- It's acceptable to respectfully decline suggestions that add verbosity without clear benefit, over-engineer for hypothetical edge cases, or contradict the project's pragmatic approach.\n- After addressing (or deciding not to address) inline review comments, mark the corresponding review threads as resolved.\n- Before resolving a thread, leave a reply comment that either explains the reason for dismissing the feedback or references the specific commit (e.g., commit SHA) that addressed the issue.\n- Prefer resolving threads only once fixes are pushed or a clear decision is documented.\n- Use the GitHub GraphQL API to reply to and resolve review threads (see below).\n\n## Resolving Review Threads via GraphQL\n\nThe CI check `Review Thread Gate/unresolved-review-threads` will fail if there are unresolved review threads. To resolve threads programmatically:\n\n1. Get the thread IDs (replace ``, ``, ``):\n```bash\ngh api graphql -f query='\n{\n repository(owner: \"\", name: \"\") {\n pullRequest(number: ) {\n reviewThreads(first: 20) {\n nodes {\n id\n isResolved\n comments(first: 1) {\n nodes { body }\n }\n }\n }\n }\n }\n}'\n```\n\n2. Reply to the thread explaining how the feedback was addressed:\n```bash\ngh api graphql -f query='\nmutation {\n addPullRequestReviewThreadReply(input: {\n pullRequestReviewThreadId: \"\"\n body: \"Fixed in \"\n }) {\n comment { id }\n }\n}'\n```\n\n3. Resolve the thread:\n```bash\ngh api graphql -f query='\nmutation {\n resolveReviewThread(input: {threadId: \"\"}) {\n thread { isResolved }\n }\n}'\n```\n\n4. Get the failed workflow run ID and rerun it:\n```bash\n# Find the run ID from the failed check URL, or use:\ngh run list --repo / --branch --limit 5\n\n# Rerun failed jobs\ngh run rerun --repo / --failed\n```\n\n\n\n\n- Avoid hacky trick like `sys.path.insert` when resolving package dependency\n- Use existing packages/libraries instead of implementing yourselves whenever possible.\n- Avoid using # type: ignore. Treat it only as a last resort. In most cases, issues should be resolved by improving type annotations, adding assertions, or adjusting code/tests—rather than silencing the type checker.\n - Please AVOID using # type: ignore[attr-defined] unless absolutely necessary. If the issue can be addressed by adding a few extra assert statements to verify types, prefer that approach instead!\n - For issue like # type: ignore[call-arg]: if you discover that the argument doesn’t actually exist, do not try to mock it again in tests. Instead, simply remove it.\n- Avoid doing in-line imports unless absolutely necessary (e.g., circular dependency).\n- Avoid getattr/hasattr guards and instead enforce type correctness by relying on explicit type assertions and proper object usage, ensuring functions only receive the expected Pydantic models or typed inputs. Prefer type hints and validated models over runtime shape checks.\n- Prefer accessing typed attributes directly. If necessary, convert inputs up front into a canonical shape; avoid purely hypothetical fallbacks.\n- Use real newlines in commit messages; do not write literal \"\\n\".\n\n\n\n\n- AFTER you edit ONE file, you should run pre-commit hook on that file via `uv run pre-commit run --files [filepath]` to make sure you didn't break it.\n- Don't write TOO MUCH test, you should write just enough to cover edge cases.\n- Check how we perform tests in .github/workflows/tests.yml\n- Put unit tests under the corresponding domain folder in `tests/` (e.g., `tests/sdk`, `tests/tools`, `tests/workspace`). For example, changes to `openhands-sdk/openhands/sdk/tool/tool.py` should be covered in `tests/sdk/tool/test_tool.py`.\n- DON'T write TEST CLASSES unless absolutely necessary!\n- If you find yourself duplicating logics in preparing mocks, loading data etc, these logic should be fixtures in conftest.py!\n- Please test only the logic implemented in the current codebase. Do not test functionality (e.g., BaseModel.model_dumps()) that is not implemented in this repository.\n- For changes to prompt templates, tool descriptions, or agent decision logic, add the `integration-test` label to trigger integration tests and verify no unexpected impact on benchmark performance.\n\n# Stress Tests\n\n`tests/agent_server/stress/` contains an opt-in stress/scale suite for the agent-server, excluded from default collection via the `stress` pytest marker. Run with `uv run pytest -m stress`. For full details on running, infrastructure, and adding new stress tests, see [openhands-agent-server/AGENTS.md](openhands-agent-server/AGENTS.md).\n\n# Behavior Tests\n\nBehavior tests (prefix `b##_*`) in `tests/integration/tests/` are designed to verify that agents exhibit desired behaviors in realistic scenarios. These tests are distinct from functional tests (prefix `t##_*`) and have specific requirements.\n\nBefore adding or modifying behavior tests, review `tests/integration/BEHAVIOR_TESTS.md` for the latest workflow, expectations, and examples.\n\n\n\n# Agent Temporary Directory Convention\n\nWhen tools need to store observation files (e.g., browser session recordings, task tracker data), use `.agent_tmp` as the directory name for consistency.\n\nThe browser session recording tool saves recordings to `.agent_tmp/observations/recording-{timestamp}/`.\n\nThis convention ensures tool-generated observation files are stored in a predictable location that can be easily:\n- Added to `.gitignore`\n- Cleaned up after agent sessions\n- Identified as agent-generated artifacts\n\nNote: This is separate from `persistence_dir` which is used for conversation state persistence.\n\n\n\n\n- This is a `uv`-managed Python monorepo (single `uv.lock` at repo root) with multiple distributable packages: `openhands-sdk/` (SDK), `openhands-tools/` (built-in tools), `openhands-workspace/` (workspace impls), and `openhands-agent-server/` (server runtime).\n- `examples/` contains runnable patterns; `tests/` is split by domain (`tests/sdk`, `tests/tools`, `tests/workspace`, `tests/agent_server`, etc.).\n- Python namespace is `openhands.*` across packages; keep new modules within the matching package and mirror test paths under `tests/`.\n\n\n\n- Set up the dev environment: `make build` (runs `uv sync --dev` and installs pre-commit; requires uv >= 0.8.13)\n- Lint/format: `make lint`, `make format`\n- Run tests: `uv run pytest`\n- Run agent-server stress tests: `uv run pytest -m stress` (see [openhands-agent-server/AGENTS.md](openhands-agent-server/AGENTS.md))\n- Build agent-server: `make build-server` (output: `dist/agent-server/`)\n- Clean caches: `make clean`\n- Run SDK examples: see [openhands-sdk/openhands/sdk/AGENTS.md](openhands-sdk/openhands/sdk/AGENTS.md).\n- The example workflow runs `uv run pytest tests/examples/test_examples.py --run-examples`; each successful example must print an `EXAMPLE_COST: ...` line to stdout (use `EXAMPLE_COST: 0` for non-LLM examples).\n- Example scripts in `examples/` should use top-level code flow (e.g. `with` blocks, bare statements) rather than wrapping logic in a `def main()` function. The `def main` pattern creates unnecessary nesting that makes examples harder to read; keep the code flat and script-like.\n- Conversation plugins passed via `plugins=[...]` are lazy-loaded on the first `send_message()` or `run()`, so example code should inspect plugin-added skills or `resolved_plugins` only after that first interaction.\n- Programmatic settings live in `openhands-sdk/openhands/sdk/settings/`. Keep the exported schema focused on neutral config structure and semantics; downstream apps should own client-specific ordering, icons, widgets, and slash-command presentation.\n\n\n\n- Ruff: `line-length = 88`, `target-version = \"py312\"` (see `pyproject.toml`).\n- Ruff ignores `ARG` (unused arguments) under `tests/**/*.py` to allow pytest fixtures.\n- Repository guidance lives in the project root AGENTS.md (loaded as a third-party skill file).\n\n\n\n" }, { "path": "CONTRIBUTING.md", "content": "# Contributing\n\nThank you for helping improve the OpenHands Software Agent SDK.\n\nThis repo is a foundation. We want the SDK to stay stable and extensible so that many\napplications can build on it safely.\n\nDownstream applications we actively keep in mind:\n- [OpenHands-CLI](https://github.com/OpenHands/OpenHands-CLI) (client)\n- [OpenHands app-server](https://github.com/OpenHands/OpenHands/blob/main/openhands/app_server/README.md) (client)\n- [OpenHands Enterprise](https://github.com/OpenHands/OpenHands/blob/main/enterprise/README.md) (client)\n\nThe SDK itself has a Python interface. In addition, the\n[agent-server](https://docs.openhands.dev/sdk/guides/agent-server/overview) is the\nREST/WebSocket server component that exposes the SDK for remote execution and integrations.\nChanges should keep both interfaces stable and consistent.\n\n## A lesson we learned (why we care about architecture)\n\nIn earlier iterations, we repeatedly ran into a failure mode: needs from downstream applications\n(or assumptions) would leak into core logic.\n\nThat kind of coupling can feel convenient in the moment, but it tends to create subtle\nbreakage elsewhere: different environments, different workspaces, different execution modes,\nand different evaluation setups.\n\nThe architecture of OpenHands V0 was too monolithic to support multiple applications built into it,\nas CLI, evaluation scripts, web server were, and built on it, as OpenHands Cloud was.\n\nIf you’re interested in the deeper background and lessons learned, see our write-up:\n[OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://arxiv.org/abs/2511.03690)\n\nThis SDK exists (as a separate, rebuilt foundation) to avoid that failure mode.\n\n## Principles we review PRs with\n\nWe welcome all contributions, big or small, to improve or extend the software agent SDK.\n\nYou may find that occasionally we are opinionated about several things:\n\n- **OpenHands SDK is its own thing**: its downstream are client applications.\n- **Prefer interfaces over special cases**: if a client needs something, add or improve a\n clean, reusable interface/extension point instead of adding a shortcut.\n- **Extensibility over one-off patches**: design features so multiple clients can adopt them\n without rewriting core logic.\n- **Avoid hidden assumptions**: don’t rely on particular env vars, workspace layouts, request\n contexts, or runtime quirks that only exist in one app.\n - Workspaces *do* encode environment specifics (local/Docker/remote), but keep those assumptions\n explicit (params + validation) and contained to the `workspace` layer.\n- **No client-specific code paths**: avoid logic that only makes sense for one\n downstream app.\n - It’s fine to have multiple workspace implementations; it’s not fine for SDK core behavior to\n branch on whether the caller is CLI/app-server/SaaS. Prefer capabilities/config over app-identity.\n- **Keep the agent loop stable**: treat stability as a feature; be cautious with control-flow\n changes and \"small\" behavior tweaks.\n- **Compatibility is part of the API**: if something could break downstream clients, call it\n out explicitly and consider a migration path. We have a deprecation mechanism you may want to use.\n\nIf you’re not sure whether a change crosses these lines, please ask early. We’re happy to help think\nthrough the shape of a clean interface.\n\n## Practical pointers\n\nThis file is mostly about principles. For the mechanics, please see:\n- [AGENTS.md](AGENTS.md) for AI agents\n- [DEVELOPMENT.md](DEVELOPMENT.md) for humans\n\n## Questions / discussion\n\nJoin us on Slack: https://openhands.dev/joinslack\n" }, { "path": "DEVELOPMENT.md", "content": "# Development Guide\n\n## Setup\n\n```bash\ngit clone https://github.com/OpenHands/agent-sdk.git\ncd agent-sdk\nmake build\n```\n\n## Code Quality\n\n```bash\nmake format # Format code\nmake lint # Lint code\nuv run pre-commit run --all-files # Run all checks\n```\n\nPre-commit hooks run automatically on commit with type checking and linting.\n\n## Testing\n\n```bash\nuv run pytest # All tests\nuv run pytest tests/sdk/ # SDK tests only\nuv run pytest tests/tools/ # Tools tests only\n```\n\n## Project Structure\n\n```\nagent-sdk/\n├── openhands-sdk/ # Core SDK package\n├── openhands-tools/ # Built-in tools\n├── openhands-workspace/ # Workspace management\n├── openhands-agent-server/ # Agent server\n├── examples/ # Usage examples\n└── tests/ # Test suites\n```\n\n## Contributing\n\n1. Create a new branch\n2. Make your changes\n3. Run tests and checks\n4. Push and create a pull request\n\nFor questions, join our [Slack community](https://openhands.dev/joinslack).\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2026 OpenHands contributors\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "MAINTAINERS", "content": "# Repository Maintainers\n#\n# Format: Each maintainer on a new line starting with \"- @username\"\n# This file is read by .github/workflows/assign-reviews.yml for automated triage\n#\n\nThe following people are maintainers of this repository and are responsible for triage and review:\n\n- @xingyaoww\n- @neubig\n- @enyst\n" }, { "path": "MANIFEST.in", "content": "# This MANIFEST.in file tells setuptools which files to include \n# in the sdist package distribution used for building docker image\n\n# ==============================================================================\n# Root-level workspace files\n# ==============================================================================\ninclude pyproject.toml\ninclude uv.lock\n\n# ==============================================================================\n# openhands-sdk\n# ==============================================================================\ninclude openhands-sdk/pyproject.toml\nrecursive-include openhands-sdk *.py\nrecursive-include openhands-sdk *.j2\nrecursive-include openhands-sdk py.typed\n\n# ==============================================================================\n# openhands-tools\n# ==============================================================================\ninclude openhands-tools/pyproject.toml\nrecursive-include openhands-tools *.py\nrecursive-include openhands-tools *.j2\nrecursive-include openhands-tools py.typed\n\n# ==============================================================================\n# openhands-workspace\n# ==============================================================================\ninclude openhands-workspace/pyproject.toml\nrecursive-include openhands-workspace *.py\nrecursive-include openhands-workspace py.typed\n\n# ==============================================================================\n# openhands-agent-server\n# ==============================================================================\ninclude openhands-agent-server/pyproject.toml\nrecursive-include openhands-agent-server *.py\nrecursive-include openhands-agent-server py.typed\n\n# Docker build files\ninclude openhands-agent-server/openhands/agent_server/docker/Dockerfile\ninclude openhands-agent-server/openhands/agent_server/docker/wallpaper.svg\n\n# PyInstaller spec\ninclude openhands-agent-server/openhands/agent_server/agent-server.spec\n\n# VSCode extensions\nrecursive-include openhands-agent-server/openhands/agent_server/vscode_extensions *\n" }, { "path": "Makefile", "content": "SHELL := /usr/bin/env bash\n.SHELLFLAGS := -eu -o pipefail -c\n\n# Colors for output\nECHO := printf '%b\\n'\nGREEN := \\033[32m\nYELLOW := \\033[33m\nRED := \\033[31m\nCYAN := \\033[36m\nRESET := \\033[0m\nUNDERLINE := \\033[4m\n\n# Required uv version\nREQUIRED_UV_VERSION := 0.8.13\nPKGS ?= openhands-sdk openhands-tools openhands-workspace openhands-agent-server\n\n.PHONY: build format lint clean help check-uv-version\n\n# Default target\n.DEFAULT_GOAL := help\n\n\ncheck-uv-version:\n\t@$(ECHO) \"$(YELLOW)Checking uv version...$(RESET)\"\n\t@UV_VERSION=$$(uv --version | cut -d' ' -f2); \\\n\tREQUIRED_VERSION=$(REQUIRED_UV_VERSION); \\\n\tif [ \"$$(printf '%s\\n' \"$$REQUIRED_VERSION\" \"$$UV_VERSION\" | sort -V | head -n1)\" != \"$$REQUIRED_VERSION\" ]; then \\\n\t\t$(ECHO) \"$(RED)Error: uv version $$UV_VERSION is less than required $$REQUIRED_VERSION$(RESET)\"; \\\n\t\t$(ECHO) \"$(YELLOW)Please update uv with: uv self update$(RESET)\"; \\\n\t\texit 1; \\\n\tfi; \\\n\t$(ECHO) \"$(GREEN)uv version $$UV_VERSION meets requirements$(RESET)\"\n\nbuild: check-uv-version\n\t@$(ECHO) \"$(CYAN)Setting up OpenHands V1 development environment...$(RESET)\"\n\t@$(ECHO) \"$(YELLOW)Installing dependencies with uv sync --dev...$(RESET)\"\n\t@uv sync --dev\n\t@$(ECHO) \"$(GREEN)Dependencies installed successfully.$(RESET)\"\n\t@$(ECHO) \"$(YELLOW)Setting up pre-commit hooks...$(RESET)\"\n\t@uv run pre-commit install\n\t@$(ECHO) \"$(GREEN)Pre-commit hooks installed successfully.$(RESET)\"\n\t@$(ECHO) \"$(GREEN)Build complete! Development environment is ready.$(RESET)\"\n\nformat:\n\t@$(ECHO) \"$(YELLOW)Formatting code with uv format...$(RESET)\"\n\t@uv run ruff format\n\t@$(ECHO) \"$(GREEN)Code formatted successfully.$(RESET)\"\n\nlint:\n\t@$(ECHO) \"$(YELLOW)Linting code with ruff...$(RESET)\"\n\t@uv run ruff check --fix\n\t@$(ECHO) \"$(GREEN)Linting completed.$(RESET)\"\n\npre-commit:\n\t@$(ECHO) \"$(YELLOW)Run pre-commit...$(RESET)\"\n\tuv run pre-commit run --all-files\n\t@$(ECHO) \"$(GREEN)Pre-commit run successfully.$(RESET)\"\n\nclean:\n\t@$(ECHO) \"$(YELLOW)Cleaning up cache files...$(RESET)\"\n\t@find . -type d -name \"__pycache__\" -exec rm -rf {} + 2>/dev/null || true\n\t@find . -type f -name \"*.pyc\" -delete 2>/dev/null || true\n\t@rm -rf .pytest_cache .ruff_cache .mypy_cache 2>/dev/null || true\n\t@$(ECHO) \"$(GREEN)Cache files cleaned.$(RESET)\"\n\n\n# Show help\nhelp:\n\t@$(ECHO) \"$(CYAN)OpenHands V1 Makefile$(RESET)\"\n\t@$(ECHO) \"\"\n\t@$(ECHO) \"$(UNDERLINE)Usage:$(RESET) make \"\n\t@$(ECHO) \"\"\n\t@$(ECHO) \"$(UNDERLINE)Commands:$(RESET)\"\n\t@$(ECHO) \" $(GREEN)build$(RESET) Setup development environment (install deps + hooks)\"\n\t@$(ECHO) \" $(GREEN)build-server$(RESET) Build agent-server executable\"\n\t@$(ECHO) \" $(GREEN)test-server-schema$(RESET) Test server schema\"\n\t@$(ECHO) \" $(GREEN)format$(RESET) Format code with uv format\"\n\t@$(ECHO) \" $(GREEN)lint$(RESET) Lint code with ruff\"\n\t@$(ECHO) \" $(GREEN)pre-commit$(RESET) Run the pre-commit\"\n\t@$(ECHO) \" $(GREEN)clean$(RESET) Clean up cache files\"\n\t@$(ECHO) \" $(GREEN)help$(RESET) Show this help message\"\n\nbuild-server: check-uv-version\n\t@$(ECHO) \"$(CYAN)Building agent-server executable...$(RESET)\"\n\t@uv run pyinstaller openhands-agent-server/openhands/agent_server/agent-server.spec\n\t@$(ECHO) \"$(GREEN)Build complete! Executable is in dist/agent-server/$(RESET)\"\n\ntest-server-schema: check-uv-version\n\tset -euo pipefail;\n\t# Generate OpenAPI JSON inline (no file left in repo)\n\tuv run python -c 'import os,json; from openhands.agent_server.api import api; open(\"openapi.json\",\"w\").write(json.dumps(api.openapi(), indent=2))'\n\tnpx --yes @apidevtools/swagger-cli@^4 validate openapi.json\n\t# Clean up temp schema\n\trm -f openapi.json\n\trm -rf .client\n\n\n.PHONY: set-package-version\nset-package-version: check-uv-version\n\t@if [ -z \"$(version)\" ]; then \\\n\t\t$(ECHO) \"$(RED)Error: missing version. Use: make set-package-version version=1.2.3$(RESET)\"; \\\n\t\texit 1; \\\n\tfi\n\t@$(ECHO) \"$(CYAN)Setting version to $(version) for: $(PKGS)$(RESET)\"\n\t@for PKG in $(PKGS); do \\\n\t\t$(ECHO) \"$(YELLOW)bumping $$PKG -> $(version)$(RESET)\"; \\\n\t\tuv version --package $$PKG $(version); \\\n\tdone\n\t@$(ECHO) \"$(GREEN)Version updated in all selected packages.$(RESET)\"\n" }, { "path": "README.md", "content": "\n\n
\n \"Logo\"\n

OpenHands Software Agent SDK

\n
\n\n\n
\n \"MIT\n \"Join\n
\n \"Check\n \"Tech\n \"Benchmark\n
\n \n Deutsch |\n Español |\n français |\n 日本語 |\n 한국어 |\n Português |\n Русский |\n 中文\n\n
\n
\n\nThe OpenHands Software Agent SDK is a set of Python and REST APIs for **building agents that work with code**.\n\nYou can use the OpenHands Software Agent SDK for:\n* One-off tasks, like building a README for your repo\n* Routine maintenance tasks, like updating dependencies\n* Major tasks that involve multiple agents, like refactors and rewrites\n\nImportantly, agents can either use the local machine as their workspace, or run inside ephemeral workspaces\n(e.g. in Docker or Kubernetes) using the Agent Server.\n\nYou can even use the SDK to build new developer experiences: it’s the engine behind the\n[OpenHands CLI](https://github.com/OpenHands/OpenHands-CLI) and [OpenHands Cloud](https://github.com/OpenHands/OpenHands).\n\nGet started with some [examples](https://docs.openhands.dev/sdk/guides/hello-world) or [check out the docs](https://docs.openhands.dev/sdk) to learn more.\n\n## Quick Start\n\nHere's what building with the SDK looks like:\n\n```python\nimport os\n\nfrom openhands.sdk import LLM, Agent, Conversation, Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.task_tracker import TaskTrackerTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nllm = LLM(\n model=\"anthropic/claude-sonnet-4-5-20250929\",\n api_key=os.getenv(\"LLM_API_KEY\"),\n)\n\nagent = Agent(\n llm=llm,\n tools=[\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n Tool(name=TaskTrackerTool.name),\n ],\n)\n\ncwd = os.getcwd()\nconversation = Conversation(agent=agent, workspace=cwd)\n\nconversation.send_message(\"Write 3 facts about the current project into FACTS.txt.\")\nconversation.run()\nprint(\"All done!\")\n```\n\nFor installation instructions and detailed setup, see the [Getting Started Guide](https://docs.openhands.dev/sdk/getting-started).\nFor local development from this repository, run `make build` to install the workspace dependencies and pre-commit hooks.\n\n## Documentation\n\nFor detailed documentation, tutorials, and API reference, visit:\n\n**[https://docs.openhands.dev/sdk](https://docs.openhands.dev/sdk)**\n\nThe documentation includes:\n- [Getting Started Guide](https://docs.openhands.dev/sdk/getting-started) - Installation and setup\n- [Architecture & Core Concepts](https://docs.openhands.dev/sdk/arch/overview) - Agents, tools, workspaces, and more\n- [Guides](https://docs.openhands.dev/sdk/guides/hello-world) - Hello World, custom tools, MCP, skills, and more\n- [Agent Server API Reference](https://docs.openhands.dev/sdk/guides/agent-server/api-reference/server-details/alive) - REST API reference for the remote agent server\n\n## Examples\n\nThe `examples/` directory contains comprehensive usage examples:\n\n- **Standalone SDK** (`examples/01_standalone_sdk/`) - Basic agent usage, custom tools, and skills\n- **Remote Agent Server** (`examples/02_remote_agent_server/`) - Client-server architecture and WebSocket connections\n- **GitHub Workflows** (`examples/03_github_workflows/`) - CI/CD integration and automated workflows\n\n## Skills for modern package tooling\n\nIf you enable public skills with `AgentContext(load_public_skills=True)`, the default\n`OpenHands/extensions` marketplace includes, for example, `uv` and `deno` skills.\nAgents can automatically pick up current package-management guidance for repositories\nthat use markers like `uv.lock`, `deno.json`, `deno.jsonc`, or `deno.lock`.\n\nSee `examples/01_standalone_sdk/03_activate_skill.py` for a minimal example that\nturns on public skill loading.\n\n## Contributing\n\nFor development setup, testing, and contribution guidelines, see [DEVELOPMENT.md](DEVELOPMENT.md).\n\n## Community\n\n- [Join Slack](https://openhands.dev/joinslack) - Connect with the OpenHands community\n- [GitHub Repository](https://github.com/OpenHands/software-agent-sdk) - Source code and issues\n- [Documentation](https://docs.openhands.dev/sdk) - Complete documentation\n\n## Cite\n\n```\n@misc{wang2025openhandssoftwareagentsdk,\n title={The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents}, \n author={Xingyao Wang and Simon Rosenberg and Juan Michelini and Calvin Smith and Hoang Tran and Engel Nyst and Rohit Malhotra and Xuhui Zhou and Valerie Chen and Robert Brennan and Graham Neubig},\n year={2025},\n eprint={2511.03690},\n archivePrefix={arXiv},\n primaryClass={cs.SE},\n url={https://arxiv.org/abs/2511.03690}, \n}\n```\n
\n\n### Thank You to Our Contributors\n\n[![Contributors](https://assets.openhands.dev/readme/openhands-software-agent-sdk-contributors.svg)](https://github.com/OpenHands/software-agent-sdk/graphs/contributors)\n\n
\n\n### Trusted by Engineers at\n\n
\n

\n\n \n \"TikTok\"\n\n\n \n \"VMware\"\n\n\n \n \"Roche\"\n\n\n \n \"Amazon\"\n\n\n \n \"C3\n\n\n \n \"Netflix\"\n\n\n \n \"Mastercard\"\n\n\n \n \"Red\n\n\n \n \"MongoDB\"\n\n\n \n \"Apple\"\n\n\n \n \"NVIDIA\"\n\n\n \n \"Google\"\n\n
\n\n" }, { "path": "examples/01_standalone_sdk/01_hello_world.py", "content": "import os\n\nfrom openhands.sdk import LLM, Agent, Conversation, Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.task_tracker import TaskTrackerTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nllm = LLM(\n model=os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\"),\n api_key=os.getenv(\"LLM_API_KEY\"),\n base_url=os.getenv(\"LLM_BASE_URL\", None),\n)\n\nagent = Agent(\n llm=llm,\n tools=[\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n Tool(name=TaskTrackerTool.name),\n ],\n)\n\ncwd = os.getcwd()\nconversation = Conversation(agent=agent, workspace=cwd)\n\nconversation.send_message(\"Write 3 facts about the current project into FACTS.txt.\")\nconversation.run()\nprint(\"All done!\")\n" }, { "path": "examples/01_standalone_sdk/02_custom_tools.py", "content": "\"\"\"Advanced example showing explicit executor usage and custom grep tool.\"\"\"\n\nimport os\nimport shlex\nfrom collections.abc import Sequence\n\nfrom pydantic import Field, SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Action,\n Agent,\n Conversation,\n Event,\n ImageContent,\n LLMConvertibleEvent,\n Observation,\n TextContent,\n ToolDefinition,\n get_logger,\n)\nfrom openhands.sdk.tool import (\n Tool,\n ToolExecutor,\n register_tool,\n)\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import (\n TerminalAction,\n TerminalExecutor,\n TerminalTool,\n)\n\n\nlogger = get_logger(__name__)\n\n# --- Action / Observation ---\n\n\nclass GrepAction(Action):\n pattern: str = Field(description=\"Regex to search for\")\n path: str = Field(\n default=\".\", description=\"Directory to search (absolute or relative)\"\n )\n include: str | None = Field(\n default=None, description=\"Optional glob to filter files (e.g. '*.py')\"\n )\n\n\nclass GrepObservation(Observation):\n matches: list[str] = Field(default_factory=list)\n files: list[str] = Field(default_factory=list)\n count: int = 0\n\n @property\n def to_llm_content(self) -> Sequence[TextContent | ImageContent]:\n if not self.count:\n return [TextContent(text=\"No matches found.\")]\n files_list = \"\\n\".join(f\"- {f}\" for f in self.files[:20])\n sample = \"\\n\".join(self.matches[:10])\n more = \"\\n...\" if self.count > 10 else \"\"\n ret = (\n f\"Found {self.count} matching lines.\\n\"\n f\"Files:\\n{files_list}\\n\"\n f\"Sample:\\n{sample}{more}\"\n )\n return [TextContent(text=ret)]\n\n\n# --- Executor ---\n\n\nclass GrepExecutor(ToolExecutor[GrepAction, GrepObservation]):\n def __init__(self, terminal: TerminalExecutor):\n self.terminal: TerminalExecutor = terminal\n\n def __call__(self, action: GrepAction, conversation=None) -> GrepObservation: # noqa: ARG002\n root = os.path.abspath(action.path)\n pat = shlex.quote(action.pattern)\n root_q = shlex.quote(root)\n\n # Use grep -r; add --include when provided\n if action.include:\n inc = shlex.quote(action.include)\n cmd = f\"grep -rHnE --include {inc} {pat} {root_q} 2>/dev/null | head -100\"\n else:\n cmd = f\"grep -rHnE {pat} {root_q} 2>/dev/null | head -100\"\n\n result = self.terminal(TerminalAction(command=cmd))\n\n matches: list[str] = []\n files: set[str] = set()\n\n # grep returns exit code 1 when no matches; treat as empty\n output_text = result.text\n\n if output_text.strip():\n for line in output_text.strip().splitlines():\n matches.append(line)\n # Expect \"path:line:content\" — take the file part before first \":\"\n file_path = line.split(\":\", 1)[0]\n if file_path:\n files.add(os.path.abspath(file_path))\n\n return GrepObservation(matches=matches, files=sorted(files), count=len(matches))\n\n\n# Tool description\n_GREP_DESCRIPTION = \"\"\"Fast content search tool.\n* Searches file contents using regular expressions\n* Supports full regex syntax (eg. \"log.*Error\", \"function\\\\s+\\\\w+\", etc.)\n* Filter files by pattern with the include parameter (eg. \"*.js\", \"*.{ts,tsx}\")\n* Returns matching file paths sorted by modification time.\n* Only the first 100 results are returned. Consider narrowing your search with stricter regex patterns or provide path parameter if you need more results.\n* Use this tool when you need to find files containing specific patterns\n* When you are doing an open ended search that may require multiple rounds of globbing and grepping, use the Agent tool instead\n\"\"\" # noqa: E501\n\n\n# --- Tool Definition ---\n\n\nclass GrepTool(ToolDefinition[GrepAction, GrepObservation]):\n \"\"\"A custom grep tool that searches file contents using regular expressions.\"\"\"\n\n @classmethod\n def create(\n cls, conv_state, terminal_executor: TerminalExecutor | None = None\n ) -> Sequence[ToolDefinition]:\n \"\"\"Create GrepTool instance with a GrepExecutor.\n\n Args:\n conv_state: Conversation state to get working directory from.\n terminal_executor: Optional terminal executor to reuse. If not provided,\n a new one will be created.\n\n Returns:\n A sequence containing a single GrepTool instance.\n \"\"\"\n if terminal_executor is None:\n terminal_executor = TerminalExecutor(\n working_dir=conv_state.workspace.working_dir\n )\n grep_executor = GrepExecutor(terminal_executor)\n\n return [\n cls(\n description=_GREP_DESCRIPTION,\n action_type=GrepAction,\n observation_type=GrepObservation,\n executor=grep_executor,\n )\n ]\n\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools - demonstrating both simplified and advanced patterns\ncwd = os.getcwd()\n\n\nclass BashAndGrepToolSet(ToolDefinition[Action, Observation]):\n \"\"\"Create terminal and grep tools sharing one terminal executor.\"\"\"\n\n @classmethod\n def create(cls, conv_state, **params) -> Sequence[ToolDefinition]:\n terminal_executor = TerminalExecutor(\n working_dir=conv_state.workspace.working_dir\n )\n terminal_tool = TerminalTool.create(\n conv_state, executor=terminal_executor, **params\n )[0]\n grep_tool = GrepTool.create(\n conv_state,\n terminal_executor=terminal_executor,\n )[0]\n return [terminal_tool, grep_tool]\n\n\nregister_tool(BashAndGrepToolSet.name, BashAndGrepToolSet)\n\ntools = [\n Tool(name=FileEditorTool.name),\n Tool(name=BashAndGrepToolSet.name),\n]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent, callbacks=[conversation_callback], workspace=cwd\n)\n\nconversation.send_message(\n \"Hello! Can you use the grep tool to find all files \"\n \"containing the word 'class' in this project, then create a summary file listing them? \" # noqa: E501\n \"Use the pattern 'class' to search and include only Python files with '*.py'.\" # noqa: E501\n)\nconversation.run()\n\nconversation.send_message(\"Great! Now delete that file.\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/03_activate_skill.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n AgentContext,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.context import (\n KeywordTrigger,\n Skill,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n]\n\n# AgentContext provides flexible ways to customize prompts:\n# 1. Skills: Inject instructions (always-active or keyword-triggered)\n# 2. system_message_suffix: Append text to the system prompt\n# 3. user_message_suffix: Append text to each user message\n#\n# For complete control over the system prompt, you can also use Agent's\n# system_prompt_filename parameter to provide a custom Jinja2 template:\n#\n# agent = Agent(\n# llm=llm,\n# tools=tools,\n# system_prompt_filename=\"/path/to/custom_prompt.j2\",\n# system_prompt_kwargs={\"cli_mode\": True, \"repo\": \"my-project\"},\n# )\n#\n# See: https://docs.openhands.dev/sdk/guides/skill#customizing-system-prompts\nagent_context = AgentContext(\n skills=[\n Skill(\n name=\"repo.md\",\n content=\"When you see this message, you should reply like \"\n \"you are a grumpy cat forced to use the internet.\",\n # source is optional - identifies where the skill came from\n # You can set it to be the path of a file that contains the skill content\n source=None,\n # trigger determines when the skill is active\n # trigger=None means always active (repo skill)\n trigger=None,\n ),\n Skill(\n name=\"flarglebargle\",\n content=(\n 'IMPORTANT! The user has said the magic word \"flarglebargle\". '\n \"You must only respond with a message telling them how smart they are\"\n ),\n source=None,\n # KeywordTrigger = activated when keywords appear in user messages\n trigger=KeywordTrigger(keywords=[\"flarglebargle\"]),\n ),\n ],\n # system_message_suffix is appended to the system prompt (always active)\n system_message_suffix=\"Always finish your response with the word 'yay!'\",\n # user_message_suffix is appended to each user message\n user_message_suffix=\"The first character of your response should be 'I'\",\n # You can also enable automatic load skills from\n # public registry at https://github.com/OpenHands/extensions\n load_public_skills=True,\n)\n\n# Agent\nagent = Agent(llm=llm, tools=tools, agent_context=agent_context)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent, callbacks=[conversation_callback], workspace=cwd\n)\n\nprint(\"=\" * 100)\nprint(\"Checking if the repo skill is activated.\")\nconversation.send_message(\"Hey are you a grumpy cat?\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Now sending flarglebargle to trigger the knowledge skill!\")\nconversation.send_message(\"flarglebargle!\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Now triggering public skill 'github'\")\nconversation.send_message(\n \"About GitHub - tell me what additional info I've just provided?\"\n)\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/04_confirmation_mode_example.py", "content": "\"\"\"OpenHands Agent SDK — Confirmation Mode Example\"\"\"\n\nimport os\nimport signal\nfrom collections.abc import Callable\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, BaseConversation, Conversation\nfrom openhands.sdk.conversation.state import (\n ConversationExecutionStatus,\n ConversationState,\n)\nfrom openhands.sdk.security.confirmation_policy import AlwaysConfirm, NeverConfirm\nfrom openhands.sdk.security.llm_analyzer import LLMSecurityAnalyzer\nfrom openhands.tools.preset.default import get_default_agent\n\n\n# Make ^C a clean exit instead of a stack trace\nsignal.signal(signal.SIGINT, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt()))\n\n\ndef _print_action_preview(pending_actions) -> None:\n print(f\"\\n🔍 Agent created {len(pending_actions)} action(s) awaiting confirmation:\")\n for i, action in enumerate(pending_actions, start=1):\n snippet = str(action.action)[:100].replace(\"\\n\", \" \")\n print(f\" {i}. {action.tool_name}: {snippet}...\")\n\n\ndef confirm_in_console(pending_actions) -> bool:\n \"\"\"\n Return True to approve, False to reject.\n Default to 'no' on EOF/KeyboardInterrupt (matches original behavior).\n \"\"\"\n _print_action_preview(pending_actions)\n while True:\n try:\n ans = (\n input(\"\\nDo you want to execute these actions? (yes/no): \")\n .strip()\n .lower()\n )\n except (EOFError, KeyboardInterrupt):\n print(\"\\n❌ No input received; rejecting by default.\")\n return False\n\n if ans in (\"yes\", \"y\"):\n print(\"✅ Approved — executing actions…\")\n return True\n if ans in (\"no\", \"n\"):\n print(\"❌ Rejected — skipping actions…\")\n return False\n print(\"Please enter 'yes' or 'no'.\")\n\n\ndef run_until_finished(conversation: BaseConversation, confirmer: Callable) -> None:\n \"\"\"\n Drive the conversation until FINISHED.\n If WAITING_FOR_CONFIRMATION, ask the confirmer;\n on reject, call reject_pending_actions().\n Preserves original error if agent waits but no actions exist.\n \"\"\"\n while conversation.state.execution_status != ConversationExecutionStatus.FINISHED:\n if (\n conversation.state.execution_status\n == ConversationExecutionStatus.WAITING_FOR_CONFIRMATION\n ):\n pending = ConversationState.get_unmatched_actions(conversation.state.events)\n if not pending:\n raise RuntimeError(\n \"⚠️ Agent is waiting for confirmation but no pending actions \"\n \"were found. This should not happen.\"\n )\n if not confirmer(pending):\n conversation.reject_pending_actions(\"User rejected the actions\")\n # Let the agent produce a new step or finish\n continue\n\n print(\"▶️ Running conversation.run()…\")\n conversation.run()\n\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\nagent = get_default_agent(llm=llm)\nconversation = Conversation(agent=agent, workspace=os.getcwd())\n\n# Conditionally add security analyzer based on environment variable\nadd_security_analyzer = bool(os.getenv(\"ADD_SECURITY_ANALYZER\", \"\").strip())\nif add_security_analyzer:\n print(\"Agent security analyzer added.\")\n conversation.set_security_analyzer(LLMSecurityAnalyzer())\n\n# 1) Confirmation mode ON\nconversation.set_confirmation_policy(AlwaysConfirm())\nprint(\"\\n1) Command that will likely create actions…\")\nconversation.send_message(\"Please list the files in the current directory using ls -la\")\nrun_until_finished(conversation, confirm_in_console)\n\n# 2) A command the user may choose to reject\nprint(\"\\n2) Command the user may choose to reject…\")\nconversation.send_message(\"Please create a file called 'dangerous_file.txt'\")\nrun_until_finished(conversation, confirm_in_console)\n\n# 3) Simple greeting (no actions expected)\nprint(\"\\n3) Simple greeting (no actions expected)…\")\nconversation.send_message(\"Just say hello to me\")\nrun_until_finished(conversation, confirm_in_console)\n\n# 4) Disable confirmation mode and run commands directly\nprint(\"\\n4) Disable confirmation mode and run a command…\")\nconversation.set_confirmation_policy(NeverConfirm())\nconversation.send_message(\"Please echo 'Hello from confirmation mode example!'\")\nconversation.run()\n\nconversation.send_message(\n \"Please delete any file that was created during this conversation.\"\n)\nconversation.run()\n\nprint(\"\\n=== Example Complete ===\")\nprint(\"Key points:\")\nprint(\n \"- conversation.run() creates actions; confirmation mode \"\n \"sets execution_status=WAITING_FOR_CONFIRMATION\"\n)\nprint(\"- User confirmation is handled via a single reusable function\")\nprint(\"- Rejection uses conversation.reject_pending_actions() and the loop continues\")\nprint(\"- Simple responses work normally without actions\")\nprint(\"- Confirmation policy is toggled with conversation.set_confirmation_policy()\")\n" }, { "path": "examples/01_standalone_sdk/05_use_llm_registry.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n LLMRegistry,\n Message,\n TextContent,\n get_logger,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM using LLMRegistry\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\n\n# Create LLM instance\nmain_llm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Create LLM registry and add the LLM\nllm_registry = LLMRegistry()\nllm_registry.add(main_llm)\n\n# Get LLM from registry\nllm = llm_registry.get(\"agent\")\n\n# Tools\ncwd = os.getcwd()\ntools = [Tool(name=TerminalTool.name)]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent, callbacks=[conversation_callback], workspace=cwd\n)\n\nconversation.send_message(\"Please echo 'Hello!'\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\nprint(\"=\" * 100)\nprint(f\"LLM Registry usage IDs: {llm_registry.list_usage_ids()}\")\n\n# Demonstrate getting the same LLM instance from registry\nsame_llm = llm_registry.get(\"agent\")\nprint(f\"Same LLM instance: {llm is same_llm}\")\n\n# Demonstrate requesting a completion directly from an LLM\nresp = llm.completion(\n messages=[\n Message(role=\"user\", content=[TextContent(text=\"Say hello in one word.\")])\n ]\n)\n# Access the response content via OpenHands LLMResponse\nmsg = resp.message\ntexts = [c.text for c in msg.content if isinstance(c, TextContent)]\nprint(f\"Direct completion response: {texts[0] if texts else str(msg)}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/06_interactive_terminal_w_reasoning.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(\n name=TerminalTool.name,\n params={\"no_change_timeout_seconds\": 3},\n )\n]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent, callbacks=[conversation_callback], workspace=cwd\n)\n\nconversation.send_message(\n \"Enter python interactive mode by directly running `python3`, then tell me \"\n \"the current time, and exit python interactive mode.\"\n)\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n" }, { "path": "examples/01_standalone_sdk/07_mcp_integration.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.security.llm_analyzer import LLMSecurityAnalyzer\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\ncwd = os.getcwd()\ntools = [\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n]\n\n# Add MCP Tools\nmcp_config = {\n \"mcpServers\": {\n \"fetch\": {\"command\": \"uvx\", \"args\": [\"mcp-server-fetch\"]},\n \"repomix\": {\"command\": \"npx\", \"args\": [\"-y\", \"repomix@1.4.2\", \"--mcp\"]},\n }\n}\n# Agent\nagent = Agent(\n llm=llm,\n tools=tools,\n mcp_config=mcp_config,\n # This regex filters out all repomix tools except pack_codebase\n filter_tools_regex=\"^(?!repomix)(.*)|^repomix.*pack_codebase.*$\",\n)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\n# Conversation\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n workspace=cwd,\n)\nconversation.set_security_analyzer(LLMSecurityAnalyzer())\n\nlogger.info(\"Starting conversation with MCP integration...\")\nconversation.send_message(\n \"Read https://github.com/OpenHands/OpenHands and write 3 facts \"\n \"about the project into FACTS.txt.\"\n)\nconversation.run()\n\nconversation.send_message(\"Great! Now delete that file.\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/08_mcp_with_oauth.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\ncwd = os.getcwd()\ntools = [\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n]\n\nmcp_config = {\n \"mcpServers\": {\"Notion\": {\"url\": \"https://mcp.notion.com/mcp\", \"auth\": \"oauth\"}}\n}\nagent = Agent(llm=llm, tools=tools, mcp_config=mcp_config)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\n# Conversation\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n)\n\nlogger.info(\"Starting conversation with MCP integration...\")\nconversation.send_message(\"Can you search about OpenHands V1 in my notion workspace?\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n" }, { "path": "examples/01_standalone_sdk/09_pause_example.py", "content": "import os\nimport threading\nimport time\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ntools = [\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\nconversation = Conversation(agent, workspace=os.getcwd())\n\nprint(\"=\" * 60)\nprint(\"Pause and Continue Example\")\nprint(\"=\" * 60)\nprint()\n\n# Phase 1: Start a long-running task\nprint(\"Phase 1: Starting agent with a task...\")\nconversation.send_message(\n \"Create a file called countdown.txt and write numbers from 100 down to 1, \"\n \"one number per line. After you finish, summarize what you did.\"\n)\n\nprint(f\"Initial status: {conversation.state.execution_status}\")\nprint()\n\n# Start the agent in a background thread\nthread = threading.Thread(target=conversation.run)\nthread.start()\n\n# Let the agent work for a few seconds\nprint(\"Letting agent work for 2 seconds...\")\ntime.sleep(2)\n\n# Phase 2: Pause the agent\nprint()\nprint(\"Phase 2: Pausing the agent...\")\nconversation.pause()\n\n# Wait for the thread to finish (it will stop when paused)\nthread.join()\n\nprint(f\"Agent status after pause: {conversation.state.execution_status}\")\nprint()\n\n# Phase 3: Send a new message while paused\nprint(\"Phase 3: Sending a new message while agent is paused...\")\nconversation.send_message(\n \"Actually, stop working on countdown.txt. Instead, create a file called \"\n \"hello.txt with just the text 'Hello, World!' in it.\"\n)\nprint()\n\n# Phase 4: Resume the agent with .run()\nprint(\"Phase 4: Resuming agent with .run()...\")\nprint(f\"Status before resume: {conversation.state.execution_status}\")\n\n# Resume execution\nconversation.run()\n\nprint(f\"Final status: {conversation.state.execution_status}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/10_persistence.py", "content": "import os\nimport uuid\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n]\n\n# Add MCP Tools\nmcp_config = {\n \"mcpServers\": {\n \"fetch\": {\"command\": \"uvx\", \"args\": [\"mcp-server-fetch\"]},\n }\n}\n# Agent\nagent = Agent(llm=llm, tools=tools, mcp_config=mcp_config)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation_id = uuid.uuid4()\npersistence_dir = \"./.conversations\"\n\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n workspace=cwd,\n persistence_dir=persistence_dir,\n conversation_id=conversation_id,\n)\nconversation.send_message(\n \"Read https://github.com/OpenHands/OpenHands. Then write 3 facts \"\n \"about the project into FACTS.txt.\"\n)\nconversation.run()\n\nconversation.send_message(\"Great! Now delete that file.\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n# Conversation persistence\nprint(\"Serializing conversation...\")\n\ndel conversation\n\n# Deserialize the conversation\nprint(\"Deserializing conversation...\")\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n workspace=cwd,\n persistence_dir=persistence_dir,\n conversation_id=conversation_id,\n)\n\nprint(\"Sending message to deserialized conversation...\")\nconversation.send_message(\"Hey what did you create? Return an agent finish action\")\nconversation.run()\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/11_async.py", "content": "\"\"\"\nThis example demonstrates usage of a Conversation in an async context\n(e.g.: From a fastapi server). The conversation is run in a background\nthread and a callback with results is executed in the main runloop\n\"\"\"\n\nimport asyncio\nimport os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.conversation.types import ConversationCallbackType\nfrom openhands.sdk.tool import Tool\nfrom openhands.sdk.utils.async_utils import AsyncCallbackWrapper\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.task_tracker import TaskTrackerTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n Tool(name=TaskTrackerTool.name),\n]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\n\nllm_messages = [] # collect raw LLM messages\n\n\n# Callback coroutine\nasync def callback_coro(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\n# Synchronous run conversation\ndef run_conversation(callback: ConversationCallbackType):\n conversation = Conversation(agent=agent, callbacks=[callback])\n\n conversation.send_message(\n \"Hello! Can you create a new Python file named hello.py that prints \"\n \"'Hello, World!'? Use task tracker to plan your steps.\"\n )\n conversation.run()\n\n conversation.send_message(\"Great! Now delete that file.\")\n conversation.run()\n\n\nasync def main():\n loop = asyncio.get_running_loop()\n\n # Create the callback\n callback = AsyncCallbackWrapper(callback_coro, loop)\n\n # Run the conversation in a background thread and wait for it to finish...\n await loop.run_in_executor(None, run_conversation, callback)\n\n print(\"=\" * 100)\n print(\"Conversation finished. Got the following LLM messages:\")\n for i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n # Report cost\n cost = llm.metrics.accumulated_cost\n print(f\"EXAMPLE_COST: {cost}\")\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "examples/01_standalone_sdk/12_custom_secrets.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n)\nfrom openhands.sdk.secret import SecretSource\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ntools = [\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\nconversation = Conversation(agent)\n\n\nclass MySecretSource(SecretSource):\n def get_value(self) -> str:\n return \"callable-based-secret\"\n\n\nconversation.update_secrets(\n {\"SECRET_TOKEN\": \"my-secret-token-value\", \"SECRET_FUNCTION_TOKEN\": MySecretSource()}\n)\n\nconversation.send_message(\"just echo $SECRET_TOKEN\")\n\nconversation.run()\n\nconversation.send_message(\"just echo $SECRET_FUNCTION_TOKEN\")\n\nconversation.run()\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/13_get_llm_metrics.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\ncwd = os.getcwd()\ntools = [\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n]\n\n# Add MCP Tools\nmcp_config = {\"mcpServers\": {\"fetch\": {\"command\": \"uvx\", \"args\": [\"mcp-server-fetch\"]}}}\n\n# Agent\nagent = Agent(llm=llm, tools=tools, mcp_config=mcp_config)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\n# Conversation\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n workspace=cwd,\n)\n\nlogger.info(\"Starting conversation with MCP integration...\")\nconversation.send_message(\n \"Read https://github.com/OpenHands/OpenHands and write 3 facts \"\n \"about the project into FACTS.txt.\"\n)\nconversation.run()\n\nconversation.send_message(\"Great! Now delete that file.\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\nassert llm.metrics is not None\nprint(\n f\"Conversation finished. Final LLM metrics with details: {llm.metrics.model_dump()}\"\n)\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/14_context_condenser.py", "content": "\"\"\"\nTo manage context in long-running conversations, the agent can use a context condenser\nthat keeps the conversation history within a specified size limit. This example\ndemonstrates using the `LLMSummarizingCondenser`, which automatically summarizes\nolder parts of the conversation when the history exceeds a defined threshold.\n\"\"\"\n\nimport os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.context.condenser import LLMSummarizingCondenser\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.task_tracker import TaskTrackerTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n Tool(name=TaskTrackerTool.name),\n]\n\n# Create a condenser to manage the context. The condenser will automatically truncate\n# conversation history when it exceeds max_size, and replaces the dropped events with an\n# LLM-generated summary. This condenser triggers when there are more than ten events in\n# the conversation history, and always keeps the first two events (system prompts,\n# initial user messages) to preserve important context.\ncondenser = LLMSummarizingCondenser(\n llm=llm.model_copy(update={\"usage_id\": \"condenser\"}), max_size=10, keep_first=2\n)\n\n# Agent with condenser\nagent = Agent(llm=llm, tools=tools, condenser=condenser)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n persistence_dir=\"./.conversations\",\n workspace=\".\",\n)\n\n# Send multiple messages to demonstrate condensation\nprint(\"Sending multiple messages to demonstrate LLM Summarizing Condenser...\")\n\nconversation.send_message(\n \"Hello! Can you create a Python file named math_utils.py with functions for \"\n \"basic arithmetic operations (add, subtract, multiply, divide)?\"\n)\nconversation.run()\n\nconversation.send_message(\n \"Great! Now add a function to calculate the factorial of a number.\"\n)\nconversation.run()\n\nconversation.send_message(\"Add a function to check if a number is prime.\")\nconversation.run()\n\nconversation.send_message(\n \"Add a function to calculate the greatest common divisor (GCD) of two numbers.\"\n)\nconversation.run()\n\nconversation.send_message(\n \"Now create a test file to verify all these functions work correctly.\"\n)\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n# Conversation persistence\nprint(\"Serializing conversation...\")\n\ndel conversation\n\n# Deserialize the conversation\nprint(\"Deserializing conversation...\")\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n persistence_dir=\"./.conversations\",\n workspace=\".\",\n)\n\nprint(\"Sending message to deserialized conversation...\")\nconversation.send_message(\"Finally, clean up by deleting both files.\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished with LLM Summarizing Condenser.\")\nprint(f\"Total LLM messages collected: {len(llm_messages)}\")\nprint(\"\\nThe condenser automatically summarized older conversation history\")\nprint(\"when the conversation exceeded the configured max_size threshold.\")\nprint(\"This helps manage context length while preserving important information.\")\n\n# Report cost\ncost = conversation.conversation_stats.get_combined_metrics().accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/15_browser_use.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.browser_use import BrowserToolSet\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n Tool(name=BrowserToolSet.name),\n]\n\n# If you need fine-grained browser control, you can manually register individual browser\n# tools by creating a BrowserToolExecutor and providing factories that return customized\n# Tool instances before constructing the Agent.\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent, callbacks=[conversation_callback], workspace=cwd\n)\n\nconversation.send_message(\n \"Could you go to https://openhands.dev/ blog page and summarize main \"\n \"points of the latest blog?\"\n)\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n" }, { "path": "examples/01_standalone_sdk/16_llm_security_analyzer.py", "content": "\"\"\"OpenHands Agent SDK — LLM Security Analyzer Example (Simplified)\n\nThis example shows how to use the LLMSecurityAnalyzer to automatically\nevaluate security risks of actions before execution.\n\"\"\"\n\nimport os\nimport signal\nfrom collections.abc import Callable\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, Agent, BaseConversation, Conversation\nfrom openhands.sdk.conversation.state import (\n ConversationExecutionStatus,\n ConversationState,\n)\nfrom openhands.sdk.security.confirmation_policy import ConfirmRisky\nfrom openhands.sdk.security.llm_analyzer import LLMSecurityAnalyzer\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Clean ^C exit: no stack trace noise\nsignal.signal(signal.SIGINT, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt()))\n\n\ndef _print_blocked_actions(pending_actions) -> None:\n print(f\"\\n🔒 Security analyzer blocked {len(pending_actions)} high-risk action(s):\")\n for i, action in enumerate(pending_actions, start=1):\n snippet = str(action.action)[:100].replace(\"\\n\", \" \")\n print(f\" {i}. {action.tool_name}: {snippet}...\")\n\n\ndef confirm_high_risk_in_console(pending_actions) -> bool:\n \"\"\"\n Return True to approve, False to reject.\n Matches original behavior: default to 'no' on EOF/KeyboardInterrupt.\n \"\"\"\n _print_blocked_actions(pending_actions)\n while True:\n try:\n ans = (\n input(\n \"\\nThese actions were flagged as HIGH RISK. \"\n \"Do you want to execute them anyway? (yes/no): \"\n )\n .strip()\n .lower()\n )\n except (EOFError, KeyboardInterrupt):\n print(\"\\n❌ No input received; rejecting by default.\")\n return False\n\n if ans in (\"yes\", \"y\"):\n print(\"✅ Approved — executing high-risk actions...\")\n return True\n if ans in (\"no\", \"n\"):\n print(\"❌ Rejected — skipping high-risk actions...\")\n return False\n print(\"Please enter 'yes' or 'no'.\")\n\n\ndef run_until_finished_with_security(\n conversation: BaseConversation, confirmer: Callable[[list], bool]\n) -> None:\n \"\"\"\n Drive the conversation until FINISHED.\n - If WAITING_FOR_CONFIRMATION: ask the confirmer.\n * On approve: set execution_status = IDLE (keeps original example’s behavior).\n * On reject: conversation.reject_pending_actions(...).\n - If WAITING but no pending actions: print warning and set IDLE (matches original).\n \"\"\"\n while conversation.state.execution_status != ConversationExecutionStatus.FINISHED:\n if (\n conversation.state.execution_status\n == ConversationExecutionStatus.WAITING_FOR_CONFIRMATION\n ):\n pending = ConversationState.get_unmatched_actions(conversation.state.events)\n if not pending:\n raise RuntimeError(\n \"⚠️ Agent is waiting for confirmation but no pending actions \"\n \"were found. This should not happen.\"\n )\n if not confirmer(pending):\n conversation.reject_pending_actions(\"User rejected high-risk actions\")\n continue\n\n print(\"▶️ Running conversation.run()...\")\n conversation.run()\n\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"security-analyzer\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ntools = [\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\n\n# Conversation with persisted filestore\nconversation = Conversation(\n agent=agent, persistence_dir=\"./.conversations\", workspace=\".\"\n)\nconversation.set_security_analyzer(LLMSecurityAnalyzer())\nconversation.set_confirmation_policy(ConfirmRisky())\n\nprint(\"\\n1) Safe command (LOW risk - should execute automatically)...\")\nconversation.send_message(\"List files in the current directory\")\nconversation.run()\n\nprint(\"\\n2) Potentially risky command (may require confirmation)...\")\nconversation.send_message(\n \"Please echo 'hello world' -- PLEASE MARK THIS AS A HIGH RISK ACTION\"\n)\nrun_until_finished_with_security(conversation, confirm_high_risk_in_console)\n" }, { "path": "examples/01_standalone_sdk/17_image_input.py", "content": "\"\"\"OpenHands Agent SDK — Image Input Example.\n\nThis script mirrors the basic setup from ``examples/01_hello_world.py`` but adds\nvision support by sending an image to the agent alongside text instructions.\n\nIt also demonstrates multi-image input with base64-encoded images that exercise\nthe Anthropic many-image resizing path (>20 images are automatically downscaled\nto 2000×2000 px).\n\"\"\"\n\nimport base64\nimport io\nimport os\n\nfrom PIL import Image\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n ImageContent,\n LLMConvertibleEvent,\n Message,\n TextContent,\n get_logger,\n)\nfrom openhands.sdk.tool.spec import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.task_tracker import TaskTrackerTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n\ndef _make_png_data_url(width: int, height: int, color: str = \"red\") -> str:\n \"\"\"Create a base64 PNG data URL with the given dimensions and colour.\"\"\"\n image = Image.new(\"RGB\", (width, height), color=color)\n buffer = io.BytesIO()\n image.save(buffer, format=\"PNG\")\n encoded = base64.b64encode(buffer.getvalue()).decode(\"ascii\")\n return f\"data:image/png;base64,{encoded}\"\n\n\n# Configure LLM (vision-capable model)\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"vision-llm\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\nassert llm.vision_is_active(), \"The selected LLM model does not support vision input.\"\n\ncwd = os.getcwd()\n\nagent = Agent(\n llm=llm,\n tools=[\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n Tool(name=TaskTrackerTool.name),\n ],\n)\n\nllm_messages = [] # collect raw LLM messages for inspection\n\n\ndef conversation_callback(event: Event) -> None:\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent, callbacks=[conversation_callback], workspace=cwd\n)\n\n# ── Part 1: single URL image ──────────────────────────────────────────────\nIMAGE_URL = \"https://github.com/OpenHands/docs/raw/main/openhands/static/img/logo.png\"\n\nconversation.send_message(\n Message(\n role=\"user\",\n content=[\n TextContent(\n text=(\n \"Study this image and describe the key elements you see. \"\n \"Summarize them in a short paragraph and suggest a catchy caption.\"\n )\n ),\n ImageContent(image_urls=[IMAGE_URL]),\n ],\n )\n)\nconversation.run()\n\nconversation.send_message(\n \"Great! Please save your description and caption into image_report.md.\"\n)\nconversation.run()\n\n# ── Part 2: many oversized base64 images (exercises Anthropic resize) ─────\n# Generate 21 base64 images at 2500×100 px — just above the 20-image threshold\n# that triggers Anthropic's many-image limit (2000×2000 px per image).\n# The SDK will automatically downscale these before sending to the provider.\nCOLORS = [\n \"red\",\n \"green\",\n \"blue\",\n \"yellow\",\n \"cyan\",\n \"magenta\",\n \"orange\",\n \"purple\",\n \"pink\",\n \"brown\",\n \"gray\",\n \"white\",\n \"navy\",\n \"teal\",\n \"olive\",\n \"maroon\",\n \"lime\",\n \"aqua\",\n \"coral\",\n \"gold\",\n \"indigo\",\n]\noversized_data_urls = [\n _make_png_data_url(2500, 100, color=COLORS[i % len(COLORS)]) for i in range(21)\n]\n\nconversation.send_message(\n Message(\n role=\"user\",\n content=[\n TextContent(\n text=(\n \"I'm sending you 21 solid-colour test images. \"\n \"List the dominant colour of each image in order, \"\n \"one per line.\"\n )\n ),\n ImageContent(image_urls=oversized_data_urls),\n ],\n )\n)\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/18_send_message_while_processing.py", "content": "\"\"\"\nExample demonstrating that user messages can be sent and processed while\nan agent is busy.\n\nThis example demonstrates a key capability of the OpenHands agent system: the ability\nto receive and process new user messages even while the agent is actively working on\na previous task. This is made possible by the agent's event-driven architecture.\n\nDemonstration Flow:\n1. Send initial message asking agent to:\n - Write \"Message 1 sent at [time], written at [CURRENT_TIME]\"\n - Wait 3 seconds\n - Write \"Message 2 sent at [time], written at [CURRENT_TIME]\"\n [time] is the time the message was sent to the agent\n [CURRENT_TIME] is the time the agent writes the line\n2. Start agent processing in a background thread\n3. While agent is busy (during the 3-second delay), send a second message asking to add:\n - \"Message 3 sent at [time], written at [CURRENT_TIME]\"\n4. Verify that all three lines are processed and included in the final document\n\nExpected Evidence:\nThe final document will contain three lines with dual timestamps:\n- \"Message 1 sent at HH:MM:SS, written at HH:MM:SS\" (from initial message, written immediately)\n- \"Message 2 sent at HH:MM:SS, written at HH:MM:SS\" (from initial message, written after 3-second delay)\n- \"Message 3 sent at HH:MM:SS, written at HH:MM:SS\" (from second message sent during delay)\n\nThe timestamps will show that Message 3 was sent while the agent was running,\nbut was still successfully processed and written to the document.\n\nThis proves that:\n- The second user message was sent while the agent was processing the first task\n- The agent successfully received and processed the second message\n- The agent's event system allows for real-time message integration during processing\n\nKey Components Demonstrated:\n- Conversation.send_message(): Adds messages to events list immediately\n- Agent.step(): Processes all events including newly added messages\n- Threading: Allows message sending while agent is actively processing\n\"\"\" # noqa\n\nimport os\nimport threading\nimport time\nfrom datetime import datetime\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(\n name=TerminalTool.name,\n ),\n Tool(name=FileEditorTool.name),\n]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\nconversation = Conversation(agent)\n\n\ndef timestamp() -> str:\n return datetime.now().strftime(\"%H:%M:%S\")\n\n\nprint(\"=== Send Message While Processing Example ===\")\n\n# Step 1: Send initial message\nstart_time = timestamp()\nconversation.send_message(\n f\"Create a file called document.txt and write this first sentence: \"\n f\"'Message 1 sent at {start_time}, written at [CURRENT_TIME].' \"\n f\"Replace [CURRENT_TIME] with the actual current time when you write the line. \"\n f\"Then wait 3 seconds and write 'Message 2 sent at {start_time}, written at [CURRENT_TIME].'\" # noqa\n)\n\n# Step 2: Start agent processing in background\nthread = threading.Thread(target=conversation.run)\nthread.start()\n\n# Step 3: Wait then send second message while agent is processing\ntime.sleep(2) # Give agent time to start working\n\nsecond_time = timestamp()\n\nconversation.send_message(\n f\"Please also add this second sentence to document.txt: \"\n f\"'Message 3 sent at {second_time}, written at [CURRENT_TIME].' \"\n f\"Replace [CURRENT_TIME] with the actual current time when you write this line.\"\n)\n\n# Wait for completion\nthread.join()\n\n# Verification\ndocument_path = os.path.join(cwd, \"document.txt\")\nif os.path.exists(document_path):\n with open(document_path) as f:\n content = f.read()\n\n print(\"\\nDocument contents:\")\n print(\"─────────────────────\")\n print(content)\n print(\"─────────────────────\")\n\n # Check if both messages were processed\n if \"Message 1\" in content and \"Message 2\" in content:\n print(\"\\nSUCCESS: Agent processed both messages!\")\n print(\n \"This proves the agent received the second message while processing the first task.\" # noqa\n )\n else:\n print(\"\\nWARNING: Agent may not have processed the second message\")\n\n # Clean up\n os.remove(document_path)\nelse:\n print(\"WARNING: Document.txt was not created\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/19_llm_routing.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n ImageContent,\n LLMConvertibleEvent,\n Message,\n TextContent,\n get_logger,\n)\nfrom openhands.sdk.llm.router import MultimodalRouter\nfrom openhands.tools.preset.default import get_default_tools\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"openhands/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\n\nprimary_llm = LLM(\n usage_id=\"agent-primary\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\nsecondary_llm = LLM(\n usage_id=\"agent-secondary\",\n model=\"openhands/devstral-small-2507\",\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\nmultimodal_router = MultimodalRouter(\n usage_id=\"multimodal-router\",\n llms_for_routing={\"primary\": primary_llm, \"secondary\": secondary_llm},\n)\n\n# Tools\ntools = get_default_tools() # Use our default openhands experience\n\n# Agent\nagent = Agent(llm=multimodal_router, tools=tools)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent, callbacks=[conversation_callback], workspace=os.getcwd()\n)\n\nconversation.send_message(\n message=Message(\n role=\"user\",\n content=[TextContent(text=(\"Hi there, who trained you?\"))],\n )\n)\nconversation.run()\n\nconversation.send_message(\n message=Message(\n role=\"user\",\n content=[\n ImageContent(\n image_urls=[\"http://images.cocodataset.org/val2017/000000039769.jpg\"]\n ),\n TextContent(text=(\"What do you see in the image above?\")),\n ],\n )\n)\nconversation.run()\n\nconversation.send_message(\n message=Message(\n role=\"user\",\n content=[TextContent(text=(\"Who trained you as an LLM?\"))],\n )\n)\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n# Report cost\ncost = conversation.conversation_stats.get_combined_metrics().accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/20_stuck_detector.py", "content": "import os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.tools.preset.default import get_default_agent\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\nagent = get_default_agent(llm=llm)\n\nllm_messages = []\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\n# Create conversation with built-in stuck detection\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n workspace=os.getcwd(),\n # This is by default True, shown here for clarity of the example\n stuck_detection=True,\n)\n\n# Send a task that will be caught by stuck detection\nconversation.send_message(\n \"Please execute 'ls' command 5 times, each in its own \"\n \"action without any thought and then exit at the 6th step.\"\n)\n\n# Run the conversation - stuck detection happens automatically\nconversation.run()\n\nassert conversation.stuck_detector is not None\nfinal_stuck_check = conversation.stuck_detector.is_stuck()\nprint(f\"Final stuck status: {final_stuck_check}\")\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n print(f\"Message {i}: {str(message)[:200]}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/21_generate_extraneous_conversation_costs.py", "content": "import os\n\nfrom pydantic import SecretStr\nfrom tabulate import tabulate\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n LLMSummarizingCondenser,\n Message,\n TextContent,\n get_logger,\n)\nfrom openhands.sdk.tool.spec import Tool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM using LLMRegistry\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\n\n# Create LLM instance\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\nllm_condenser = LLM(\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n usage_id=\"condenser\",\n)\n\n# Tools\ncondenser = LLMSummarizingCondenser(llm=llm_condenser, max_size=10, keep_first=2)\n\ncwd = os.getcwd()\nagent = Agent(\n llm=llm,\n tools=[\n Tool(\n name=TerminalTool.name,\n ),\n ],\n condenser=condenser,\n)\n\nconversation = Conversation(agent=agent, workspace=cwd)\nconversation.send_message(\n message=Message(\n role=\"user\",\n content=[TextContent(text=\"Please echo 'Hello!'\")],\n )\n)\nconversation.run()\n\n# Demonstrate extraneous costs part of the conversation\nsecond_llm = LLM(\n usage_id=\"demo-secondary\",\n model=model,\n base_url=os.getenv(\"LLM_BASE_URL\"),\n api_key=SecretStr(api_key),\n)\nconversation.llm_registry.add(second_llm)\ncompletion_response = second_llm.completion(\n messages=[Message(role=\"user\", content=[TextContent(text=\"echo 'More spend!'\")])]\n)\n\n# Access total spend\nspend = conversation.conversation_stats.get_combined_metrics()\nprint(\"\\n=== Total Spend for Conversation ===\\n\")\nprint(f\"Accumulated Cost: ${spend.accumulated_cost:.6f}\")\nif spend.accumulated_token_usage:\n print(f\"Prompt Tokens: {spend.accumulated_token_usage.prompt_tokens}\")\n print(f\"Completion Tokens: {spend.accumulated_token_usage.completion_tokens}\")\n print(f\"Cache Read Tokens: {spend.accumulated_token_usage.cache_read_tokens}\")\n print(f\"Cache Write Tokens: {spend.accumulated_token_usage.cache_write_tokens}\")\n\nspend_per_usage = conversation.conversation_stats.usage_to_metrics\nprint(\"\\n=== Spend Breakdown by Usage ID ===\\n\")\nrows = []\nfor usage_id, metrics in spend_per_usage.items():\n rows.append(\n [\n usage_id,\n f\"${metrics.accumulated_cost:.6f}\",\n metrics.accumulated_token_usage.prompt_tokens\n if metrics.accumulated_token_usage\n else 0,\n metrics.accumulated_token_usage.completion_tokens\n if metrics.accumulated_token_usage\n else 0,\n ]\n )\n\nprint(\n tabulate(\n rows,\n headers=[\"Usage ID\", \"Cost\", \"Prompt Tokens\", \"Completion Tokens\"],\n tablefmt=\"github\",\n )\n)\n\n# Report cost\ncost = conversation.conversation_stats.get_combined_metrics().accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/22_anthropic_thinking.py", "content": "\"\"\"Example demonstrating Anthropic's extended thinking feature with thinking blocks.\"\"\"\n\nimport os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n RedactedThinkingBlock,\n ThinkingBlock,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Configure LLM for Anthropic Claude with extended thinking\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\n\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Setup agent with bash tool\nagent = Agent(llm=llm, tools=[Tool(name=TerminalTool.name)])\n\n\n# Callback to display thinking blocks\ndef show_thinking(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n message = event.to_llm_message()\n if hasattr(message, \"thinking_blocks\") and message.thinking_blocks:\n print(f\"\\n🧠 Found {len(message.thinking_blocks)} thinking blocks\")\n for i, block in enumerate(message.thinking_blocks):\n if isinstance(block, RedactedThinkingBlock):\n print(f\" Block {i + 1}: {block.data}\")\n elif isinstance(block, ThinkingBlock):\n print(f\" Block {i + 1}: {block.thinking}\")\n\n\nconversation = Conversation(\n agent=agent, callbacks=[show_thinking], workspace=os.getcwd()\n)\n\nconversation.send_message(\n \"Calculate compound interest for $10,000 at 5% annually, \"\n \"compounded quarterly for 3 years. Show your work.\",\n)\nconversation.run()\n\nconversation.send_message(\n \"Now, write that number to RESULTs.txt.\",\n)\nconversation.run()\nprint(\"✅ Done!\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/23_responses_reasoning.py", "content": "\"\"\"\nExample: Responses API path via LiteLLM in a Real Agent Conversation\n\n- Runs a real Agent/Conversation to verify /responses path works\n- Demonstrates rendering of Responses reasoning within normal conversation events\n\"\"\"\n\nfrom __future__ import annotations\n\nimport os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.llm import LLM\nfrom openhands.tools.preset.default import get_default_agent\n\n\nlogger = get_logger(__name__)\n\napi_key = os.getenv(\"LLM_API_KEY\") or os.getenv(\"OPENAI_API_KEY\")\nassert api_key, \"Set LLM_API_KEY or OPENAI_API_KEY in your environment.\"\n\nmodel = \"openhands/gpt-5-mini-2025-08-07\" # Use a model that supports Responses API\nbase_url = os.getenv(\"LLM_BASE_URL\")\n\nllm = LLM(\n model=model,\n api_key=SecretStr(api_key),\n base_url=base_url,\n # Responses-path options\n reasoning_effort=\"high\",\n # Logging / behavior tweaks\n log_completions=False,\n usage_id=\"agent\",\n)\n\nprint(\"\\n=== Agent Conversation using /responses path ===\")\nagent = get_default_agent(\n llm=llm,\n cli_mode=True, # disable browser tools for env simplicity\n)\n\nllm_messages = [] # collect raw LLM-convertible messages for inspection\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n workspace=os.getcwd(),\n)\n\n# Keep the tasks short for demo purposes\nconversation.send_message(\"Read the repo and write one fact into FACTS.txt.\")\nconversation.run()\n\nconversation.send_message(\"Now delete FACTS.txt.\")\nconversation.run()\n\nprint(\"=\" * 100)\nprint(\"Conversation finished. Got the following LLM messages:\")\nfor i, message in enumerate(llm_messages):\n ms = str(message)\n print(f\"Message {i}: {ms[:200]}{'...' if len(ms) > 200 else ''}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/24_planning_agent_workflow.py", "content": "#!/usr/bin/env python3\n\"\"\"\nPlanning Agent Workflow Example\n\nThis example demonstrates a two-stage workflow:\n1. Planning Agent: Analyzes the task and creates a detailed implementation plan\n2. Execution Agent: Implements the plan with full editing capabilities\n\nThe task: Create a Python web scraper that extracts article titles and URLs\nfrom a news website, handles rate limiting, and saves results to JSON.\n\"\"\"\n\nimport os\nimport tempfile\nfrom pathlib import Path\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, Conversation\nfrom openhands.sdk.llm import content_to_str\nfrom openhands.tools.preset.default import get_default_agent\nfrom openhands.tools.preset.planning import get_planning_agent\n\n\ndef get_event_content(event):\n \"\"\"Extract content from an event.\"\"\"\n if hasattr(event, \"llm_message\"):\n return \"\".join(content_to_str(event.llm_message.content))\n return str(event)\n\n\n\"\"\"Run the planning agent workflow example.\"\"\"\n\n# Create a temporary workspace\nworkspace_dir = Path(tempfile.mkdtemp())\nprint(f\"Working in: {workspace_dir}\")\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n usage_id=\"agent\",\n)\n\n# Task description\ntask = \"\"\"\nCreate a Python web scraper with the following requirements:\n- Scrape article titles and URLs from a news website\n- Handle HTTP errors gracefully with retry logic\n- Save results to a JSON file with timestamp\n- Use requests and BeautifulSoup for scraping\n\nDo NOT ask for any clarifying questions. Directly create your implementation plan.\n\"\"\"\n\nprint(\"=\" * 80)\nprint(\"PHASE 1: PLANNING\")\nprint(\"=\" * 80)\n\n# Create Planning Agent with read-only tools\nplanning_agent = get_planning_agent(llm=llm)\n\n# Create conversation for planning\nplanning_conversation = Conversation(\n agent=planning_agent,\n workspace=str(workspace_dir),\n)\n\n# Run planning phase\nprint(\"Planning Agent is analyzing the task and creating implementation plan...\")\nplanning_conversation.send_message(\n f\"Please analyze this web scraping task and create a detailed \"\n f\"implementation plan:\\n\\n{task}\"\n)\nplanning_conversation.run()\n\nprint(\"\\n\" + \"=\" * 80)\nprint(\"PLANNING COMPLETE\")\nprint(\"=\" * 80)\nprint(f\"Implementation plan saved to: {workspace_dir}/PLAN.md\")\n\nprint(\"\\n\" + \"=\" * 80)\nprint(\"PHASE 2: EXECUTION\")\nprint(\"=\" * 80)\n\n# Create Execution Agent with full editing capabilities\nexecution_agent = get_default_agent(llm=llm, cli_mode=True)\n\n# Create conversation for execution\nexecution_conversation = Conversation(\n agent=execution_agent,\n workspace=str(workspace_dir),\n)\n\n# Prepare execution prompt with reference to the plan file\nexecution_prompt = f\"\"\"\nPlease implement the web scraping project according to the implementation plan.\n\nThe detailed implementation plan has been created and saved at: {workspace_dir}/PLAN.md\n\nPlease read the plan from PLAN.md and implement all components according to it.\n\nCreate all necessary files, implement the functionality, and ensure everything\nworks together properly.\n\"\"\"\n\nprint(\"Execution Agent is implementing the plan...\")\nexecution_conversation.send_message(execution_prompt)\nexecution_conversation.run()\n\n# Get the last message from the conversation\nexecution_result = execution_conversation.state.events[-1]\n\nprint(\"\\n\" + \"=\" * 80)\nprint(\"EXECUTION RESULT:\")\nprint(\"=\" * 80)\nprint(get_event_content(execution_result))\n\nprint(\"\\n\" + \"=\" * 80)\nprint(\"WORKFLOW COMPLETE\")\nprint(\"=\" * 80)\nprint(f\"Project files created in: {workspace_dir}\")\n\n# List created files\nprint(\"\\nCreated files:\")\nfor file_path in workspace_dir.rglob(\"*\"):\n if file_path.is_file():\n print(f\" - {file_path.relative_to(workspace_dir)}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/25_agent_delegation.py", "content": "\"\"\"\nAgent Delegation Example\n\nThis example demonstrates the agent delegation feature where a main agent\ndelegates tasks to sub-agents for parallel processing.\nEach sub-agent runs independently and returns its results to the main agent,\nwhich then merges both analyses into a single consolidated report.\n\"\"\"\n\nimport os\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n AgentContext,\n Conversation,\n Tool,\n get_logger,\n)\nfrom openhands.sdk.context import Skill\nfrom openhands.sdk.subagent import register_agent\nfrom openhands.sdk.tool import register_tool\nfrom openhands.tools import register_builtins_agents\nfrom openhands.tools.delegate import (\n DelegateTool,\n DelegationVisualizer,\n)\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM and agent\nllm = LLM(\n model=os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\"),\n api_key=os.getenv(\"LLM_API_KEY\"),\n base_url=os.environ.get(\"LLM_BASE_URL\", None),\n usage_id=\"agent\",\n)\n\n\ndef create_lodging_planner(llm: LLM) -> Agent:\n \"\"\"Create a lodging planner focused on London stays.\"\"\"\n skills = [\n Skill(\n name=\"lodging_planning\",\n content=(\n \"You specialize in finding great places to stay in London. \"\n \"Provide 3-4 hotel recommendations with neighborhoods, quick \"\n \"pros/cons, \"\n \"and notes on transit convenience. Keep options varied by budget.\"\n ),\n trigger=None,\n )\n ]\n return Agent(\n llm=llm,\n tools=[],\n agent_context=AgentContext(\n skills=skills,\n system_message_suffix=\"Focus only on London lodging recommendations.\",\n ),\n )\n\n\ndef create_activities_planner(llm: LLM) -> Agent:\n \"\"\"Create an activities planner focused on London itineraries.\"\"\"\n skills = [\n Skill(\n name=\"activities_planning\",\n content=(\n \"You design concise London itineraries. Suggest 2-3 daily \"\n \"highlights, grouped by proximity to minimize travel time. \"\n \"Include food/coffee stops \"\n \"and note required tickets/reservations.\"\n ),\n trigger=None,\n )\n ]\n return Agent(\n llm=llm,\n tools=[],\n agent_context=AgentContext(\n skills=skills,\n system_message_suffix=\"Plan practical, time-efficient days in London.\",\n ),\n )\n\n\n# Register user-defined agent types (default agent type is always available)\nregister_agent(\n name=\"lodging_planner\",\n factory_func=create_lodging_planner,\n description=\"Finds London lodging options with transit-friendly picks.\",\n)\nregister_agent(\n name=\"activities_planner\",\n factory_func=create_activities_planner,\n description=\"Creates time-efficient London activity itineraries.\",\n)\nregister_builtins_agents()\n\n# Make the delegation tool available to the main agent\nregister_tool(\"DelegateTool\", DelegateTool)\n\nmain_agent = Agent(\n llm=llm,\n tools=[Tool(name=\"DelegateTool\")],\n)\nconversation = Conversation(\n agent=main_agent,\n workspace=os.getcwd(),\n visualizer=DelegationVisualizer(name=\"Delegator\"),\n)\n\nprint(\"=\" * 100)\nprint(\"Demonstrating London trip delegation (lodging + activities)...\")\nprint(\"=\" * 100)\n\nconversation.send_message(\"\"\"\nLet's plan a trip to London. I have two specific areas to address:\n\nLodging: What are the best areas to stay in while keeping a budget in mind?\nActivities: What are the top five must-see attractions and hidden gems?\n\nPlease use delegation tools to handle these two tasks in parallel.\nEnsure the sub-agents use their own internal knowledge and do not\nrely on internet access. Keep the responses concise.\nOnce you have the results, use the bash sub-agent to write a file\nnamed london_trip_report.txt containing the findings in the working directory.\n\"\"\")\nconversation.run()\n\nconversation.send_message(\n \"Ask the lodging sub-agent what it thinks about Covent Garden.\"\n)\nconversation.run()\n\n# Report cost for user-defined agent types example\ncost_user_defined = (\n conversation.conversation_stats.get_combined_metrics().accumulated_cost\n)\nprint(f\"EXAMPLE_COST: {cost_user_defined}\")\n\nprint(\"All done!\")\n" }, { "path": "examples/01_standalone_sdk/26_custom_visualizer.py", "content": "\"\"\"Custom Visualizer Example\n\nThis example demonstrates how to create and use a custom visualizer by subclassing\nConversationVisualizer. This approach provides:\n- Clean, testable code with class-based state management\n- Direct configuration (just pass the visualizer instance to visualizer parameter)\n- Reusable visualizer that can be shared across conversations\n\nThis demonstrates how you can pass a ConversationVisualizer instance directly\nto the visualizer parameter for clean, reusable visualization logic.\n\"\"\"\n\nimport logging\nimport os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, Conversation\nfrom openhands.sdk.conversation.visualizer import ConversationVisualizerBase\nfrom openhands.sdk.event import (\n Event,\n)\nfrom openhands.tools.preset.default import get_default_agent\n\n\nclass MinimalVisualizer(ConversationVisualizerBase):\n \"\"\"A minimal visualizer that print the raw events as they occur.\"\"\"\n\n def on_event(self, event: Event) -> None:\n \"\"\"Handle events for minimal progress visualization.\"\"\"\n print(f\"\\n\\n[EVENT] {type(event).__name__}: {event.model_dump_json()[:200]}...\")\n\n\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n model=model,\n api_key=SecretStr(api_key),\n base_url=base_url,\n usage_id=\"agent\",\n)\nagent = get_default_agent(llm=llm, cli_mode=True)\n\n# ============================================================================\n# Configure Visualization\n# ============================================================================\n# Set logging level to reduce verbosity\nlogging.getLogger().setLevel(logging.WARNING)\n\n# Start a conversation with custom visualizer\ncwd = os.getcwd()\nconversation = Conversation(\n agent=agent,\n workspace=cwd,\n visualizer=MinimalVisualizer(),\n)\n\n# Send a message and let the agent run\nprint(\"Sending task to agent...\")\nconversation.send_message(\"Write 3 facts about the current project into FACTS.txt.\")\nconversation.run()\nprint(\"Task completed!\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost:.4f}\")\n" }, { "path": "examples/01_standalone_sdk/27_observability_laminar.py", "content": "\"\"\"\nObservability & Laminar example\n\nThis example demonstrates enabling OpenTelemetry tracing with Laminar in the\nOpenHands SDK. Set LMNR_PROJECT_API_KEY and run the script to see traces.\n\"\"\"\n\nimport os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, Agent, Conversation, Tool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Tip: Set LMNR_PROJECT_API_KEY in your environment before running, e.g.:\n# export LMNR_PROJECT_API_KEY=\"your-laminar-api-key\"\n# For non-Laminar OTLP backends, set OTEL_* variables instead.\n\n# Configure LLM and Agent\napi_key = os.getenv(\"LLM_API_KEY\")\nmodel = os.getenv(\"LLM_MODEL\", \"openhands/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n model=model,\n api_key=SecretStr(api_key) if api_key else None,\n base_url=base_url,\n usage_id=\"agent\",\n)\n\nagent = Agent(\n llm=llm,\n tools=[Tool(name=TerminalTool.name)],\n)\n\n# Create conversation and run a simple task\nconversation = Conversation(agent=agent, workspace=\".\")\nconversation.send_message(\"List the files in the current directory and print them.\")\nconversation.run()\nprint(\n \"All done! Check your Laminar dashboard for traces \"\n \"(session is the conversation UUID).\"\n)\n" }, { "path": "examples/01_standalone_sdk/28_ask_agent_example.py", "content": "\"\"\"\nExample demonstrating the ask_agent functionality for getting sidebar replies\nfrom the agent for a running conversation.\n\nThis example shows how to use ask_agent() to get quick responses from the agent\nabout the current conversation state without interrupting the main execution flow.\n\"\"\"\n\nimport os\nimport threading\nimport time\nfrom datetime import datetime\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n)\nfrom openhands.sdk.conversation import ConversationVisualizerBase\nfrom openhands.sdk.event import Event\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.task_tracker import TaskTrackerTool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n Tool(name=TaskTrackerTool.name),\n]\n\n\nclass MinimalVisualizer(ConversationVisualizerBase):\n \"\"\"A minimal visualizer that print the raw events as they occur.\"\"\"\n\n count = 0\n\n def on_event(self, event: Event) -> None:\n \"\"\"Handle events for minimal progress visualization.\"\"\"\n print(f\"\\n\\n[EVENT {self.count}] {type(event).__name__}\")\n self.count += 1\n\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\nconversation = Conversation(\n agent=agent, workspace=cwd, visualizer=MinimalVisualizer, max_iteration_per_run=5\n)\n\n\ndef timestamp() -> str:\n return datetime.now().strftime(\"%H:%M:%S\")\n\n\nprint(\"=== Ask Agent Example ===\")\nprint(\"This example demonstrates asking questions during conversation execution\")\n\n# Step 1: Build conversation context\nprint(f\"\\n[{timestamp()}] Building conversation context...\")\nconversation.send_message(\"Explore the current directory and describe the architecture\")\n\n# Step 2: Start conversation in background thread\nprint(f\"[{timestamp()}] Starting conversation in background thread...\")\nthread = threading.Thread(target=conversation.run)\nthread.start()\n\n# Give the agent time to start processing\ntime.sleep(2)\n\n# Step 3: Use ask_agent while conversation is running\nprint(f\"\\n[{timestamp()}] Using ask_agent while conversation is processing...\")\n\n# Ask context-aware questions\nquestions_and_responses = []\n\nquestion_1 = \"Summarize the activity so far in 1 sentence.\"\nprint(f\"\\n[{timestamp()}] Asking: {question_1}\")\nresponse1 = conversation.ask_agent(question_1)\nquestions_and_responses.append((question_1, response1))\nprint(f\"Response: {response1}\")\n\ntime.sleep(1)\n\nquestion_2 = \"How's the progress?\"\nprint(f\"\\n[{timestamp()}] Asking: {question_2}\")\nresponse2 = conversation.ask_agent(question_2)\nquestions_and_responses.append((question_2, response2))\nprint(f\"Response: {response2}\")\n\ntime.sleep(1)\n\nquestion_3 = \"Have you finished running?\"\nprint(f\"\\n[{timestamp()}] {question_3}\")\nresponse3 = conversation.ask_agent(question_3)\nquestions_and_responses.append((question_3, response3))\nprint(f\"Response: {response3}\")\n\n# Step 4: Wait for conversation to complete\nprint(f\"\\n[{timestamp()}] Waiting for conversation to complete...\")\nthread.join()\n\n# Step 5: Verify conversation state wasn't affected\nfinal_event_count = len(conversation.state.events)\n# Step 6: Ask a final question after conversation completion\nprint(f\"\\n[{timestamp()}] Asking final question after completion...\")\nfinal_response = conversation.ask_agent(\n \"Can you summarize what you accomplished in this conversation?\"\n)\nprint(f\"Final response: {final_response}\")\n\n# Step 7: Summary\nprint(\"\\n\" + \"=\" * 60)\nprint(\"SUMMARY OF ASK_AGENT DEMONSTRATION\")\nprint(\"=\" * 60)\n\nprint(\"\\nQuestions and Responses:\")\nfor i, (question, response) in enumerate(questions_and_responses, 1):\n print(f\"\\n{i}. Q: {question}\")\n print(f\" A: {response[:100]}{'...' if len(response) > 100 else ''}\")\n\nfinal_truncated = final_response[:100] + (\"...\" if len(final_response) > 100 else \"\")\nprint(f\"\\nFinal Question Response: {final_truncated}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost:.4f}\")\n" }, { "path": "examples/01_standalone_sdk/29_llm_streaming.py", "content": "import os\nimport sys\nfrom typing import Literal\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n Conversation,\n get_logger,\n)\nfrom openhands.sdk.llm import LLM\nfrom openhands.sdk.llm.streaming import ModelResponseStream\nfrom openhands.tools.preset.default import get_default_agent\n\n\nlogger = get_logger(__name__)\n\n\napi_key = os.getenv(\"LLM_API_KEY\") or os.getenv(\"OPENAI_API_KEY\")\nif not api_key:\n raise RuntimeError(\"Set LLM_API_KEY or OPENAI_API_KEY in your environment.\")\n\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n model=model,\n api_key=SecretStr(api_key),\n base_url=base_url,\n usage_id=\"stream-demo\",\n stream=True,\n)\n\nagent = get_default_agent(llm=llm, cli_mode=True)\n\n\n# Define streaming states\nStreamingState = Literal[\"thinking\", \"content\", \"tool_name\", \"tool_args\"]\n# Track state across on_token calls for boundary detection\n_current_state: StreamingState | None = None\n\n\ndef on_token(chunk: ModelResponseStream) -> None:\n \"\"\"\n Handle all types of streaming tokens including content,\n tool calls, and thinking blocks with dynamic boundary detection.\n \"\"\"\n global _current_state\n\n choices = chunk.choices\n for choice in choices:\n delta = choice.delta\n if delta is not None:\n # Handle thinking blocks (reasoning content)\n reasoning_content = getattr(delta, \"reasoning_content\", None)\n if isinstance(reasoning_content, str) and reasoning_content:\n if _current_state != \"thinking\":\n if _current_state is not None:\n sys.stdout.write(\"\\n\")\n sys.stdout.write(\"THINKING: \")\n _current_state = \"thinking\"\n sys.stdout.write(reasoning_content)\n sys.stdout.flush()\n\n # Handle regular content\n content = getattr(delta, \"content\", None)\n if isinstance(content, str) and content:\n if _current_state != \"content\":\n if _current_state is not None:\n sys.stdout.write(\"\\n\")\n sys.stdout.write(\"CONTENT: \")\n _current_state = \"content\"\n sys.stdout.write(content)\n sys.stdout.flush()\n\n # Handle tool calls\n tool_calls = getattr(delta, \"tool_calls\", None)\n if tool_calls:\n for tool_call in tool_calls:\n tool_name = (\n tool_call.function.name if tool_call.function.name else \"\"\n )\n tool_args = (\n tool_call.function.arguments\n if tool_call.function.arguments\n else \"\"\n )\n if tool_name:\n if _current_state != \"tool_name\":\n if _current_state is not None:\n sys.stdout.write(\"\\n\")\n sys.stdout.write(\"TOOL NAME: \")\n _current_state = \"tool_name\"\n sys.stdout.write(tool_name)\n sys.stdout.flush()\n if tool_args:\n if _current_state != \"tool_args\":\n if _current_state is not None:\n sys.stdout.write(\"\\n\")\n sys.stdout.write(\"TOOL ARGS: \")\n _current_state = \"tool_args\"\n sys.stdout.write(tool_args)\n sys.stdout.flush()\n\n\nconversation = Conversation(\n agent=agent,\n workspace=os.getcwd(),\n token_callbacks=[on_token],\n)\n\nstory_prompt = (\n \"Tell me a long story about LLM streaming, write it a file, \"\n \"make sure it has multiple paragraphs. \"\n)\nconversation.send_message(story_prompt)\nprint(\"Token Streaming:\")\nprint(\"-\" * 100 + \"\\n\")\nconversation.run()\n\ncleanup_prompt = (\n \"Thank you. Please delete the streaming story file now that I've read it, \"\n \"then confirm the deletion.\"\n)\nconversation.send_message(cleanup_prompt)\nprint(\"Token Streaming:\")\nprint(\"-\" * 100 + \"\\n\")\nconversation.run()\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/30_tom_agent.py", "content": "\"\"\"Example demonstrating Tom agent with Theory of Mind capabilities.\n\nThis example shows how to set up an agent with Tom tools for getting\npersonalized guidance based on user modeling. Tom tools include:\n- TomConsultTool: Get guidance for vague or unclear tasks\n- SleeptimeComputeTool: Index conversations for user modeling\n\"\"\"\n\nimport os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, Agent, Conversation\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.preset.default import get_default_tools\nfrom openhands.tools.tom_consult import (\n SleeptimeComputeAction,\n SleeptimeComputeObservation,\n SleeptimeComputeTool,\n TomConsultTool,\n)\n\n\n# Configure LLM\napi_key: str | None = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\n\nllm: LLM = LLM(\n model=os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\"),\n api_key=os.getenv(\"LLM_API_KEY\"),\n base_url=os.getenv(\"LLM_BASE_URL\", None),\n usage_id=\"agent\",\n drop_params=True,\n)\n\n# Build tools list with Tom tools\n# Note: Tom tools are automatically registered on import (PR #862)\ntools = get_default_tools(enable_browser=False)\n\n# Configure Tom tools with parameters\ntom_params: dict[str, bool | str] = {\n \"enable_rag\": True, # Enable RAG in Tom agent\n}\n\n# Add LLM configuration for Tom tools (uses same LLM as main agent)\ntom_params[\"llm_model\"] = llm.model\nif llm.api_key:\n if isinstance(llm.api_key, SecretStr):\n tom_params[\"api_key\"] = llm.api_key.get_secret_value()\n else:\n tom_params[\"api_key\"] = llm.api_key\nif llm.base_url:\n tom_params[\"api_base\"] = llm.base_url\n\n# Add both Tom tools to the agent\ntools.append(Tool(name=TomConsultTool.name, params=tom_params))\ntools.append(Tool(name=SleeptimeComputeTool.name, params=tom_params))\n\n# Create agent with Tom capabilities\n# This agent can consult Tom for personalized guidance\n# Note: Tom's user modeling data will be stored in ~/.openhands/\nagent: Agent = Agent(llm=llm, tools=tools)\n\n# Start conversation\ncwd: str = os.getcwd()\nPERSISTENCE_DIR = os.path.expanduser(\"~/.openhands\")\nCONVERSATIONS_DIR = os.path.join(PERSISTENCE_DIR, \"conversations\")\nconversation = Conversation(\n agent=agent, workspace=cwd, persistence_dir=CONVERSATIONS_DIR\n)\n\n# Optionally run sleeptime compute to index existing conversations\n# This builds user preferences and patterns from conversation history\n# Using execute_tool allows running tools before conversation.run()\nprint(\"\\nRunning sleeptime compute to index conversations...\")\ntry:\n sleeptime_result = conversation.execute_tool(\n \"sleeptime_compute\", SleeptimeComputeAction()\n )\n # Cast to the expected observation type for type-safe access\n if isinstance(sleeptime_result, SleeptimeComputeObservation):\n print(f\"Result: {sleeptime_result.message}\")\n print(f\"Sessions processed: {sleeptime_result.sessions_processed}\")\n else:\n print(f\"Result: {sleeptime_result.text}\")\nexcept KeyError as e:\n print(f\"Tool not available: {e}\")\n\n# Send a potentially vague message where Tom consultation might help\nconversation.send_message(\n \"I need to debug some code but I'm not sure where to start. \"\n + \"Can you help me figure out the best approach?\"\n)\nconversation.run()\n\nprint(\"\\n\" + \"=\" * 80)\nprint(\"Tom agent consultation example completed!\")\nprint(\"=\" * 80)\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n\n\n# Optional: Index this conversation for Tom's user modeling\n# This builds user preferences and patterns from conversation history\n# Uncomment the lines below to index the conversation:\n#\n# conversation.send_message(\"Please index this conversation using sleeptime_compute\")\n# conversation.run()\n# print(\"\\nConversation indexed for user modeling!\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/31_iterative_refinement.py", "content": "#!/usr/bin/env python3\n\"\"\"\nIterative Refinement Example: COBOL to Java Refactoring\n\nThis example demonstrates an iterative refinement workflow where:\n1. A refactoring agent converts COBOL files to Java files\n2. A critique agent evaluates the quality of each conversion and provides scores\n3. If the average score is below 90%, the process repeats with feedback\n\nThe workflow continues until the refactoring meets the quality threshold.\n\nSource COBOL files can be obtained from:\nhttps://github.com/aws-samples/aws-mainframe-modernization-carddemo/tree/main/app/cbl\n\"\"\"\n\nimport os\nimport re\nimport tempfile\nfrom pathlib import Path\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, Conversation\nfrom openhands.tools.preset.default import get_default_agent\n\n\nQUALITY_THRESHOLD = float(os.getenv(\"QUALITY_THRESHOLD\", \"90.0\"))\nMAX_ITERATIONS = int(os.getenv(\"MAX_ITERATIONS\", \"5\"))\n\n\ndef setup_workspace() -> tuple[Path, Path, Path]:\n \"\"\"Create workspace directories for the refactoring workflow.\"\"\"\n workspace_dir = Path(tempfile.mkdtemp())\n cobol_dir = workspace_dir / \"cobol\"\n java_dir = workspace_dir / \"java\"\n critique_dir = workspace_dir / \"critiques\"\n\n cobol_dir.mkdir(parents=True, exist_ok=True)\n java_dir.mkdir(parents=True, exist_ok=True)\n critique_dir.mkdir(parents=True, exist_ok=True)\n\n return workspace_dir, cobol_dir, java_dir\n\n\ndef create_sample_cobol_files(cobol_dir: Path) -> list[str]:\n \"\"\"Create sample COBOL files for demonstration.\n\n In a real scenario, you would clone files from:\n https://github.com/aws-samples/aws-mainframe-modernization-carddemo/tree/main/app/cbl\n \"\"\"\n sample_files = {\n \"CBACT01C.cbl\": \"\"\" IDENTIFICATION DIVISION.\n PROGRAM-ID. CBACT01C.\n *****************************************************************\n * Program: CBACT01C - Account Display Program\n * Purpose: Display account information for a given account number\n *****************************************************************\n ENVIRONMENT DIVISION.\n DATA DIVISION.\n WORKING-STORAGE SECTION.\n 01 WS-ACCOUNT-ID PIC 9(11).\n 01 WS-ACCOUNT-STATUS PIC X(1).\n 01 WS-ACCOUNT-BALANCE PIC S9(13)V99.\n 01 WS-CUSTOMER-NAME PIC X(50).\n 01 WS-ERROR-MSG PIC X(80).\n\n PROCEDURE DIVISION.\n PERFORM 1000-INIT.\n PERFORM 2000-PROCESS.\n PERFORM 3000-TERMINATE.\n STOP RUN.\n\n 1000-INIT.\n INITIALIZE WS-ACCOUNT-ID\n INITIALIZE WS-ACCOUNT-STATUS\n INITIALIZE WS-ACCOUNT-BALANCE\n INITIALIZE WS-CUSTOMER-NAME.\n\n 2000-PROCESS.\n DISPLAY \"ENTER ACCOUNT NUMBER: \"\n ACCEPT WS-ACCOUNT-ID\n IF WS-ACCOUNT-ID = ZEROS\n MOVE \"INVALID ACCOUNT NUMBER\" TO WS-ERROR-MSG\n DISPLAY WS-ERROR-MSG\n ELSE\n DISPLAY \"ACCOUNT: \" WS-ACCOUNT-ID\n DISPLAY \"STATUS: \" WS-ACCOUNT-STATUS\n DISPLAY \"BALANCE: \" WS-ACCOUNT-BALANCE\n END-IF.\n\n 3000-TERMINATE.\n DISPLAY \"PROGRAM COMPLETE\".\n\"\"\",\n \"CBCUS01C.cbl\": \"\"\" IDENTIFICATION DIVISION.\n PROGRAM-ID. CBCUS01C.\n *****************************************************************\n * Program: CBCUS01C - Customer Information Program\n * Purpose: Manage customer data operations\n *****************************************************************\n ENVIRONMENT DIVISION.\n DATA DIVISION.\n WORKING-STORAGE SECTION.\n 01 WS-CUSTOMER-ID PIC 9(9).\n 01 WS-FIRST-NAME PIC X(25).\n 01 WS-LAST-NAME PIC X(25).\n 01 WS-ADDRESS PIC X(100).\n 01 WS-PHONE PIC X(15).\n 01 WS-EMAIL PIC X(50).\n 01 WS-OPERATION PIC X(1).\n 88 OP-ADD VALUE 'A'.\n 88 OP-UPDATE VALUE 'U'.\n 88 OP-DELETE VALUE 'D'.\n 88 OP-DISPLAY VALUE 'V'.\n\n PROCEDURE DIVISION.\n PERFORM 1000-MAIN-PROCESS.\n STOP RUN.\n\n 1000-MAIN-PROCESS.\n DISPLAY \"CUSTOMER MANAGEMENT SYSTEM\"\n DISPLAY \"A-ADD U-UPDATE D-DELETE V-VIEW\"\n ACCEPT WS-OPERATION\n EVALUATE TRUE\n WHEN OP-ADD\n PERFORM 2000-ADD-CUSTOMER\n WHEN OP-UPDATE\n PERFORM 3000-UPDATE-CUSTOMER\n WHEN OP-DELETE\n PERFORM 4000-DELETE-CUSTOMER\n WHEN OP-DISPLAY\n PERFORM 5000-DISPLAY-CUSTOMER\n WHEN OTHER\n DISPLAY \"INVALID OPERATION\"\n END-EVALUATE.\n\n 2000-ADD-CUSTOMER.\n DISPLAY \"ADDING NEW CUSTOMER\"\n ACCEPT WS-CUSTOMER-ID\n ACCEPT WS-FIRST-NAME\n ACCEPT WS-LAST-NAME\n DISPLAY \"CUSTOMER ADDED: \" WS-CUSTOMER-ID.\n\n 3000-UPDATE-CUSTOMER.\n DISPLAY \"UPDATING CUSTOMER\"\n ACCEPT WS-CUSTOMER-ID\n DISPLAY \"CUSTOMER UPDATED: \" WS-CUSTOMER-ID.\n\n 4000-DELETE-CUSTOMER.\n DISPLAY \"DELETING CUSTOMER\"\n ACCEPT WS-CUSTOMER-ID\n DISPLAY \"CUSTOMER DELETED: \" WS-CUSTOMER-ID.\n\n 5000-DISPLAY-CUSTOMER.\n DISPLAY \"DISPLAYING CUSTOMER\"\n ACCEPT WS-CUSTOMER-ID\n DISPLAY \"ID: \" WS-CUSTOMER-ID\n DISPLAY \"NAME: \" WS-FIRST-NAME \" \" WS-LAST-NAME.\n\"\"\",\n \"CBTRN01C.cbl\": \"\"\" IDENTIFICATION DIVISION.\n PROGRAM-ID. CBTRN01C.\n *****************************************************************\n * Program: CBTRN01C - Transaction Processing Program\n * Purpose: Process financial transactions\n *****************************************************************\n ENVIRONMENT DIVISION.\n DATA DIVISION.\n WORKING-STORAGE SECTION.\n 01 WS-TRANS-ID PIC 9(16).\n 01 WS-TRANS-TYPE PIC X(2).\n 88 TRANS-CREDIT VALUE 'CR'.\n 88 TRANS-DEBIT VALUE 'DB'.\n 88 TRANS-TRANSFER VALUE 'TR'.\n 01 WS-TRANS-AMOUNT PIC S9(13)V99.\n 01 WS-FROM-ACCOUNT PIC 9(11).\n 01 WS-TO-ACCOUNT PIC 9(11).\n 01 WS-TRANS-DATE PIC 9(8).\n 01 WS-TRANS-STATUS PIC X(10).\n\n PROCEDURE DIVISION.\n PERFORM 1000-INITIALIZE.\n PERFORM 2000-PROCESS-TRANSACTION.\n PERFORM 3000-FINALIZE.\n STOP RUN.\n\n 1000-INITIALIZE.\n MOVE ZEROS TO WS-TRANS-ID\n MOVE SPACES TO WS-TRANS-TYPE\n MOVE ZEROS TO WS-TRANS-AMOUNT\n MOVE \"PENDING\" TO WS-TRANS-STATUS.\n\n 2000-PROCESS-TRANSACTION.\n DISPLAY \"ENTER TRANSACTION TYPE (CR/DB/TR): \"\n ACCEPT WS-TRANS-TYPE\n DISPLAY \"ENTER AMOUNT: \"\n ACCEPT WS-TRANS-AMOUNT\n EVALUATE TRUE\n WHEN TRANS-CREDIT\n PERFORM 2100-PROCESS-CREDIT\n WHEN TRANS-DEBIT\n PERFORM 2200-PROCESS-DEBIT\n WHEN TRANS-TRANSFER\n PERFORM 2300-PROCESS-TRANSFER\n WHEN OTHER\n MOVE \"INVALID\" TO WS-TRANS-STATUS\n END-EVALUATE.\n\n 2100-PROCESS-CREDIT.\n DISPLAY \"PROCESSING CREDIT\"\n ACCEPT WS-TO-ACCOUNT\n MOVE \"COMPLETED\" TO WS-TRANS-STATUS\n DISPLAY \"CREDIT APPLIED TO: \" WS-TO-ACCOUNT.\n\n 2200-PROCESS-DEBIT.\n DISPLAY \"PROCESSING DEBIT\"\n ACCEPT WS-FROM-ACCOUNT\n MOVE \"COMPLETED\" TO WS-TRANS-STATUS\n DISPLAY \"DEBIT FROM: \" WS-FROM-ACCOUNT.\n\n 2300-PROCESS-TRANSFER.\n DISPLAY \"PROCESSING TRANSFER\"\n ACCEPT WS-FROM-ACCOUNT\n ACCEPT WS-TO-ACCOUNT\n MOVE \"COMPLETED\" TO WS-TRANS-STATUS\n DISPLAY \"TRANSFER FROM \" WS-FROM-ACCOUNT \" TO \" WS-TO-ACCOUNT.\n\n 3000-FINALIZE.\n DISPLAY \"TRANSACTION STATUS: \" WS-TRANS-STATUS.\n\"\"\",\n }\n\n created_files = []\n for filename, content in sample_files.items():\n file_path = cobol_dir / filename\n file_path.write_text(content)\n created_files.append(filename)\n\n return created_files\n\n\ndef get_refactoring_prompt(\n cobol_dir: Path,\n java_dir: Path,\n cobol_files: list[str],\n critique_file: Path | None = None,\n) -> str:\n \"\"\"Generate the prompt for the refactoring agent.\"\"\"\n files_list = \"\\n\".join(f\" - {f}\" for f in cobol_files)\n\n base_prompt = f\"\"\"Convert the following COBOL files to Java:\n\nCOBOL Source Directory: {cobol_dir}\nJava Target Directory: {java_dir}\n\nFiles to convert:\n{files_list}\n\nRequirements:\n1. Create a Java class for each COBOL program\n2. Preserve the business logic and data structures\n3. Use appropriate Java naming conventions (camelCase for methods, PascalCase)\n4. Convert COBOL data types to appropriate Java types\n5. Implement proper error handling with try-catch blocks\n6. Add JavaDoc comments explaining the purpose of each class and method\n7. In JavaDoc comments, include traceability to the original COBOL source using\n the format: @source : (e.g., @source CBACT01C.cbl:73-77)\n8. Create a clean, maintainable object-oriented design\n9. Each Java file should be compilable and follow Java best practices\n\nRead each COBOL file and create the corresponding Java file in the target directory.\n\"\"\"\n\n if critique_file and critique_file.exists():\n base_prompt += f\"\"\"\n\nIMPORTANT: A previous refactoring attempt was evaluated and needs improvement.\nPlease review the critique at: {critique_file}\nAddress all issues mentioned in the critique to improve the conversion quality.\n\"\"\"\n\n return base_prompt\n\n\ndef get_critique_prompt(\n cobol_dir: Path,\n java_dir: Path,\n cobol_files: list[str],\n) -> str:\n \"\"\"Generate the prompt for the critique agent.\"\"\"\n files_list = \"\\n\".join(f\" - {f}\" for f in cobol_files)\n\n return f\"\"\"Evaluate the quality of COBOL to Java refactoring.\n\nCOBOL Source Directory: {cobol_dir}\nJava Target Directory: {java_dir}\n\nOriginal COBOL files:\n{files_list}\n\nPlease evaluate each converted Java file against its original COBOL source.\n\nFor each file, assess:\n1. Correctness: Does the Java code preserve the original business logic? (0-25 pts)\n2. Code Quality: Is the code clean, readable, following Java conventions? (0-25 pts)\n3. Completeness: Are all COBOL features properly converted? (0-25 pts)\n4. Best Practices: Does it use proper OOP, error handling, documentation? (0-25 pts)\n\nCreate a critique report in the following EXACT format:\n\n# COBOL to Java Refactoring Critique Report\n\n## Summary\n[Brief overall assessment]\n\n## File Evaluations\n\n### [Original COBOL filename]\n- **Java File**: [corresponding Java filename or \"NOT FOUND\"]\n- **Correctness**: [score]/25 - [brief explanation]\n- **Code Quality**: [score]/25 - [brief explanation]\n- **Completeness**: [score]/25 - [brief explanation]\n- **Best Practices**: [score]/25 - [brief explanation]\n- **File Score**: [total]/100\n- **Issues to Address**:\n - [specific issue 1]\n - [specific issue 2]\n ...\n\n[Repeat for each file]\n\n## Overall Score\n- **Average Score**: [calculated average of all file scores]\n- **Recommendation**: [PASS if average >= 90, NEEDS_IMPROVEMENT otherwise]\n\n## Priority Improvements\n1. [Most critical improvement needed]\n2. [Second priority]\n3. [Third priority]\n\nSave this report to: {java_dir.parent}/critiques/critique_report.md\n\"\"\"\n\n\ndef parse_critique_score(critique_file: Path) -> float:\n \"\"\"Parse the average score from the critique report.\"\"\"\n if not critique_file.exists():\n return 0.0\n\n content = critique_file.read_text()\n\n # Look for \"Average Score: X\" pattern\n patterns = [\n r\"\\*\\*Average Score\\*\\*:\\s*(\\d+(?:\\.\\d+)?)\",\n r\"Average Score:\\s*(\\d+(?:\\.\\d+)?)\",\n r\"average.*?(\\d+(?:\\.\\d+)?)\\s*(?:/100|%|$)\",\n ]\n\n for pattern in patterns:\n match = re.search(pattern, content, re.IGNORECASE)\n if match:\n return float(match.group(1))\n\n return 0.0\n\n\ndef run_iterative_refinement() -> None:\n \"\"\"Run the iterative refinement workflow.\"\"\"\n # Setup\n api_key = os.getenv(\"LLM_API_KEY\")\n assert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\n model = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\n base_url = os.getenv(\"LLM_BASE_URL\")\n\n llm = LLM(\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n usage_id=\"iterative_refinement\",\n )\n\n workspace_dir, cobol_dir, java_dir = setup_workspace()\n critique_dir = workspace_dir / \"critiques\"\n\n print(f\"Workspace: {workspace_dir}\")\n print(f\"COBOL Directory: {cobol_dir}\")\n print(f\"Java Directory: {java_dir}\")\n print(f\"Critique Directory: {critique_dir}\")\n print()\n\n # Create sample COBOL files\n cobol_files = create_sample_cobol_files(cobol_dir)\n print(f\"Created {len(cobol_files)} sample COBOL files:\")\n for f in cobol_files:\n print(f\" - {f}\")\n print()\n\n critique_file = critique_dir / \"critique_report.md\"\n current_score = 0.0\n iteration = 0\n\n while current_score < QUALITY_THRESHOLD and iteration < MAX_ITERATIONS:\n iteration += 1\n print(\"=\" * 80)\n print(f\"ITERATION {iteration}\")\n print(\"=\" * 80)\n\n # Phase 1: Refactoring\n print(\"\\n--- Phase 1: Refactoring Agent ---\")\n refactoring_agent = get_default_agent(llm=llm, cli_mode=True)\n refactoring_conversation = Conversation(\n agent=refactoring_agent,\n workspace=str(workspace_dir),\n )\n\n previous_critique = critique_file if iteration > 1 else None\n refactoring_prompt = get_refactoring_prompt(\n cobol_dir, java_dir, cobol_files, previous_critique\n )\n\n refactoring_conversation.send_message(refactoring_prompt)\n refactoring_conversation.run()\n print(\"Refactoring phase complete.\")\n\n # Phase 2: Critique\n print(\"\\n--- Phase 2: Critique Agent ---\")\n critique_agent = get_default_agent(llm=llm, cli_mode=True)\n critique_conversation = Conversation(\n agent=critique_agent,\n workspace=str(workspace_dir),\n )\n\n critique_prompt = get_critique_prompt(cobol_dir, java_dir, cobol_files)\n critique_conversation.send_message(critique_prompt)\n critique_conversation.run()\n print(\"Critique phase complete.\")\n\n # Parse the score\n current_score = parse_critique_score(critique_file)\n print(f\"\\nCurrent Score: {current_score:.1f}%\")\n\n if current_score >= QUALITY_THRESHOLD:\n print(f\"\\n✓ Quality threshold ({QUALITY_THRESHOLD}%) met!\")\n else:\n print(\n f\"\\n✗ Score below threshold ({QUALITY_THRESHOLD}%). \"\n \"Continuing refinement...\"\n )\n\n # Final summary\n print(\"\\n\" + \"=\" * 80)\n print(\"ITERATIVE REFINEMENT COMPLETE\")\n print(\"=\" * 80)\n print(f\"Total iterations: {iteration}\")\n print(f\"Final score: {current_score:.1f}%\")\n print(f\"Workspace: {workspace_dir}\")\n\n # List created Java files\n print(\"\\nCreated Java files:\")\n for java_file in java_dir.glob(\"*.java\"):\n print(f\" - {java_file.name}\")\n\n # Show critique file location\n if critique_file.exists():\n print(f\"\\nFinal critique report: {critique_file}\")\n\n # Report cost\n cost = llm.metrics.accumulated_cost\n print(f\"\\nEXAMPLE_COST: {cost}\")\n\n\nif __name__ == \"__main__\":\n run_iterative_refinement()\n" }, { "path": "examples/01_standalone_sdk/32_configurable_security_policy.py", "content": "\"\"\"OpenHands Agent SDK — Configurable Security Policy Example\n\nThis example demonstrates how to use a custom security policy template\nwith an agent. Security policies define risk assessment guidelines that\nhelp agents evaluate the safety of their actions.\n\nBy default, agents use the built-in security_policy.j2 template. This\nexample shows how to:\n1. Use the default security policy\n2. Provide a custom security policy template embedded in the script\n3. Apply the custom policy to guide agent behavior\n\"\"\"\n\nimport os\nimport tempfile\nfrom pathlib import Path\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nlogger = get_logger(__name__)\n\n# Define a custom security policy template inline\nCUSTOM_SECURITY_POLICY = (\n \"# 🔐 Custom Security Risk Policy\\n\"\n \"When using tools that support the security_risk parameter, assess the \"\n \"safety risk of your actions:\\n\"\n \"\\n\"\n \"- **LOW**: Safe read-only actions.\\n\"\n \" - Viewing files, calculations, documentation.\\n\"\n \"- **MEDIUM**: Moderate container-scoped actions.\\n\"\n \" - File modifications, package installations.\\n\"\n \"- **HIGH**: Potentially dangerous actions.\\n\"\n \" - Network access, system modifications, data exfiltration.\\n\"\n \"\\n\"\n \"**Custom Rules**\\n\"\n \"- Always prioritize user data safety.\\n\"\n \"- Escalate to **HIGH** for any external data transmission.\\n\"\n)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools\ncwd = os.getcwd()\ntools = [\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n]\n\n# Example 1: Agent with default security policy\nprint(\"=\" * 100)\nprint(\"Example 1: Agent with default security policy\")\nprint(\"=\" * 100)\ndefault_agent = Agent(llm=llm, tools=tools)\nprint(f\"Security policy filename: {default_agent.security_policy_filename}\")\nprint(\"\\nDefault security policy is embedded in the agent's system message.\")\n\n# Example 2: Agent with custom security policy\nprint(\"\\n\" + \"=\" * 100)\nprint(\"Example 2: Agent with custom security policy\")\nprint(\"=\" * 100)\n\n# Create a temporary file for the custom security policy\nwith tempfile.NamedTemporaryFile(\n mode=\"w\", suffix=\".j2\", delete=False, encoding=\"utf-8\"\n) as temp_file:\n temp_file.write(CUSTOM_SECURITY_POLICY)\n custom_policy_path = temp_file.name\n\ntry:\n # Create agent with custom security policy (using absolute path)\n custom_agent = Agent(\n llm=llm,\n tools=tools,\n security_policy_filename=custom_policy_path,\n )\n print(f\"Security policy filename: {custom_agent.security_policy_filename}\")\n print(\"\\nCustom security policy loaded from temporary file.\")\n\n # Verify the custom policy is in the system message\n system_message = custom_agent.static_system_message\n if \"Custom Security Risk Policy\" in system_message:\n print(\"✓ Custom security policy successfully embedded in system message.\")\n else:\n print(\"✗ Custom security policy not found in system message.\")\n\n # Run a conversation with the custom agent\n print(\"\\n\" + \"=\" * 100)\n print(\"Running conversation with custom security policy\")\n print(\"=\" * 100)\n\n llm_messages = [] # collect raw LLM messages\n\n def conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n conversation = Conversation(\n agent=custom_agent,\n callbacks=[conversation_callback],\n workspace=\".\",\n )\n\n conversation.send_message(\n \"Please create a simple Python script named hello.py that prints \"\n \"'Hello, World!'. Make sure to follow security best practices.\"\n )\n conversation.run()\n\n print(\"\\n\" + \"=\" * 100)\n print(\"Conversation finished.\")\n print(f\"Total LLM messages: {len(llm_messages)}\")\n print(\"=\" * 100)\n\n # Report cost\n cost = conversation.conversation_stats.get_combined_metrics().accumulated_cost\n print(f\"EXAMPLE_COST: {cost}\")\n\nfinally:\n # Clean up temporary file\n Path(custom_policy_path).unlink(missing_ok=True)\n\nprint(\"\\n\" + \"=\" * 100)\nprint(\"Example Summary\")\nprint(\"=\" * 100)\nprint(\"This example demonstrated:\")\nprint(\"1. Using the default security policy (security_policy.j2)\")\nprint(\"2. Creating a custom security policy template\")\nprint(\"3. Applying the custom policy via security_policy_filename parameter\")\nprint(\"4. Running a conversation with the custom security policy\")\nprint(\n \"\\nYou can customize security policies to match your organization's \"\n \"specific requirements.\"\n)\n" }, { "path": "examples/01_standalone_sdk/33_hooks/README.md", "content": "# Hooks Examples\n\nThis folder demonstrates the OpenHands hooks system.\n\n## Example\n\n- **main.py** - Complete hooks demo showing all four hook types\n\n## Scripts\n\nThe `hook_scripts/` directory contains reusable hook script examples:\n\n- `block_dangerous.sh` - Blocks rm -rf commands (PreToolUse)\n- `log_tools.sh` - Logs tool usage to a file (PostToolUse)\n- `inject_git_context.sh` - Injects git status into prompts (UserPromptSubmit)\n- `require_summary.sh` - Requires summary.txt before stopping (Stop)\n\n## Running\n\n```bash\n# Set your LLM credentials\nexport LLM_API_KEY=\"your-key\"\nexport LLM_MODEL=\"anthropic/claude-sonnet-4-5-20250929\" # optional\nexport LLM_BASE_URL=\"https://your-endpoint\" # optional\n\n# Run example\npython main.py\n```\n\n## Hook Types\n\n| Hook | When it runs | Can block? |\n|------|--------------|------------|\n| PreToolUse | Before tool execution | Yes (exit 2) |\n| PostToolUse | After tool execution | No |\n| UserPromptSubmit | Before processing user message | Yes (exit 2) |\n| Stop | When agent tries to finish | Yes (exit 2) |\n| SessionStart | When conversation starts | No |\n| SessionEnd | When conversation ends | No |\n\n## Exit Codes\n\nHook scripts signal their result via the exit code (matching the Claude Code\nhook contract):\n\n- **`0` — success.** The operation proceeds. `stdout` is parsed as JSON for\n structured output (`decision`, `reason`, `additionalContext`).\n- **`2` — block.** The operation is denied. For `Stop` hooks, this prevents\n the agent from finishing and the agent continues running. `stderr` /\n `reason` is surfaced as feedback.\n- **Any other non-zero exit code — non-blocking error.** The error is\n logged, but the operation still proceeds.\n\n> **Note:** Only exit code `2` blocks. Exit code `1` (the conventional Unix\n> failure code) is treated as a non-blocking error. A hook that is meant to\n> enforce a policy must exit with `2`.\n" }, { "path": "examples/01_standalone_sdk/33_hooks/hook_scripts/block_dangerous.sh", "content": "#!/bin/bash\n# PreToolUse hook: Block dangerous rm -rf commands\n# Uses grep on raw JSON input (no jq needed)\n\ninput=$(cat)\n\n# Block rm -rf commands by checking if the input contains the pattern\nif echo \"$input\" | grep -q \"rm -rf\"; then\n echo '{\"decision\": \"deny\", \"reason\": \"rm -rf commands are blocked for safety\"}'\n exit 2 # Exit code 2 = block the operation\nfi\n\nexit 0 # Exit code 0 = allow the operation\n" }, { "path": "examples/01_standalone_sdk/33_hooks/hook_scripts/inject_git_context.sh", "content": "#!/bin/bash\n# UserPromptSubmit hook: Inject git status when user asks about code changes\n\ninput=$(cat)\n\n# Check if user is asking about changes, diff, or git\nif echo \"$input\" | grep -qiE \"(changes|diff|git|commit|modified)\"; then\n # Get git status if in a git repo\n if git rev-parse --git-dir > /dev/null 2>&1; then\n status=$(git status --short 2>/dev/null | head -10)\n if [ -n \"$status\" ]; then\n # Escape for JSON\n escaped=$(echo \"$status\" | sed 's/\"/\\\\\"/g' | tr '\\n' ' ')\n echo \"{\\\"additionalContext\\\": \\\"Current git status: $escaped\\\"}\"\n fi\n fi\nfi\nexit 0\n" }, { "path": "examples/01_standalone_sdk/33_hooks/hook_scripts/log_tools.sh", "content": "#!/bin/bash\n# PostToolUse hook: Log all tool usage\n# Uses OPENHANDS_TOOL_NAME env var (no jq/python needed!)\n\n# LOG_FILE should be set by the calling script\nLOG_FILE=\"${LOG_FILE:-/tmp/tool_usage.log}\"\n\necho \"[$(date)] Tool used: $OPENHANDS_TOOL_NAME\" >> \"$LOG_FILE\"\nexit 0\n" }, { "path": "examples/01_standalone_sdk/33_hooks/hook_scripts/require_summary.sh", "content": "#!/bin/bash\n# Stop hook: Require a summary.txt file before allowing agent to finish\n# SUMMARY_FILE should be set by the calling script\n\nSUMMARY_FILE=\"${SUMMARY_FILE:-./summary.txt}\"\n\nif [ ! -f \"$SUMMARY_FILE\" ]; then\n echo '{\"decision\": \"deny\", \"additionalContext\": \"Create summary.txt first.\"}'\n exit 2\nfi\nexit 0\n" }, { "path": "examples/01_standalone_sdk/33_hooks/main.py", "content": "\"\"\"OpenHands Agent SDK — Hooks Example\n\nDemonstrates the OpenHands hooks system.\nHooks are shell scripts that run at key lifecycle events:\n\n- PreToolUse: Block dangerous commands before execution\n- PostToolUse: Log tool usage after execution\n- UserPromptSubmit: Inject context into user messages\n- Stop: Enforce task completion criteria\n\nThe hook scripts are in the scripts/ directory alongside this file.\n\"\"\"\n\nimport os\nimport signal\nimport tempfile\nfrom pathlib import Path\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, Conversation\nfrom openhands.sdk.hooks import HookConfig, HookDefinition, HookMatcher\nfrom openhands.tools.preset.default import get_default_agent\n\n\nsignal.signal(signal.SIGINT, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt()))\n\nSCRIPT_DIR = Path(__file__).parent / \"hook_scripts\"\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\n\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Create temporary workspace with git repo\nwith tempfile.TemporaryDirectory() as tmpdir:\n workspace = Path(tmpdir)\n os.system(f\"cd {workspace} && git init -q && echo 'test' > file.txt\")\n\n log_file = workspace / \"tool_usage.log\"\n summary_file = workspace / \"summary.txt\"\n\n # Configure hooks using the typed approach (recommended)\n # This provides better type safety and IDE support\n hook_config = HookConfig(\n pre_tool_use=[\n HookMatcher(\n matcher=\"terminal\",\n hooks=[\n HookDefinition(\n command=str(SCRIPT_DIR / \"block_dangerous.sh\"),\n timeout=10,\n )\n ],\n )\n ],\n post_tool_use=[\n HookMatcher(\n matcher=\"*\",\n hooks=[\n HookDefinition(\n command=(f\"LOG_FILE={log_file} {SCRIPT_DIR / 'log_tools.sh'}\"),\n timeout=5,\n )\n ],\n )\n ],\n user_prompt_submit=[\n HookMatcher(\n hooks=[\n HookDefinition(\n command=str(SCRIPT_DIR / \"inject_git_context.sh\"),\n )\n ],\n )\n ],\n stop=[\n HookMatcher(\n hooks=[\n HookDefinition(\n command=(\n f\"SUMMARY_FILE={summary_file} \"\n f\"{SCRIPT_DIR / 'require_summary.sh'}\"\n ),\n )\n ],\n )\n ],\n )\n\n # Alternative: You can also use .from_dict() for loading from JSON config files\n # Example with a single hook matcher:\n # hook_config = HookConfig.from_dict({\n # \"hooks\": {\n # \"PreToolUse\": [{\n # \"matcher\": \"terminal\",\n # \"hooks\": [{\"command\": \"path/to/script.sh\", \"timeout\": 10}]\n # }]\n # }\n # })\n\n agent = get_default_agent(llm=llm)\n conversation = Conversation(\n agent=agent,\n workspace=str(workspace),\n hook_config=hook_config,\n )\n\n # Demo 1: Safe command (PostToolUse logs it)\n print(\"=\" * 60)\n print(\"Demo 1: Safe command - logged by PostToolUse\")\n print(\"=\" * 60)\n conversation.send_message(\"Run: echo 'Hello from hooks!'\")\n conversation.run()\n\n if log_file.exists():\n print(f\"\\n[Log: {log_file.read_text().strip()}]\")\n\n # Demo 2: Dangerous command (PreToolUse blocks it)\n print(\"\\n\" + \"=\" * 60)\n print(\"Demo 2: Dangerous command - blocked by PreToolUse\")\n print(\"=\" * 60)\n conversation.send_message(\"Run: rm -rf /tmp/test\")\n conversation.run()\n\n # Demo 3: Context injection + Stop hook enforcement\n print(\"\\n\" + \"=\" * 60)\n print(\"Demo 3: Context injection + Stop hook\")\n print(\"=\" * 60)\n print(\"UserPromptSubmit injects git status; Stop requires summary.txt\\n\")\n conversation.send_message(\n \"Check what files have changes, then create summary.txt describing the repo.\"\n )\n conversation.run()\n\n if summary_file.exists():\n print(f\"\\n[summary.txt: {summary_file.read_text()[:80]}...]\")\n\n print(\"\\n\" + \"=\" * 60)\n print(\"Example Complete!\")\n print(\"=\" * 60)\n\n cost = conversation.conversation_stats.get_combined_metrics().accumulated_cost\n print(f\"\\nEXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/34_critic_example.py", "content": "\"\"\"Iterative Refinement with Critic Model Example.\n\nThis is EXPERIMENTAL.\n\nThis example demonstrates how to use a critic model to shepherd an agent through\ncomplex, multi-step tasks. The critic evaluates the agent's progress and provides\nfeedback that can trigger follow-up prompts when the agent hasn't completed the\ntask successfully.\n\nKey concepts demonstrated:\n1. Setting up a critic with IterativeRefinementConfig for automatic retry\n2. Conversation.run() automatically handles retries based on critic scores\n3. Custom follow-up prompt generation via critic.get_followup_prompt()\n4. Iterating until the task is completed successfully or max iterations reached\n\nFor All-Hands LLM proxy (llm-proxy.*.all-hands.dev), the critic is auto-configured\nusing the same base_url with /vllm suffix and \"critic\" as the model name.\n\"\"\"\n\nimport os\nimport re\nimport tempfile\nfrom pathlib import Path\n\nfrom openhands.sdk import LLM, Agent, Conversation, Tool\nfrom openhands.sdk.critic import APIBasedCritic, IterativeRefinementConfig\nfrom openhands.sdk.critic.base import CriticBase\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.task_tracker import TaskTrackerTool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Configuration\n# Higher threshold (70%) makes it more likely the agent needs multiple iterations,\n# which better demonstrates how iterative refinement works.\n# Adjust as needed to see different behaviors.\nSUCCESS_THRESHOLD = float(os.getenv(\"CRITIC_SUCCESS_THRESHOLD\", \"0.7\"))\nMAX_ITERATIONS = int(os.getenv(\"MAX_ITERATIONS\", \"3\"))\n\n\ndef get_required_env(name: str) -> str:\n value = os.getenv(name)\n if value:\n return value\n raise ValueError(\n f\"Missing required environment variable: {name}. \"\n f\"Set {name} before running this example.\"\n )\n\n\ndef get_default_critic(llm: LLM) -> CriticBase | None:\n \"\"\"Auto-configure critic for All-Hands LLM proxy.\n\n When the LLM base_url matches `llm-proxy.*.all-hands.dev`, returns an\n APIBasedCritic configured with:\n - server_url: {base_url}/vllm\n - api_key: same as LLM\n - model_name: \"critic\"\n\n Args:\n llm: The LLM instance to derive critic configuration from.\n\n Returns:\n An APIBasedCritic if the LLM is configured for All-Hands proxy,\n None otherwise.\n\n Example:\n llm = LLM(\n model=\"anthropic/claude-sonnet-4-5\",\n api_key=api_key,\n base_url=\"https://llm-proxy.eval.all-hands.dev\",\n )\n critic = get_default_critic(llm)\n if critic is None:\n # Fall back to explicit configuration\n critic = APIBasedCritic(\n server_url=\"https://my-critic-server.com\",\n api_key=\"my-api-key\",\n model_name=\"my-critic-model\",\n )\n \"\"\"\n base_url = llm.base_url\n api_key = llm.api_key\n if base_url is None or api_key is None:\n return None\n\n # Match: llm-proxy.{env}.all-hands.dev (e.g., staging, prod, eval)\n pattern = r\"^https?://llm-proxy\\.[^./]+\\.all-hands\\.dev\"\n if not re.match(pattern, base_url):\n return None\n\n return APIBasedCritic(\n server_url=f\"{base_url.rstrip('/')}/vllm\",\n api_key=api_key,\n model_name=\"critic\",\n )\n\n\n# Task prompt designed to be moderately complex with subtle requirements.\n# The task is simple enough to complete in 1-2 iterations, but has specific\n# requirements that are easy to miss - triggering critic feedback.\nINITIAL_TASK_PROMPT = \"\"\"\\\nCreate a Python word statistics tool called `wordstats` that analyzes text files.\n\n## Structure\n\nCreate directory `wordstats/` with:\n- `stats.py` - Main module with `analyze_file(filepath)` function\n- `cli.py` - Command-line interface\n- `tests/test_stats.py` - Unit tests\n\n## Requirements for stats.py\n\nThe `analyze_file(filepath)` function must return a dict with these EXACT keys:\n- `lines`: total line count (including empty lines)\n- `words`: word count\n- `chars`: character count (including whitespace)\n- `unique_words`: count of unique words (case-insensitive)\n\n### Important edge cases (often missed!):\n1. Empty files must return all zeros, not raise an exception\n2. Hyphenated words count as ONE word (e.g., \"well-known\" = 1 word)\n3. Numbers like \"123\" or \"3.14\" are NOT counted as words\n4. Contractions like \"don't\" count as ONE word\n5. File not found must raise FileNotFoundError with a clear message\n\n## Requirements for cli.py\n\nWhen run as `python cli.py `:\n- Print each stat on its own line: \"Lines: X\", \"Words: X\", etc.\n- Exit with code 1 if file not found, printing error to stderr\n- Exit with code 0 on success\n\n## Required Tests (test_stats.py)\n\nWrite tests that verify:\n1. Basic counting on normal text\n2. Empty file returns all zeros\n3. Hyphenated words counted correctly\n4. Numbers are excluded from word count\n5. FileNotFoundError raised for missing files\n\n## Verification Steps\n\n1. Create a sample file `sample.txt` with this EXACT content (no trailing newline):\n```\nHello world!\nThis is a well-known test file.\n\nIt has 5 lines, including empty ones.\nNumbers like 42 and 3.14 don't count as words.\n```\n\n2. Run: `python wordstats/cli.py sample.txt`\n Expected output:\n - Lines: 5\n - Words: 21\n - Chars: 130\n - Unique words: 21\n\n3. Run the tests: `python -m pytest wordstats/tests/ -v`\n ALL tests must pass.\n\nThe task is complete ONLY when:\n- All files exist\n- The CLI outputs the correct stats for sample.txt\n- All 5+ tests pass\n\"\"\"\n\n\nllm_api_key = get_required_env(\"LLM_API_KEY\")\n# Use a weaker model to increase likelihood of needing multiple iterations\nllm_model = os.getenv(\"LLM_MODEL\", \"anthropic/claude-haiku-4-5-20251001\")\nllm = LLM(\n model=llm_model,\n api_key=llm_api_key,\n top_p=0.95,\n base_url=os.getenv(\"LLM_BASE_URL\"),\n)\n\n# Setup critic with iterative refinement config\n# The IterativeRefinementConfig tells Conversation.run() to automatically\n# retry the task if the critic score is below the threshold\niterative_config = IterativeRefinementConfig(\n success_threshold=SUCCESS_THRESHOLD,\n max_iterations=MAX_ITERATIONS,\n)\n\n# Auto-configure critic for All-Hands proxy or use explicit env vars\ncritic = get_default_critic(llm)\nif critic is None:\n print(\"⚠️ No All-Hands LLM proxy detected, trying explicit env vars...\")\n critic = APIBasedCritic(\n server_url=get_required_env(\"CRITIC_SERVER_URL\"),\n api_key=get_required_env(\"CRITIC_API_KEY\"),\n model_name=get_required_env(\"CRITIC_MODEL_NAME\"),\n iterative_refinement=iterative_config,\n )\nelse:\n # Add iterative refinement config to the auto-configured critic\n critic = critic.model_copy(update={\"iterative_refinement\": iterative_config})\n\n# Create agent with critic (iterative refinement is built into the critic)\nagent = Agent(\n llm=llm,\n tools=[\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n Tool(name=TaskTrackerTool.name),\n ],\n critic=critic,\n)\n\n# Create workspace\nworkspace = Path(tempfile.mkdtemp(prefix=\"critic_demo_\"))\nprint(f\"📁 Created workspace: {workspace}\")\n\n# Create conversation - iterative refinement is handled automatically\n# by Conversation.run() based on the critic's config\nconversation = Conversation(\n agent=agent,\n workspace=str(workspace),\n)\n\nprint(\"\\n\" + \"=\" * 70)\nprint(\"🚀 Starting Iterative Refinement with Critic Model\")\nprint(\"=\" * 70)\nprint(f\"Success threshold: {SUCCESS_THRESHOLD:.0%}\")\nprint(f\"Max iterations: {MAX_ITERATIONS}\")\n\n# Send the task and run - Conversation.run() handles retries automatically\nconversation.send_message(INITIAL_TASK_PROMPT)\nconversation.run()\n\n# Print additional info about created files\nprint(\"\\nCreated files:\")\nfor path in sorted(workspace.rglob(\"*\")):\n if path.is_file():\n relative = path.relative_to(workspace)\n print(f\" - {relative}\")\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"\\nEXAMPLE_COST: {cost:.4f}\")\n" }, { "path": "examples/01_standalone_sdk/35_subscription_login.py", "content": "\"\"\"Example: Using ChatGPT subscription for Codex models.\n\nThis example demonstrates how to use your ChatGPT Plus/Pro subscription\nto access OpenAI's Codex models without consuming API credits.\n\nThe subscription_login() method handles:\n- OAuth PKCE authentication flow\n- Device-code authentication for remote/headless environments\n- Credential caching (~/.openhands/auth/)\n- Automatic token refresh\n\nSupported models:\n- gpt-5.2-codex\n- gpt-5.2\n- gpt-5.1-codex-max\n- gpt-5.1-codex-mini\n\nRequirements:\n- Active ChatGPT Plus or Pro subscription\n- Browser access for initial OAuth login, or another browser/device for\n device-code login\n\nEnvironment variables:\n- OPENHANDS_SUBSCRIPTION_MODEL: Model to use (default: gpt-5.2-codex)\n- OPENHANDS_SUBSCRIPTION_AUTH_METHOD: \"browser\" or \"device_code\"\n (default: browser)\n- OPENHANDS_SUBSCRIPTION_FORCE_LOGIN: Set to \"1\" to force fresh login\n- SUBSCRIPTION_LOGIN_ONLY: Set to \"1\" to verify login without running an agent\n\"\"\"\n\nimport os\nfrom typing import Literal\n\nfrom openhands.sdk import LLM, Agent, Conversation, Tool\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\nAuthMethod = Literal[\"browser\", \"device_code\"]\n\n\n# First time: Opens browser for OAuth login\n# Subsequent calls: Reuses cached credentials (auto-refreshes if expired)\nmodel = os.getenv(\"OPENHANDS_SUBSCRIPTION_MODEL\", \"gpt-5.2-codex\")\nauth_method_env = os.getenv(\"OPENHANDS_SUBSCRIPTION_AUTH_METHOD\", \"browser\")\nif auth_method_env not in (\"browser\", \"device_code\"):\n raise ValueError(\n \"OPENHANDS_SUBSCRIPTION_AUTH_METHOD must be 'browser' or 'device_code'\"\n )\nauth_method: AuthMethod = auth_method_env\nforce_login = os.getenv(\"OPENHANDS_SUBSCRIPTION_FORCE_LOGIN\") == \"1\"\n\nllm = LLM.subscription_login(\n vendor=\"openai\",\n model=model, # or \"gpt-5.2\", \"gpt-5.1-codex-max\", \"gpt-5.1-codex-mini\"\n auth_method=auth_method,\n force_login=force_login,\n)\n\n# Alternative: Force a fresh login (useful if credentials are stale)\n# llm = LLM.subscription_login(vendor=\"openai\", model=\"gpt-5.2-codex\", force_login=True)\n\n# Alternative: Disable auto-opening browser (prints URL to console instead)\n# llm = LLM.subscription_login(\n# vendor=\"openai\", model=\"gpt-5.2-codex\", open_browser=False\n# )\n#\n# Alternative: Use device-code login for remote/headless environments\n# llm = LLM.subscription_login(\n# vendor=\"openai\",\n# model=\"gpt-5.2-codex\",\n# auth_method=\"device_code\",\n# force_login=True,\n# )\n\n# Verify subscription mode is active\nprint(f\"Using subscription mode: {llm.is_subscription}\")\nprint(f\"Model: {llm.model}\")\nprint(f\"Auth method: {auth_method}\")\n\nif os.getenv(\"SUBSCRIPTION_LOGIN_ONLY\") == \"1\":\n print(\"Login verified; skipping agent run because SUBSCRIPTION_LOGIN_ONLY=1.\")\n raise SystemExit(0)\n\n# Use the LLM with an agent as usual\nagent = Agent(\n llm=llm,\n tools=[\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n ],\n)\n\ncwd = os.getcwd()\nconversation = Conversation(agent=agent, workspace=cwd)\n\nconversation.send_message(\"List the files in the current directory.\")\nconversation.run()\nprint(\"Done!\")\n" }, { "path": "examples/01_standalone_sdk/36_event_json_to_openai_messages.py", "content": "\"\"\"Load persisted events and convert them into LLM-ready messages.\"\"\"\n\nimport json\nimport os\nimport uuid\nfrom pathlib import Path\n\nfrom pydantic import SecretStr\n\n\nconversation_id = uuid.uuid4()\npersistence_root = Path(\".conversations\")\nlog_dir = (\n persistence_root / \"logs\" / \"event-json-to-openai-messages\" / conversation_id.hex\n)\n\nos.environ.setdefault(\"LOG_JSON\", \"true\")\nos.environ.setdefault(\"LOG_TO_FILE\", \"true\")\nos.environ.setdefault(\"LOG_DIR\", str(log_dir))\nos.environ.setdefault(\"LOG_LEVEL\", \"INFO\")\n\nfrom openhands.sdk import ( # noqa: E402\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n Tool,\n)\nfrom openhands.sdk.logger import get_logger, setup_logging # noqa: E402\nfrom openhands.tools.terminal import TerminalTool # noqa: E402\n\n\nsetup_logging(log_to_file=True, log_dir=str(log_dir))\nlogger = get_logger(__name__)\n\napi_key = os.getenv(\"LLM_API_KEY\")\nif not api_key:\n raise RuntimeError(\"LLM_API_KEY environment variable is not set.\")\n\nllm = LLM(\n usage_id=\"agent\",\n model=os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\"),\n base_url=os.getenv(\"LLM_BASE_URL\"),\n api_key=SecretStr(api_key),\n)\n\nagent = Agent(\n llm=llm,\n tools=[Tool(name=TerminalTool.name)],\n)\n\n######\n# Create a conversation that persists its events\n######\n\nconversation = Conversation(\n agent=agent,\n workspace=os.getcwd(),\n persistence_dir=str(persistence_root),\n conversation_id=conversation_id,\n)\n\nconversation.send_message(\n \"Use the terminal tool to run `pwd` and write the output to tool_output.txt. \"\n \"Reply with a short confirmation once done.\"\n)\nconversation.run()\n\nconversation.send_message(\n \"Without using any tools, summarize in one sentence what you did.\"\n)\nconversation.run()\n\nassert conversation.state.persistence_dir is not None\npersistence_dir = Path(conversation.state.persistence_dir)\nevent_dir = persistence_dir / \"events\"\n\nevent_paths = sorted(event_dir.glob(\"event-*.json\"))\n\nif not event_paths:\n raise RuntimeError(\"No event files found. Was persistence enabled?\")\n\n######\n# Read from serialized events\n######\n\n\nevents = [Event.model_validate_json(path.read_text()) for path in event_paths]\n\nconvertible_events = [\n event for event in events if isinstance(event, LLMConvertibleEvent)\n]\nllm_messages = LLMConvertibleEvent.events_to_messages(convertible_events)\n\nif llm.uses_responses_api():\n logger.info(\"Formatting messages for the OpenAI Responses API.\")\n instructions, input_items = llm.format_messages_for_responses(llm_messages)\n logger.info(\"Responses instructions:\\n%s\", instructions)\n logger.info(\"Responses input:\\n%s\", json.dumps(input_items, indent=2))\nelse:\n logger.info(\"Formatting messages for the OpenAI Chat Completions API.\")\n chat_messages = llm.format_messages_for_llm(llm_messages)\n logger.info(\"Chat Completions messages:\\n%s\", json.dumps(chat_messages, indent=2))\n\n# Report cost\ncost = llm.metrics.accumulated_cost\nprint(f\"EXAMPLE_COST: {cost}\")\n" }, { "path": "examples/01_standalone_sdk/37_llm_profile_store/main.py", "content": "\"\"\"Example: Using LLMProfileStore to save and reuse LLM configurations.\n\nThis example ships with one pre-generated profile JSON file and creates another\nprofile at runtime. The checked-in profile comes from a normal save, so secrets\nare masked instead of exposed and non-secret fields like `base_url` are kept\nwhen present.\n\"\"\"\n\nimport os\nimport shutil\nimport tempfile\nfrom pathlib import Path\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, LLMProfileStore\n\n\nSCRIPT_DIR = Path(__file__).parent\nEXAMPLE_PROFILES_DIR = SCRIPT_DIR / \"profiles\"\nDEFAULT_MODEL = \"anthropic/claude-sonnet-4-5-20250929\"\n\n\nprofile_store_dir = Path(tempfile.mkdtemp()) / \"profiles\"\nshutil.copytree(EXAMPLE_PROFILES_DIR, profile_store_dir)\nstore = LLMProfileStore(base_dir=profile_store_dir)\n\nprint(f\"Seeded profiles: {store.list()}\")\n\napi_key = os.getenv(\"LLM_API_KEY\")\ncreative_llm = LLM(\n usage_id=\"creative\",\n model=os.getenv(\"LLM_MODEL\", DEFAULT_MODEL),\n api_key=SecretStr(api_key) if api_key else None,\n base_url=os.getenv(\"LLM_BASE_URL\"),\n temperature=0.9,\n)\n\n# The checked-in fast.json was generated with a normal save, so its api_key is\n# masked and any configured base_url would be preserved. This runtime profile\n# also avoids persisting the real API key because secrets are masked by default.\nstore.save(\"creative\", creative_llm)\ncreative_profile_json = (profile_store_dir / \"creative.json\").read_text()\nif api_key is not None:\n assert api_key not in creative_profile_json\n\nprint(f\"Stored profiles: {store.list()}\")\n\nfast_profile = store.load(\"fast\")\ncreative_profile = store.load(\"creative\")\n\nprint(\n \"Loaded fast profile. \"\n f\"usage: {fast_profile.usage_id}, \"\n f\"model: {fast_profile.model}, \"\n f\"temperature: {fast_profile.temperature}.\"\n)\nprint(\n \"Loaded creative profile. \"\n f\"usage: {creative_profile.usage_id}, \"\n f\"model: {creative_profile.model}, \"\n f\"temperature: {creative_profile.temperature}.\"\n)\n\nstore.delete(\"creative\")\nprint(f\"After deletion: {store.list()}\")\n\nprint(\"EXAMPLE_COST: 0\")\n" }, { "path": "examples/01_standalone_sdk/37_llm_profile_store/profiles/fast.json", "content": "{\n \"model\": \"anthropic/claude-sonnet-4-5-20250929\",\n \"api_key\": \"**********\",\n \"openrouter_site_url\": \"https://docs.all-hands.dev/\",\n \"openrouter_app_name\": \"OpenHands\",\n \"num_retries\": 5,\n \"retry_multiplier\": 8.0,\n \"retry_min_wait\": 8,\n \"retry_max_wait\": 64,\n \"timeout\": 300,\n \"max_message_chars\": 30000,\n \"temperature\": 0.0,\n \"max_input_tokens\": 200000,\n \"max_output_tokens\": 64000,\n \"stream\": false,\n \"drop_params\": true,\n \"modify_params\": true,\n \"disable_stop_word\": false,\n \"caching_prompt\": true,\n \"log_completions\": false,\n \"log_completions_folder\": \"logs/completions\",\n \"native_tool_calling\": true,\n \"reasoning_effort\": \"high\",\n \"enable_encrypted_reasoning\": true,\n \"prompt_cache_retention\": \"24h\",\n \"extended_thinking_budget\": 200000,\n \"usage_id\": \"fast\",\n \"litellm_extra_body\": {}\n}\n" }, { "path": "examples/01_standalone_sdk/38_browser_session_recording.py", "content": "\"\"\"Browser Session Recording Example\n\nThis example demonstrates how to use the browser session recording feature\nto capture and save a recording of the agent's browser interactions using rrweb.\n\nThe recording can be replayed later using rrweb-player to visualize the agent's\nbrowsing session.\n\nThe recording will be automatically saved to the persistence directory when\nbrowser_stop_recording is called. You can replay it with:\n - rrweb-player: https://github.com/rrweb-io/rrweb/tree/master/packages/rrweb-player\n - Online viewer: https://www.rrweb.io/demo/\n\"\"\"\n\nimport json\nimport os\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import (\n LLM,\n Agent,\n Conversation,\n Event,\n LLMConvertibleEvent,\n get_logger,\n)\nfrom openhands.sdk.tool import Tool\nfrom openhands.tools.browser_use import BrowserToolSet\nfrom openhands.tools.browser_use.definition import (\n BROWSER_RECORDING_OUTPUT_DIR,\n BrowserNavigateAction,\n)\n\n\nlogger = get_logger(__name__)\n\n# Configure LLM\napi_key = os.getenv(\"LLM_API_KEY\")\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nmodel = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\nbase_url = os.getenv(\"LLM_BASE_URL\")\nllm = LLM(\n usage_id=\"agent\",\n model=model,\n base_url=base_url,\n api_key=SecretStr(api_key),\n)\n\n# Tools - including browser tools with recording capability\ncwd = os.getcwd()\ntools = [\n Tool(name=BrowserToolSet.name),\n]\n\n# Agent\nagent = Agent(llm=llm, tools=tools)\n\nllm_messages = [] # collect raw LLM messages\n\n\ndef conversation_callback(event: Event):\n if isinstance(event, LLMConvertibleEvent):\n llm_messages.append(event.to_llm_message())\n\n\n# Create conversation with persistence_dir set to save browser recordings\nconversation = Conversation(\n agent=agent,\n callbacks=[conversation_callback],\n workspace=cwd,\n persistence_dir=\"./.conversations\",\n)\n\n# The prompt instructs the agent to:\n# 1. Start recording the browser session\n# 2. Navigate to a page and get its content\n# 3. Stop recording (auto-saves to file)\nPROMPT = \"\"\"\nPlease complete the following task to demonstrate browser session recording:\n\n1. Use `browser_start_recording` to begin recording.\n2. Navigate to https://docs.openhands.dev/ and:\n - Get the page content\n - Scroll down the page\n - Get the browser state to see interactive elements\n3. Use `browser_stop_recording` to stop and save the recording.\n\"\"\"\n\nprint(\"=\" * 80)\nprint(\"Browser Session Recording Example\")\nprint(\"=\" * 80)\nprint(\"\\nTask: Record an agent's browser session and save it for replay\")\n\n# Pre-initialize the browser so CDP is ready before the agent starts.\n# This avoids wasting LLM calls if the browser fails to connect.\nprint(\"\\nInitializing browser...\")\n\ninit_obs = conversation.execute_tool(\n \"browser_navigate\",\n BrowserNavigateAction(url=\"about:blank\"),\n)\nif init_obs.is_error:\n print(f\"Browser initialization failed: {init_obs.text}\")\n print(\"Ensure Chrome/Chromium is installed and accessible.\")\n exit(1)\nprint(\"Browser initialized successfully.\\n\")\n\nprint(\"Starting conversation with agent...\\n\")\n\nconversation.send_message(PROMPT)\nconversation.run()\n\nprint(\"\\n\" + \"=\" * 80)\nprint(\"Conversation finished!\")\nprint(\"=\" * 80)\n\n# Check if the recording files were created\n# Recordings are saved in BROWSER_RECORDING_OUTPUT_DIR/recording-{timestamp}/\nif os.path.exists(BROWSER_RECORDING_OUTPUT_DIR):\n # Find recording subdirectories (they start with \"recording-\")\n recording_dirs = sorted(\n [\n d\n for d in os.listdir(BROWSER_RECORDING_OUTPUT_DIR)\n if d.startswith(\"recording-\")\n and os.path.isdir(os.path.join(BROWSER_RECORDING_OUTPUT_DIR, d))\n ]\n )\n\n if recording_dirs:\n # Process the most recent recording directory\n latest_recording = recording_dirs[-1]\n recording_path = os.path.join(BROWSER_RECORDING_OUTPUT_DIR, latest_recording)\n json_files = sorted(\n [f for f in os.listdir(recording_path) if f.endswith(\".json\")]\n )\n\n print(f\"\\n✓ Recording saved to: {recording_path}\")\n print(f\"✓ Number of files: {len(json_files)}\")\n\n # Count total events across all files\n total_events = 0\n all_event_types: dict[int | str, int] = {}\n total_size = 0\n\n for json_file in json_files:\n filepath = os.path.join(recording_path, json_file)\n file_size = os.path.getsize(filepath)\n total_size += file_size\n\n with open(filepath) as f:\n events = json.load(f)\n\n # Events are stored as a list in each file\n if isinstance(events, list):\n total_events += len(events)\n for event in events:\n event_type = event.get(\"type\", \"unknown\")\n all_event_types[event_type] = all_event_types.get(event_type, 0) + 1\n\n print(f\" - {json_file}: {len(events)} events, {file_size} bytes\")\n\n print(f\"✓ Total events: {total_events}\")\n print(f\"✓ Total size: {total_size} bytes\")\n if all_event_types:\n print(f\"✓ Event types: {all_event_types}\")\n\n print(\"\\nTo replay this recording, you can use:\")\n print(\n \" - rrweb-player: \"\n \"https://github.com/rrweb-io/rrweb/tree/master/packages/rrweb-player\"\n )\n else:\n print(f\"\\n✗ No recording directories found in: {BROWSER_RECORDING_OUTPUT_DIR}\")\n print(\" The agent may not have completed the recording task.\")\nelse:\n print(f\"\\n✗ Observations directory not found: {BROWSER_RECORDING_OUTPUT_DIR}\")\n print(\" The agent may not have completed the recording task.\")\n\nprint(\"\\n\" + \"=\" * 100)\nprint(\"Conversation finished.\")\nprint(f\"Total LLM messages: {len(llm_messages)}\")\nprint(\"=\" * 100)\n\n# Report cost\ncost = conversation.conversation_stats.get_combined_metrics().accumulated_cost\nprint(f\"Conversation ID: {conversation.id}\")\nprint(f\"EXAMPLE_COST: {cost}\")\n\n# Close conversation to shut down browser and other tool executors\nconversation.close()\n" }, { "path": "examples/01_standalone_sdk/39_llm_fallback.py", "content": "\"\"\"Example: Using FallbackStrategy for LLM resilience.\n\nWhen the primary LLM fails with a transient error (rate limit, timeout, etc.),\nFallbackStrategy automatically tries alternate LLMs in order. Fallback is\nper-call: each new request starts with the primary model. Token usage and\ncost from fallback calls are merged into the primary LLM's metrics.\n\nThis example:\n 1. Saves two fallback LLM profiles to a temporary store.\n 2. Configures a primary LLM with a FallbackStrategy pointing at those profiles.\n 3. Runs a conversation — if the primary model is unavailable, the agent\n transparently falls back to the next available model.\n\"\"\"\n\nimport os\nimport tempfile\n\nfrom pydantic import SecretStr\n\nfrom openhands.sdk import LLM, Agent, Conversation, LLMProfileStore, Tool\nfrom openhands.sdk.llm import FallbackStrategy\nfrom openhands.tools.file_editor import FileEditorTool\nfrom openhands.tools.terminal import TerminalTool\n\n\n# Read configuration from environment\napi_key = os.getenv(\"LLM_API_KEY\", None)\nassert api_key is not None, \"LLM_API_KEY environment variable is not set.\"\nbase_url = os.getenv(\"LLM_BASE_URL\")\nprimary_model = os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\")\n\n# Use a temporary directory so this example doesn't pollute your home folder.\n# In real usage you can omit base_dir to use the default (~/.openhands/profiles).\nprofile_store_dir = tempfile.mkdtemp()\nstore = LLMProfileStore(base_dir=profile_store_dir)\n\nfallback_1 = LLM(\n usage_id=\"fallback-1\",\n model=os.getenv(\"LLM_FALLBACK_MODEL_1\", \"openai/gpt-4o\"),\n api_key=SecretStr(os.getenv(\"LLM_FALLBACK_API_KEY_1\", api_key)),\n base_url=os.getenv(\"LLM_FALLBACK_BASE_URL_1\", base_url),\n)\nstore.save(\"fallback-1\", fallback_1, include_secrets=True)\n\nfallback_2 = LLM(\n usage_id=\"fallback-2\",\n model=os.getenv(\"LLM_FALLBACK_MODEL_2\", \"openai/gpt-4o-mini\"),\n api_key=SecretStr(os.getenv(\"LLM_FALLBACK_API_KEY_2\", api_key)),\n base_url=os.getenv(\"LLM_FALLBACK_BASE_URL_2\", base_url),\n)\nstore.save(\"fallback-2\", fallback_2, include_secrets=True)\n\nprint(f\"Saved fallback profiles: {store.list()}\")\n\n\n# Configure the primary LLM with a FallbackStrategy\nprimary_llm = LLM(\n usage_id=\"agent-primary\",\n model=primary_model,\n api_key=SecretStr(api_key),\n base_url=base_url,\n fallback_strategy=FallbackStrategy(\n fallback_llms=[\"fallback-1\", \"fallback-2\"],\n profile_store_dir=profile_store_dir,\n ),\n)\n\n\n# Run a conversation\nagent = Agent(\n llm=primary_llm,\n tools=[\n Tool(name=TerminalTool.name),\n Tool(name=FileEditorTool.name),\n ],\n)\n\nconversation = Conversation(agent=agent, workspace=os.getcwd())\nconversation.send_message(\"Write a haiku about resilience into HAIKU.txt.\")\nconversation.run()\n\n\n# Inspect metrics (includes any fallback usage)\nmetrics = primary_llm.metrics\nprint(f\"Total cost (including fallbacks): ${metrics.accumulated_cost:.6f}\")\nprint(f\"Token usage records: {len(metrics.token_usages)}\")\nfor usage in metrics.token_usages:\n print(\n f\" model={usage.model}\"\n f\" prompt={usage.prompt_tokens}\"\n f\" completion={usage.completion_tokens}\"\n )\n\nprint(f\"EXAMPLE_COST: {metrics.accumulated_cost}\")\n" }, { "path": "examples/01_standalone_sdk/40_acp_agent_example.py", "content": "\"\"\"Example: Using ACPAgent with Claude Code ACP server.\n\nThis example shows how to use an ACP-compatible server (claude-agent-acp)\nas the agent backend instead of direct LLM calls. It also demonstrates\n``ask_agent()`` — a stateless side-question that forks the ACP session\nand leaves the main conversation untouched — and sending an image alongside\ntext to verify multimodal (vision) input support.\n\nPrerequisites:\n - Node.js / npx available\n - ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY set (can point to LiteLLM proxy)\n\nUsage:\n uv run python examples/01_standalone_sdk/40_acp_agent_example.py\n\"\"\"\n\nimport os\n\nfrom openhands.sdk import ImageContent, Message, TextContent\nfrom openhands.sdk.agent import ACPAgent\nfrom openhands.sdk.conversation import Conversation\n\n\nIMAGE_URL = \"https://github.com/OpenHands/docs/raw/main/openhands/static/img/logo.png\"\n\nagent = ACPAgent(acp_command=[\"npx\", \"-y\", \"@agentclientprotocol/claude-agent-acp\"])\n\ntry:\n cwd = os.getcwd()\n conversation = Conversation(agent=agent, workspace=cwd)\n\n # --- Main conversation turn (text only) ---\n conversation.send_message(\n \"List the Python source files under openhands-sdk/openhands/sdk/agent/, \"\n \"then read the __init__.py and summarize what agent classes are exported.\"\n )\n conversation.run()\n\n # --- Image input turn (text + image) ---\n print(\"\\n--- image input ---\")\n conversation.send_message(\n Message(\n role=\"user\",\n content=[\n TextContent(\n text=\"Describe what you see in this image in one sentence.\"\n ),\n ImageContent(image_urls=[IMAGE_URL]),\n ],\n )\n )\n conversation.run()\n\n # --- ask_agent: stateless side-question via fork_session ---\n print(\"\\n--- ask_agent ---\")\n response = conversation.ask_agent(\n \"Based on what you just saw, which agent class is the newest addition?\"\n )\n print(f\"ask_agent response: {response}\")\n # Report cost (ACP server reports usage via session_update notifications)\n cost = agent.llm.metrics.accumulated_cost\n print(f\"EXAMPLE_COST: {cost:.4f}\")\nfinally:\n # Clean up the ACP server subprocess\n agent.close()\n\ncost = conversation.conversation_stats.get_combined_metrics().accumulated_cost\nprint(f\"\\nEXAMPLE_COST: {cost}\")\nprint(\"Done!\")\n" }, { "path": "examples/01_standalone_sdk/41_task_tool_set.py", "content": "\"\"\"\nAnimal Quiz with Task Tool Set\n\nDemonstrates the TaskToolSet with a main agent delegating to an\nanimal-expert sub-agent. The flow is:\n\n1. Main agent picks an animal and delegates to the \"animal_expert\"\n sub-agent to generate a multiple-choice question about it.\n2. Main agent thinks about the question and picks an answer.\n3. Main agent resumes the same sub-agent conversation to ask whether\n its answer is correct. The sub-agent confirms or corrects it.\n\"\"\"\n\nimport os\n\nfrom openhands.sdk import LLM, Agent, AgentContext, Conversation, Tool\nfrom openhands.sdk.context import Skill\nfrom openhands.sdk.subagent import register_agent\nfrom openhands.tools.delegate import DelegationVisualizer\nfrom openhands.tools.task import TaskToolSet\n\n\nllm = LLM(\n model=os.getenv(\"LLM_MODEL\", \"anthropic/claude-sonnet-4-5-20250929\"),\n api_key=os.getenv(\"LLM_API_KEY\"),\n base_url=os.getenv(\"LLM_BASE_URL\", None),\n)\n# ── Register the animal expert sub-agent ─────────────────────────────\n\n\ndef create_animal_expert(llm: LLM) -> Agent:\n \"\"\"Factory for the animal-expert sub-agent.\"\"\"\n return Agent(\n llm=llm,\n tools=[], # no tools needed – pure knowledge\n agent_context=AgentContext(\n skills=[\n Skill(\n name=\"animal_expertise\",\n content=(\n \"You are a world-class zoologist. \"\n \"When asked to generate a quiz question, respond with \"\n \"EXACTLY this format and nothing else:\\n\\n\"\n \"Question: \\n\"\n \"A)