[
{
"path": ".gitignore",
"content": "# Development artifacts\n*.pyc\n__pycache__/\n.pytest_cache/\n*.egg-info/\ndist/\nbuild/\n\n# Environment files\n.env\n.env.local\n.env.*.local\n\n# IDE and editor files\n.vscode/\n.idea/\n*.swp\n*.swo\n*~\n\n# OS generated files\n.DS_Store\n.DS_Store?\n._*\n.Spotlight-V100\n.Trashes\nehthumbs.db\nThumbs.db\n\n# Python virtual environments\nvenv/\nenv/\nENV/\n\n# Analysis outputs\nanalysis_results/\n*.log\n\n# Docker volumes (if persisted locally)\nprometheus_data/\ngrafana_data/\nloki_data/\n\n# Temporary files\ntmp/\ntemp/ "
},
{
"path": "CLAUDE_OBSERVABILITY.md",
"content": "# Monitoring\n\n> Learn how to enable and configure OpenTelemetry for Claude Code.\n\nClaude Code supports OpenTelemetry (OTel) metrics and events for monitoring and observability.\n\nAll metrics are time series data exported via OpenTelemetry's standard metrics protocol, and events are exported via OpenTelemetry's logs/events protocol. It is the user's responsibility to ensure their metrics and logs backends are properly configured and that the aggregation granularity meets their monitoring requirements.\n\n\n OpenTelemetry support is currently in beta and details are subject to change.\n\n\n## Quick Start\n\nConfigure OpenTelemetry using environment variables:\n\n```bash\n# 1. Enable telemetry\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\n\n# 2. Choose exporters (both are optional - configure only what you need)\nexport OTEL_METRICS_EXPORTER=otlp # Options: otlp, prometheus, console\nexport OTEL_LOGS_EXPORTER=otlp # Options: otlp, console\n\n# 3. Configure OTLP endpoint (for OTLP exporter)\nexport OTEL_EXPORTER_OTLP_PROTOCOL=grpc\nexport OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317\n\n# 4. Set authentication (if required)\nexport OTEL_EXPORTER_OTLP_HEADERS=\"Authorization=Bearer your-token\"\n\n# 5. For debugging: reduce export intervals\nexport OTEL_METRIC_EXPORT_INTERVAL=10000 # 10 seconds (default: 60000ms)\nexport OTEL_LOGS_EXPORT_INTERVAL=5000 # 5 seconds (default: 5000ms)\n\n# 6. Run Claude Code\nclaude\n```\n\n\n The default export intervals are 60 seconds for metrics and 5 seconds for logs. During setup, you may want to use shorter intervals for debugging purposes. Remember to reset these for production use.\n\n\nFor full configuration options, see the [OpenTelemetry specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md#configuration-options).\n\n## Administrator Configuration\n\nAdministrators can configure OpenTelemetry settings for all users through the managed settings file. This allows for centralized control of telemetry settings across an organization. See the [settings precedence](/en/docs/claude-code/settings#settings-precedence) for more information about how settings are applied.\n\nThe managed settings file is located at:\n\n* macOS: `/Library/Application Support/ClaudeCode/managed-settings.json`\n* Linux: `/etc/claude-code/managed-settings.json`\n\nExample managed settings configuration:\n\n```json\n{\n \"env\": {\n \"CLAUDE_CODE_ENABLE_TELEMETRY\": \"1\",\n \"OTEL_METRICS_EXPORTER\": \"otlp\",\n \"OTEL_LOGS_EXPORTER\": \"otlp\",\n \"OTEL_EXPORTER_OTLP_PROTOCOL\": \"grpc\",\n \"OTEL_EXPORTER_OTLP_ENDPOINT\": \"http://collector.company.com:4317\",\n \"OTEL_EXPORTER_OTLP_HEADERS\": \"Authorization=Bearer company-token\"\n }\n}\n```\n\n\n Managed settings can be distributed via MDM (Mobile Device Management) or other device management solutions. Environment variables defined in the managed settings file have high precedence and cannot be overridden by users.\n\n\n## Configuration Details\n\n### Common Configuration Variables\n\n| Environment Variable | Description | Example Values |\n| ----------------------------------------------- | --------------------------------------------------------- | ------------------------------------ |\n| `CLAUDE_CODE_ENABLE_TELEMETRY` | Enables telemetry collection (required) | `1` |\n| `OTEL_METRICS_EXPORTER` | Metrics exporter type(s) (comma-separated) | `console`, `otlp`, `prometheus` |\n| `OTEL_LOGS_EXPORTER` | Logs/events exporter type(s) (comma-separated) | `console`, `otlp` |\n| `OTEL_EXPORTER_OTLP_PROTOCOL` | Protocol for OTLP exporter (all signals) | `grpc`, `http/json`, `http/protobuf` |\n| `OTEL_EXPORTER_OTLP_ENDPOINT` | OTLP collector endpoint (all signals) | `http://localhost:4317` |\n| `OTEL_EXPORTER_OTLP_METRICS_PROTOCOL` | Protocol for metrics (overrides general) | `grpc`, `http/json`, `http/protobuf` |\n| `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | OTLP metrics endpoint (overrides general) | `http://localhost:4318/v1/metrics` |\n| `OTEL_EXPORTER_OTLP_LOGS_PROTOCOL` | Protocol for logs (overrides general) | `grpc`, `http/json`, `http/protobuf` |\n| `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | OTLP logs endpoint (overrides general) | `http://localhost:4318/v1/logs` |\n| `OTEL_EXPORTER_OTLP_HEADERS` | Authentication headers for OTLP | `Authorization=Bearer token` |\n| `OTEL_EXPORTER_OTLP_METRICS_CLIENT_KEY` | Client key for mTLS authentication | Path to client key file |\n| `OTEL_EXPORTER_OTLP_METRICS_CLIENT_CERTIFICATE` | Client certificate for mTLS authentication | Path to client cert file |\n| `OTEL_METRIC_EXPORT_INTERVAL` | Export interval in milliseconds (default: 60000) | `5000`, `60000` |\n| `OTEL_LOGS_EXPORT_INTERVAL` | Logs export interval in milliseconds (default: 5000) | `1000`, `10000` |\n| `OTEL_LOG_USER_PROMPTS` | Enable logging of user prompt content (default: disabled) | `1` to enable |\n\n### Metrics Cardinality Control\n\nThe following environment variables control which attributes are included in metrics to manage cardinality:\n\n| Environment Variable | Description | Default Value | Example to Disable |\n| ----------------------------------- | ----------------------------------------------- | ------------- | ------------------ |\n| `OTEL_METRICS_INCLUDE_SESSION_ID` | Include session.id attribute in metrics | `true` | `false` |\n| `OTEL_METRICS_INCLUDE_VERSION` | Include app.version attribute in metrics | `false` | `true` |\n| `OTEL_METRICS_INCLUDE_ACCOUNT_UUID` | Include user.account\\_uuid attribute in metrics | `true` | `false` |\n\nThese variables help control the cardinality of metrics, which affects storage requirements and query performance in your metrics backend. Lower cardinality generally means better performance and lower storage costs but less granular data for analysis.\n\n### Example Configurations\n\n```bash\n# Console debugging (1-second intervals)\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\nexport OTEL_METRICS_EXPORTER=console\nexport OTEL_METRIC_EXPORT_INTERVAL=1000\n\n# OTLP/gRPC\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\nexport OTEL_METRICS_EXPORTER=otlp\nexport OTEL_EXPORTER_OTLP_PROTOCOL=grpc\nexport OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317\n\n# Prometheus\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\nexport OTEL_METRICS_EXPORTER=prometheus\n\n# Multiple exporters\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\nexport OTEL_METRICS_EXPORTER=console,otlp\nexport OTEL_EXPORTER_OTLP_PROTOCOL=http/json\n\n# Different endpoints/backends for metrics and logs\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\nexport OTEL_METRICS_EXPORTER=otlp\nexport OTEL_LOGS_EXPORTER=otlp\nexport OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=http/protobuf\nexport OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://metrics.company.com:4318\nexport OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=grpc\nexport OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://logs.company.com:4317\n\n# Metrics only (no events/logs)\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\nexport OTEL_METRICS_EXPORTER=otlp\nexport OTEL_EXPORTER_OTLP_PROTOCOL=grpc\nexport OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317\n\n# Events/logs only (no metrics)\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\nexport OTEL_LOGS_EXPORTER=otlp\nexport OTEL_EXPORTER_OTLP_PROTOCOL=grpc\nexport OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317\n```\n\n## Available Metrics and Events\n\n### Metrics\n\nClaude Code exports the following metrics:\n\n| Metric Name | Description | Unit |\n| ------------------------------------- | ----------------------------------------------- | ------ |\n| `claude_code.session.count` | Count of CLI sessions started | count |\n| `claude_code.lines_of_code.count` | Count of lines of code modified | count |\n| `claude_code.pull_request.count` | Number of pull requests created | count |\n| `claude_code.commit.count` | Number of git commits created | count |\n| `claude_code.cost.usage` | Cost of the Claude Code session | USD |\n| `claude_code.token.usage` | Number of tokens used | tokens |\n| `claude_code.code_edit_tool.decision` | Count of code editing tool permission decisions | count |\n\n### Metric Details\n\nAll metrics share these standard attributes:\n\n* `session.id`: Unique session identifier (controlled by `OTEL_METRICS_INCLUDE_SESSION_ID`)\n* `app.version`: Current Claude Code version (controlled by `OTEL_METRICS_INCLUDE_VERSION`)\n* `organization.id`: Organization UUID (when authenticated)\n* `user.account_uuid`: Account UUID (when authenticated, controlled by `OTEL_METRICS_INCLUDE_ACCOUNT_UUID`)\n\n#### Session Counter\n\nEmitted at the start of each session.\n\n#### Lines of Code Counter\n\nEmitted when code is added or removed.\n\nAdditional attribute: `type` (`\"added\"` or `\"removed\"`)\n\n#### Pull Request Counter\n\nEmitted when creating pull requests via Claude Code.\n\n#### Commit Counter\n\nEmitted when creating git commits via Claude Code.\n\n#### Cost Counter\n\nEmitted after each API request.\n\nAdditional attribute: `model`\n\n#### Token Counter\n\nEmitted after each API request.\n\nAdditional attributes: `type` (`\"input\"`, `\"output\"`, `\"cacheRead\"`, `\"cacheCreation\"`) and `model`\n\n#### Code Edit Tool Decision Counter\n\nEmitted when user accepts or rejects Edit, MultiEdit, Write, or NotebookEdit tool usage.\n\nAdditional attributes: `tool` (tool name: `\"Edit\"`, `\"MultiEdit\"`, `\"Write\"`, `\"NotebookEdit\"`) and `decision` (`\"accept\"`, `\"reject\"`)\n\n### Events\n\nClaude Code exports the following events via OpenTelemetry logs/events (when `OTEL_LOGS_EXPORTER` is configured):\n\n#### User Prompt Event\n\n* **Event Name**: `claude_code.user_prompt`\n* **Description**: Logged when a user submits a prompt\n* **Attributes**:\n * All standard attributes (user.id, session.id, etc.)\n * `event.name`: `\"user_prompt\"`\n * `event.timestamp`: ISO 8601 timestamp\n * `prompt_length`: Length of the prompt\n * `prompt`: Prompt content (redacted by default, enable with `OTEL_LOG_USER_PROMPTS=1`)\n\n#### Tool Result Event\n\n* **Event Name**: `claude_code.tool_result`\n* **Description**: Logged when a tool completes execution\n* **Attributes**:\n * All standard attributes\n * `event.name`: `\"tool_result\"`\n * `event.timestamp`: ISO 8601 timestamp\n * `name`: Name of the tool\n * `success`: `\"true\"` or `\"false\"`\n * `duration_ms`: Execution time in milliseconds\n * `error`: Error message (if failed)\n\n#### API Request Event\n\n* **Event Name**: `claude_code.api_request`\n* **Description**: Logged for each API request to Claude\n* **Attributes**:\n * All standard attributes\n * `event.name`: `\"api_request\"`\n * `event.timestamp`: ISO 8601 timestamp\n * `model`: Model used (e.g., \"claude-3-5-sonnet-20241022\")\n * `cost_usd`: Estimated cost in USD\n * `duration_ms`: Request duration in milliseconds\n * `input_tokens`: Number of input tokens\n * `output_tokens`: Number of output tokens\n * `cache_read_tokens`: Number of tokens read from cache\n * `cache_creation_tokens`: Number of tokens used for cache creation\n\n#### API Error Event\n\n* **Event Name**: `claude_code.api_error`\n* **Description**: Logged when an API request to Claude fails\n* **Attributes**:\n * All standard attributes\n * `event.name`: `\"api_error\"`\n * `event.timestamp`: ISO 8601 timestamp\n * `model`: Model used (e.g., \"claude-3-5-sonnet-20241022\")\n * `error`: Error message\n * `status_code`: HTTP status code (if applicable)\n * `duration_ms`: Request duration in milliseconds\n * `attempt`: Attempt number (for retried requests)\n\n#### Tool Decision Event\n\n* **Event Name**: `claude_code.tool_decision`\n* **Description**: Logged when a tool permission decision is made (accept/reject)\n* **Attributes**:\n * All standard attributes\n * `event.name`: `\"tool_decision\"`\n * `event.timestamp`: ISO 8601 timestamp\n * `tool_name`: Name of the tool (e.g., \"Read\", \"Edit\", \"MultiEdit\", \"Write\", \"NotebookEdit\", etc.)\n * `decision`: Either `\"accept\"` or `\"reject\"`\n * `source`: Decision source - `\"config\"`, `\"user_permanent\"`, `\"user_temporary\"`, `\"user_abort\"`, or `\"user_reject\"`\n\n## Interpreting Metrics and Events Data\n\nThe metrics exported by Claude Code provide valuable insights into usage patterns and productivity. Here are some common visualizations and analyses you can create:\n\n### Usage Monitoring\n\n| Metric | Analysis Opportunity |\n| ------------------------------------------------------------- | --------------------------------------------------------- |\n| `claude_code.token.usage` | Break down by `type` (input/output), user, team, or model |\n| `claude_code.session.count` | Track adoption and engagement over time |\n| `claude_code.lines_of_code.count` | Measure productivity by tracking code additions/removals |\n| `claude_code.commit.count` & `claude_code.pull_request.count` | Understand impact on development workflows |\n\n### Cost Monitoring\n\nThe `claude_code.cost.usage` metric helps with:\n\n* Tracking usage trends across teams or individuals\n* Identifying high-usage sessions for optimization\n\n\n Cost metrics are approximations. For official billing data, refer to your API provider (Anthropic Console, AWS Bedrock, or Google Cloud Vertex).\n\n\n### Alerting and Segmentation\n\nCommon alerts to consider:\n\n* Cost spikes\n* Unusual token consumption\n* High session volume from specific users\n\nAll metrics can be segmented by `user.account_uuid`, `organization.id`, `session.id`, `model`, and `app.version`.\n\n### Event Analysis\n\nThe event data provides detailed insights into Claude Code interactions:\n\n**Tool Usage Patterns**: Analyze tool result events to identify:\n\n* Most frequently used tools\n* Tool success rates\n* Average tool execution times\n* Error patterns by tool type\n\n**Performance Monitoring**: Track API request durations and tool execution times to identify performance bottlenecks.\n\n## Backend Considerations\n\nYour choice of metrics and logs backends will determine the types of analyses you can perform:\n\n### For Metrics:\n\n* **Time series databases (e.g., Prometheus)**: Rate calculations, aggregated metrics\n* **Columnar stores (e.g., ClickHouse)**: Complex queries, unique user analysis\n* **Full-featured observability platforms (e.g., Honeycomb, Datadog)**: Advanced querying, visualization, alerting\n\n### For Events/Logs:\n\n* **Log aggregation systems (e.g., Elasticsearch, Loki)**: Full-text search, log analysis\n* **Columnar stores (e.g., ClickHouse)**: Structured event analysis\n* **Full-featured observability platforms (e.g., Honeycomb, Datadog)**: Correlation between metrics and events\n\nFor organizations requiring Daily/Weekly/Monthly Active User (DAU/WAU/MAU) metrics, consider backends that support efficient unique value queries.\n\n## Service Information\n\nAll metrics are exported with:\n\n* Service Name: `claude-code`\n* Service Version: Current Claude Code version\n* Meter Name: `com.anthropic.claude_code`\n\n## Security/Privacy Considerations\n\n* Telemetry is opt-in and requires explicit configuration\n* Sensitive information like API keys or file contents are never included in metrics or events\n* User prompt content is redacted by default - only prompt length is recorded. To enable user prompt logging, set `OTEL_LOG_USER_PROMPTS=1`\n"
},
{
"path": "CONTRIBUTING.md",
"content": "# Contributing to Claude Code Observability Stack\n\nThank you for your interest in contributing to the Claude Code Observability Stack! This project helps developers monitor and analyze their Claude Code usage through comprehensive dashboards and metrics.\n\n## ๐ Getting Started\n\n### Prerequisites\n- Docker and Docker Compose\n- Basic understanding of OpenTelemetry, Prometheus, and Grafana\n- Familiarity with Claude Code and its telemetry features\n\n### Development Setup\n1. Fork the repository\n2. Clone your fork: `git clone https://github.com/your-username/claude-code-otel.git`\n - Original repository: `git clone https://github.com/ColeMurray/claude-code-otel.git`\n3. Start the development stack: `make up`\n4. Access Grafana at http://localhost:3000 (admin/admin)\n\n## ๐ How to Contribute\n\n### Dashboard Improvements\n- Add new panels for additional Claude Code metrics\n- Improve existing visualizations for better insights\n- Optimize queries for better performance\n- Enhance color schemes and layouts\n\n### Configuration Enhancements\n- Improve OpenTelemetry collector configurations\n- Add new Prometheus recording rules\n- Optimize data retention and storage\n\n### Documentation\n- Improve setup instructions\n- Add troubleshooting guides\n- Create usage examples\n- Update metric documentation\n\n## ๐ Project Structure\n\n```\nโโโ claude-code-dashboard.json # Main Grafana dashboard\nโโโ collector-config.yaml # OpenTelemetry collector config\nโโโ docker-compose.yml # Main stack configuration\nโโโ prometheus.yml # Prometheus configuration\nโโโ grafana-*.yml # Grafana configuration files\nโโโ Makefile # Management commands\nโโโ README.md # Project documentation\n```\n\n## ๐ Contribution Guidelines\n\n### Code Quality\n1. **Follow OpenTelemetry Standards**: Use proper metric naming conventions as defined in the Claude Code documentation\n2. **Test Your Changes**: Verify that new configurations work with actual Claude Code telemetry data\n3. **Documentation**: Update relevant documentation for any changes\n4. **Security**: Never commit sensitive information (API keys, credentials, etc.)\n\n### Dashboard Guidelines\n1. **Consistent Styling**: Follow the existing color schemes and layout patterns\n2. **Performance**: Optimize queries to avoid excessive resource usage\n3. **Accessibility**: Use clear labels and legends for all visualizations\n4. **Mobile-Friendly**: Ensure dashboards work well on different screen sizes\n\n### Commit Messages\nUse clear, descriptive commit messages:\n- `feat: add API request count panel to cost analysis`\n- `fix: correct token usage query in performance dashboard`\n- `docs: update setup instructions for macOS`\n- `refactor: optimize Prometheus queries for better performance`\n\n## ๐งช Testing Your Changes\n\n### Before Submitting a PR\n1. **Start the Stack**: `make up`\n2. **Generate Test Data**: Use Claude Code with telemetry enabled\n3. **Verify Dashboards**: Check that all panels display data correctly\n4. **Test Configuration**: Ensure `make validate-config` passes\n5. **Check Documentation**: Verify all links and instructions work\n\n### Configuration Validation\n```bash\n# Validate all configurations\nmake validate-config\n\n# Test individual components\ndocker compose config # Validate docker-compose.yml\ncurl -f http://localhost:9090/-/healthy # Test Prometheus\ncurl -f http://localhost:3000/api/health # Test Grafana\n```\n\n## ๐ Types of Contributions\n\n### Dashboard Enhancements\n- **New Metric Panels**: Add visualizations for additional Claude Code metrics\n- **Improved Layouts**: Better organization of dashboard sections\n- **Enhanced Queries**: More efficient or insightful Prometheus/LogQL queries\n- **Visual Improvements**: Better color schemes, legends, and formatting\n\n### Infrastructure Improvements\n- **Performance Optimization**: Faster queries and reduced resource usage\n- **Configuration Management**: Better default configurations\n- **Documentation**: Clearer setup and troubleshooting guides\n- **Compatibility**: Support for different environments and setups\n\n### Feature Additions\n- **New Metrics**: Support for additional Claude Code telemetry data\n- **Export Options**: Additional data export formats or integrations\n- **Monitoring Enhancements**: Better health checks and status monitoring\n\n## ๐ Reporting Issues\n\nWhen reporting issues, please include:\n1. **Environment Details**: OS, Docker version, Claude Code version\n2. **Steps to Reproduce**: Clear instructions to replicate the issue\n3. **Expected vs Actual**: What you expected to happen vs what actually happened\n4. **Logs**: Relevant logs from the observability stack\n5. **Configuration**: Any custom configurations you're using\n\n## ๐ Resources\n\n- [Claude Code Observability Documentation](CLAUDE_OBSERVABILITY.md)\n- [OpenTelemetry Documentation](https://opentelemetry.io/docs/)\n- [Prometheus Query Language](https://prometheus.io/docs/prometheus/latest/querying/)\n- [Grafana Dashboard Best Practices](https://grafana.com/docs/grafana/latest/best-practices/)\n- [LogQL Documentation](https://grafana.com/docs/loki/latest/logql/)\n\n## ๐ฌ Getting Help\n\n- **Issues**: Open a GitHub issue for bugs or feature requests\n- **Discussions**: Use GitHub Discussions for questions and ideas\n- **Documentation**: Check the README and CLAUDE_OBSERVABILITY.md first\n\n## ๐ Pull Request Process\n\n1. **Fork & Branch**: Create a feature branch from `main`\n2. **Develop**: Make your changes following the guidelines above\n3. **Test**: Verify your changes work as expected\n4. **Document**: Update documentation if needed\n5. **Submit**: Create a pull request with a clear description\n\n### PR Description Template\n```markdown\n## Summary\nBrief description of changes\n\n## Type of Change\n- [ ] Dashboard improvement\n- [ ] Configuration enhancement\n- [ ] Documentation update\n- [ ] Bug fix\n- [ ] New feature\n\n## Testing\n- [ ] Tested with actual Claude Code telemetry data\n- [ ] Verified all dashboard panels work correctly\n- [ ] Configuration validation passes\n- [ ] Documentation is accurate\n\n## Screenshots (if applicable)\nInclude screenshots of dashboard changes\n```\n\nThank you for contributing to the Claude Code Observability Stack! ๐ "
},
{
"path": "LICENSE",
"content": "MIT License\n\nCopyright (c) 2024 Claude Code Observability Stack\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE. "
},
{
"path": "Makefile",
"content": "# Claude Code Observability Stack\n.PHONY: help up down logs restart clean validate-config\n\nhelp: ## Show this help message\n\t@echo \"Claude Code Observability Stack\"\n\t@echo \"================================\"\n\t@echo \"\"\n\t@echo \"Available commands:\"\n\t@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = \":.*?## \"}; {printf \"\\033[36m%-20s\\033[0m %s\\n\", $$1, $$2}'\n\nup: ## Start the observability stack\n\t@echo \"๐ Starting Claude Code observability stack...\"\n\tdocker compose up -d\n\t@echo \"โ
Stack started!\"\n\t@echo \"๐ Grafana: http://localhost:3000 (admin/admin)\"\n\t@echo \"๐ Prometheus: http://localhost:9090\"\n\t@echo \"๐ Loki: http://localhost:3100\"\n\n\ndown: ## Stop the observability stack\n\t@echo \"๐ Stopping Claude Code observability stack...\"\n\tdocker compose down\n\t@echo \"โ
Stack stopped!\"\n\nrestart: ## Restart the observability stack\n\t@echo \"๐ Restarting Claude Code observability stack...\"\n\tdocker compose restart\n\t@echo \"โ
Stack restarted!\"\n\nlogs: ## Show logs from all services\n\tdocker compose logs -f\n\nlogs-collector: ## Show OpenTelemetry collector logs\n\tdocker compose logs -f otel-collector\n\nlogs-prometheus: ## Show Prometheus logs\n\tdocker compose logs -f prometheus\n\nlogs-grafana: ## Show Grafana logs\n\tdocker compose logs -f grafana\n\nclean: ## Clean up containers and volumes\n\t@echo \"๐งน Cleaning up...\"\n\tdocker compose down -v\n\tdocker system prune -f\n\t@echo \"โ
Cleanup complete!\"\n\n\n\n\n\nvalidate-config: ## Validate all configuration files\n\t@echo \"โ
Validating configurations...\"\n\t@echo \"๐ Checking docker compose.yml...\"\n\tdocker compose config > /dev/null && echo \"โ
docker compose.yml is valid\"\n\t@echo \"๐ Checking collector-config.yaml...\"\n\t@if command -v otelcol-contrib >/dev/null 2>&1; then \\\n\t\totelcol-contrib --config-validate --config=collector-config.yaml; \\\n\telse \\\n\t\techo \"โน๏ธ Install otelcol-contrib to validate collector config\"; \\\n\tfi\n\n\nstatus: ## Show stack status\n\t@echo \"๐ Claude Code Observability Stack Status\"\n\t@echo \"===========================================\"\n\t@docker compose ps\n\t@echo \"\"\n\t@echo \"๐ Service URLs:\"\n\t@echo \" Grafana: http://localhost:3000\"\n\t@echo \" Prometheus: http://localhost:9090\"\n\t@echo \" Loki: http://localhost:3100\"\n\n\t@echo \" Collector: http://localhost:4317 (gRPC), http://localhost:4318 (HTTP)\"\n\nsetup-claude: ## Display Claude Code telemetry setup instructions\n\t@echo \"๐ค Claude Code Telemetry Setup\"\n\t@echo \"===============================\"\n\t@echo \"\"\n\t@echo \"To enable telemetry in Claude Code, set these environment variables:\"\n\t@echo \"\"\n\t@echo \"export CLAUDE_CODE_ENABLE_TELEMETRY=1\"\n\t@echo \"export OTEL_METRICS_EXPORTER=otlp\"\n\t@echo \"export OTEL_LOGS_EXPORTER=otlp\"\n\t@echo \"export OTEL_EXPORTER_OTLP_PROTOCOL=grpc\"\n\t@echo \"export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317\"\n\t@echo \"\"\n\t@echo \"For debugging (faster export intervals):\"\n\t@echo \"export OTEL_METRIC_EXPORT_INTERVAL=10000\"\n\t@echo \"export OTEL_LOGS_EXPORT_INTERVAL=5000\"\n\t@echo \"\"\n\t@echo \"Then run: claude\"\n\ndemo-metrics: ## Generate demo metrics for testing\n\t@echo \"๐ฏ This would generate demo metrics if Claude Code was running\"\n\t@echo \"๐ก To see real metrics, ensure Claude Code is configured with telemetry enabled\"\n\t@echo \"๐ Run 'make setup-claude' for setup instructions\" "
},
{
"path": "README.md",
"content": "# Claude Code Observability Stack\n\n[](https://github.com/ColeMurray/claude-code-otel)\n[](LICENSE)\n[](docker-compose.yml)\n\nA comprehensive observability solution for monitoring Claude Code usage, performance, and costs. This setup implements the recommendations from the [Claude Code Observability Documentation](CLAUDE_OBSERVABILITY.md) to provide deep insights into AI-assisted development workflows.\n\n## ๐ธ Dashboard Screenshots\n\n### ๐ฐ Cost & Usage Analysis\nTrack spending across different Claude models with detailed breakdowns of costs, API requests, and token usage patterns.\n\n![\"Cost](\"docs/images/cost-usage-analytics.png\")
\n\n*Features: Model cost comparison, API request tracking, token usage breakdown by type*\n\n### ๐ User Activity & Productivity \nMonitor development productivity with comprehensive session analytics, tool usage patterns, and code change metrics.\n\n![\"User](\"docs/images/user-activity.png\")
\n\n*Features: Session tracking, tool performance metrics, code productivity insights*\n\n## ๐ฏ Features\n\n### ๐ **Comprehensive Monitoring**\n- **Cost Analysis**: Track usage costs by model, user, and time periods\n- **User Analytics**: Daily/Weekly/Monthly Active Users (DAU/WAU/MAU)\n- **Tool Usage**: Monitor which Claude Code tools are used most frequently\n- **Performance Metrics**: API latency, success rates, and bottleneck identification\n- **Productivity Insights**: Lines of code changes, commits, and pull requests\n\n### ๐ **Enhanced Analytics**\n- **API Request Tracking**: Monitor actual request counts by model version\n- **Token Efficiency**: Track cost-per-token across different models\n- **Session Analytics**: Comprehensive session and productivity tracking\n- **Real-time Monitoring**: Live dashboards with 30-second refresh rates\n\n### ๐ **Rich Dashboards**\n- **Executive Overview**: High-level KPIs and trends\n- **Cost Management**: Detailed cost breakdowns and projections\n- **Tool Performance**: Success rates and execution times\n- **User Activity**: Productivity and engagement metrics\n- **Error Analysis**: Comprehensive error tracking and investigation\n\n## ๐๏ธ Architecture\n\n```\nClaude Code โ OpenTelemetry Collector โ Prometheus (metrics) + Loki (events/logs)\n โ\n Grafana (visualization & analysis)\n```\n\n### Components\n\n| Service | Purpose | Port | UI |\n|---------|---------|------|----| \n| **OpenTelemetry Collector** | Metrics/logs ingestion | 4317 (gRPC), 4318 (HTTP) | - |\n| **Prometheus** | Metrics storage & querying | 9090 | http://localhost:9090 |\n| **Loki** | Log aggregation & storage | 3100 | - |\n| **Grafana** | Dashboards & visualization | 3000 | http://localhost:3000 |\n\n## ๐ Quick Start\n\n### 1. Start the Stack\n```bash\n# Start all services\nmake up\n\n# Check status\nmake status\n```\n\n### 2. Configure Claude Code\n```bash\n# Enable telemetry\nexport CLAUDE_CODE_ENABLE_TELEMETRY=1\n\n# Configure exporters\nexport OTEL_METRICS_EXPORTER=otlp\nexport OTEL_LOGS_EXPORTER=otlp\nexport OTEL_EXPORTER_OTLP_PROTOCOL=grpc\nexport OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317\n\n# For debugging (faster export intervals)\nexport OTEL_METRIC_EXPORT_INTERVAL=10000\nexport OTEL_LOGS_EXPORT_INTERVAL=5000\n\n# Run Claude Code\nclaude\n```\n\n### 3. Access Dashboards\n- **Grafana**: http://localhost:3000 (admin/admin)\n- **Prometheus**: http://localhost:9090\n\n> ๐ผ๏ธ **Visual Guide**: Check out the [Dashboard Screenshots](#-dashboard-screenshots) to see what your dashboards will look like!\n\n## ๐ Available Metrics\n\nBased on the [Claude Code Observability Documentation](CLAUDE_OBSERVABILITY.md), this stack monitors:\n\n### Core Metrics\n- `claude_code.session.count` - CLI sessions started\n- `claude_code.lines_of_code.count` - Lines of code modified (added/removed)\n- `claude_code.pull_request.count` - Pull requests created\n- `claude_code.commit.count` - Git commits created\n- `claude_code.cost.usage` - Cost of sessions by model\n- `claude_code.token.usage` - Token usage (input/output/cache/creation)\n- `claude_code.code_edit_tool.decision` - Tool permission decisions\n\n### Event Data\n- `claude_code.user_prompt` - User prompt submissions\n- `claude_code.tool_result` - Tool execution results and timings\n- `claude_code.api_request` - API requests with duration and tokens\n- `claude_code.api_error` - API errors with status codes\n- `claude_code.tool_decision` - Tool permission decisions\n\n## ๐ Usage Analysis\n\n### Real-time Dashboard Analysis\n\nAccess comprehensive analytics through the Grafana dashboard at http://localhost:3000:\n\n- **Cost Analysis**: Real-time cost tracking with model breakdowns\n- **Request Monitoring**: API request counts and patterns by model\n- **Token Efficiency**: Track token usage and cost-per-token metrics\n- **Tool Performance**: Success rates and execution time analysis\n- **Session Analytics**: User activity and productivity insights\n\n### Key Metrics Available\n- Total and per-model costs with trending\n- API request counts independent of cost variations\n- Token usage breakdown (input/output/cache/creation)\n- Tool usage patterns and success rates\n- Session activity and code productivity metrics\n\n## ๐ Key Dashboard Features\n\n> ๐ก **See [Dashboard Screenshots](#-dashboard-screenshots) above for visual examples**\n\n### ๐ฐ Cost & Usage Analysis\n- **Cost by Model**: Track spending across different Claude models\n- **API Request Tracking**: Monitor actual request counts by model version \n- **Token Usage Breakdown**: Detailed analysis by token type (input/output/cache)\n\n### ๐ง Tool Performance\n- **Usage Patterns**: Most frequently used Claude Code tools\n- **Success Rates**: Tool execution success percentages\n- **Performance Metrics**: Average execution times and bottleneck identification\n\n### โก Real-time Monitoring\n- **Live Metrics**: 30-second refresh rate for current activity\n- **Session Tracking**: Active sessions and productivity metrics\n- **Error Analysis**: API errors and troubleshooting information\n\n## ๐ Dashboard Sections\n\nThe Grafana dashboard is organized into sections reflecting the observability documentation recommendations:\n\n### ๐ Overview\n- Active sessions, cost, token usage, lines of code changed\n\n### ๐ฐ Cost & Usage Analysis \n- Cost trends by model, token usage breakdown\n- **NEW**: API request count tracking by model version\n- Implements cost monitoring recommendations\n\n### ๐ง Tool Usage & Performance\n- Tool frequency and success rates\n- Performance bottleneck identification\n\n### โก Performance & Errors\n- API latency by model, error rate tracking\n- Performance monitoring as recommended\n\n### ๐ User Activity & Productivity\n- Code changes, commits, pull requests\n- Productivity measurement insights\n\n### ๐ Event Logs\n- Real-time tool execution events and API errors\n- Structured log analysis for troubleshooting\n\n## ๐ง Advanced Configuration\n\n### Environment Variables\n\nKey configuration options (see [CLAUDE_OBSERVABILITY.md](CLAUDE_OBSERVABILITY.md) for complete reference):\n\n```bash\n# Core telemetry\nCLAUDE_CODE_ENABLE_TELEMETRY=1\n\n# Exporter configuration\nOTEL_METRICS_EXPORTER=otlp,prometheus # Multiple exporters\nOTEL_LOGS_EXPORTER=otlp\n\n# Protocol and endpoints\nOTEL_EXPORTER_OTLP_PROTOCOL=grpc\nOTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317\nOTEL_EXPORTER_OTLP_HEADERS=\"Authorization=Bearer token\"\n\n# Export intervals\nOTEL_METRIC_EXPORT_INTERVAL=60000 # 1 minute (production)\nOTEL_LOGS_EXPORT_INTERVAL=5000 # 5 seconds\n\n# Privacy controls\nOTEL_LOG_USER_PROMPTS=1 # Enable prompt content logging\n\n# Cardinality control\nOTEL_METRICS_INCLUDE_SESSION_ID=true\nOTEL_METRICS_INCLUDE_VERSION=false\nOTEL_METRICS_INCLUDE_ACCOUNT_UUID=true\n```\n\n### Collector Configuration\n\nThe OpenTelemetry collector is configured with:\n- **Processors**: Resource enrichment and event filtering\n- **Multiple Pipelines**: Separate routing for metrics and different event types\n- **Metric Relabeling**: Cardinality control for better performance\n\n### Backend Considerations\n\nFollowing the documentation recommendations:\n\n- **Metrics Backend**: Prometheus (time series) + optional columnar stores\n- **Events Backend**: Loki (log aggregation) with JSON parsing\n- **Cardinality Management**: Configurable attribute inclusion\n- **Retention**: Configure based on your analysis needs\n\n## ๐ ๏ธ Management Commands\n\n```bash\n# Stack management\nmake up # Start all services\nmake down # Stop all services \nmake restart # Restart services\nmake clean # Clean up containers and volumes\n\n# Monitoring\nmake logs # View all logs\nmake logs-collector # View collector logs only\nmake status # Show service status\n\n# Validation\nmake validate-config # Validate all configs\nmake setup-claude # Show Claude Code setup instructions\n```\n\n## ๐ฏ Use Cases\n\n### For Engineering Teams\n- **Cost Management**: Track AI assistance costs by team/project\n- **Productivity Measurement**: Quantify development velocity improvements\n- **Tool Adoption**: Understand which Claude Code features drive value\n- **Performance Optimization**: Identify and resolve usage bottlenecks\n\n### For Platform Teams\n- **Capacity Planning**: Predict infrastructure needs based on usage growth\n- **SLA Monitoring**: Track API performance and availability\n- **Security**: Monitor unusual usage patterns\n- **Resource Optimization**: Optimize token usage and reduce costs\n\n### For Management\n- **ROI Analysis**: Measure productivity gains from AI assistance\n- **Usage Insights**: Understand adoption patterns across teams\n- **Cost Control**: Monitor and optimize AI assistance spending\n- **Strategic Planning**: Data-driven decisions on AI tool investments\n\n## ๐ Security & Privacy\n\n- **User Privacy**: Prompt content logging is disabled by default\n- **Data Isolation**: All data stays within your infrastructure\n- **Access Control**: Configure Grafana authentication as needed\n- **Audit Trail**: Complete logging of all tool usage and decisions\n\n## ๐ Resources\n\n- [Claude Code Observability Documentation](CLAUDE_OBSERVABILITY.md) - Complete reference\n- [OpenTelemetry Documentation](https://opentelemetry.io/docs/) - OTel specification\n- [Prometheus Documentation](https://prometheus.io/docs/) - Metrics and alerting\n- [Grafana Documentation](https://grafana.com/docs/) - Dashboards and visualization\n- [Loki Documentation](https://grafana.com/docs/loki/) - Log aggregation\n\n## ๐ค Contributing\n\nThis observability stack implements the patterns and recommendations from the official Claude Code documentation. To contribute:\n\n1. Follow the metric naming conventions in the documentation\n2. Update dashboards to reflect new data sources and metrics\n3. Test configurations before submitting changes\n4. Ensure all sensitive information is excluded from commits\n5. Update documentation for any new features or configuration changes\n\n## ๐ License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## ๐ Acknowledgments\n\n- Built following the [Claude Code Observability Documentation](CLAUDE_OBSERVABILITY.md)\n- Uses OpenTelemetry standards for metrics and events\n- Implements industry best practices for observability stack architecture "
},
{
"path": "claude-code-dashboard.json",
"content": "{\n \"annotations\": {\n \"list\": [\n {\n \"builtIn\": 1,\n \"datasource\": {\n \"type\": \"grafana\",\n \"uid\": \"-- Grafana --\"\n },\n \"enable\": true,\n \"hide\": true,\n \"iconColor\": \"rgba(0, 211, 255, 1)\",\n \"name\": \"Annotations & Alerts\",\n \"type\": \"dashboard\"\n }\n ]\n },\n \"description\": \"Claude Code Observability Dashboard - Monitor usage, costs, and performance\",\n \"editable\": true,\n \"fiscalYearStartMonth\": 0,\n \"graphTooltip\": 0,\n \"id\": null,\n \"links\": [],\n \"liveNow\": false,\n \"panels\": [\n {\n \"title\": \"๐ Overview\",\n \"type\": \"row\",\n \"gridPos\": {\"h\": 1, \"w\": 24, \"x\": 0, \"y\": 0},\n \"collapsed\": false\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"yellow\",\n \"value\": 10\n },\n {\n \"color\": \"red\",\n \"value\": 50\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 4,\n \"w\": 6,\n \"x\": 0,\n \"y\": 1\n },\n \"id\": 1,\n \"options\": {\n \"colorMode\": \"background\",\n \"graphMode\": \"area\",\n \"justifyMode\": \"center\",\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"values\": false,\n \"calcs\": [\"lastNotNull\"],\n \"fields\": \"\"\n },\n \"textMode\": \"auto\"\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum(increase(claude_code_session_count_total{job=\\\"otel-collector\\\"}[1h]))\",\n \"interval\": \"\",\n \"legendFormat\": \"Sessions (1h)\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Active Sessions (1h)\",\n \"type\": \"stat\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"yellow\",\n \"value\": 5\n },\n {\n \"color\": \"red\",\n \"value\": 20\n }\n ]\n },\n \"unit\": \"currencyUSD\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 4,\n \"w\": 6,\n \"x\": 6,\n \"y\": 1\n },\n \"id\": 2,\n \"options\": {\n \"colorMode\": \"background\",\n \"graphMode\": \"area\",\n \"justifyMode\": \"center\",\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"values\": false,\n \"calcs\": [\"lastNotNull\"],\n \"fields\": \"\"\n },\n \"textMode\": \"auto\"\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum(increase(claude_code_cost_usage_USD_total{job=\\\"otel-collector\\\"}[1h]))\",\n \"interval\": \"\",\n \"legendFormat\": \"Cost (1h)\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Cost (Last Hour)\",\n \"type\": \"stat\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"yellow\",\n \"value\": 10000\n },\n {\n \"color\": \"red\",\n \"value\": 50000\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 4,\n \"w\": 6,\n \"x\": 12,\n \"y\": 1\n },\n \"id\": 3,\n \"options\": {\n \"colorMode\": \"background\",\n \"graphMode\": \"area\",\n \"justifyMode\": \"center\",\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"values\": false,\n \"calcs\": [\"lastNotNull\"],\n \"fields\": \"\"\n },\n \"textMode\": \"auto\"\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum(increase(claude_code_token_usage_tokens_total{job=\\\"otel-collector\\\"}[1h]))\",\n \"interval\": \"\",\n \"legendFormat\": \"Tokens (1h)\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Token Usage (1h)\",\n \"type\": \"stat\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"thresholds\"\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"yellow\",\n \"value\": 100\n },\n {\n \"color\": \"red\",\n \"value\": 500\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 4,\n \"w\": 6,\n \"x\": 18,\n \"y\": 1\n },\n \"id\": 4,\n \"options\": {\n \"colorMode\": \"background\",\n \"graphMode\": \"area\",\n \"justifyMode\": \"center\",\n \"orientation\": \"auto\",\n \"reduceOptions\": {\n \"values\": false,\n \"calcs\": [\"lastNotNull\"],\n \"fields\": \"\"\n },\n \"textMode\": \"auto\"\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum(increase(claude_code_lines_of_code_count_total{job=\\\"otel-collector\\\"}[1h]))\",\n \"interval\": \"\",\n \"legendFormat\": \"Lines Changed (1h)\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Lines of Code (1h)\",\n \"type\": \"stat\"\n },\n {\n \"title\": \"๐ฐ Cost & Usage Analysis\",\n \"type\": \"row\",\n \"gridPos\": {\"h\": 1, \"w\": 24, \"x\": 0, \"y\": 5},\n \"collapsed\": false\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"USD\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 2,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"currencyUSD\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 6\n },\n \"id\": 5,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"max\", \"mean\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum by (model) (increase(claude_code_cost_usage_USD_total{job=\\\"otel-collector\\\"}[1h]))\",\n \"interval\": \"\",\n \"legendFormat\": \"{{model}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Cost by Model (Hourly)\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"Tokens (log scale)\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 2,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"log\",\n \"log\": 10\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": [\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"cacheRead\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"blue\"\n }\n },\n {\n \"id\": \"custom.lineWidth\",\n \"value\": 3\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"output\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"green\"\n }\n },\n {\n \"id\": \"custom.lineWidth\",\n \"value\": 3\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"input\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"orange\"\n }\n },\n {\n \"id\": \"custom.lineWidth\",\n \"value\": 2\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"cacheCreation\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"purple\"\n }\n },\n {\n \"id\": \"custom.lineWidth\",\n \"value\": 2\n }\n ]\n }\n ]\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 6\n },\n \"id\": 6,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"max\", \"mean\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum by (type) (rate(claude_code_token_usage_tokens_total{job=\\\"otel-collector\\\"}[5m]) * 60)\",\n \"interval\": \"\",\n \"legendFormat\": \"{{type}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Token Usage Rate by Type\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"Requests\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 2,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": [\n {\n \"matcher\": {\n \"id\": \"byRegexp\",\n \"options\": \".*sonnet.*\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"purple\"\n }\n },\n {\n \"id\": \"custom.lineWidth\",\n \"value\": 3\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byRegexp\",\n \"options\": \".*haiku.*\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"blue\"\n }\n },\n {\n \"id\": \"custom.lineWidth\",\n \"value\": 3\n }\n ]\n }\n ]\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 24,\n \"x\": 0,\n \"y\": 14\n },\n \"id\": 15,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"lastNotNull\", \"max\", \"mean\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum by (model) (changes(claude_code_cost_usage_USD_total{job=\\\"otel-collector\\\"}[5m]))\",\n \"interval\": \"\",\n \"legendFormat\": \"{{model}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"API Requests by Model (5min rate)\",\n \"type\": \"timeseries\"\n },\n {\n \"title\": \"๐ง Tool Usage & Performance\",\n \"type\": \"row\",\n \"gridPos\": {\"h\": 1, \"w\": 24, \"x\": 0, \"y\": 22},\n \"collapsed\": false\n },\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"Tool Usage Rate (per 5min)\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"stepAfter\",\n \"lineWidth\": 2,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"auto\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"min\": 0,\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"yellow\",\n \"value\": 5\n },\n {\n \"color\": \"red\",\n \"value\": 10\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": [\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Bash\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"red\"\n }\n },\n {\n \"id\": \"custom.lineWidth\",\n \"value\": 3\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Read\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"blue\"\n }\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Write\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"green\"\n }\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Grep\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"orange\"\n }\n }\n ]\n }\n ]\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 23\n },\n \"id\": 7,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"lastNotNull\", \"mean\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\",\n \"showLegend\": true\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"expr\": \"sum by (tool_name) (count_over_time({service_name=\\\"claude-code\\\"} |= \\\"claude_code.tool_result\\\" [5m]))\",\n \"interval\": \"\",\n \"legendFormat\": \"{{tool_name}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Tool Usage Rate Over Time\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"Total Tool Count\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 0,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"stepAfter\",\n \"lineWidth\": 2,\n \"pointSize\": 4,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"min\": 0,\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": [\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Bash\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"red\"\n }\n },\n {\n \"id\": \"custom.lineWidth\",\n \"value\": 3\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Read\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"blue\"\n }\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Write\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"green\"\n }\n }\n ]\n },\n {\n \"matcher\": {\n \"id\": \"byName\",\n \"options\": \"Grep\"\n },\n \"properties\": [\n {\n \"id\": \"color\",\n \"value\": {\n \"mode\": \"fixed\",\n \"fixedColor\": \"orange\"\n }\n }\n ]\n }\n ]\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 23\n },\n \"id\": 14,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"lastNotNull\", \"max\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\",\n \"showLegend\": true\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"expr\": \"sum by (tool_name) (count_over_time({service_name=\\\"claude-code\\\"} |= \\\"claude_code.tool_result\\\" [$__range]))\",\n \"interval\": \"\",\n \"legendFormat\": \"{{tool_name}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Cumulative Tool Usage\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"Success Rate %\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 2,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"auto\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"line\"\n }\n },\n \"mappings\": [],\n \"max\": 100,\n \"min\": 0,\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"red\",\n \"value\": 0\n },\n {\n \"color\": \"yellow\",\n \"value\": 80\n },\n {\n \"color\": \"green\",\n \"value\": 95\n }\n ]\n },\n \"unit\": \"percent\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 23\n },\n \"id\": 8,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"last\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"expr\": \"100 * (sum by (tool_name) (count_over_time({service_name=\\\"claude-code\\\"} |= \\\"claude_code.tool_result\\\" | json | success=\\\"true\\\" [15m]))) / (sum by (tool_name) (count_over_time({service_name=\\\"claude-code\\\"} |= \\\"claude_code.tool_result\\\" [15m])))\",\n \"interval\": \"\",\n \"legendFormat\": \"{{tool_name}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Tool Success Rate\",\n \"type\": \"timeseries\"\n },\n {\n \"title\": \"โก Performance & Errors\",\n \"type\": \"row\",\n \"gridPos\": {\"h\": 1, \"w\": 24, \"x\": 0, \"y\": 39},\n \"collapsed\": false\n },\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"Milliseconds\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 2,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"none\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"ms\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 40\n },\n \"id\": 9,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"mean\", \"max\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"expr\": \"avg by (model) (avg_over_time({service_name=\\\"claude-code\\\"} |= \\\"claude_code.api_request\\\" | unwrap duration_ms [$__interval]))\",\n \"interval\": \"\",\n \"legendFormat\": \"{{model}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"API Request Duration by Model\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"Errors/min\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"bars\",\n \"fillOpacity\": 80,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"normal\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 40\n },\n \"id\": 10,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"sum\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"expr\": \"sum by (status_code) (rate({service_name=\\\"claude-code\\\"} |= \\\"claude_code.api_error\\\" | json | __error__ = \\\"\\\" [$__interval]))\",\n \"interval\": \"\",\n \"legendFormat\": \"HTTP {{status_code}}\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"API Error Rate\",\n \"type\": \"timeseries\"\n },\n {\n \"title\": \"๐ User Activity & Productivity\",\n \"type\": \"row\",\n \"gridPos\": {\"h\": 1, \"w\": 24, \"x\": 0, \"y\": 48},\n \"collapsed\": false\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"line\",\n \"fillOpacity\": 10,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 2,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"normal\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 49\n },\n \"id\": 11,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"sum\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum by (type) (rate(claude_code_lines_of_code_count_total{job=\\\"otel-collector\\\"}[5m]) * 60)\",\n \"interval\": \"\",\n \"legendFormat\": \"{{type}} lines/min\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Code Changes Rate\",\n \"type\": \"timeseries\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"fieldConfig\": {\n \"defaults\": {\n \"color\": {\n \"mode\": \"palette-classic\"\n },\n \"custom\": {\n \"axisLabel\": \"\",\n \"axisPlacement\": \"auto\",\n \"barAlignment\": 0,\n \"drawStyle\": \"bars\",\n \"fillOpacity\": 80,\n \"gradientMode\": \"none\",\n \"hideFrom\": {\n \"legend\": false,\n \"tooltip\": false,\n \"vis\": false\n },\n \"lineInterpolation\": \"linear\",\n \"lineWidth\": 1,\n \"pointSize\": 5,\n \"scaleDistribution\": {\n \"type\": \"linear\"\n },\n \"showPoints\": \"never\",\n \"spanNulls\": false,\n \"stacking\": {\n \"group\": \"A\",\n \"mode\": \"normal\"\n },\n \"thresholdsStyle\": {\n \"mode\": \"off\"\n }\n },\n \"mappings\": [],\n \"thresholds\": {\n \"mode\": \"absolute\",\n \"steps\": [\n {\n \"color\": \"green\",\n \"value\": null\n },\n {\n \"color\": \"red\",\n \"value\": 80\n }\n ]\n },\n \"unit\": \"short\"\n },\n \"overrides\": []\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 49\n },\n \"id\": 12,\n \"options\": {\n \"legend\": {\n \"calcs\": [\"sum\"],\n \"displayMode\": \"table\",\n \"placement\": \"bottom\"\n },\n \"tooltip\": {\n \"mode\": \"multi\",\n \"sort\": \"desc\"\n }\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum(increase(claude_code_commit_count_total{job=\\\"otel-collector\\\"}[1h])) or vector(0)\",\n \"interval\": \"\",\n \"legendFormat\": \"Commits\",\n \"refId\": \"A\"\n },\n {\n \"datasource\": {\n \"type\": \"prometheus\",\n \"uid\": \"prometheus\"\n },\n \"expr\": \"sum(increase(claude_code_pull_request_count_total{job=\\\"otel-collector\\\"}[1h])) or vector(0)\",\n \"interval\": \"\",\n \"legendFormat\": \"Pull Requests\",\n \"refId\": \"B\"\n }\n ],\n \"title\": \"Development Activity (Hourly)\",\n \"type\": \"timeseries\"\n },\n {\n \"title\": \"๐ Event Logs\",\n \"type\": \"row\",\n \"gridPos\": {\"h\": 1, \"w\": 24, \"x\": 0, \"y\": 57},\n \"collapsed\": false\n },\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 0,\n \"y\": 58\n },\n \"id\": 13,\n \"options\": {\n \"dedupStrategy\": \"none\",\n \"enableLogDetails\": true,\n \"prettifyLogMessage\": false,\n \"showCommonLabels\": false,\n \"showLabels\": false,\n \"showTime\": true,\n \"sortOrder\": \"Descending\",\n \"wrapLogMessage\": false\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"expr\": \"{service_name=\\\"claude-code\\\"} |= \\\"claude_code.tool_result\\\" | line_format \\\"{{.event_timestamp}} [{{.tool_name}}] {{if eq .success \\\\\\\"true\\\\\\\"}}โ
{{else}}โ{{end}} {{.duration_ms}}ms {{if .error}}ERROR: {{.error}}{{end}}\\\"\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"Tool Execution Events\",\n \"type\": \"logs\"\n },\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"gridPos\": {\n \"h\": 8,\n \"w\": 12,\n \"x\": 12,\n \"y\": 58\n },\n \"id\": 17,\n \"options\": {\n \"dedupStrategy\": \"none\",\n \"enableLogDetails\": true,\n \"prettifyLogMessage\": false,\n \"showCommonLabels\": false,\n \"showLabels\": false,\n \"showTime\": true,\n \"sortOrder\": \"Descending\",\n \"wrapLogMessage\": false\n },\n \"targets\": [\n {\n \"datasource\": {\n \"type\": \"loki\",\n \"uid\": \"loki\"\n },\n \"expr\": \"{service_name=\\\"claude-code\\\"} |= \\\"claude_code.api_error\\\" | line_format \\\"{{.event_timestamp}} [{{.model}}] โ HTTP {{.status_code}} {{.duration_ms}}ms ERROR: {{.error}}\\\"\",\n \"refId\": \"A\"\n }\n ],\n \"title\": \"API Error Events\",\n \"type\": \"logs\"\n }\n ],\n \"refresh\": \"30s\",\n \"schemaVersion\": 27,\n \"style\": \"dark\",\n \"tags\": [\"claude-code\", \"observability\"],\n \"templating\": {\n \"list\": []\n },\n \"time\": {\n \"from\": \"now-1h\",\n \"to\": \"now\"\n },\n \"timepicker\": {},\n \"timezone\": \"\",\n \"title\": \"Claude Code Observability\",\n \"uid\": \"claude-code-obs\",\n \"version\": 1\n}"
},
{
"path": "collector-config.yaml",
"content": "receivers:\n otlp:\n protocols:\n grpc:\n endpoint: 0.0.0.0:4317\n http:\n endpoint: 0.0.0.0:4318\n\nprocessors:\n # Add resource attributes for better analysis\n resource:\n attributes:\n - key: environment\n value: \"production\"\n action: upsert\n\nexporters:\n prometheus:\n endpoint: \"0.0.0.0:8889\"\n\n debug:\n verbosity: normal\n\n otlphttp:\n endpoint: http://loki:3100/otlp\n\nservice:\n pipelines:\n metrics: \n receivers: [otlp]\n processors: [resource]\n exporters: [prometheus, debug]\n \n logs: \n receivers: [otlp]\n processors: [resource]\n exporters: [debug, otlphttp] "
},
{
"path": "docker-compose-lgtm.yml",
"content": "version: \"3.9\"\n\nservices:\n lgtm:\n image: grafana/otel-lgtm:1.4.0\n container_name: lgtm\n ports:\n - \"3000:3000\" # Grafana\n - \"4317:4317\" # OTLP gRPC\n - \"4318:4318\" # OTLP HTTP\n restart: unless-stopped "
},
{
"path": "docker-compose.yml",
"content": "networks:\n otel-network:\n driver: bridge\n\nservices:\n otel-collector:\n image: otel/opentelemetry-collector-contrib:latest # โ 50 MB\n container_name: otel-collector\n command: [\"--config=/etc/otel/collector-config.yaml\"]\n volumes:\n - ./collector-config.yaml:/etc/otel/collector-config.yaml:ro\n ports:\n - \"4317:4317\" # OTLP gRPC in\n - \"4318:4318\" # OTLP HTTP in\n - \"8889:8889\" # Prometheus scrape out\n restart: unless-stopped\n networks:\n - otel-network\n\n prometheus:\n image: prom/prometheus:latest\n container_name: prometheus\n volumes:\n - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro\n ports:\n - \"9090:9090\"\n restart: unless-stopped\n depends_on: [otel-collector]\n networks:\n - otel-network\n\n loki:\n image: grafana/loki:latest\n container_name: loki\n ports:\n - \"3100:3100\"\n command: -config.file=/etc/loki/local-config.yaml\n restart: unless-stopped\n networks:\n - otel-network\n\n grafana:\n image: grafana/grafana-oss:latest\n container_name: grafana\n environment:\n - GF_SECURITY_ADMIN_PASSWORD=admin\n - GF_USERS_ALLOW_SIGN_UP=false\n - GF_FEATURE_TOGGLES_ENABLE=logsSampleInExplore\n volumes:\n - ./grafana-datasources.yml:/etc/grafana/provisioning/datasources/datasources.yml:ro\n - ./grafana-dashboards.yml:/etc/grafana/provisioning/dashboards/dashboards.yml:ro\n - ./claude-code-dashboard.json:/var/lib/grafana/dashboards/claude-code-dashboard.json:ro\n ports:\n - \"3000:3000\"\n restart: unless-stopped\n depends_on: [prometheus, loki]\n networks:\n - otel-network "
},
{
"path": "grafana-dashboards.yml",
"content": "apiVersion: 1\n\nproviders:\n - name: 'default'\n orgId: 1\n folder: ''\n type: file\n disableDeletion: false\n updateIntervalSeconds: 10\n allowUiUpdates: true\n options:\n path: /var/lib/grafana/dashboards"
},
{
"path": "grafana-datasources.yml",
"content": "apiVersion: 1\n\ndatasources:\n - name: prometheus\n type: prometheus\n access: proxy\n url: http://prometheus:9090\n isDefault: true\n\n - name: loki\n type: loki\n access: proxy\n url: http://loki:3100\n \n - name: alertmanager\n type: alertmanager\n access: proxy\n url: http://alertmanager:9093"
},
{
"path": "prometheus.yml",
"content": "global:\n scrape_interval: 15s\n\nscrape_configs:\n - job_name: 'otel-collector'\n static_configs:\n - targets: ['otel-collector:8889'] "
}
]