Showing preview only (305K chars total). Download the full file or copy to clipboard to get everything.
Repository: getsentry/self-hosted
Branch: master
Commit: 6365d51868a1
Files: 119
Total size: 278.1 KB
Directory structure:
gitextract_uhvx5ma5/
├── .craft.yml
├── .editorconfig
├── .gitattributes
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── config.yml
│ │ ├── feature-request.yml
│ │ ├── problem-report.yml
│ │ └── release.yml
│ ├── PULL_REQUEST_TEMPLATE.md
│ ├── dependabot.yml
│ └── workflows/
│ ├── changelog-preview.yml
│ ├── enforce-license-compliance.yml
│ ├── fast-revert.yml
│ ├── lock.yml
│ ├── pre-commit.yml
│ ├── release.yml
│ ├── shellcheck.yml
│ └── test.yml
├── .gitignore
├── .pre-commit-config.yaml
├── .python-version
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE.md
├── README.md
├── _integration-test/
│ ├── conftest.py
│ ├── custom-ca-roots/
│ │ ├── custom-ca-roots-test.py
│ │ └── docker-compose.test.yml
│ ├── fixtures/
│ │ ├── envelope-with-profile
│ │ └── envelope-with-transaction
│ ├── nodejs/
│ │ ├── .gitignore
│ │ ├── instrument.js
│ │ ├── package.json
│ │ └── user-feedback.js
│ ├── test_01_basics.py
│ └── test_02_backup.py
├── _unit-test/
│ ├── _test_setup.sh
│ ├── bootstrap-s3-nodestore-test.sh
│ ├── bootstrap-s3-profiles-test.sh
│ ├── create-docker-volumes-test.sh
│ ├── ensure-relay-credentials-test.sh
│ ├── error-handling-test.sh
│ ├── geoip-test.sh
│ ├── js-sdk-assets-test.sh
│ ├── merge-env-file-test.sh
│ ├── migrate-pgbouncer-test.sh
│ ├── multiple-seaweedfs-bucket-test.sh
│ └── snapshots/
│ └── sentry-envelope-f73e4da437c42a1d28b86a81ebcff35d
├── action.yaml
├── certificates/
│ └── .gitignore
├── clickhouse/
│ ├── Dockerfile
│ ├── config.xml
│ └── default-password.xml
├── codecov.yml
├── cron/
│ ├── Dockerfile
│ └── entrypoint.sh
├── docker-compose.yml
├── geoip/
│ └── GeoLite2-City.mmdb.empty
├── get-compose-action/
│ └── action.yaml
├── install/
│ ├── _detect-container-engine.sh
│ ├── _lib.sh
│ ├── _logging.sh
│ ├── _min-requirements.sh
│ ├── bootstrap-s3-nodestore.sh
│ ├── bootstrap-s3-profiles.sh
│ ├── bootstrap-snuba.sh
│ ├── build-docker-images.sh
│ ├── check-latest-commit.sh
│ ├── check-memcached-backend.sh
│ ├── check-minimum-requirements.sh
│ ├── create-docker-volumes.sh
│ ├── dc-detect-version.sh
│ ├── detect-platform.sh
│ ├── ensure-correct-permissions-profiles-dir.sh
│ ├── ensure-files-from-examples.sh
│ ├── ensure-relay-credentials.sh
│ ├── error-handling.sh
│ ├── generate-secret-key.sh
│ ├── geoip.sh
│ ├── migrate-pgbouncer.sh
│ ├── parse-cli.sh
│ ├── set-up-and-migrate-database.sh
│ ├── setup-js-sdk-assets.sh
│ ├── turn-things-off.sh
│ ├── update-docker-images.sh
│ ├── upgrade-clickhouse.sh
│ ├── upgrade-postgres.sh
│ └── wrap-up.sh
├── install.sh
├── jq/
│ └── Dockerfile
├── nginx.conf
├── optional-modifications/
│ ├── README.md
│ └── patches/
│ └── external-kafka/
│ ├── config.example.yml.patch
│ ├── docker-compose.yml.patch
│ └── sentry.conf.example.py.patch
├── pyproject.toml
├── redis.conf
├── relay/
│ └── config.example.yml
├── scripts/
│ ├── _lib.sh
│ ├── backup.sh
│ ├── bump-version.sh
│ ├── post-release.sh
│ ├── reset.sh
│ └── restore.sh
├── sentry/
│ ├── Dockerfile
│ ├── config.example.yml
│ ├── enhance-image.example.sh
│ ├── entrypoint.sh
│ ├── requirements.example.txt
│ └── sentry.conf.example.py
├── sentry-admin.sh
├── symbolicator/
│ └── config.example.yml
├── unit-test.sh
└── workstation/
├── 200_download-self-hosted.sh
├── 201_install-self-hosted.sh
├── 299_setup-completed.sh
├── README.md
├── commands.sh
├── postinstall/
│ └── Dockerfile
└── preinstall/
└── Dockerfile
================================================
FILE CONTENTS
================================================
================================================
FILE: .craft.yml
================================================
minVersion: 2.21.6
changelogPolicy: auto
preReleaseCommand: bash scripts/bump-version.sh
artifactProvider:
name: none
targets:
- name: github
versioning:
policy: calver
================================================
FILE: .editorconfig
================================================
root = true
[*]
charset = utf-8
end_of_line = lf
indent_style = space
insert_final_newline = true
[*.sh]
indent_size = 2
[*.yml]
indent_size = 2
[nginx/*.conf]
indent_style = tab
================================================
FILE: .gitattributes
================================================
/.gitattributes export-ignore
/.gitignore export-ignore
/.github export-ignore
/.editorconfig export-ignore
/.craft.yml export-ignore
================================================
FILE: .github/ISSUE_TEMPLATE/config.yml
================================================
blank_issues_enabled: false
contact_links:
- name: Report a security vulnerability
url: https://sentry.io/security/#vulnerability-disclosure
about: Please see our guide for responsible disclosure.
================================================
FILE: .github/ISSUE_TEMPLATE/feature-request.yml
================================================
name: 💡 Feature Request
description: Tell us about a problem our software could solve but doesn't.
body:
- type: textarea
id: problem
attributes:
label: Problem Statement
description: What problem could `self-hosted` solve that it doesn't?
placeholder: |-
I want to make whirled peas, but `self-hosted` doesn't blend.
validations:
required: true
- type: textarea
id: expected
attributes:
label: Solution Brainstorm
description: We know you have bright ideas to share ... share away, friend.
placeholder: |-
Add a blender to `self-hosted`.
validations:
required: false
- type: markdown
attributes:
value: |-
## Thanks
Check our [triage docs](https://open.sentry.io/triage/) for what to expect next.
validations:
required: false
================================================
FILE: .github/ISSUE_TEMPLATE/problem-report.yml
================================================
name: 🐞 Problem Report
description: Tell us about something that's not working the way you expect.
body:
- type: input
id: self_hosted_version
attributes:
label: Self-Hosted Version
placeholder: 21.7.0 ← should look like this (check the footer)
description: What version of self-hosted Sentry are you running?
validations:
required: true
- type: input
id: cpu_architecture
attributes:
label: CPU Architecture
placeholder: x86_64 ← should look like this
description: |
What cpu architecture are you running self-hosted on?
e.g: (docker info --format '{{.Architecture}}')
validations:
required: true
- type: input
id: docker_version
attributes:
label: Docker Version
placeholder: 20.10.16 ← should look like this (docker --version)
description: |
What version of docker are you using to run self-hosted?
e.g: (docker --version)
validations:
required: true
- type: input
id: docker_compose_version
attributes:
label: Docker Compose Version
placeholder: 2.6.0 ← should look like this (docker compose version)
description: |
What version of docker compose are you using to run self-hosted?
e.g: (docker compose version)
validations:
required: true
- type: checkboxes
id: machine_specification
attributes:
label: Machine Specification
description: Make sure your system meets the [minimum system requirements of Sentry](https://develop.sentry.dev/self-hosted/#required-minimum-system-resources).
options:
- label: My system meets the minimum system requirements of Sentry
required: true
validations:
required: true
- type: input
id: installation_type
attributes:
label: Installation Type
placeholder: Fresh install / Upgrade from 24.8.0 to 25.5.1
description: |
Are you filing this issue for a fresh install or an upgrade?
validations:
required: true
- type: textarea
id: repro
attributes:
label: Steps to Reproduce
description: How can we see what you're seeing? Specific is terrific.
placeholder: |-
1. foo
2. bar
3. baz
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected Result
description: |
What did you expect to happen?
validations:
required: true
- type: textarea
id: actual
attributes:
label: Actual Result
description: |
Logs? Screenshots? Yes, please.
e.g.:
- latest install logs: `ls -1 sentry_install_log-*.txt | tail -1 | xargs cat`
- `docker compose logs` output
placeholder: |-
e.g.:
- logs output
validations:
required: true
- type: input
id: event_id
attributes:
label: Event ID
description: |
If you opted into sending errors to our error monitoring and the error has an event ID, enter it here!
placeholder: c2d85058-d3b0-4d85-a509-e2ba965845d7
- type: markdown
attributes:
value: |-
## Thanks
Check our [triage docs](https://open.sentry.io/triage/) for what to expect next.
If you're reporting a security issue, please follow our [security policy](https://github.com/getsentry/.github/blob/main/SECURITY.md) instead.
validations:
required: false
================================================
FILE: .github/ISSUE_TEMPLATE/release.yml
================================================
name: 📦 Release Issue
description: Start a new self-hosted Sentry release
title: Release YY.M.N
body:
- type: textarea
attributes:
label: Body
description: "Edit YY.M.N in the title and three times in the first line of the body, then submit. 👍"
value: |
[previous YY.M.N](https://github.com/getsentry/self-hosted/issues) | ***YY.M.N*** | [next YY.M.N](https://github.com/getsentry/self-hosted/issues)
- [ ] Release all components (_replace items with [publish repo issue links](https://github.com/getsentry/publish/issues)_).
- [ ] [`relay`](https://github.com/getsentry/relay/actions/workflows/release_binary.yml)
- [ ] [`sentry`](https://github.com/getsentry/sentry/actions/workflows/release.yml)
- [ ] [`snuba`](https://github.com/getsentry/snuba/actions/workflows/release.yml)
- [ ] [`symbolicator`](https://github.com/getsentry/symbolicator/actions/workflows/release.yml)
- [ ] [`vroom`](https://github.com/getsentry/vroom/actions/workflows/release.yaml)
- [ ] [`uptime-checker`](https://github.com/getsentry/uptime-checker/actions/workflows/release.yml)
- [ ] [`taskbroker`](https://github.com/getsentry/taskbroker/actions/workflows/release.yml)
- [ ] Release self-hosted.
- [ ] [Prepare the `self-hosted` release](https://github.com/getsentry/self-hosted/actions/workflows/release.yml) (_replace with publish issue repo link_).
- [ ] Check to make sure the new release branch in self-hosted includes the appropriate CalVer images.
- [ ] Accept (publish) the release.
- [ ] Edit [release notes](https://github.com/getsentry/self-hosted/releases).
- [ ] Follow up.
- [ ] [Create the next release issue](https://github.com/getsentry/self-hosted/issues/new?assignees=&labels=&projects=&template=release.yml) and link it from this one.
- _replace with link_
- [ ] Update the [release issue template](https://github.com/getsentry/self-hosted/blob/master/.github/ISSUE_TEMPLATE/release.yml).
- [ ] Create a PR to update relocation release tests to add the new version.
- _replace with link_
validations:
required: true
================================================
FILE: .github/PULL_REQUEST_TEMPLATE.md
================================================
<!-- Describe your PR here. -->
<!--
Sentry employees and contractors can delete or ignore the following.
-->
### Legal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.
================================================
FILE: .github/dependabot.yml
================================================
version: 2
updates:
- package-ecosystem: docker
directory: "/"
schedule:
interval: daily
open-pull-requests-limit: 0 # only security updates
reviewers:
- "@getsentry/dev-infra"
- "@getsentry/security"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
# Check for updates to GitHub Actions every week
interval: "weekly"
reviewers:
- "@getsentry/dev-infra"
- "@getsentry/security"
================================================
FILE: .github/workflows/changelog-preview.yml
================================================
name: Changelog Preview
on:
pull_request_target:
types:
- opened
- synchronize
- reopened
- edited
- labeled
- unlabeled
permissions:
contents: write
pull-requests: write
statuses: write
jobs:
changelog-preview:
uses: getsentry/craft/.github/workflows/changelog-preview.yml@v2
secrets: inherit
================================================
FILE: .github/workflows/enforce-license-compliance.yml
================================================
name: Enforce License Compliance
on:
push:
branches: [master]
pull_request:
branches: [master]
permissions:
contents: read
jobs:
enforce-license-compliance:
if: github.repository_owner == 'getsentry'
runs-on: ubuntu-latest
steps:
- name: 'Enforce License Compliance'
uses: getsentry/action-enforce-license-compliance@main
with:
fossa_api_key: ${{ secrets.FOSSA_API_KEY }}
================================================
FILE: .github/workflows/fast-revert.yml
================================================
on:
pull_request_target:
types: [labeled]
workflow_dispatch:
inputs:
pr:
required: true
description: pr number
co_authored_by:
required: true
description: '`name <email>` for triggering user'
# disable all permissions -- we use the PAT's permissions instead
permissions: {}
jobs:
revert:
runs-on: ubuntu-latest
if: |
github.event_name == 'workflow_dispatch' || github.event.label.name == 'Trigger: Revert'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
token: ${{ secrets.BUMP_SENTRY_TOKEN }}
- uses: getsentry/action-fast-revert@35b4b6c1f8f91b5911159568b3b15e531b5b8174 # v2.0.1
with:
pr: ${{ github.event.number || github.event.inputs.pr }}
co_authored_by: ${{ github.event.inputs.co_authored_by || format('{0} <{1}+{0}@users.noreply.github.com>', github.event.sender.login, github.event.sender.id) }}
committer_name: getsentry-bot
committer_email: bot@sentry.io
token: ${{ secrets.BUMP_SENTRY_TOKEN }}
- name: comment on failure
run: |
curl \
--silent \
-X POST \
-H 'Authorization: token ${{ secrets.BUMP_SENTRY_TOKEN }}' \
-d'{"body": "revert failed (conflict? already reverted?) -- [check the logs](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})"}' \
https://api.github.com/repositories/${{ github.event.repository.id }}/issues/${{ github.event.number || github.event.inputs.pr }}/comments
if: failure()
================================================
FILE: .github/workflows/lock.yml
================================================
name: 'Lock closed issues/PRs'
on:
schedule:
- cron: '11 3 * * *'
workflow_dispatch:
jobs:
lock:
if: github.repository_owner == 'getsentry'
runs-on: ubuntu-latest
steps:
- uses: getsentry/forked-action-lock-threads@master
with:
github-token: ${{ github.token }}
issue-lock-inactive-days: 15
issue-lock-reason: ''
pr-lock-inactive-days: 15
pr-lock-reason: ''
================================================
FILE: .github/workflows/pre-commit.yml
================================================
name: pre-commit
on:
pull_request:
push:
branches: [master]
permissions:
contents: read
jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: 3.x
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
================================================
FILE: .github/workflows/release.yml
================================================
name: Release
on:
workflow_dispatch:
inputs:
version:
description: Version to release (or "auto")
required: false
force:
description: Force a release even when there are release-blockers (optional)
required: false
schedule:
# We want the release to be at 10 or 11am Pacific Time
# We also make this an hour after all others such as Sentry,
# Snuba, and Relay to make sure their releases finish.
- cron: "0 18 15 * *"
permissions:
contents: read
jobs:
release:
if: github.repository_owner == 'getsentry'
runs-on: ubuntu-latest
name: "Release a new version"
steps:
- name: Get auth token
id: token
uses: actions/create-github-app-token@29824e69f54612133e76f7eaac726eef6c875baf # v2.2.1
with:
app-id: ${{ vars.SENTRY_RELEASE_BOT_CLIENT_ID }}
private-key: ${{ secrets.SENTRY_RELEASE_BOT_PRIVATE_KEY }}
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
token: ${{ steps.token.outputs.token }}
fetch-depth: 0
- name: Prepare release
id: prepare-release
uses: getsentry/craft@013a7b2113c2cac0ff32d5180cfeaefc7c9ce5b6 # v2.24.1
env:
GITHUB_TOKEN: ${{ steps.token.outputs.token }}
with:
version: ${{ github.event.inputs.version }}
force: ${{ github.event.inputs.force }}
outputs:
release-version: ${{ env.RELEASE_VERSION }}
dogfood-release:
if: github.repository_owner == 'getsentry'
runs-on: ubuntu-latest
name: Create release on self-hosted dogfood instance
needs: release
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- uses: getsentry/action-release@dab6548b3c03c4717878099e43782cf5be654289 # v3.5.0
env:
SENTRY_ORG: self-hosted
SENTRY_PROJECT: installer
SENTRY_URL: https://self-hosted.getsentry.net/
SENTRY_AUTH_TOKEN: ${{ secrets.SELF_HOSTED_RELEASE_TOKEN }}
with:
environment: production
version: ${{ needs.release.outputs.release-version }}
ignore_empty: true
ignore_missing: true
================================================
FILE: .github/workflows/shellcheck.yml
================================================
name: "ShellCheck"
on:
push:
paths:
- "**.sh"
branches: [master]
pull_request:
paths:
- "**.sh"
branches: [master]
permissions:
contents: read
jobs:
shellcheck:
name: ShellCheck
runs-on: ubuntu-latest
steps:
- name: Repository checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
- name: Run ShellCheck
run: |
git diff --name-only -z `git merge-base origin/master HEAD` -- \
install/_lib.sh \
'optional-modifications/**.sh' \
'scripts/**.sh' \
unit-test.sh \
'workstation/**.sh' \
| xargs -0 -r -- \
shellcheck \
--shell=bash \
--format=json1 \
--external-sources \
| jq -r '
.comments
| map(.level |= if ([.] | inside(["info", "style"])) then "notice" else . end)
| .[] as $note
| "::\($note.level) file=\($note.file),line=\($note.line),endLine=\($note.endLine),col=\($note.column),endColumn=\($note.endColumn)::[SC\($note.code)] \($note.message)"
' \
| grep . >&2 && exit 1
exit 0
================================================
FILE: .github/workflows/test.yml
================================================
name: Test
on:
# Run CI on all pushes to the master and release/** branches, and on all new
# pull requests, and on all pushes to pull requests (even if a pull request
# is not against master).
push:
branches:
- "master"
- "release/**"
pull_request:
schedule:
- cron: "0 0,12 * * *"
concurrency:
group: ${{ github.ref_name || github.sha }}
cancel-in-progress: true
permissions:
contents: read
defaults:
run:
shell: bash
jobs:
unit-test:
if: github.repository_owner == 'getsentry'
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-24.04, ubuntu-24.04-arm]
name: ${{ matrix.os == 'ubuntu-24.04-arm' && 'unit tests (arm64)' || 'unit tests' }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Unit Tests
run: ./unit-test.sh
integration-test:
if: github.repository_owner == 'getsentry'
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, ubuntu-24.04-arm]
container_engine: ['docker'] # TODO: add 'podman' into the list
compose_profiles: ['feature-complete', 'errors-only']
name: ${{ format('integration test{0}{1}{2}', matrix.os == 'ubuntu-24.04-arm' && ' (arm64)' || '', matrix.container_engine == 'podman' && ' (podman)' || '', matrix.compose_profiles == 'errors-only' && ' (errors-only)' || '') }}
env:
REPORT_SELF_HOSTED_ISSUES: 0
SELF_HOSTED_TESTING_DSN: ${{ vars.SELF_HOSTED_TESTING_DSN }}
CONTAINER_ENGINE_PODMAN: ${{ matrix.container_engine == 'podman' && '1' || '0' }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install Podman
if: matrix.container_engine == 'podman'
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends podman
# TODO: Replace below with podman-compose
# We need this commit to be able to work: https://github.com/containers/podman-compose/commit/8206cc3ea277eee6c2e87d4cd66eba8eae3d44eb
pip3 install --user https://github.com/containers/podman-compose/archive/main.tar.gz
echo "PODMAN_COMPOSE_PROVIDER=podman-compose" >> $GITHUB_ENV
echo "PODMAN_COMPOSE_WARNING_LOGS=false" >> $GITHUB_ENV
- name: Use action from local checkout
uses: './'
with:
compose_profiles: ${{ matrix.compose_profiles }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
================================================
FILE: .gitignore
================================================
# Error reporting choice cache
.reporterrors
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
sentry_install_log*.txt
sentry_reset_log*.txt
sentry_restore_log*.txt
sentry_backup_log*.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Ipython Notebook
.ipynb_checkpoints
# https://docs.docker.com/compose/extends/
docker-compose.override.yml
# https://docs.docker.com/compose/environment-variables/#using-the---env-file--option
.env.custom
*.tar
data/
# Editor / IDE
.vscode/tags
.idea
# custom Sentry config
sentry/sentry.conf.py
sentry/config.yml
sentry/*.bak
sentry/backup.json
sentry/enhance-image.sh
sentry/requirements.txt
relay/credentials.json
relay/config.yml
symbolicator/config.yml
geoip/GeoIP.conf
geoip/*.mmdb
geoip/.geoipupdate.lock
# integration testing
_integration-test/custom-ca-roots/nginx/*
sentry/test-custom-ca-roots.py
# OSX minutia
.DS_Store
================================================
FILE: .pre-commit-config.yaml
================================================
exclude: '\.patch$'
repos:
- repo: local
hooks:
# Based on https://github.com/scop/pre-commit-shfmt/blob/main/.pre-commit-hooks.yaml
# Customized to also work on ARM, and give diff for CI on failure.
- id: shfmt
name: shfmt
description: Format shell source code
language: docker_image
entry: --net none mvdan/shfmt:v3.5.1
args: [-w, -d]
files: .*\.sh
stages: [commit, merge-commit, push, manual]
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-merge-conflict
- id: check-symlinks
- id: end-of-file-fixer
- id: trailing-whitespace
- id: check-yaml
================================================
FILE: .python-version
================================================
3.12
================================================
FILE: CHANGELOG.md
================================================
# Changelog
## 26.3.1
- No documented changes.
## 26.3.0
### New Features ✨
- Reorder pull images by @aldy505 in [#4202](https://github.com/getsentry/self-hosted/pull/4202)
### Bug Fixes 🐛
- Manual image tags rollback to nightly by @aldy505 in [#4204](https://github.com/getsentry/self-hosted/pull/4204)
### Internal Changes 🔧
#### Deps
- Bump actions/setup-node from 6.2.0 to 6.3.0 by @dependabot in [#4206](https://github.com/getsentry/self-hosted/pull/4206)
- Bump getsentry/craft from 2.21.7 to 2.23.2 by @dependabot in [#4207](https://github.com/getsentry/self-hosted/pull/4207)
- Bump minimatch from 9.0.5 to 9.0.7 in /_integration-test/nodejs by @dependabot in [#4189](https://github.com/getsentry/self-hosted/pull/4189)
## 26.2.1
### Bug Fixes 🐛
#### Release
- Restore version bumping for releases by @BYK in [#4191](https://github.com/getsentry/self-hosted/pull/4191)
- Be explicit about pre release and post release command by @hubertdeng123 in [#4190](https://github.com/getsentry/self-hosted/pull/4190)
#### Other
- Prevent script injection vulnerability in get-compose-action by @fix-it-felix-sentry in [#4179](https://github.com/getsentry/self-hosted/pull/4179)
### Internal Changes 🔧
- (deps) Bump getsentry/craft from 2.21.4 to 2.21.7 by @dependabot in [#4188](https://github.com/getsentry/self-hosted/pull/4188)
### Other
- Use docker-compose shipped in GHA runners by @aminvakil in [#4184](https://github.com/getsentry/self-hosted/pull/4184)
## 26.1.0
### New Features ✨
- Switch nodestore-s3 package to getsentry org by @aldy505 in [#4119](https://github.com/getsentry/self-hosted/pull/4119)
### Build / dependencies / internal 🔧
- (deps) Bump astral-sh/setup-uv from 7.1.5 to 7.1.6 by @dependabot in [#4100](https://github.com/getsentry/self-hosted/pull/4100)
### Other
- Include SDK version 10 when using local JS SDK assets by @hsgt-brice in [#4130](https://github.com/getsentry/self-hosted/pull/4130)
## 25.12.1
### Build / dependencies / internal 🔧
- chore(deps): bump seaweedfs version to 3.97 by @kostirez1 in [#4120](https://github.com/getsentry/self-hosted/pull/4120)
## 25.12.0
### New Features ✨
- feat(release): Manually run post release script by @hubertdeng123 in [#4073](https://github.com/getsentry/self-hosted/pull/4073)
- feat: bump action-setup-venv to use working-directory by @aldy505 in [#4098](https://github.com/getsentry/self-hosted/pull/4098)
### Bug Fixes 🐛
- fix: Provide useful info on permission errors by @BYK in [#4096](https://github.com/getsentry/self-hosted/pull/4096)
- fix: missing 'SENTRY_SYSTEM_SECRET_KEY' declaration on Docker Compose block by @aldy505 in [#4087](https://github.com/getsentry/self-hosted/pull/4087)
- fix: grep seaweedfs bucket name instead of using awk by @aldy505 in [#4076](https://github.com/getsentry/self-hosted/pull/4076)
### Build / dependencies / internal 🔧
#### Deps
- build(deps): bump actions/create-github-app-token from 2.2.0 to 2.2.1 by @dependabot in [#4090](https://github.com/getsentry/self-hosted/pull/4090)
- build(deps): bump actions/setup-node from 4.4.0 to 6.1.0 by @dependabot in [#4091](https://github.com/getsentry/self-hosted/pull/4091)
- build(deps): bump astral-sh/setup-uv from 7.1.4 to 7.1.5 by @dependabot in [#4093](https://github.com/getsentry/self-hosted/pull/4093)
- chore: guard unit-test.sh from being invoked by users by @aldy505 in [#4085](https://github.com/getsentry/self-hosted/pull/4085)
- chore: ask installation type on problem report template by @aldy505 in [#4086](https://github.com/getsentry/self-hosted/pull/4086)
### Other
- Revert "Revert "ref: migrate to uv (#4061)"" by @aldy505 in [#4097](https://github.com/getsentry/self-hosted/pull/4097)
- Revert "ref: migrate to uv (#4061)" by @BYK in [#4094](https://github.com/getsentry/self-hosted/pull/4094)
- test: integration test for user feedback by @aldy505 in [#3880](https://github.com/getsentry/self-hosted/pull/3880)
- ref: migrate to uv by @aldy505 in [#4061](https://github.com/getsentry/self-hosted/pull/4061)
- Add shm_size configuration back to postgres by @max-wittig in [#4072](https://github.com/getsentry/self-hosted/pull/4072)
## 25.11.1
### Build / dependencies / internal 🔧
- chore: remove some more unused directories by @aldy505 in [#4046](https://github.com/getsentry/self-hosted/pull/4046)
- build(deps): bump actions/create-github-app-token from 2.1.4 to 2.2.0 by @dependabot in [#4058](https://github.com/getsentry/self-hosted/pull/4058)
- build(deps): bump actions/checkout from 5 to 6 by @dependabot in [#4059](https://github.com/getsentry/self-hosted/pull/4059)
- chore: uptime checker is missing on the release issue by @aldy505 in [#4047](https://github.com/getsentry/self-hosted/pull/4047)
### Bug Fixes 🐛
- fix(profiling): Ingest profile file path by @Zylphrex in [#4060](https://github.com/getsentry/self-hosted/pull/4060)
- fix: ensure seaweedfs lifecycle policy is set correctly by @kodebach in [#4040](https://github.com/getsentry/self-hosted/pull/4040)
- fix: missing SYMBOLICATOR_STATSD_ADDR environment var for symbolicator-cleanup by @aldy505 in [#4042](https://github.com/getsentry/self-hosted/pull/4042)
### Other
- Add `dcx` shortcut for docker compose exec with HTTP proxy env vars by @copilot-swe-agent in [#4067](https://github.com/getsentry/self-hosted/pull/4067)
## 25.11.0
### Various fixes & improvements
- feat: statsd configuration through environment variables (#4031) by @aldy505
- ref: sound SENTRY_DISALLOWED_IPS on the configuration file (#3981) by @aldy505
- fix: broken link to Errors-Only Mode docs (#4032) by @mariansimecek
- Fix Clickhouse max_server_memory_usage_to_ram_ratio setting (#4025) by @otoriphoenix
- Increase default max_suspicious_broken_parts to 100 (#4011) by @stevenobird
- fix(install): add migrate-pgbouncer.sh to install.sh (#4030) by @kodebach
- fix: remove snuba uptime results consumer (#4027) by @aldy505
## 25.10.0
### Various fixes & improvements
- fix: geoip standalone script should check on CONTAINER_ENGINE variable first (#3982) by @aldy505
- fix: missing `-dir` flag for seaweedfs (#3991) by @aldy505
- Remove symbolicator volume once (#3994) by @aminvakil
- Remove symbolicator external volume (#3992) by @aminvakil
- chore(spans): Remove old snuba-spans consumer (#3989) by @jjbayer
- Bump redis 6.2.20-alpine (#3988) by @aminvakil
- ref: add `continue-on-error` for codecov action on self-hosted integration tests (#3978) by @aldy505
- ref: use dedicated `healthcheck` command for symbolicator & remove cron for `symbolicator-cleanup` (#3979) by @aldy505
- fix(actions): include arch and compose_profiles information on cache keys (#3974) by @aldy505
- ref: Remove proxy_next_upstream directives (#3973) by @aminvakil
- fix: Unset the proxy when performing the seaweedfs health check (#3959) by @SteppingHat
- fix: logic error in s3 install script (#3965) by @kodebach
- Fix swap allocation in integration test (#3972) by @aminvakil
- chore(tasks) Remove reference to celery (#3962) by @markstory
- Respect uppercase proxy variables (#3949) by @aminvakil
- chore(tasks): Remove the worker and cron containers (#3946) by @markstory
- fix: install behind a proxy (#3944) by @moroine
## 25.9.0
### Various fixes & improvements
- fix: able to setup nodestore multiple times (#3940) by @aldy505
- build(deps): bump actions/create-github-app-token from 2.1.1 to 2.1.4 (#3936) by @dependabot
- docs: provide information for SENTRY_AIR_GAP flag on Django config file (#3935) by @aldy505
- feat: Use S3 node store with seaweedfs (#3498) by @BYK
- feat(tasks): Remove taskworker option override and add worker healthcheck (#3933) by @markstory
- feat: install script to migrate sentry.conf.py config to use pgbouncer (#3898) by @aldy505
- chore(deps): bump clickhouse to 25.3 (#3878) by @aldy505
- feat: enable `issue-views` flag (#3922) by @aldy505
- feat: query against `eap` dataset instead of `metrics` dataset for spans (#3923) by @aldy505
- build(deps): bump actions/setup-python from 5 to 6 (#3927) by @dependabot
- Add restart policy to pgbouncer service (#3925) by @frederikspang
- fix(tests): skip logs event test for errors-only (#3915) by @aldy505
- Improve nginx depends_on policy (#3914) by @aminvakil
- test: run errors-only integration tests (#3910) by @aldy505
- feat: enable Logs feature (#3912) by @aldy505
- fix: ensuring vroom permission should be skipped on errors-only (#3911) by @aldy505
- chore(deps): bump patches version (#3879) by @aldy505
- Revert "increase postgres max_connections above 100 connections (#2740)" (#3899) by @aminvakil
- Add pgbouncer (#3884) by @frederikspang
- chore: resolve GHA code scanning alerts (#3889) by @aldy505
- fix(enhancement): search for permissions on docker container instead of host and combine it in one command for performance enhancement (#3890) by @LvckyAPI
- build(deps): bump actions/create-github-app-token from 2.1.0 to 2.1.1 (#3885) by @dependabot
- build(deps): bump actions/checkout from 4 to 5 (#3883) by @dependabot
## 25.8.0
### Various fixes & improvements
- feat: Relay healthcheck (#3875) by @aldy505
- fix: setup swapfile only if runner architecture is X64 or X86 (#3876) by @aldy505
- Set minimum bash version to 4.4.0 (#3873) by @aminvakil
- fix: adjust file healthcheck durations (#3874) by @mzglinski
- feat: healthchecks for sentry components (#3859) by @mzglinski
- fix(eap): Fix dataset parameter to target spans (#3866) by @phacops
- build(deps): bump actions/create-github-app-token from 2.0.6 to 2.1.0 (#3865) by @dependabot
- fix(scripts): use `env` to find `bash` interpreter (#3861) by @Zaczero
- fix(scripts): every known flags should be shifted before executing the sentry <foo> command (#3831) by @aldy505
- fix: uptime checker image should be bumped to the tagged release (#3858) by @aldy505
- fix(enhancement): ensure correct ownership check before setting permissions of profiles (#3855) by @LvckyAPI
- chore(features): cleanup feature flags grouped by its' category (#3843) by @aldy505
- fix: add schedulers for generic metrics subscriptions (#3847) by @mzglinski
- feat: Continue using celery in self-hosted for now (#3845) by @markstory
- feat(features): add `profiling-view` flag (#3837) by @aldy505
- Potential fix for code scanning alert no. 12: Workflow does not contain permissions (#3822) by @aldy505
- docs: clearly state that `system.internal-url-prefix` shouldn't be changed (#3829) by @aldy505
- feat(install): Adds support for podman(compose) (#3673) by @DuncanConroy
- fix(action): missing project directory path for failure inspection (#3825) by @aldy505
- Cleanup unused feature flags (#3820) by @doc-sheet
- feat: inspect docker compose failure on self-hosted e2e action (#3817) by @aldy505
## 25.7.0
### Various fixes & improvements
- feat: Swap `trace-view-v1` feature flag with `visibility-explore-view` (#3801) by @aldy505
- fix: set harakiri Django option to 30s (#3792) by @aldy505
- feat(images):Cutover images to ghcr (#3800) by @hubertdeng123
- docs: encourage community patches (#3794) by @aldy505
- feat: run EAP-related containers (#3778) by @aldy505
- feat(uptime): Enable uptime in self-hosted (#3787) by @evanpurkhiser
- feat: make `system.secret-key` configurable from environment variables (#3783) by @aldy505
- ci: run tests on arm64 (#3750) by @aldy505
## 25.6.2
### Various fixes & improvements
- fix: Increase timeout for flakey test (#3781) by @tobias-wilfert
- chore: provide detailed note for sentry endpoint settings (#3780) by @aldy505
## 25.6.1
### Various fixes & improvements
- fix(taskworker) Remove num-brokers (#3769) by @markstory
- feat: enable customization sentry DSN endpoint (#3747) by @yildizozgur
- ref(js-assets): Simplify how we call nginx container (#3761) by @BYK
- Revert "fix(vroom): Explicitly set PROFILES_DIR for upcoming change" (#3760) by @hubertdeng123
- fix(vroom): Explicitly set PROFILES_DIR for upcoming change (#3759) by @BYK
## 25.6.0
### Various fixes & improvements
- enable shell linter for more scripts (#3748) by @doc-sheet
- feat: migrate to arm64-compatible smtp image (#3746) by @ezhevita
- Introduce patches with external kafka (#3521) by @aldy505
- add shellcheck action to lint bash scripts (#3710) by @doc-sheet
- tests: Install version 2.x of Python SDK (#3745) by @sentrivana
- feat(features): enable continuous profiling (#3742) by @aldy505
- feat: Add taskbroker + worker + scheduler (#3738) by @markstory
- fix(profiles): Run the profile chunks consumer (#3739) by @phacops
- chore: prune removed feature flags on main repository (#3731) by @aldy505
- remove index workaround (#3730) by @asottile-sentry
- Make usage of Python SDK future proof (#3714) by @antonpirker
## 25.5.1
### Various fixes & improvements
- Add missing lib script to sentry-admin.sh (#3693) by @djakielski
- chore: cleanup obsolete feature flags (#3701) by @doc-sheet
## 25.5.0
### Various fixes & improvements
- build(deps): bump actions/create-github-app-token from 2.0.2 to 2.0.6 (#3690) by @dependabot
- Resolve datetime deprecation warnings (#3686) by @emmanuel-ferdman
- ref: remove SENTRY_USE_BIG_INTS (always True) (#3687) by @asottile-sentry
## 25.4.0
### Stand-alone Docker Compose Fixes
By: @aminvakil (#3658, #3654)
### Various fixes & improvements
- chore(relay): specify spool.enveloppe.max_backpressure_memory_percent configuration for handling relay's failing healthcheck (#3635) by @aldy505
- build(deps): bump actions/create-github-app-token from 1.12.0 to 2.0.2 (#3649) by @dependabot
- build(deps): bump actions/create-github-app-token from 1.11.7 to 1.12.0 (#3639) by @dependabot
- Minimum requirements for 'errors-only' profile (#3634) by @madest92
- build(deps): bump actions/create-github-app-token from 1.11.6 to 1.11.7 (#3632) by @dependabot
- feat(sentry): add dynamic sampling feature to config (#3631) by @aldy505
- docs(config): add example config for Google Auth (#3623) by @junsung-cho
- fix: js-sdk directory/file permission should be set correctly (#3616) by @aldy505
- feat(features): enable session replay canvas (#3619) by @aldy505
## 25.3.0
### Various fixes & improvements
- feat(features): enable trace view (#3617) by @aldy505
- feat: provide monitoring-related configurations (#3611) by @aldy505
- Enforce license compliance only on getsentry repository (#3606) by @aminvakil
- Fix unbound variable error in install script (#3601) by @brettdh
- Add --short to docker-compose version (#3605) by @aminvakil
- ref: Less complicated docker compose detection (#3604) by @BYK
- Use docker-compose if version is gte docker compose (#3595) by @aminvakil
- build(deps): bump actions/create-github-app-token from 1.11.3 to 1.11.6 (#3598) by @dependabot
- build(deps): bump getsentry/action-release from 1 to 3 (#3599) by @dependabot
- Bump docker-compose 2.33.1 (#3597) by @aminvakil
- refactor: move system.url-prefix under systems settings section (#3588) by @leeoocca
## 25.2.0
### Various fixes & improvements
- build(deps): bump actions/create-github-app-token from 1.11.2 to 1.11.3 (#3569) by @dependabot
- feat: merge `.env` and `.env.custom` file during installation (#3564) by @aldy505
- build(deps): bump actions/create-github-app-token from 1.11.1 to 1.11.2 (#3561) by @dependabot
- feat: Require both inputs to be set on action (#3554) by @BYK
- ref: Simpler and more accurate cache keys (#3553) by @BYK
- Hand off open-source to dev-infra (#3549) by @chadwhitacre
- ci: Remove obsolete `dcr up -w` from import test (#3544) by @BYK
- fix: github.action_path may not have trailing slash (#3547) by @BYK
- chore: Remove upgrade test (#3541) by @hubertdeng123
- fix: Use correct path for get compose action (#3539) by @hubertdeng123
- fix: Caching of sentry migrations should cover additional folders (#3542) by @hubertdeng123
- ci: Move self-contained action reference to master branch (#3538) by @BYK
- breaking: Upgrade min Compose version to 2.23.2 (#3535) by @BYK
- ci: Even better cache keys and granular caching (#3534) by @BYK
- test: Reorganize backup/restore tests for speed and reliability (#3537) by @BYK
## 25.1.0
### Various fixes & improvements
- ci: Use generic Docker volume cache action (#3524) by @BYK
- ci: Less volatile cache keys (#3522) by @BYK
- docs: include regular env file on wrap-up (#3523) by @aldy505
- ci: Faster and smarter backup/restore tests (#3516) by @BYK
- fix: Fix the new e2e action to be portable (#3520) by @BYK
- ci: Move e2e test action into the repo (#3519) by @BYK
- ci: Only test on compose 2.26 w/ customizations (#3506) by @BYK
- ci: Skip DB ops during install completely on cache hit (#3496) by @BYK
- chore: Remove everything zookeeper (#3499) by @hubertdeng123
- ci: Cache postgres volume after first migration (#3488) by @BYK
- fix: Remove the extra space in the log file names (#3212) by @melnele
- ref(snuba): Combine bootstrap & migrate for faster bootstrap (#3491) by @BYK
- ref(geoip): Remove geoipupdate from compose (#3490) by @BYK
- build(deps): bump actions/create-github-app-token from 1.11.0 to 1.11.1 (#3492) by @dependabot
## 24.12.1
### Various fixes & improvements
- chore: clearer message for errors-only mode (#3487) by @aldy505
- chore(relay): provide opt-in max_memory_percent config as workaround for failing healthcheck (#3486) by @aldy505
- fix(nginx): _assets should rewrite to _static/sentry/dist (#3483) by @BYK
## 24.12.0
- No documented changes.
## 24.11.2
### Various fixes & improvements
- fix(redis): Actually use custom config (#3459) by @BYK
- feat(release): Replace release bot with GH app (#3458) by @Jeffreyhung
- chore(issue-template): ask for machine specification and provide link to security policy (#3447) by @aldy505
- add sentry/backup.json to gitignore (#3450) by @niklassc7
- ref: remove suggested fix (#3446) by @aldy505
## 24.11.1
### Various fixes & improvements
- fix(redis): Use a safer eviction rule (#3432) by @BYK
- feat: add Redis configuration for improved memory management (#3427) by @Hassanzadeh-sd
- build(deps): bump codecov/codecov-action from 4 to 5 (#3429) by @dependabot
## 24.11.0
### Various fixes & improvements
- feat(healthcheck): Improve redis healthcheck (#3422) by @hubertdeng123
- fix: missing mime types and turning off autoindex for js-sdk endpoint (#3395) by @aldy505
- fix: Use js.sentry-cdn.com for JS SDK downloads (#3417) by @BYK
- fix(loader): provide js sdk assets from 4.x (#3415) by @aldy505
- Revert "Revert "ref(feedback): remove issue platform flags after releasing issue types"" (#3403) by @BYK
- Revert "ref(feedback): remove issue platform flags after releasing issue types" (#3402) by @BYK
- ref(feedback): remove issue platform flags after releasing issue types (#3397) by @aliu39
- fix(sentry-admin): Do not wait for command finish to display output (#3390) by @Makhonya
## 24.10.0
### Various fixes & improvements
- chore: Disable codecov for master/release branches (#3384) by @hubertdeng123
- chore: replace old URLs of the repo with the new docs (#3375) by @victorelec14
- ref: span normalization allowed host config (#3245) by @aldy505
- docs: explicitly specify `mail.use-{tls,ssl}` is mutually exclusive (#3368) by @aldy505
- ref: allow hosted js sdk bundles (#3365) by @aldy505
- fix(clickhouse): Allow nullable key (#3354) by @nikhars
## 24.9.0
### Various fixes & improvements
- docs: link to develop docs (#3307) by @joshuarli
- fix: more leeway for minimum RAM (#3290) by @joshuarli
- Mandate minimum requirements for ram/cpu (#3275) by @hubertdeng123
- ref(feedback): cleanup topic rollout option (#3276) by @aliu39
- Update release template (#3270) by @hubertdeng123
## 24.8.0
### Various fixes & improvements
- Migrate to zookeeper-less kafka (#3263) by @hubertdeng123
- Revert "ref(feedback): cleanup topic rollout option" (#3262) by @aliu39
- ref(feedback): cleanup topic rollout option (#3256) by @aliu39
- Remove cdc and wal2json and use the default postgres entrypoint (#3260) by @beezz
- add `-euo pipefail` to enhance-image.example.sh (#3246) by @asottile-sentry
- remove python-dev (#3242) by @asottile-sentry
- feat: enable user feedback feature (#3193) by @aldy505
- Use CDN by default for JS SDK Loader (#3213) by @stayallive
## 24.7.1
### Various fixes & improvements
- Fix: errors only config flag (#3220) by @hubertdeng123
- Add errors only self-hosted infrastructure (#3190) by @hubertdeng123
- feat(generic-metrics): Add gauges to docker compose, re-try (#3177) by @ayirr7
## 24.7.0
### Various fixes & improvements
- Check postgres os before proceeding with install (#3197) by @hubertdeng123
- Update sentry-admin.sh to select its own working directory (#3184) by @theoriginalgri
- feat: add insights feature flags (#3152) by @aldy505
- feat(relay): Forward /api/0/relays/* to inner relays (#3144) by @iambriccardo
## 24.6.0
### Various fixes & improvements
- Use general kafka topic creation in self-hosted (#3121) by @hubertdeng123
- Use non-alpine postgres (#3116) by @hubertdeng123
- Bump Python SDK version used in tests (#3108) by @sentrivana
## 24.5.1
### Various fixes & improvements
- Update consumer flags (#3112) by @hubertdeng123
- feat: Add crons task consumers (#3106) by @wedamija
- Update minimum docker compose requirement (#3078) by @JannKleen
- Different approach to editing permissions of docker volumes (#3084) by @hubertdeng123
- ref(spans): Add new feature flags needed (#3092) by @phacops
- chore: Add comment explaining the one liner in clickhouse config (#3085) by @hubertdeng123
- Fix install: use dynamic docker root dir instead of hardcoded one (#3064) by @boutetnico
- Typo in config.example.yml (#3063) by @luchaninov
## 24.5.0
### Various fixes & improvements
- fix: Make docker volume script respect compose project name (#3039) by @hubertdeng123
- remove ref to skip writes (#3041) by @john-z-yang
- Add clickhouse healthchecks to upgrade (#3024) by @hubertdeng123
- Upgrade clickhouse to 23.8 (#3009) by @hubertdeng123
- fix: use nginx realip module (#2977) by @oioki
- Add upgrade test (#3012) by @hubertdeng123
- Bump kafka and zookeeper versions (#2988) by @hubertdeng123
## 24.4.2
### Various fixes & improvements
- Edit test file name (#3002) by @hubertdeng123
- Revert "Sampling: Run e2e tests every 5 minutes" (#2999) by @hubertdeng123
- Fix master test failures (#3000) by @hubertdeng123
- Sampling: Run e2e tests every 5 minutes (#2994) by @hubertdeng123
- Tweak e2e test github action (#2987) by @hubertdeng123
- fix(performance): Add spans-first-ui flag to enable starfish/performance module views in ui (#2993) by @edwardgou-sentry
- Bump docker compose version in CI (#2980) by @hubertdeng123
- Upgrade postgres to 14.11 (#2975) by @mdtro
- Add workstation configuration (#2968) by @azaslavsky
## 24.4.1
### Various fixes & improvements
- chore(deps): bump memcached and redis to latest patch versions (#2973) by @mdtro
- Use docker compose exec to create additional kafka topics (#2904) by @saz
- Add example to docker compose version in problem report (#2959) by @edgariscoding
- Port last integration tests to python (#2966) by @hubertdeng123
## 24.4.0
### Various fixes & improvements
- Use python for e2e tests (#2953) by @hubertdeng123
- feat: adds group attributes consumer (#2927) by @scefali
- fix(spans): Adds organizations:standalone-span-ingestion flag to default config (#2936) by @edwardgou-sentry
- Bump ubuntu version for tests (#2923) by @hubertdeng123
- Write Customization tests in python (#2918) by @hubertdeng123
- feat(clickhouse): Added max_suspicious_broken_parts to the config.xml (#2853) by @victorelec14
- Port backup tests to python (#2907) by @hubertdeng123
- Fix defunct java processes (#2914) by @hubertdeng123
- Integration tests in python (#2892) by @hubertdeng123
- feat: run outcomes-billing consumer (#2909) by @lynnagara
- Remove duplicate feature flags (#2899) by @JannKleen
## 24.3.0
### Various fixes & improvements
- feat(spans): Ingest spans (#2861) by @phacops
- Integration test improvements (#2858) by @hubertdeng123
- increase postgres max_connections above 100 connections (#2740) by @erfantkerfan
- deps: bump maxmind/geoipupdate to 6.1.0 (#2859) by @victorelec14
- Enable proxy buffering in nginx (#2844) by @RexTim
- Add snuba rust consumers (#2831) by @hubertdeng123
- simplify if for open-ai-suggestion (#2732) by @LvckyAPI
- Upgrade to FSL-1.1 (#2835) by @chadwhitacre
- chore: provide clearer csrf url example (#2833) by @aldy505
- chore: Use django ORM to perform sql commands (#2827) by @hubertdeng123
- revert changes in 3067683f6c0e1c6dd9ceb72cb5155c1dbf3bf501 (#2829) by @hubertdeng123
- use rust consumers in self-hosted (3067683f) by @hubertdeng123
## 24.2.0
### Various fixes & improvements
- Bump nginx version (#2797) by @hubertdeng123
- build(deps): bump pre-commit/action from 3.0.0 to 3.0.1 (#2788) by @dependabot
- Tweak postgres indexing fix (#2792) by @hubertdeng123
- fix: DB migration script (#2779) by @hubertdeng123
## 24.1.2
### Various fixes & improvements
- Check memcached backend in Django (#2778) by @chadwhitacre
- Fix groupedmessage indexing error (#2777) by @hubertdeng123
- build(deps): bump actions/setup-python from 4 to 5 (#2644) by @dependabot
- feat: provide csrf settings information for sentry config (#2762) by @aldy505
- Fix apt config generation when http_proxy is set (#2725) (#2734) by @lemrouch
## 24.1.1
### Various fixes & improvements
- Revert "Move open ai key from env variables" (#2724) by @hubertdeng123
- Fix cache error self hosted (#2722) by @hubertdeng123
## 24.1.0
### Various fixes & improvements
- Enable crons (#2712) by @hubertdeng123
- Parameterize backup restore script (#2412) by @hubertdeng123
- Run tests only on getsentry repository (#2681) by @aminvakil
- Tweak the template now that we can see it (#2670) by @chadwhitacre
- Nginx client request body is buffered to a temporary file (#2630) by @zKoz210
## 23.12.1
### Various fixes & improvements
- Make a release issue template (#2666) by @chadwhitacre
## 23.12.0
### Various fixes & improvements
- test(backup): Use --no-prompt for backup tests (#2618) by @azaslavsky
## 23.11.2
- No documented changes.
## 23.11.1
### Various fixes & improvements
- feat: Add sentry-admin.sh tool (#2594) by @azaslavsky
- Patch for dev self-hosted environments (#2592) by @hubertdeng123
- Relicense under FSL-1.0-Apache-2.0 (#2586) by @chadwhitacre
- Bump minimum ram usage (#2585) by @hubertdeng123
## 23.11.0
### Various fixes & improvements
- feat: provide a toggle to enable discord integration (#2548) by @aldy505
- ref: fix a typo (#2556) by @asottile-sentry
- ref: use `git branch --show-current` instead of sed (#2550) by @asottile-sentry
- Remove sessions infra (#2514) by @hubertdeng123
- Upgrade Clickhouse to 21.8 (#2536) by @hubertdeng123
- [Snyk] Security upgrade debian from bullseye-slim to bookworm-20231009-slim (#2511) by @Indigi-managed
- snuba: Remove deprecated CLI arg (#2515) by @lynnagara
## 23.10.1
### Various fixes & improvements
- Revert "feat: upgrade to zookeeper-less kafka (#2445)" (#2500) by @hubertdeng123
- Add fast revert GH workflow (#2499) by @hubertdeng123
- build(deps): bump actions/checkout from 3 to 4 (#2493) by @dependabot
- configure dependabot (#2491) by @mdtro
- deps: bump nginx to 1.25.2 (#2490) by @mdtro
- feat: upgrade to zookeeper-less kafka (#2445) by @joshuarli
- Update outdated install option in README (#2440) by @hubertdeng123
## 23.10.0
### Various fixes & improvements
- Switch geoipupdate image to ghcr.io (#2442) by @hkraal
- Add system.url-prefix to config for visibility (#2426) by @hubertdeng123
- Remove CSPMiddleware since it is enabled by default in the upstream sentry (#2434) by @oioki
- Update nginx.conf (#2455) by @mwarkentin
- Update Redis container image to 6.2.13 (#2432) by @mencarellic
## 23.9.1
### Various fixes & improvements
- fix: e2e test jq bug (#2410) by @azaslavsky
- feat(backup): Support new backup script (#2407) by @azaslavsky
- Decrease frequency of e2e tests (#2383) by @hubertdeng123
- Reduce logs coming from clickhouse (#2382) by @hubertdeng123
- Increase frequency of e2e test runs (#2375) by @hubertdeng123
- Remove nginx content-disposition hack for safari (#2381) by @hubertdeng123
- Attempt to fix integration test flakiness (#2372) by @hubertdeng123
- change health check for kafka service (#2371) by @johnatannvmd
- Add metrics and generic metrics backend (#2355) by @hubertdeng123
- Bump self-hosted e2e action commit sha (#2369) by @hubertdeng123
## 23.8.0
### Various fixes & improvements
- Add issue platform infra (#2309) by @hubertdeng123
## 23.7.2
### Various fixes & improvements
- Ignore fixture-custom-ca-roots service in integration test (#2321) by @hubertdeng123
- Update GeoIpUpdate to v6.0.0 (#2287) by @victorelec14
- Bump healthcheck timeout (#2300) by @hubertdeng123
## 23.7.1
### Various fixes & improvements
- Resolve Safari Content-Disposition header bug (#2297) by @azaslavsky
- feat: vroom cleanup script that respects default retention days (#2211) by @aldy505
## 23.7.0
### Various fixes & improvements
- Remove nc -q option (#2275) by @hubertdeng123
- Move open ai key from env variables (#2274) by @hubertdeng123
- Fix command called in reset script (#2254) by @stayallive
- Remove stale-bot in self-hosted (#2255) by @hubertdeng123
- Update geoipupdate to 5.1.1 (#2236) by @williamdes
## 23.6.2
### Various fixes & improvements
- Update nginx to 1.25.1 (#2235) by @williamdes
- Fix error fingerprinting (#2237) by @chadwhitacre
- A couple unit testing improvements (#2238) by @chadwhitacre
- Fix #1684 (#2234) by @azaslavsky
- Update memcached to 1.6.21 (#2231) by @williamdes
- Update redis to 6.2.12 (#2230) by @williamdes
- ref: Move all consumers to unified consumer CLI (#2224) by @hubertdeng123
- Revert "ref: Move most consumers to unified consumer CLI" (#2223) by @hubertdeng123
- ref: Move most consumers to unified consumer CLI (#2203) by @untitaker
- Release 23.6.1 cleanup (#2209) by @hubertdeng123
## 23.6.1
### Various fixes & improvements
- Fix bump version script (#2207) by @hubertdeng123
## 23.6.0
### Various fixes & improvements
- Remove docker compose v1 (#2187) by @hubertdeng123
- ref(compose): Separate ingest consumers (#2193) by @jan-auer
- feat(profiling): Run profiling on self-hosted (#2154) by @phacops
## 23.5.2
- No documented changes.
## 23.5.1
### Various fixes & improvements
- fix(suggested-fix): key should be 'key', not 'token' (#2146) by @aldy505
## 23.5.0
### Various fixes & improvements
- Add no strict offset reset options to consumers (#2144) by @hubertdeng123
- Add settings for enabling CSP to config file (#2134) by @hubertdeng123
- feat: add suggested fix feature (#2115) by @aldy505
- adding ulimits for zookeeper, kafka, and web (#2123) by @jamincollins
- Uninstall Docker Compose v1 from CI so it's not used for tests (#2114) by @hubertdeng123
- Fixed docker compose issue in backup/restore (#2110) by @montaniasystemab
- Enable upstream keepalive (#2099) by @otbutz
- Bump commit sha for e2e test action (#2104) by @hubertdeng123
- Use docker compose exec to account for differences in container names for Postgres upgrade (#2096) by @hubertdeng123
- Change symbolicator to use CalVer for release (#2091) by @hubertdeng123
## 23.4.0
### Postgres 14 Upgrade
We've now included an upgrade from Postgres 9.6 to 14.5 that will automatically be run via the `./install.sh` script.
By: @hubertdeng123 (#2074)
### Various fixes & improvements
- Remove clean function testing line (#2082) by @hubertdeng123
- Fix command to get docker compose version in problem report template (#2080) by @hubertdeng123
- Tweak permissioning of backup file in backup script to read/write for all users (#2043) by @hubertdeng123
- Remove commit-batch-size parameter (#2058) by @hubertdeng123
- Support external sourcemaps bigger, than 1Mb (#2050) by @le0pard
- Add github setup instructions to config.example.yml (#2051) by @tm1000
- ref(snuba): Use snuba self-hosted settings (#2039) by @enochtangg
## 23.3.1
### Various fixes & improvements
- Bump Kafka version to keep up with SaaS (#2037) by @chadwhitacre
- Add Backup/restore scripts (#2029) by @hubertdeng123
- Add opt in error monitoring to reset and clean scripts (#2021) by @hubertdeng123
## 23.3.0
### Various fixes & improvements
- Remove ZooKeeper snapshot (#2020) by @dereckson
- feat(snuba): Add snuba sessions subscription service (#2006) by @klboke
- Add backup/restore integration tests (#2012) by @hubertdeng123
- ref(snuba): Remove snuba-cleanup, snuba-transactions-cleanup jobs (#2003) by @klboke
- ref(replays): Remove the session-replay-ui flag (#2010) by @ryan953
- Remove broken replay integration test (#2011) by @hubertdeng123
- Bump self-hosted-e2e-tests action commit sha (#2008) by @hubertdeng123
- Revert symbolicator tests (#2004) by @hubertdeng123
- post-process-forwarder: Update CLI command (#1999) by @lynnagara
- feat(replays): add replays to self hosted (#1990) by @JoshFerge
- Remove issue status helper automation (#1989) by @hubertdeng123
- Add proxy buffer size config to fix Bad Gateway (#1984) by @SCjona
- Reference paths relative to project root (#1800) by @spawnia
- Run close stale issues/PRs only on getsentry (#1969) by @aminvakil
## 23.2.0
### Various fixes & improvements
- Run lock issues/PRs only on getsentry (#1966) by @aminvakil
- Updates Redis to 6.2.10 (#1937) by @danielhartnell
- Handle missing example files gracefully (#1950) by @chadwhitacre
- Fix post-release.sh for `git pull` (#1938) by @BYK
- Manually change 23.1.1 to nightly (#1936) by @hubertdeng123
## 23.1.1
### Various fixes & improvements
- ci: Add test for symbolicator pipeline (#1916) by @ethanhs
## 23.1.0
### Various fixes & improvements
- ci: Check health of services after running integration tests and fix snuba-replacer (#1897) by @ethanhs
- Add wal2json debugging (#1906) by @chadwhitacre
- Pick up CI bugfix (#1905) by @chadwhitacre
- ref: Move jq build to error-handling.sh, and use proxy config (#1895) by @ethanhs
- fix(CI): use default curl retry mechanism for wal2json install (#1890) by @volokluev
- ref: Retry wal2json download in installer (#1881) by @ethanhs
- ci: Remove GCB and update Github Action SHA (#1880) by @ethanhs
## 22.12.0
### Various fixes & improvements
- Build each service image individually (#1858) by @ethanhs
- Set higher kafka healthcheck timeout and fix clickhouse timeout (#1855) by @ethanhs
- Add --skip-sse42-requirements to install.sh and enable SKIP_SSE42_REQUIREMENTS override (#1790) by @erinaceous
- Fix commit-log-topic parameter configuration problem (#1817) by @klboke
- Add .idea to .gitignore (#1803) by @spawnia
- Add USE_X_FORWARDED_HOST to example config (#1804) by @crinjes
- (fix): Fix contributor PR e2e tests (#1820) by @hubertdeng123
## 22.11.0
### Various fixes & improvements
- Fix jq usage (#1814) by @ethanhs
- Try adding end to end tests using new action (#1806) by @ethanhs
- Add context line, error msg to envelope (#1784) by @hubertdeng123
- Update to actions/checkoutv3 to address upcoming github deprecations (#1792) by @mattgauntseo-sentry
- ref: upgrade actions/setup-python to avoid set-output deprecation (#1789) by @asottile-sentry
- Enforce error reporting (#1777) by @hubertdeng123
- Upload end of log as breadcrumbs, use exceptions and stacktrace (#1775) by @ethanhs
- Fix sentry release for dogfood instance (#1768) by @hubertdeng123
- Add pre-commit config (#1738) by @ethanhs
- Do not send event on INT signal (#1773) by @hubertdeng123
## 22.10.0
### Various fixes & improvements
- Split post process forwarders (#1759) by @chadwhitacre
- Revert "Enforce error reporting for self-hosted" (#1755) by @hubertdeng123
- Enforce error reporting for self-hosted (#1753) by @hubertdeng123
- ref: Remove unused scripts and code (#1710) by @BYK
- Check to see if docker compose exists, else error out (#1733) by @hubertdeng123
- Fix minimum version requirements for docker and docker compose (#1732) by @hubertdeng123
- Factor out clean and use it in unit-test (#1731) by @chadwhitacre
- Reorganize unit test layout (#1729) by @hubertdeng123
- Request event ID in issue template (#1723) by @ethanhs
- Tag releases with sentry-cli (#1718) by @hubertdeng123
- Send full logs as an attachment to our dogfood instance (#1715) by @hubertdeng123
## 22.9.0
### Various fixes & improvements
- Fix traceback hash for error monitoring (#1700) by @hubertdeng123
- Add section about error monitoring to the README (#1699) by @ethanhs
- Switch from .reporterrors file to flag + envvar (#1697) by @chadwhitacre
- Rename flag to --skip-user-creation (#1696) by @chadwhitacre
- Default to not sending data to Sentry for now (#1695) by @chadwhitacre
- fix(e2e tests): Pull branch that initially triggers gcp build for PRs (#1694) by @hubertdeng123
- fix(e2e tests): Add .reporterrors file for GCP run of e2e tests (#1691) by @hubertdeng123
- Error monitoring of the self-hosted installer (#1679) by @ethanhs
- added docker commands in the description (#1673) by @victorelec14
- Use docker-compose 2.7.0 instead of 2.2.3 in CI (#1591) by @aminvakil
## 22.8.0
- No documented changes.
## 22.7.0
### Various fixes & improvements
- ref: use sort -V to check minimum versions (#1553) by @ethanhs
- Get more data from users in issue templates (#1497) by @aminvakil
- Add ARM support (#1538) by @chadwhitacre
- do not use gosu for snuba-transactions-cleanup and snuba-cleanup (#1564) by @goganchic
- ref: Replace regex with --short flag to get compose version (#1551) by @ethanhs
- Improve installation through proxy (#1543) by @goganchic
- Cleanup .env{,.custom} handling (#1539) by @chadwhitacre
- Bump nginx:1.22.0-alpine (#1506) by @aminvakil
- Run release a new version job only on getsentry (#1529) by @aminvakil
## 22.6.0
### Various fixes & improvements
- fix "services.web.healthcheck.retries must be a number" (#1482) by @yuval1986
- Add volume for nginx cache (#1511) by @glensc
- snuba: New subscriptions infrastructure rollout (#1507) by @lynnagara
- Ease modification of base image (#1479) by @spawnia
## 22.5.0
### Various fixes & improvements
- ref: reset user to root for installation (#1469) by @asottile-sentry
- Document From email display name (#1446) by @chadwhitacre
- Bring in CLA Lite (#1439) by @chadwhitacre
- fix: replace git.io links with redirect targets (#1430) by @asottile-sentry
## 22.4.0
### Various fixes & improvements
- Use better API key when available (#1408) by @chadwhitacre
- Use a custom action (#1407) by @chadwhitacre
- Add some debug logging (#1340) by @chadwhitacre
- meta(gha): Deploy workflow enforce-license-compliance.yml (#1388) by @chadwhitacre
- Turn off containers under old name as well (#1384) by @chadwhitacre
## 22.3.0
### Various fixes & improvements
- Run CI every night (#1334) by @aminvakil
- Docker-Compose: Avoid setting hostname to '' (#1365) by @glensc
- meta(gha): Deploy workflow enforce-license-compliance.yml (#1375) by @chadwhitacre
- ci: Change stale GitHub workflow to run once a day (#1371) by @kamilogorek
- ci: Temporary fix for interactive prompt on createuser (#1370) by @BYK
- meta(gha): Deploy workflow enforce-license-compliance.yml (#1347) by @chadwhitacre
- Add SaaS nudge to README (#1327) by @chadwhitacre
## 22.2.0
### Various fixes & improvements
- fix: unbound variable _group in reset/dc-detect-version script (#1283) (#1284) by @lovetodream
- Remove routing helper (#1323) by @chadwhitacre
- Bump nginx:1.21.6-alpine (#1319) by @aminvakil
- Add a cloudbuild.yaml for GCB (#1315) by @chadwhitacre
- Update set-up-and-migrate-database.sh (#1308) by @drmrbrewer
- Pull relay explicitly to avoid garbage in creds (#1301) by @chadwhitacre
- Improve logging of docker versions and relay creds (#1298) by @chadwhitacre
- Remove file again (#1299) by @chadwhitacre
- Clean up relay credentials generation (#1289) by @chadwhitacre
- Add CI compose version 1.29.2 / 2.0.1 / 2.2.3 (#1290) by @chadwhitacre
- Revert "Add CI compose version 1.29.2 / 2.0.1 / 2.2.3 (#1251)" (#1272) by @chadwhitacre
- Add CI compose version 1.29.2 / 2.0.1 / 2.2.3 (#1251) by @aminvakil
## 22.1.0
### Various fixes & improvements
- Make healthcheck variables configurable in .env (#1248) by @aminvakil
- Take some actions to avoid unhealthy containers (#1241) by @chadwhitacre
- Install: setup umask (#1222) by @glensc
- Deprecated /docker-entrypoint.sh call (#1218) by @marcinroman
- Bump nginx:1.21.5-alpine (#1230) by @aminvakil
- Fix reset.sh docker-compose call (#1215) by @aminvakil
- Set worker_processes to auto (#1207) by @aminvakil
## 21.12.0
### Support Docker Compose v2 (ongoing)
Self-hosted Sentry mostly works with Docker Compose v2 (in addition to v1 >= 1.28.0). There is [one more bug](https://github.com/getsentry/self-hosted/issues/1133) we are trying to squash.
By: @chadwhitacre (#1179)
### Prevent Component Drift
When a user runs the `install.sh` script, they get the latest version of the Sentry, Snuba, Relay and Symbolicator projects. However there is no guarantee they have pulled the latest `self-hosted` version first, and running an old one may cause problems. To mitigate this, we now perform a check during installation that the user is on the latest commit if they are on the `master` branch. You can disable this check with `--skip-commit-check`.
By: @chadwhitacre (#1191), @aminvakil (#1186)
### React to log4shell
Self-hosted Sentry is [not vulnerable](https://github.com/getsentry/self-hosted/issues/1196) to the [log4shell](https://log4shell.com/) vulnerability.
By: @chadwhitacre (#1203)
### Forum → Issues
In the interest of reducing sources of truth, providing better support, and restarting the fire of the self-hosted Sentry community, we [deprecated the Discourse forum in favor of GitHub Issues](https://github.com/getsentry/self-hosted/issues/1151).
By: @chadwhitacre (#1167, #1160, #1159)
### Rename onpremise to self-hosted (ongoing)
In the beginning we used the term "on-premise" and over time we introduced the term "self-hosted." In an effort to regain some consistency for both branding and developer mental overhead purposes, we are standardizing on the term "self-hosted." This release includes a fair portion of the work towards this across multiple repos, hopefully a future release will include the remainder. Some orphaned containers / volumes / networks are [expected](https://github.com/getsentry/self-hosted/pull/1169#discussion_r756401917). You may clean them up with `docker-compose down --remove-orphans`.
By: @chadwhitacre (#1169)
### Add support for custom DotEnv file
There are several ways to [configure self-hosted Sentry](https://develop.sentry.dev/self-hosted/#configuration) and one of them is the `.env` file. In this release we add support for a `.env.custom` file that is git-ignored to make it easier for you to override keys configured this way with custom values. Thanks to @Sebi94nbg for the contribution!
By: @Sebi94nbg (#1113)
### Various fixes & improvements
- Revert "Rename onpremise to self-hosted" (5495fe2e) by @chadwhitacre
- Rename onpremise to self-hosted (9ad05d87) by @chadwhitacre
## 21.11.0
### Various fixes & improvements
- Fix #1079 - bug in reset.sh (#1134) by @chadwhitacre
- ci: Enable parallel tests again, increase timeouts (#1125) by @BYK
- fix: Hide compose errors during version check (#1124) by @BYK
- build: Omit nightly bump commit from changelog (#1120) by @BYK
- build: Set master version to nightly (d3e77857)
## 21.10.0
### Support for Docker Compose v2 (ongoing)
You asked for it and you did it! Sentry self-hosted now can work with Docker Compose v2 thanks to our community's contributions.
PRs: #1116
### Various fixes & improvements
- docs: simplify Linux `sudo` instructions in README (#1096)
- build: Set master version to nightly (58874cf9)
## 21.9.0
- fix(healthcheck): Increase retries to 5 (#1072)
- fix(requirements): Make compose version check bw-compatible (#1068)
- ci: Test with the required minimum docker-compose (#1066)
Run tests using docker-compose `1.28.0` instead of latest
- fix(clickhouse): Use correct HTTP port for healthcheck (#1069)
Fixes the regular `Unexpected packet` errors in Clickhouse
## 21.8.0
- feat: Support custom CA roots ([#27062](https://github.com/getsentry/sentry/pull/27062)), see the [docs](https://develop.sentry.dev/self-hosted/custom-ca-roots/) for more details.
- fix: Fix `curl` image to version 7.77.0
- upgrade: docker-compose version to 1.29.2
- feat: Leverage health checks for depends_on
## 21.7.0
- No documented changes.
## 21.6.3
- No documented changes.
## 21.6.2
- BREAKING CHANGE: The frontend bundle will be loaded asynchronously (via [#25744](https://github.com/getsentry/sentry/pull/25744)). This is a breaking change that can affect custom plugins that access certain globals in the django template. Please see https://forum.sentry.io/t/breaking-frontend-changes-for-custom-plugins/14184 for more information.
## 21.6.1
- No documented changes.
## 21.6.0
- feat: Add healthchecks for redis, memcached and postgres (#975)
================================================
FILE: CONTRIBUTING.md
================================================
## Testing
### Running Tests with Pytest
We use pytest for running tests. To run the tests:
1) Ensure that you are in the root directory of the project.
2) Run the following command:
```bash
pytest
```
This will automatically discover and run all test cases in the project.
================================================
FILE: LICENSE.md
================================================
# Functional Source License, Version 1.1, Apache 2.0 Future License
## Abbreviation
FSL-1.1-Apache-2.0
## Notice
Copyright 2016-2024 Functional Software, Inc. dba Sentry
## Terms and Conditions
### Licensor ("We")
The party offering the Software under these Terms and Conditions.
### The Software
The "Software" is each version of the software that we make available under
these Terms and Conditions, as indicated by our inclusion of these Terms and
Conditions with the Software.
### License Grant
Subject to your compliance with this License Grant and the Patents,
Redistribution and Trademark clauses below, we hereby grant you the right to
use, copy, modify, create derivative works, publicly perform, publicly display
and redistribute the Software for any Permitted Purpose identified below.
### Permitted Purpose
A Permitted Purpose is any purpose other than a Competing Use. A Competing Use
means making the Software available to others in a commercial product or
service that:
1. substitutes for the Software;
2. substitutes for any other product or service we offer using the Software
that exists as of the date we make the Software available; or
3. offers the same or substantially similar functionality as the Software.
Permitted Purposes specifically include using the Software:
1. for your internal use and access;
2. for non-commercial education;
3. for non-commercial research; and
4. in connection with professional services that you provide to a licensee
using the Software in accordance with these Terms and Conditions.
### Patents
To the extent your use for a Permitted Purpose would necessarily infringe our
patents, the license grant above includes a license under our patents. If you
make a claim against any party that the Software infringes or contributes to
the infringement of any patent, then your patent license to the Software ends
immediately.
### Redistribution
The Terms and Conditions apply to all copies, modifications and derivatives of
the Software.
If you redistribute any copies, modifications or derivatives of the Software,
you must include a copy of or a link to these Terms and Conditions and not
remove any copyright notices provided in or with the Software.
### Disclaimer
THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR
PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT.
IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE
SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES,
EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE.
### Trademarks
Except for displaying the License Details and identifying us as the origin of
the Software, you have no right under these Terms and Conditions to use our
trademarks, trade names, service marks or product names.
## Grant of Future License
We hereby irrevocably grant you an additional license to use the Software under
the Apache License, Version 2.0 that is effective on the second anniversary of
the date we make the Software available. On or after that date, you may use the
Software under the Apache License, Version 2.0, in which case the following
will apply:
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
================================================
FILE: README.md
================================================
# Self-Hosted Sentry 26.3.1
[Sentry](https://sentry.io/), feature-complete and packaged up for low-volume deployments and proofs-of-concept.
Documentation [here](https://develop.sentry.dev/self-hosted/).
================================================
FILE: _integration-test/conftest.py
================================================
import os
from os.path import join
import subprocess
import pytest
SENTRY_CONFIG_PY = "sentry/sentry.conf.py"
SENTRY_TEST_HOST = os.getenv("SENTRY_TEST_HOST", "http://localhost:9000")
TEST_USER = "test@example.com"
TEST_PASS = "test123TEST"
@pytest.fixture(scope="session", autouse=True)
def configure_self_hosted_environment(request):
subprocess.run(
["docker", "compose", "--ansi", "never", "up", "--wait"],
check=True,
capture_output=True,
)
# Create test user
subprocess.run(
[
"docker",
"compose",
"exec",
"-T",
"web",
"sentry",
"createuser",
"--force-update",
"--superuser",
"--email",
TEST_USER,
"--password",
TEST_PASS,
"--no-input",
],
check=True,
text=True,
)
@pytest.fixture()
def setup_backup_restore_env_variables():
os.environ["SENTRY_DOCKER_IO_DIR"] = os.path.join(os.getcwd(), "sentry")
os.environ["SKIP_USER_CREATION"] = "1"
================================================
FILE: _integration-test/custom-ca-roots/custom-ca-roots-test.py
================================================
import unittest
import requests
class CustomCATests(unittest.TestCase):
def test_valid_self_signed(self):
self.assertEqual(requests.get("https://self.test").text, "ok")
def test_invalid_self_signed(self):
with self.assertRaises(requests.exceptions.SSLError):
requests.get("https://fail.test")
if __name__ == "__main__":
unittest.main()
================================================
FILE: _integration-test/custom-ca-roots/docker-compose.test.yml
================================================
version: '3.4'
services:
fixture-custom-ca-roots:
image: nginx:1.21.0-alpine
restart: unless-stopped
volumes:
- ./_integration-test/custom-ca-roots/nginx:/etc/nginx:ro
networks:
default:
aliases:
- self.test
- fail.test
================================================
FILE: _integration-test/fixtures/envelope-with-profile
================================================
{"event_id":"66578634d48d433db0ad52882d1efe5b","sent_at":"2023-05-17T14:54:31.057Z","sdk":{"name":"sentry.javascript.node","version":"7.46.0"},"trace":{"environment":"production","transaction":"fib: sourcemaps here","public_key":"05ab86aebbe14a24bcab62caa839cf27","trace_id":"33321bfbd5304bcc9663d1b53b08f84e","sample_rate":"1"}}
{"type":"transaction"}
{"contexts":{"profile":{"profile_id":"e73aaf1f29b24812be60132f32d09f92"},"trace":{"op":"test","span_id":"b38f2b24537c3858","trace_id":"33321bfbd5304bcc9663d1b53b08f84e"},"runtime":{"name":"node","version":"v16.16.0"},"app":{"app_start_time":"2023-05-17T14:54:27.678Z","app_memory":57966592},"os":{"kernel_version":"22.3.0","name":"macOS","version":"13.2","build":"22D49"},"device":{"boot_time":"2023-05-12T15:08:41.047Z","arch":"arm64","memory_size":34359738368,"free_memory":6861651968,"processor_count":10,"cpu_description":"Apple M1 Pro","processor_frequency":24},"culture":{"locale":"en-US","timezone":"America/New_York"}},"spans":[],"start_timestamp":1684335267.744,"tags":{},"timestamp":1684335271.033,"transaction":"fib: sourcemaps here","type":"transaction","transaction_info":{"source":"custom"},"platform":"node","server_name":"TK6G745PW1.local","event_id":"66578634d48d433db0ad52882d1efe5b","environment":"production","sdk":{"integrations":["InboundFilters","FunctionToString","Console","Http","OnUncaughtException","OnUnhandledRejection","ContextLines","LocalVariables","Context","Modules","RequestData","LinkedErrors","ProfilingIntegration"],"name":"sentry.javascript.node","version":"7.46.0","packages":[{"name":"npm:@sentry/node","version":"7.46.0"}]},"debug_meta":{"images":[]},"modules":{}}
{"type":"profile"}
{"event_id":"e73aaf1f29b24812be60132f32d09f92","timestamp":"2023-05-17T14:54:27.744Z","platform":"node","version":"1","release":"","environment":"production","runtime":{"name":"node","version":"16.16.0"},"os":{"name":"darwin","version":"22.3.0","build_number":"Darwin Kernel Version 22.3.0: Thu Jan 5 20:48:54 PST 2023; root:xnu-8792.81.2~2/RELEASE_ARM64_T6000"},"device":{"locale":"en_US.UTF-8","model":"arm64","manufacturer":"Darwin","architecture":"arm64","is_emulator":false},"debug_meta":{"images":[]},"profile":{"samples":[{"stack_id":0,"thread_id":"0","elapsed_since_start_ns":125000},{"stack_id":0,"thread_id":"0","elapsed_since_start_ns":13958000}],"frames":[{"lineno":14129,"colno":17,"function":"startProfiling","abs_path":"/Users/jonasbadalic/code/node-profiler/lib/index.js"}],"stacks":[[0]],"thread_metadata":{"0":{"name":"main"}}},"transaction":{"name":"fib: sourcemaps here","id":"66578634d48d433db0ad52882d1efe5b","trace_id":"33321bfbd5304bcc9663d1b53b08f84e","active_thread_id":"0"}}
================================================
FILE: _integration-test/fixtures/envelope-with-transaction
================================================
{"event_id":"66578634d48d433db0ad52882d1efe5b","sent_at":"2023-05-17T14:54:31.057Z","sdk":{"name":"sentry.javascript.node","version":"7.46.0"},"trace":{"environment":"production","transaction":"fib: sourcemaps here","public_key":"05ab86aebbe14a24bcab62caa839cf27","trace_id":"33321bfbd5304bcc9663d1b53b08f84e","sample_rate":"1"}}
{"type":"transaction"}
{"contexts":{"trace":{"op":"test","span_id":"b38f2b24537c3858","trace_id":"33321bfbd5304bcc9663d1b53b08f84e"},"runtime":{"name":"node","version":"v16.16.0"},"app":{"app_start_time":"2023-05-17T14:54:27.678Z","app_memory":57966592},"os":{"kernel_version":"22.3.0","name":"macOS","version":"13.2","build":"22D49"},"device":{"boot_time":"2023-05-12T15:08:41.047Z","arch":"arm64","memory_size":34359738368,"free_memory":6861651968,"processor_count":10,"cpu_description":"Apple M1 Pro","processor_frequency":24},"culture":{"locale":"en-US","timezone":"America/New_York"}},"spans":[],"start_timestamp":1684335267.744,"tags":{},"timestamp":1684335271.033,"transaction":"fib: sourcemaps here","type":"transaction","transaction_info":{"source":"custom"},"platform":"node","server_name":"TK6G745PW1.local","event_id":"66578634d48d433db0ad52882d1efe5b","environment":"production","sdk":{"integrations":["InboundFilters","FunctionToString","Console","Http","OnUncaughtException","OnUnhandledRejection","ContextLines","LocalVariables","Context","Modules","RequestData","LinkedErrors","ProfilingIntegration"],"name":"sentry.javascript.node","version":"7.46.0","packages":[{"name":"npm:@sentry/node","version":"7.46.0"}]},"debug_meta":{"images":[]},"modules":{}}
================================================
FILE: _integration-test/nodejs/.gitignore
================================================
node_modules
================================================
FILE: _integration-test/nodejs/instrument.js
================================================
import * as Sentry from "@sentry/node";
Sentry.init({
dsn: process.env.SENTRY_DSN,
sampleRate: 1.0,
tracesSampleRate: 1.0,
enableLogs: true,
profileLifecycle: "manual",
sendClientReports: true,
sendDefaultPii: true,
debug: true,
});
================================================
FILE: _integration-test/nodejs/package.json
================================================
{
"name": "sentry-self-hosted-integration-test-nodejs",
"version": "0.0.0",
"description": "Scripts to run for features only available on Nodejs SDK",
"author": "Sentry <oss@sentry.io>",
"type": "module",
"scripts": {
"start": "node --import ./instrument.js index.js"
},
"dependencies": {
"@sentry/node": "^10.5.0"
}
}
================================================
FILE: _integration-test/nodejs/user-feedback.js
================================================
import * as Sentry from "@sentry/node";
Sentry.captureFeedback({
message: "I love your startup!",
name: "John Doe",
email: "john@example.com",
url: "https://example.com",
});
Sentry.flush(5000);
================================================
FILE: _integration-test/test_01_basics.py
================================================
import datetime
import json
import os
import re
import shutil
import subprocess
import sys
import time
from functools import lru_cache
from typing import Callable
import httpx
import pytest
import sentry_sdk
from sentry_sdk import logger as sentry_logger
from bs4 import BeautifulSoup
from cryptography import x509
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.x509.oid import NameOID
SENTRY_CONFIG_PY = "sentry/sentry.conf.py"
SENTRY_TEST_HOST = os.getenv("SENTRY_TEST_HOST", "http://localhost:9000")
TEST_USER = "test@example.com"
TEST_PASS = "test123TEST"
TIMEOUT_SECONDS = 120
def poll_for_response(
request: str, client: httpx.Client, validator: Callable = None
) -> httpx.Response:
for i in range(TIMEOUT_SECONDS):
response = client.get(
request, follow_redirects=True, headers={"Referer": SENTRY_TEST_HOST}
)
if response.status_code == 200:
if validator is None or validator(response.text):
break
time.sleep(1)
else:
raise AssertionError(
"timeout waiting for response status code 200 or valid data"
)
return response
@lru_cache
def get_sentry_dsn(client: httpx.Client) -> str:
response = poll_for_response(
f"{SENTRY_TEST_HOST}/api/0/projects/sentry/internal/keys/",
client,
lambda x: len(json.loads(x)[0]["dsn"]["public"]) > 0,
)
sentry_dsn = json.loads(response.text)[0]["dsn"]["public"]
return sentry_dsn
@pytest.fixture()
def client_login():
client = httpx.Client()
response = client.get(SENTRY_TEST_HOST, follow_redirects=True)
parser = BeautifulSoup(response.text, "html.parser")
login_csrf_token = parser.find("input", {"name": "csrfmiddlewaretoken"})["value"]
login_response = client.post(
f"{SENTRY_TEST_HOST}/auth/login/sentry/",
follow_redirects=True,
data={
"op": "login",
"username": TEST_USER,
"password": TEST_PASS,
"csrfmiddlewaretoken": login_csrf_token,
},
headers={"Referer": f"{SENTRY_TEST_HOST}/auth/login/sentry/"},
)
assert login_response.status_code == 200
yield (client, login_response)
def test_initial_redirect():
initial_auth_redirect = httpx.get(SENTRY_TEST_HOST, follow_redirects=True)
assert initial_auth_redirect.url == f"{SENTRY_TEST_HOST}/auth/login/sentry/"
def test_asset_internal_rewrite():
"""Tests whether we correctly map `/_assets/*` to `/_static/dist/sentry` as
we don't have a CDN setup in self-hosted."""
response = httpx.get(f"{SENTRY_TEST_HOST}/_assets/entrypoints/app.js")
assert response.status_code == 200
assert response.headers["Content-Type"] == "text/javascript"
def test_login(client_login):
client, login_response = client_login
parser = BeautifulSoup(login_response.text, "html.parser")
script_tag = parser.find(
"script", string=lambda x: x and "window.__initialData" in x
)
assert script_tag is not None
json_data = json.loads(script_tag.text.split("=", 1)[1].strip().rstrip(";"))
assert json_data["isAuthenticated"] is True
assert json_data["user"]["username"] == "test@example.com"
assert json_data["user"]["isSuperuser"] is True
assert login_response.cookies["sc"] is not None
# Set up initial/required settings (InstallWizard request)
client.headers.update({"X-CSRFToken": login_response.cookies["sc"]})
response = client.put(
f"{SENTRY_TEST_HOST}/api/0/internal/options/?query=is:required",
follow_redirects=True,
headers={"Referer": SENTRY_TEST_HOST},
data={
"mail.use-tls": False,
"mail.username": "",
"mail.port": 25,
"system.admin-email": "test@example.com",
"mail.password": "",
"system.url-prefix": SENTRY_TEST_HOST,
"auth.allow-registration": False,
"beacon.anonymous": True,
},
)
assert response.status_code == 200
def test_receive_event(client_login):
event_id = None
client, _ = client_login
sentry_sdk.init(dsn=get_sentry_dsn(client))
event_id = sentry_sdk.capture_exception(Exception("a failure"))
assert event_id is not None
response = poll_for_response(
f"{SENTRY_TEST_HOST}/api/0/projects/sentry/internal/events/{event_id}/", client
)
response_json = json.loads(response.text)
assert response_json["eventID"] == event_id
assert response_json["metadata"]["value"] == "a failure"
def test_cleanup_crons_running():
docker_services = subprocess.check_output(
[
"docker",
"compose",
"--ansi",
"never",
"ps",
"-a",
],
text=True,
)
pattern = re.compile(
r"(\-cleanup\s+running)|(\-cleanup[_-].+\s+Up\s+)", re.MULTILINE
)
cleanup_crons = pattern.findall(docker_services)
assert len(cleanup_crons) > 0
def test_custom_certificate_authorities():
# Set environment variable
os.environ["COMPOSE_FILE"] = (
"docker-compose.yml:_integration-test/custom-ca-roots/docker-compose.test.yml"
)
test_nginx_conf_path = "_integration-test/custom-ca-roots/nginx"
custom_certs_path = "certificates"
# Generate tightly constrained CA
ca_key = rsa.generate_private_key(
public_exponent=65537, key_size=2048, backend=default_backend()
)
ca_name = x509.Name(
[x509.NameAttribute(NameOID.COMMON_NAME, "TEST CA *DO NOT TRUST*")]
)
ca_cert = (
x509.CertificateBuilder()
.subject_name(ca_name)
.issuer_name(ca_name)
.public_key(ca_key.public_key())
.serial_number(x509.random_serial_number())
.not_valid_before(datetime.datetime.now(datetime.timezone.utc))
.not_valid_after(
datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(days=1)
)
.add_extension(x509.BasicConstraints(ca=True, path_length=None), critical=True)
.add_extension(
x509.KeyUsage(
digital_signature=False,
key_encipherment=False,
content_commitment=False,
data_encipherment=False,
key_agreement=False,
key_cert_sign=True,
crl_sign=True,
encipher_only=False,
decipher_only=False,
),
critical=True,
)
.add_extension(
x509.NameConstraints([x509.DNSName("self.test")], None), critical=True
)
.add_extension(
x509.SubjectKeyIdentifier.from_public_key(ca_key.public_key()),
critical=False,
)
.sign(private_key=ca_key, algorithm=hashes.SHA256())
)
ca_key_path = f"{test_nginx_conf_path}/ca.key"
ca_crt_path = f"{test_nginx_conf_path}/ca.crt"
with open(ca_key_path, "wb") as key_file:
key_file.write(
ca_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.TraditionalOpenSSL,
encryption_algorithm=serialization.NoEncryption(),
)
)
with open(ca_crt_path, "wb") as cert_file:
cert_file.write(ca_cert.public_bytes(serialization.Encoding.PEM))
# Create custom certs path and copy ca.crt
os.makedirs(custom_certs_path, exist_ok=True)
shutil.copyfile(ca_crt_path, f"{custom_certs_path}/test-custom-ca-roots.crt")
# Generate server key and certificate
self_test_key_path = os.path.join(test_nginx_conf_path, "self.test.key")
self_test_csr_path = os.path.join(test_nginx_conf_path, "self.test.csr")
self_test_cert_path = os.path.join(test_nginx_conf_path, "self.test.crt")
self_test_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
self_test_req = (
x509.CertificateSigningRequestBuilder()
.subject_name(
x509.Name(
[
x509.NameAttribute(
NameOID.COMMON_NAME, "Self Signed with CA Test Server"
)
]
)
)
.add_extension(
x509.SubjectAlternativeName([x509.DNSName("self.test")]), critical=False
)
.sign(self_test_key, hashes.SHA256())
)
self_test_cert = (
x509.CertificateBuilder()
.subject_name(
x509.Name(
[
x509.NameAttribute(
NameOID.COMMON_NAME, "Self Signed with CA Test Server"
)
]
)
)
.issuer_name(ca_cert.issuer)
.serial_number(x509.random_serial_number())
.not_valid_before(datetime.datetime.now(datetime.timezone.utc))
.not_valid_after(
datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(days=1)
)
.public_key(self_test_req.public_key())
.add_extension(
x509.SubjectAlternativeName([x509.DNSName("self.test")]), critical=False
)
.add_extension(
x509.SubjectKeyIdentifier.from_public_key(self_test_req.public_key()),
critical=False,
)
.add_extension(
x509.AuthorityKeyIdentifier.from_issuer_subject_key_identifier(
ca_cert.extensions.get_extension_for_class(x509.SubjectKeyIdentifier).value
),
critical=False,
)
.sign(private_key=ca_key, algorithm=hashes.SHA256())
)
# Save server key, CSR, and certificate
with open(self_test_key_path, "wb") as key_file:
key_file.write(
self_test_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.TraditionalOpenSSL,
encryption_algorithm=serialization.NoEncryption(),
)
)
with open(self_test_csr_path, "wb") as csr_file:
csr_file.write(self_test_req.public_bytes(serialization.Encoding.PEM))
with open(self_test_cert_path, "wb") as cert_file:
cert_file.write(self_test_cert.public_bytes(serialization.Encoding.PEM))
# Generate server key and certificate for fake.test
fake_test_key_path = os.path.join(test_nginx_conf_path, "fake.test.key")
fake_test_cert_path = os.path.join(test_nginx_conf_path, "fake.test.crt")
fake_test_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
fake_test_cert = (
x509.CertificateBuilder()
.subject_name(
x509.Name(
[x509.NameAttribute(NameOID.COMMON_NAME, "Self Signed Test Server")]
)
)
.issuer_name(
x509.Name(
[x509.NameAttribute(NameOID.COMMON_NAME, "Self Signed Test Server")]
)
)
.serial_number(x509.random_serial_number())
.not_valid_before(datetime.datetime.now(datetime.timezone.utc))
.not_valid_after(
datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(days=1)
)
.public_key(fake_test_key.public_key())
.add_extension(
x509.SubjectAlternativeName([x509.DNSName("fake.test")]), critical=False
)
.add_extension(
x509.SubjectKeyIdentifier.from_public_key(fake_test_key.public_key()),
critical=False,
)
.sign(private_key=fake_test_key, algorithm=hashes.SHA256())
)
# Save server key and certificate for fake.test
with open(fake_test_key_path, "wb") as key_file:
key_file.write(
fake_test_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.TraditionalOpenSSL,
encryption_algorithm=serialization.NoEncryption(),
)
)
# Our asserts for this test case must be executed within the web container, so we are copying a python test script into the mounted sentry directory
with open(fake_test_cert_path, "wb") as cert_file:
cert_file.write(fake_test_cert.public_bytes(serialization.Encoding.PEM))
shutil.copyfile(
"_integration-test/custom-ca-roots/custom-ca-roots-test.py",
"sentry/test-custom-ca-roots.py",
)
subprocess.run(
[
"docker",
"compose",
"--ansi",
"never",
"up",
"--wait",
"fixture-custom-ca-roots",
],
check=True,
)
subprocess.run(
[
"docker",
"compose",
"--ansi",
"never",
"run",
"--no-deps",
"web",
"python3",
"/etc/sentry/test-custom-ca-roots.py",
],
check=True,
)
subprocess.run(
[
"docker",
"compose",
"--ansi",
"never",
"rm",
"-s",
"-f",
"-v",
"fixture-custom-ca-roots",
],
check=True,
)
# Remove files
os.remove(f"{custom_certs_path}/test-custom-ca-roots.crt")
os.remove("sentry/test-custom-ca-roots.py")
# Unset environment variable
if "COMPOSE_FILE" in os.environ:
del os.environ["COMPOSE_FILE"]
@pytest.mark.skipif(os.environ.get("COMPOSE_PROFILES") != "feature-complete", reason="Only run if feature-complete")
def test_receive_transaction_events(client_login):
client, _ = client_login
sentry_sdk.init(
dsn=get_sentry_dsn(client), profiles_sample_rate=1.0, traces_sample_rate=1.0
)
def placeholder_fn():
sum = 0
for i in range(5):
sum += i
time.sleep(0.25)
with sentry_sdk.start_transaction(op="task", name="Test Transactions"):
placeholder_fn()
poll_for_response(
f"{SENTRY_TEST_HOST}/api/0/organizations/sentry/events/?dataset=profiles&field=profile.id&project=1&statsPeriod=1h",
client,
lambda x: len(json.loads(x)["data"]) > 0,
)
poll_for_response(
f"{SENTRY_TEST_HOST}/api/0/organizations/sentry/events/?dataset=spans&field=id&project=1&statsPeriod=1h",
client,
lambda x: len(json.loads(x)["data"]) > 0,
)
@pytest.mark.skipif(os.environ.get("COMPOSE_PROFILES") != "feature-complete", reason="Only run if feature-complete")
def test_receive_user_feedback_events(client_login):
client, _ = client_login
sentry_dsn = get_sentry_dsn(client)
# Execute `node --import instrument.js user-feedback.js` on the `nodejs` directory with the `SENTRY_DSN` env var set
env = os.environ.copy()
env["SENTRY_DSN"] = sentry_dsn
subprocess.run(
["node", "--import", "./instrument.js", "./user-feedback.js"],
check=True,
shell=False,
env=env,
cwd="_integration-test/nodejs",
stdout=sys.stdout,
stderr=sys.stderr,
timeout=60,
)
poll_for_response(
f"{SENTRY_TEST_HOST}/api/0/organizations/sentry/issues/?query=issue.category%3Afeedback",
client,
lambda x: len(json.loads(x)) > 0,
)
poll_for_response(
f"{SENTRY_TEST_HOST}/api/0/organizations/sentry/events/?dataset=issuePlatform&field=message&field=title&field=timestamp&project=1&statsPeriod=1h",
client,
lambda x: len(json.loads(x)["data"]) > 0,
)
@pytest.mark.skipif(os.environ.get("COMPOSE_PROFILES") != "feature-complete", reason="Only run if feature-complete")
def test_receive_logs_events(client_login):
client, _ = client_login
sentry_sdk.init(
dsn=get_sentry_dsn(client), profiles_sample_rate=1.0, traces_sample_rate=1.0, enable_logs=True,
)
sentry_logger.trace('Starting database connection {database}', database="users")
sentry_logger.debug('Cache miss for user {user_id}', user_id=123)
sentry_logger.info('Updated global cache')
sentry_logger.warning('Rate limit reached for endpoint {endpoint}', endpoint='/api/results/')
sentry_logger.error('Failed to process payment. Order: {order_id}. Amount: {amount}', order_id="or_2342", amount=99.99)
sentry_logger.fatal('Database {database} connection pool exhausted', database="users")
sentry_logger.error(
'Payment processing failed',
attributes={
'payment.provider': 'stripe',
'payment.method': 'credit_card',
'payment.currency': 'USD',
'user.subscription_tier': 'premium'
}
)
poll_for_response(
f"{SENTRY_TEST_HOST}/api/0/organizations/sentry/events/?dataset=ourlogs&field=sentry.item_id&field=project.id&field=trace&field=severity_number&field=severity&field=timestamp&field=timestamp_precise&field=observed_timestamp&field=message&project=1&statsPeriod=1h",
client,
lambda x: len(json.loads(x)["data"]) > 0,
)
def test_customizations():
commands = [
[
"docker",
"compose",
"--ansi",
"never",
"run",
"--no-deps",
"web",
"bash",
"-c",
"if [ ! -e /created-by-enhance-image ]; then exit 1; fi",
],
[
"docker",
"compose",
"--ansi",
"never",
"run",
"--no-deps",
"--entrypoint=/etc/sentry/entrypoint.sh",
"sentry-cleanup",
"bash",
"-c",
"if [ ! -e /created-by-enhance-image ]; then exit 1; fi",
],
[
"docker",
"compose",
"--ansi",
"never",
"run",
"--no-deps",
"web",
"python",
"-c",
"import ldap",
],
[
"docker",
"compose",
"--ansi",
"never",
"run",
"--no-deps",
"--entrypoint=/etc/sentry/entrypoint.sh",
"sentry-cleanup",
"python",
"-c",
"import ldap",
],
]
for command in commands:
result = subprocess.run(command, check=False)
assert result.returncode == 0
================================================
FILE: _integration-test/test_02_backup.py
================================================
import os
from os.path import join
import subprocess
def test_sentry_admin(setup_backup_restore_env_variables):
sentry_admin_sh = os.path.join(os.getcwd(), "sentry-admin.sh")
output = subprocess.run(
[sentry_admin_sh, "--help"], check=True, capture_output=True, encoding="utf8"
).stdout
assert "Usage: ./sentry-admin.sh" in output
assert "SENTRY_DOCKER_IO_DIR" in output
output = subprocess.run(
[sentry_admin_sh, "permissions", "--help"],
check=True,
capture_output=True,
encoding="utf8",
).stdout
assert "Usage: ./sentry-admin.sh permissions" in output
def test_01_backup(setup_backup_restore_env_variables):
# Docker was giving me permission issues when trying to create this file and write to it even after giving read + write access
# to group and owner. Instead, try creating the empty file and then give everyone write access to the backup file
file_path = os.path.join(os.getcwd(), "sentry", "backup.json")
sentry_admin_sh = os.path.join(os.getcwd(), "sentry-admin.sh")
open(file_path, "a", encoding="utf8").close()
os.chmod(file_path, 0o666)
assert os.path.getsize(file_path) == 0
subprocess.run(
[
sentry_admin_sh,
"export",
"global",
"/sentry-admin/backup.json",
"--no-prompt",
],
check=True,
)
assert os.path.getsize(file_path) > 0
def test_02_import(setup_backup_restore_env_variables):
# Bring postgres down and recreate the docker volume
subprocess.run(["docker", "compose", "--ansi", "never", "down"], check=True)
# We reset all DB-related volumes here and not just Postgres although the backups
# are only for Postgres. The reason is to get a "clean slate" as we need the Kafka
# and Clickhouse volumes to be back to their initial state as well (without any events)
# We cannot just rm and create them as they still need the migrations.
for name in ("postgres", "clickhouse", "kafka"):
subprocess.run(["docker", "volume", "rm", f"sentry-{name}"], check=True)
subprocess.run(["docker", "volume", "create", f"sentry-{name}"], check=True)
subprocess.run(
[
"rsync",
"-aW",
"--super",
"--numeric-ids",
"--no-compress",
"--mkpath",
join(os.environ["RUNNER_TEMP"], "volumes", f"sentry-{name}", ""),
f"/var/lib/docker/volumes/sentry-{name}/",
],
check=True,
capture_output=True,
)
sentry_admin_sh = os.path.join(os.getcwd(), "sentry-admin.sh")
subprocess.run(
[
sentry_admin_sh,
"import",
"global",
"/sentry-admin/backup.json",
"--no-prompt",
],
check=True,
)
# TODO: Check something actually restored here like the test user we have from earlier
================================================
FILE: _unit-test/_test_setup.sh
================================================
set -euo pipefail
source install/_lib.sh
_ORIGIN=$(pwd)
rm -rf /tmp/sentry-self-hosted-test-sandbox.*
_SANDBOX="$(mktemp -d /tmp/sentry-self-hosted-test-sandbox.XXX)"
source install/detect-platform.sh
docker build -t sentry-self-hosted-jq-local --platform="$DOCKER_PLATFORM" jq
report_success() {
echo "$(basename $0) - Success 👍"
}
teardown() {
test "${DEBUG:-}" || rm -rf "$_SANDBOX"
cd "$_ORIGIN"
}
setup() {
# Clone the local repo into a temp dir. FWIW `git clone --local` breaks for
# me because it depends on hard-linking, which doesn't work across devices,
# and I happen to have my workspace and /tmp on separate devices.
git -c advice.detachedHead=false clone --depth=1 "file://$_ORIGIN" "$_SANDBOX"
# Now propagate any local changes from the working copy to the sandbox. This
# provides a pretty nice dev experience: edit the files in the working copy,
# then run `DEBUG=1 some-test.sh` to leave the sandbox up for interactive
# dev/debugging.
git status --porcelain | while read line; do
# $line here is something like `M some-script.sh`.
local filepath="$(cut -f2 -d' ' <(echo $line))"
local filestatus="$(cut -f1 -d' ' <(echo $line))"
case $filestatus in
D)
rm "$_SANDBOX/$filepath"
;;
A | M | AM | ??)
ln -sf "$(realpath $filepath)" "$_SANDBOX/$filepath"
;;
**)
echo "Wuh? $line"
exit 77
;;
esac
done
cd "$_SANDBOX"
trap teardown EXIT
}
setup
================================================
FILE: _unit-test/bootstrap-s3-nodestore-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
source install/dc-detect-version.sh
source install/create-docker-volumes.sh
# Set the flag to apply automatic updates
export APPLY_AUTOMATIC_CONFIG_UPDATES=1
# Here we're just gonna test to run it multiple times
# Only to make sure it doesn't break
for i in $(seq 1 5); do
source install/bootstrap-s3-nodestore.sh
done
report_success
================================================
FILE: _unit-test/bootstrap-s3-profiles-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
source install/dc-detect-version.sh
source install/create-docker-volumes.sh
source install/ensure-files-from-examples.sh
export COMPOSE_PROFILES="feature-complete"
$dc pull vroom
source install/ensure-correct-permissions-profiles-dir.sh
# Generate some random files on `sentry-vroom` volume for testing
$dc run --rm --no-deps -v sentry-vroom:/var/vroom/sentry-profiles --entrypoint /bin/bash vroom -c '
for i in $(seq 1 1000); do
echo This is test file $i > /var/vroom/sentry-profiles/test_file_$i.txt
done
'
# Set the flag to apply automatic updates
export APPLY_AUTOMATIC_CONFIG_UPDATES=1
# Here we're just gonna test to run it multiple times
# Only to make sure it doesn't break
for i in $(seq 1 5); do
source install/bootstrap-s3-profiles.sh
done
# Ensure that the files have been migrated to SeaweedFS
migrated_files_count=$($dc exec seaweedfs s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=seaweedfs:8333 --host-bucket="seaweedfs:8333/%(bucket)" ls s3://profiles/ | wc -l)
if [[ "$migrated_files_count" -ne 1000 ]]; then
echo "Error: Expected 1000 migrated files, but found $migrated_files_count"
exit 1
fi
# Manual cleanup, otherwise `create-docker-volumes.sh` will fail
$dc down -v --remove-orphans
report_success
================================================
FILE: _unit-test/create-docker-volumes-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
get_volumes() {
# If grep returns no strings, we still want to return without error
docker volume ls --quiet | { grep '^sentry-.*' || true; } | sort
}
# Maybe they exist prior, maybe they don't. Script is idempotent.
expected_volumes="sentry-clickhouse
sentry-data
sentry-kafka
sentry-postgres
sentry-redis
sentry-seaweedfs"
before=$(get_volumes)
test "$before" == "" || test "$before" == "$expected_volumes"
source install/create-docker-volumes.sh
after=$(get_volumes)
test "$after" == "$expected_volumes"
report_success
================================================
FILE: _unit-test/ensure-relay-credentials-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
source install/dc-detect-version.sh
# using _file format for these variables since there is a creds defined in dc-detect-version.sh
cfg_file=relay/config.yml
creds_file=relay/credentials.json
# Relay files don't exist in a clean clone.
test ! -f $cfg_file
test ! -f $creds_file
# Running the install script adds them.
source install/ensure-relay-credentials.sh
test -f $cfg_file
test -f $creds_file
test "$(jq -r 'keys[2]' "$creds_file")" = "secret_key"
# If the files exist we don't touch it.
echo GARBAGE >$cfg_file
echo MOAR GARBAGE >$creds_file
source install/ensure-relay-credentials.sh
test "$(cat $cfg_file)" = "GARBAGE"
test "$(cat $creds_file)" = "MOAR GARBAGE"
report_success
================================================
FILE: _unit-test/error-handling-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
export REPORT_SELF_HOSTED_ISSUES=1
# This is set up in dc-detect-version.sh, but for
# our purposes we don't care about proxies.
dbuild="docker build"
source install/error-handling.sh
# mock send_envelope
send_envelope() {
echo "Test Sending $1"
}
##########################
export -f send_envelope
echo "Testing initial send_event"
export log_file=test_log.txt
echo "Test Logs" >"$log_file"
echo "Error Msg" >>"$log_file"
breadcrumbs=$(generate_breadcrumb_json | sed '$d' | $jq -s -c)
SEND_EVENT_RESPONSE=$(
send_event \
"'foo' exited with status 1" \
"Test exited with status 1" \
"Traceback: ignore me" \
"{\"ignore\": \"me\"}" \
"$breadcrumbs"
)
rm "$log_file"
expected_filename='sentry-envelope-f73e4da437c42a1d28b86a81ebcff35d'
test "$SEND_EVENT_RESPONSE" == "Test Sending $expected_filename"
ENVELOPE_CONTENTS=$(cat "/tmp/$expected_filename")
test "$ENVELOPE_CONTENTS" == "$(cat _unit-test/snapshots/$expected_filename)"
echo "Pass."
##########################
echo "Testing send_event duplicate"
SEND_EVENT_RESPONSE=$(
send_event \
"'foo' exited with status 1" \
"Test exited with status 1" \
"Traceback: ignore me" \
"{\"ignore\": \"me\"}" \
"$breadcrumbs"
)
test "$SEND_EVENT_RESPONSE" == "Looks like you've already sent this error to us, we're on it :)"
echo "Pass."
rm "/tmp/$expected_filename"
##########################
echo "Testing cleanup without minimizing downtime"
export REPORT_SELF_HOSTED_ISSUES=0
export MINIMIZE_DOWNTIME=''
export dc=':'
echo "Test Logs" >"$log_file"
CLEANUP_RESPONSE=$(cleanup ERROR) # the linenumber of this line must match just below
rm "$log_file"
test "$CLEANUP_RESPONSE" == 'Error in _unit-test/error-handling-test.sh:62.
'\''local cmd="${BASH_COMMAND}"'\'' exited with status 0
Cleaning up...'
echo "Pass."
##########################
echo "Testing cleanup while minimizing downtime"
export REPORT_SELF_HOSTED_ISSUES=0
export MINIMIZE_DOWNTIME=1
echo "Test Logs" >"$log_file"
CLEANUP_RESPONSE=$(cleanup ERROR) # the linenumber of this line must match just below
rm "$log_file"
test "$CLEANUP_RESPONSE" == 'Error in _unit-test/error-handling-test.sh:76.
'\''local cmd="${BASH_COMMAND}"'\'' exited with status 0
*NOT* cleaning up, to clean your environment run "docker compose stop".'
echo "Pass."
##########################
report_success
================================================
FILE: _unit-test/geoip-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
mmdb=geoip/GeoLite2-City.mmdb
# Starts with no mmdb, ends up with empty.
test ! -f $mmdb
source install/geoip.sh
diff -rub $mmdb $mmdb.empty
# Doesn't clobber existing, though.
echo GARBAGE >$mmdb
source install/geoip.sh
test "$(cat $mmdb)" = "GARBAGE"
report_success
================================================
FILE: _unit-test/js-sdk-assets-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
source install/dc-detect-version.sh
$dcb --force-rm web
$dc pull nginx
export SETUP_JS_SDK_ASSETS=1
source install/setup-js-sdk-assets.sh
sdk_files=$($dcr --no-deps nginx ls -lah /var/www/js-sdk/)
sdk_tree=$($dcr --no-deps nginx tree /var/www/js-sdk/ | tail -n 1)
non_empty_file_count=$($dcr --no-deps nginx find /var/www/js-sdk/ -type f -size +1k | wc -l)
# `sdk_files` should contains 7 lines, '4.*', '5.*', '6.*', `7.*`, `8.*`, `9.*`, and `10.*`
echo $sdk_files
total_directories=$(echo "$sdk_files" | grep -c '[4-9|10]\.[0-9]*\.[0-9]*$')
echo $total_directories
test "7" == "$total_directories"
echo "Pass"
# `sdk_tree` should output "7 directories, 29 files"
echo "$sdk_tree"
test "7 directories, 29 files" == "$(echo "$sdk_tree")"
echo "Pass"
# Files should all be >1k (ensure they are not empty)
echo "Testing file sizes"
test "29" == "$non_empty_file_count"
echo "Pass"
# Files should be owned by the root user
echo "Testing file ownership"
directory_owners=$(echo "$sdk_files" | awk '$3=="root" { print $0 }' | wc -l)
echo "$directory_owners"
test "$directory_owners" == "9"
echo "Pass"
report_success
================================================
FILE: _unit-test/merge-env-file-test.sh
================================================
#!/usr/bin/env bash
# This is a test file for a part of `_lib.sh`, where we read `.env.custom` file if there is one.
# We only want to give very minimal value to the `.env.custom` file, and expect that it would
# be merged with the original `.env` file, with the `.env.custom` file taking precedence.
cat <<EOF >".env.custom"
SENTRY_EVENT_RETENTION_DAYS=10
EOF
# The `_test_setup.sh` script sources `install/_lib.sh`, so.. finger crossed this should works.
source _unit-test/_test_setup.sh
rm -f .env.custom
echo "Expecting SENTRY_EVENT_RETENTION_DAYS to be 10, got ${SENTRY_EVENT_RETENTION_DAYS}"
test "$SENTRY_EVENT_RETENTION_DAYS" == "10"
echo "Pass"
echo "Expecting SENTRY_BIND to be 9000, got ${SENTRY_BIND}"
test "$SENTRY_BIND" == "9000"
echo "Pass"
echo "Expecting COMPOSE_PROJECT_NAME to be sentry-self-hosted, got ${COMPOSE_PROJECT_NAME}"
test "$COMPOSE_PROJECT_NAME" == "sentry-self-hosted"
echo "Pass"
report_success
================================================
FILE: _unit-test/migrate-pgbouncer-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
source install/dc-detect-version.sh
source install/ensure-files-from-examples.sh
cp $SENTRY_CONFIG_PY /tmp/sentry_conf_py
# Set the flag to apply automatic updates
export APPLY_AUTOMATIC_CONFIG_UPDATES=1
# Declare expected content
expected_db_config=$(
cat <<'EOF'
DATABASES = {
"default": {
"ENGINE": "sentry.db.postgres",
"NAME": "postgres",
"USER": "postgres",
"PASSWORD": "",
"HOST": "pgbouncer",
"PORT": "",
}
}
EOF
)
echo "Test 1 (pre 25.9.0 release)"
# Modify the `DATABASES = {` to the next `}` line, with:
# DATABASES = {
# "default": {
# "ENGINE": "sentry.db.postgres",
# "NAME": "postgres",
# "USER": "postgres",
# "PASSWORD": "",
# "HOST": "postgres",
# "PORT": "",
# }
# }
# Create the replacement text in a temp file
cat >/tmp/sentry_conf_py_db_config <<'EOF'
DATABASES = {
"default": {
"ENGINE": "sentry.db.postgres",
"NAME": "postgres",
"USER": "postgres",
"PASSWORD": "",
"HOST": "postgres",
"PORT": "",
}
}
EOF
# Replace the block
sed -i '/^DATABASES = {$/,/^}$/{
/^DATABASES = {$/r /tmp/sentry_conf_py_db_config
d
}' $SENTRY_CONFIG_PY
# Clean up
rm /tmp/sentry_conf_py_db_config
source install/migrate-pgbouncer.sh
# Extract actual content
actual_db_config=$(sed -n '/^DATABASES = {$/,/^}$/p' $SENTRY_CONFIG_PY)
# Compare
if [ "$actual_db_config" = "$expected_db_config" ]; then
echo "DATABASES section is correct"
else
echo "DATABASES section does not match"
echo "Expected:"
echo "$expected_db_config"
echo "Actual:"
echo "$actual_db_config"
exit 1
fi
# Reset the file
rm $SENTRY_CONFIG_PY
cp /tmp/sentry_conf_py $SENTRY_CONFIG_PY
echo "Test 2 (post 25.9.0 release)"
# Modify the `DATABASES = {` to the next `}` line, with:
# DATABASES = {
# "default": {
# "ENGINE": "sentry.db.postgres",
# "NAME": "postgres",
# "USER": "postgres",
# "PASSWORD": "",
# "HOST": "pgbouncer",
# "PORT": "",
# }
# }
# Create the replacement text in a temp file
cat >/tmp/sentry_conf_py_db_config <<'EOF'
DATABASES = {
"default": {
"ENGINE": "sentry.db.postgres",
"NAME": "postgres",
"USER": "postgres",
"PASSWORD": "",
"HOST": "pgbouncer",
"PORT": "",
}
}
EOF
# Replace the block
sed -i '/^DATABASES = {$/,/^}$/{
/^DATABASES = {$/r /tmp/sentry_conf_py_db_config
d
}' $SENTRY_CONFIG_PY
# Clean up
rm /tmp/sentry_conf_py_db_config
source install/migrate-pgbouncer.sh
# Extract actual content
actual_db_config=$(sed -n '/^DATABASES = {$/,/^}$/p' $SENTRY_CONFIG_PY)
# Compare
if [ "$actual_db_config" = "$expected_db_config" ]; then
echo "DATABASES section is correct"
else
echo "DATABASES section does not match"
echo "Expected:"
echo "$expected_db_config"
echo "Actual:"
echo "$actual_db_config"
exit 1
fi
# Reset the file
rm $SENTRY_CONFIG_PY
cp /tmp/sentry_conf_py $SENTRY_CONFIG_PY
echo "Test 3 (custom postgres config)"
# Modify the `DATABASES = {` to the next `}` line, with:
# DATABASES = {
# "default": {
# "ENGINE": "sentry.db.postgres",
# "NAME": "postgres",
# "USER": "sentry",
# "PASSWORD": "sentry",
# "HOST": "postgres.internal",
# "PORT": "5432",
# }
# }
# Create the replacement text in a temp file
cat >/tmp/sentry_conf_py_db_config <<'EOF'
DATABASES = {
"default": {
"ENGINE": "sentry.db.postgres",
"NAME": "postgres",
"USER": "sentry",
"PASSWORD": "sentry",
"HOST": "postgres.internal",
"PORT": "5432",
}
}
EOF
# Replace the block
sed -i '/^DATABASES = {$/,/^}$/{
/^DATABASES = {$/r /tmp/sentry_conf_py_db_config
d
}' $SENTRY_CONFIG_PY
# Clean up
rm /tmp/sentry_conf_py_db_config
source install/migrate-pgbouncer.sh
# Extract actual content
actual_db_config=$(sed -n '/^DATABASES = {$/,/^}$/p' $SENTRY_CONFIG_PY)
# THe file should NOT be modified
if [ "$actual_db_config" = "$expected_db_config" ]; then
echo "DATABASES section SHOULD NOT be modified"
echo "Expected:"
echo "$expected_db_config"
echo "Actual:"
echo "$actual_db_config"
exit 1
else
echo "DATABASES section is correct"
fi
# Remove the file
rm $SENTRY_CONFIG_PY /tmp/sentry_conf_py
report_success
================================================
FILE: _unit-test/multiple-seaweedfs-bucket-test.sh
================================================
#!/usr/bin/env bash
source _unit-test/_test_setup.sh
source install/dc-detect-version.sh
source install/create-docker-volumes.sh
start_service_and_wait_ready seaweedfs
$dcx seaweedfs apk add --no-cache s3cmd
s3cmd="$dc exec seaweedfs s3cmd"
# Create multiple buckets for testing
buckets=(bucket1 bucket2 bucket3)
for bucket in "${buckets[@]}"; do
$s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' mb s3://$bucket
done
# Verify that all buckets were created successfully
bucket_list=$($s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' ls)
for bucket in "${buckets[@]}"; do
if ! echo "$bucket_list" | grep -q "s3://$bucket"; then
echo "Error: Bucket s3://$bucket was not created successfully."
exit 1
fi
done
# Can find "bucket2"
if ! echo "$bucket_list" | grep -q "s3://bucket2"; then
echo "Error: Bucket s3://bucket2 was not found."
exit 1
fi
# Can't find "bucket5", should not exist
if echo "$bucket_list" | grep -q "s3://bucket5"; then
echo "Error: Bucket s3://bucket5 should not exist."
exit 1
fi
report_success
================================================
FILE: _unit-test/snapshots/sentry-envelope-f73e4da437c42a1d28b86a81ebcff35d
================================================
{"event_id":"f73e4da437c42a1d28b86a81ebcff35d","dsn":"https://19555c489ded4769978daae92f2346ca@self-hosted.getsentry.net/3"}
{"type":"event"}
{"level":"error","exception":{"values":[{"type":"Error","value":"Test exited with status 1","stacktrace":{"frames":[{"ignore":"me"}]}}]},"breadcrumbs":{"values":[{"message":"Test Logs","category":"log","level":"info"}]},"fingerprint":["f73e4da437c42a1d28b86a81ebcff35d"]}
{"type":"attachment","length":20,"content_type":"text/plain","filename":"install_log.txt"}
Test Logs
Error Msg
================================================
FILE: action.yaml
================================================
name: "Sentry self-hosted end-to-end tests"
inputs:
project_name:
required: false
description: "e.g. snuba, sentry, relay, self-hosted"
image_url:
required: false
description: "The URL to the built relay, snuba, sentry image to test against."
compose_profiles:
required: false
description: "Docker Compose profile to use. Defaults to feature-complete."
CODECOV_TOKEN:
required: false
description: "The Codecov token to upload coverage."
runs:
using: "composite"
steps:
- name: Validate inputs and configure test image
shell: bash
env:
PROJECT_NAME: ${{ inputs.project_name }}
IMAGE_URL: ${{ inputs.image_url }}
COMPOSE_PROFILES: ${{ inputs.compose_profiles }}
run: |
if [[ -n "$PROJECT_NAME" && -n "$IMAGE_URL" ]]; then
image_var=$(echo "${PROJECT_NAME}_IMAGE" | tr '[:lower:]' '[:upper:]')
echo "${image_var}=$IMAGE_URL" >> ${{ github.action_path }}/.env
elif [[ -z "$PROJECT_NAME" && -z "$IMAGE_URL" ]]; then
echo "No project name and image URL set. Skipping image configuration."
else
echo "You must set both project_name and image_url or unset both."
echo "project_name: $PROJECT_NAME, image_url: $IMAGE_URL"
exit 1
fi
# `COMPOSE_PROFILES` may only be `feature-complete` or `errors-only`
if [[ "$COMPOSE_PROFILES" != "" && "$COMPOSE_PROFILES" != "feature-complete" && "$COMPOSE_PROFILES" != "errors-only" ]]; then
echo "COMPOSE_PROFILES must be either unset, or set to either 'feature-complete' or 'errors-only'."
exit 1
else
echo "COMPOSE_PROFILES=$COMPOSE_PROFILES" >> ${{ github.action_path }}/.env
fi
- name: Cleanup runner image
shell: bash
run: |
### Inspired by https://github.com/endersonmenezes/free-disk-space ###
sudo rm -rf /usr/local/.ghcup
sudo rm -rf /home/runner/Android /usr/local/lib/android /opt/android /usr/local/android-sdk
sudo rm -rf /usr/share/dotnet /usr/share/doc/dotnet-*
df -h
- name: Setup uv
uses: astral-sh/setup-uv@e06108dd0aef18192324c70427afc47652e63a82 # v7.5.0
with:
working-directory: ${{ github.action_path }}
version: "0.9.15"
# we just cache the venv-dir directly in action-setup-venv
enable-cache: false
- uses: getsentry/action-setup-venv@6e8aebf461914919d9de6e3082669d6bfecc400c # v3.1.0
with:
python-version: "3.12"
cache-dependency-path: uv.lock
install-cmd: uv sync --frozen --active
working-directory: ${{ github.action_path }}
- name: Setup dev environment
shell: bash
run: |
cd ${{ github.action_path }}
echo "PY_COLORS=1" >> "$GITHUB_ENV"
### pytest-sentry configuration ###
if [ "$GITHUB_REPOSITORY" = "getsentry/self-hosted" ]; then
echo "PYTEST_SENTRY_DSN=$SELF_HOSTED_TESTING_DSN" >> $GITHUB_ENV
echo "PYTEST_SENTRY_TRACES_SAMPLE_RATE=0" >> $GITHUB_ENV
# This records failures on master to sentry in order to detect flakey tests, as it's
# expected that people have failing tests on their PRs
if [ "$GITHUB_REF" = "refs/heads/master" ]; then
echo "PYTEST_SENTRY_ALWAYS_REPORT=1" >> $GITHUB_ENV
fi
fi
- name: Compute Docker Volume Cache Keys
id: cache_key
shell: bash
run: |
source ${{ github.action_path }}/.env
# See https://explainshell.com/explain?cmd=ls%20-Rv1rpq
# for that long `ls` command
SENTRY_MIGRATIONS_MD5=$(docker run --rm --entrypoint bash $SENTRY_IMAGE -c '{ cat migrations_lockfile.txt; grep -Poz "(?s)(?<=class Topic\\(Enum\\):\\n).+?(?=\\n\\n\\n)" src/sentry/conf/types/kafka_definition.py; }' | md5sum | cut -d ' ' -f 1)
echo "SENTRY_MIGRATIONS_MD5=$SENTRY_MIGRATIONS_MD5" >> $GITHUB_OUTPUT
SNUBA_MIGRATIONS_MD5=$(docker run --rm --entrypoint bash $SNUBA_IMAGE -c '{ ls -Rv1rpq snuba/snuba_migrations/**/*.py; grep -Poz "(?s)(?<=class Topic\\(Enum\\):\\n).+?(?=\\n\\n\\n)" snuba/utils/streams/topics.py; }' | md5sum | cut -d ' ' -f 1)
echo "SNUBA_MIGRATIONS_MD5=$SNUBA_MIGRATIONS_MD5" >> $GITHUB_OUTPUT
echo "ARCH=$(uname -m)" >> $GITHUB_OUTPUT
- name: Restore Sentry Volume Cache
id: restore_cache_sentry
uses: BYK/docker-volume-cache-action/restore@be89365902126f508dcae387a32ec3712df6b1cd
with:
key: db-volumes-sentry-v3-${{ steps.cache_key.outputs.ARCH }}-${{ inputs.compose_profiles }}-${{ steps.cache_key.outputs.SENTRY_MIGRATIONS_MD5 }}
restore-keys: |
key: db-volumes-sentry-v3-${{ steps.cache_key.outputs.ARCH }}-${{ inputs.compose_profiles }}
volumes: |
sentry-postgres
- name: Restore Snuba Volume Cache
id: restore_cache_snuba
uses: BYK/docker-volume-cache-action/restore@be89365902126f508dcae387a32ec3712df6b1cd
with:
key: db-volumes-snuba-v3-${{ steps.cache_key.outputs.ARCH }}-${{ inputs.compose_profiles }}-${{ steps.cache_key.outputs.SNUBA_MIGRATIONS_MD5 }}
restore-keys: |
key: db-volumes-snuba-v3-${{ steps.cache_key.outputs.ARCH }}-${{ inputs.compose_profiles }}
volumes: |
sentry-clickhouse
- name: Restore Kafka Volume Cache
id: restore_cache_kafka
uses: BYK/docker-volume-cache-action/restore@be89365902126f508dcae387a32ec3712df6b1cd
with:
key: db-volumes-kafka-v2-${{ steps.cache_key.outputs.ARCH }}-${{ inputs.compose_profiles }}-${{ steps.cache_key.outputs.SENTRY_MIGRATIONS_MD5 }}-${{ steps.cache_key.outputs.SNUBA_MIGRATIONS_MD5 }}
restore-keys: |
db-volumes-kafka-v2-${{ steps.cache_key.outputs.ARCH }}-${{ inputs.compose_profiles }}-${{ steps.cache_key.outputs.SENTRY_MIGRATIONS_MD5 }}
db-volumes-kafka-v2-${{ steps.cache_key.outputs.ARCH }}-${{ inputs.compose_profiles }}
volumes: |
sentry-kafka
- name: Install self-hosted
env:
# Note that cache keys for Sentry and Snuba have their respective Kafka configs built into them
# and the Kafka volume cache is comprises both keys. This way we can omit the Kafka cache hit
# in here to still avoid running Sentry or Snuba migrations if only one of their Kafka config has
# changed. Heats up your head a bit but if you think about it, it makes sense.
SKIP_SENTRY_MIGRATIONS: ${{ steps.restore_cache_sentry.outputs.cache-hit == 'true' && '1' || '' }}
SKIP_SNUBA_MIGRATIONS: ${{ steps.restore_cache_snuba.outputs.cache-hit == 'true' && '1' || '' }}
shell: bash
run: |
cd ${{ github.action_path }}
# Add some customizations to test that path
cat <<EOT >> sentry/enhance-image.sh
#!/bin/bash
touch /created-by-enhance-image
apt-get update
apt-get install -y gcc libsasl2-dev python-dev-is-python3 libldap2-dev libssl-dev
EOT
chmod 755 sentry/enhance-image.sh
echo "python-ldap" > sentry/requirements.txt
./install.sh --no-report-self-hosted-issues --skip-commit-check
- name: Save Sentry Volume Cache
if: steps.restore_cache_sentry.outputs.cache-hit != 'true'
uses: BYK/docker-volume-cache-action/save@be89365902126f508dcae387a32ec3712df6b1cd
with:
key: ${{ steps.restore_cache_sentry.outputs.cache-primary-key }}
volumes: |
sentry-postgres
- name: Save Snuba Volume Cache
if: steps.restore_cache_snuba.outputs.cache-hit != 'true'
uses: BYK/docker-volume-cache-action/save@be89365902126f508dcae387a32ec3712df6b1cd
with:
key: ${{ steps.restore_cache_snuba.outputs.cache-primary-key }}
volumes: |
sentry-clickhouse
- name: Save Kafka Volume Cache
if: steps.restore_cache_kafka.outputs.cache-hit != 'true'
uses: BYK/docker-volume-cache-action/save@be89365902126f508dcae387a32ec3712df6b1cd
with:
key: ${{ steps.restore_cache_kafka.outputs.cache-primary-key }}
volumes: |
sentry-kafka
- name: Setup swapfile
shell: bash
run: |
sudo swapoff -a
sudo fallocate -l 16G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo swapon --show
free -h
- name: Setup Nodejs
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "22.x"
- name: Install Nodejs dependencies
shell: bash
run: |
cd ${{ github.action_path }}/_integration-test/nodejs
npm ci
- name: Integration Test
shell: bash
env:
COMPOSE_PROFILES: ${{ inputs.compose_profiles }}
run: |
sudo chown root /usr/bin/rsync && sudo chmod u+s /usr/bin/rsync
rsync -aW --super --numeric-ids --no-compress --mkpath \
/var/lib/docker/volumes/sentry-postgres \
/var/lib/docker/volumes/sentry-clickhouse \
/var/lib/docker/volumes/sentry-kafka \
"$RUNNER_TEMP/volumes/"
cd ${{ github.action_path }}
pytest -x --cov --junitxml=junit.xml _integration-test/
- name: Upload coverage to Codecov
if: inputs.CODECOV_TOKEN
continue-on-error: true
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
with:
directory: ${{ github.action_path }}
token: ${{ inputs.CODECOV_TOKEN }}
slug: getsentry/self-hosted
- name: Upload test results to Codecov
if: inputs.CODECOV_TOKEN && !cancelled()
continue-on-error: true
uses: codecov/test-results-action@0fa95f0e1eeaafde2c782583b36b28ad0d8c77d3 # v1.2.1
with:
directory: ${{ github.action_path }}
token: ${{ inputs.CODECOV_TOKEN }}
- name: Inspect failure
if: failure()
shell: bash
run: |
echo "::group::Inspect failure - docker compose ps"
cd ${{ github.action_path }}
docker compose ps
echo "::endgroup::"
echo "::group::Inspect failure - docker compose logs"
docker compose logs
echo "::endgroup::"
================================================
FILE: certificates/.gitignore
================================================
# Add all custom CAs in this folder
*
!.gitignore
================================================
FILE: clickhouse/Dockerfile
================================================
ARG BASE_IMAGE
FROM ${BASE_IMAGE}
================================================
FILE: clickhouse/config.xml
================================================
<yandex>
<!-- This include is important! It is required for the version of Clickhouse used on ARM to read the environment variable. This must be a one-liner to avoid errors in Clickhouse. -->
<max_server_memory_usage_to_ram_ratio from_env="MAX_MEMORY_USAGE_RATIO"/>
<logger>
<level>warning</level>
<console>true</console>
</logger>
<query_thread_log remove="remove"/>
<query_log remove="remove"/>
<text_log remove="remove"/>
<trace_log remove="remove"/>
<metric_log remove="remove"/>
<asynchronous_metric_log remove="remove"/>
<!-- Update: Required for newer versions of Clickhouse -->
<session_log remove="remove"/>
<part_log remove="remove"/>
<allow_nullable_key>1</allow_nullable_key>
<profiles>
<default>
<log_queries>0</log_queries>
<log_query_threads>0</log_query_threads>
</default>
</profiles>
<merge_tree>
<enable_mixed_granularity_parts>1</enable_mixed_granularity_parts>
<!-- Increase "max_suspicious_broken_parts" in case of errors with Clickhouse like "Suspiciously many broken parts to remove".
see: https://github.com/getsentry/self-hosted/issues/2832 -->
<max_suspicious_broken_parts>100</max_suspicious_broken_parts>
</merge_tree>
</yandex>
================================================
FILE: clickhouse/default-password.xml
================================================
<clickhouse>
<users>
<default>
<password></password>
<networks>
<ip>::/0</ip>
</networks>
</default>
</users>
</clickhouse>
================================================
FILE: codecov.yml
================================================
coverage:
status:
project:
default:
only_pulls: true
patch:
default:
only_pulls: true
================================================
FILE: cron/Dockerfile
================================================
ARG BASE_IMAGE
FROM ${BASE_IMAGE}
USER 0
RUN if [ -n "${HTTP_PROXY}" ]; then echo "Acquire::http::proxy \"${HTTP_PROXY}\";" >> /etc/apt/apt.conf; fi
RUN if [ -n "${HTTPS_PROXY}" ]; then echo "Acquire::https::proxy \"${HTTPS_PROXY}\";" >> /etc/apt/apt.conf; fi
RUN if [ -n "${http_proxy}" ]; then echo "Acquire::http::proxy \"${http_proxy}\";" >> /etc/apt/apt.conf; fi
RUN if [ -n "${https_proxy}" ]; then echo "Acquire::https::proxy \"${https_proxy}\";" >> /etc/apt/apt.conf; fi
RUN apt-get update && apt-get install -y --no-install-recommends cron && \
rm -r /var/lib/apt/lists/*
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
================================================
FILE: cron/entrypoint.sh
================================================
#!/usr/bin/env bash
if [ "$(ls -A /usr/local/share/ca-certificates/)" ]; then
update-ca-certificates
fi
# Prior art:
# - https://github.com/renskiy/cron-docker-image/blob/5600db37acf841c6d7a8b4f3866741bada5b4622/debian/start-cron#L34-L36
# - https://blog.knoldus.com/running-a-cron-job-in-docker-container/
declare -p | grep -Ev 'BASHOPTS|BASH_VERSINFO|EUID|PPID|SHELLOPTS|UID' >/container.env
{ for cron_job in "$@"; do echo -e "SHELL=/bin/bash
BASH_ENV=/container.env
${cron_job} > /proc/1/fd/1 2>/proc/1/fd/2"; done; } |
sed --regexp-extended 's/\\(.)/\1/g' |
crontab -
crontab -l
exec cron -f -l -L 15
================================================
FILE: docker-compose.yml
================================================
x-restart-policy: &restart_policy
restart: unless-stopped
x-pull-policy: &pull_policy
pull_policy: never
x-depends_on-healthy: &depends_on-healthy
condition: service_healthy
x-depends_on-default: &depends_on-default
condition: service_started
x-healthcheck-defaults: &healthcheck_defaults
# Avoid setting the interval too small, as docker uses much more CPU than one would expect.
# Related issues:
# https://github.com/moby/moby/issues/39102
# https://github.com/moby/moby/issues/39388
# https://github.com/getsentry/self-hosted/issues/1000
interval: "$HEALTHCHECK_INTERVAL"
timeout: "$HEALTHCHECK_TIMEOUT"
retries: $HEALTHCHECK_RETRIES
start_period: "$HEALTHCHECK_START_PERIOD"
x-file-healthcheck: &file_healthcheck_defaults
test: ["CMD-SHELL", "rm /tmp/health.txt"]
interval: "$HEALTHCHECK_FILE_INTERVAL"
timeout: "$HEALTHCHECK_FILE_TIMEOUT"
retries: $HEALTHCHECK_FILE_RETRIES
start_period: "$HEALTHCHECK_FILE_START_PERIOD"
x-sentry-defaults: &sentry_defaults
<<: [*restart_policy, *pull_policy]
image: sentry-self-hosted-local
# Set the platform to build for linux/arm64 when needed on Apple silicon Macs.
platform: ${DOCKER_PLATFORM:-}
build:
context: ./sentry
args:
- SENTRY_IMAGE
depends_on:
redis:
<<: *depends_on-healthy
kafka:
<<: *depends_on-healthy
pgbouncer:
<<: *depends_on-healthy
memcached:
<<: *depends_on-default
smtp:
<<: *depends_on-default
seaweedfs:
<<: *depends_on-default
snuba-api:
<<: *depends_on-default
symbolicator:
<<: *depends_on-default
entrypoint: "/etc/sentry/entrypoint.sh"
command: ["run", "web"]
environment:
PYTHONUSERBASE: "/data/custom-packages"
SENTRY_CONF: "/etc/sentry"
SNUBA: "http://snuba-api:1218"
VROOM: "http://vroom:8085"
# Force everything to use the system CA bundle
# This is mostly needed to support installing custom CA certs
# This one is used by botocore
DEFAULT_CA_BUNDLE: &ca_bundle "/etc/ssl/certs/ca-certificates.crt"
# This one is used by requests
REQUESTS_CA_BUNDLE: *ca_bundle
# This one is used by grpc/google modules
GRPC_DEFAULT_SSL_ROOTS_FILE_PATH_ENV_VAR: *ca_bundle
# Leaving the value empty to just pass whatever is set
# on the host system (or in the .env file)
COMPOSE_PROFILES:
SENTRY_EVENT_RETENTION_DAYS:
SENTRY_MAIL_HOST:
SENTRY_MAX_EXTERNAL_SOURCEMAP_SIZE:
SENTRY_SYSTEM_SECRET_KEY:
SENTRY_STATSD_ADDR: "${STATSD_ADDR:-}"
volumes:
- "sentry-data:/data"
- "./sentry:/etc/sentry"
- "./geoip:/geoip:ro"
- "./certificates:/usr/local/share/ca-certificates:ro"
x-snuba-defaults: &snuba_defaults
<<: *restart_policy
depends_on:
clickhouse:
<<: *depends_on-healthy
kafka:
<<: *depends_on-healthy
redis:
<<: *depends_on-healthy
image: "$SNUBA_IMAGE"
environment:
SNUBA_SETTINGS: self_hosted
CLICKHOUSE_HOST: clickhouse
DEFAULT_BROKERS: "kafka:9092"
REDIS_HOST: redis
UWSGI_MAX_REQUESTS: "10000"
UWSGI_DISABLE_LOGGING: "true"
# Leaving the value empty to just pass whatever is set
# on the host system (or in the .env file)
SENTRY_EVENT_RETENTION_DAYS:
# If you have statsd server, you can utilize that to monitor self-hosted Snuba containers.
# To start, state these environment variables below on your `.env.` file and adjust the options as needed.
SNUBA_STATSD_ADDR: "${STATSD_ADDR:-}"
services:
smtp:
<<: *restart_policy
image: registry.gitlab.com/egos-tech/smtp
volumes:
- "sentry-smtp:/var/spool/exim4"
- "sentry-smtp-log:/var/log/exim4"
environment:
MAILNAME: ${SENTRY_MAIL_HOST:-}
# Refer to https://develop.sentry.dev/self-hosted/configuration/email/#as-aws-ses-relay
SES_USER:
SES_PASSWORD:
SES_REGION:
memcached:
<<: *restart_policy
image: "memcached:1.6.26-alpine"
command: ["-I", "${SENTRY_MAX_EXTERNAL_SOURCEMAP_SIZE:-1M}"]
healthcheck:
<<: *healthcheck_defaults
# From: https://stackoverflow.com/a/31877626/5155484
test: echo stats | nc 127.0.0.1 11211
redis:
<<: *restart_policy
image: "redis:6.2.20-alpine"
healthcheck:
<<: *healthcheck_defaults
test: redis-cli ping | grep PONG
volumes:
- "sentry-redis:/data"
- type: bind
read_only: true
source: ./redis.conf
target: /usr/local/etc/redis/redis.conf
ulimits:
nofile:
soft: 10032
hard: 10032
command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
postgres:
<<: *restart_policy
# Using the same postgres version as Sentry dev for consistency purposes
image: "postgres:14.19-bookworm"
healthcheck:
<<: *healthcheck_defaults
# Using default user "postgres" from sentry/sentry.conf.example.py or value of POSTGRES_USER if provided
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-postgres}"]
command:
["postgres"]
environment:
POSTGRES_HOST_AUTH_METHOD: "trust"
volumes:
- "sentry-postgres:/var/lib/postgresql/data"
shm_size: 256m
pgbouncer:
<<: *restart_policy
image: "edoburu/pgbouncer:v1.24.1-p1"
healthcheck:
<<: *healthcheck_defaults
# Using default user "postgres" from sentry/sentry.conf.example.py or value of POSTGRES_USER if provided
test: ["CMD-SHELL", "psql -U postgres -p 5432 -h 127.0.0.1 -tA -c \"select 1;\" -d postgres >/dev/null"]
depends_on:
postgres:
<<: *depends_on-healthy
environment:
DB_USER: ${POSTGRES_USER:-postgres}
DB_HOST: postgres
DB_NAME: postgres
AUTH_TYPE: trust
POOL_MODE: transaction
ADMIN_USERS: postgres,sentry
MAX_CLIENT_CONN: 10000
kafka:
<<: *restart_policy
image: "confluentinc/cp-kafka:7.6.6"
environment:
# https://docs.confluent.io/platform/current/installation/docker/config-reference.html#cp-kakfa-example
KAFKA_PROCESS_ROLES: "broker,controller"
KAFKA_CONTROLLER_QUORUM_VOTERS: "1001@127.0.0.1:29093"
KAFKA_CONTROLLER_LISTENER_NAMES: "CONTROLLER"
KAFKA_NODE_ID: "1001"
CLUSTER_ID: "MkU3OEVBNTcwNTJENDM2Qk"
KAFKA_LISTENERS: "PLAINTEXT://0.0.0.0:29092,INTERNAL://0.0.0.0:9093,EXTERNAL://0.0.0.0:9092,CONTROLLER://0.0.0.0:29093"
KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://127.0.0.1:29092,INTERNAL://kafka:9093,EXTERNAL://kafka:9092"
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "PLAINTEXT:PLAINTEXT,INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT"
KAFKA_INTER_BROKER_LISTENER_NAME: "PLAINTEXT"
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: "1"
KAFKA_OFFSETS_TOPIC_NUM_PARTITIONS: "1"
KAFKA_LOG_RETENTION_HOURS: "24"
KAFKA_MESSAGE_MAX_BYTES: "50000000" #50MB or bust
KAFKA_MAX_REQUEST_SIZE: "50000000" #50MB on requests apparently too
CONFLUENT_SUPPORT_METRICS_ENABLE: "false"
KAFKA_LOG4J_LOGGERS: "kafka.cluster=WARN,kafka.controller=WARN,kafka.coordinator=WARN,kafka.log=WARN,kafka.server=WARN,state.change.logger=WARN"
KAFKA_LOG4J_ROOT_LOGLEVEL: "WARN"
KAFKA_TOOLS_LOG4J_LOGLEVEL: "WARN"
ulimits:
nofile:
soft: 4096
hard: 4096
volumes:
- "sentry-kafka:/var/lib/kafka/data"
- "sentry-kafka-log:/var/lib/kafka/log"
- "sentry-secrets:/etc/kafka/secrets"
healthcheck:
<<: *healthcheck_defaults
test: ["CMD-SHELL", "nc -z localhost 9092"]
interval: 10s
timeout: 10s
retries: 30
clickhouse:
<<: [*restart_policy, *pull_policy]
image: clickhouse-self-hosted-local
build:
context: ./clickhouse
args:
BASE_IMAGE: "altinity/clickhouse-server:25.3.6.10034.altinitystable"
ulimits:
nofile:
soft: 262144
hard: 262144
volumes:
- "sentry-clickhouse:/var/lib/clickhouse"
- "sentry-clickhouse-log:/var/log/clickhouse-server"
- type: "bind"
read_only: true
source: "./clickhouse/config.xml"
target: "/etc/clickhouse-server/config.d/sentry.xml"
- type: "bind"
read_only: true
source: "./clickhouse/default-password.xml"
target: "/etc/clickhouse-server/users.d/default-password.xml"
environment:
# This limits Clickhouse's memory to 30% of the host memory
# If you have high volume and your search return incomplete results
# You might want to change this to a higher value (and ensure your host has enough memory)
MAX_MEMORY_USAGE_RATIO: 0.3
healthcheck:
test: [
"CMD-SHELL",
# Manually override any http_proxy envvar that might be set, because
# this wget does not support no_proxy. See:
# https://github.com/getsentry/self-hosted/issues/1537
"HTTP_PROXY='' http_proxy='' wget -nv -t1 --spider 'http://localhost:8123/' || exit 1",
]
interval: 10s
timeout: 10s
retries: 30
seaweedfs:
<<: *restart_policy
image: "chrislusf/seaweedfs:4.09_large_disk"
entrypoint: "weed"
command: >-
server
-dir=/data
-filer=true
-filer.port=8888
-filer.port.grpc=18888
-filer.defaultReplicaPlacement=000
-master=true
-master.port=9333
-master.port.grpc=19333
-metricsPort=9091
-s3=true
-s3.port=8333
-s3.port.grpc=18333
-volume=true
-volume.dir.idx=/data/idx
-volume.index=leveldbLarge
-volume.max=0
-volume.preStopSeconds=8
-volume.readMode=redirect
-volume.port=8080
-volume.port.grpc=18080
-ip=127.0.0.1
-ip.bind=0.0.0.0
-webdav=false
environment:
AWS_ACCESS_KEY_ID: sentry
AWS_SECRET_ACCESS_KEY: sentry
volumes:
- "sentry-seaweedfs:/data"
healthcheck:
test: [
"CMD-SHELL",
# Manually override any http_proxy envvar that might be set, because
# this wget does not support no_proxy. See:
# https://github.com/getsentry/self-hosted/issues/1537
"http_proxy='' wget -q -O- http://seaweedfs:8080/healthz http://seaweedfs:9333/cluster/healthz http://seaweedfs:8333/healthz || exit 1",
]
interval: 30s
timeout: 20s
retries: 5
start_period: 60s
snuba-api:
<<: *snuba_defaults
healthcheck:
<<: *healthcheck_defaults
test:
- "CMD"
- "/bin/bash"
- "-c"
# Courtesy of https://unix.stackexchange.com/a/234089/108960
- 'exec 3<>/dev/tcp/127.0.0.1/1218 && echo -e "GET /health HTTP/1.1\r\nhost: 127.0.0.1\r\n\r\n" >&3 && grep ok -s -m 1 <&3'
# Kafka consumer responsible for feeding events into Clickhouse
snuba-errors-consumer:
<<: *snuba_defaults
command: rust-consumer --storage errors --consumer-group snuba-consumers --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
# Kafka consumer responsible for feeding outcomes into Clickhouse
# Use --auto-offset-reset=earliest to recover up to 7 days of TSDB data
# since we did not do a proper migration
snuba-outcomes-consumer:
<<: *snuba_defaults
command: rust-consumer --storage outcomes_raw --consumer-group snuba-consumers --auto-offset-reset=earliest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
snuba-outcomes-billing-consumer:
<<: *snuba_defaults
command: rust-consumer --storage outcomes_raw --consumer-group snuba-consumers --auto-offset-reset=earliest --max-batch-time-ms 750 --no-strict-offset-reset --raw-events-topic outcomes-billing --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
snuba-group-attributes-consumer:
<<: *snuba_defaults
command: rust-consumer --storage group_attributes --consumer-group snuba-group-attributes-consumers --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
snuba-replacer:
<<: *snuba_defaults
command: replacer --storage errors --auto-offset-reset=latest --no-strict-offset-reset
snuba-subscription-consumer-events:
<<: *snuba_defaults
command: subscriptions-scheduler-executor --dataset events --entity events --auto-offset-reset=latest --no-strict-offset-reset --consumer-group=snuba-events-subscriptions-consumers --followed-consumer-group=snuba-consumers --schedule-ttl=60 --stale-threshold-seconds=900 --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
#############################################
## Feature Complete Sentry Snuba Consumers ##
#############################################
# Kafka consumer responsible for feeding transactions data into Clickhouse
snuba-transactions-consumer:
<<: *snuba_defaults
command: rust-consumer --storage transactions --consumer-group transactions_group --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-replays-consumer:
<<: *snuba_defaults
command: rust-consumer --storage replays --consumer-group snuba-consumers --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-issue-occurrence-consumer:
<<: *snuba_defaults
command: rust-consumer --storage search_issues --consumer-group generic_events_group --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-metrics-consumer:
<<: *snuba_defaults
command: rust-consumer --storage metrics_raw --consumer-group snuba-metrics-consumers --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-subscription-consumer-transactions:
<<: *snuba_defaults
command: subscriptions-scheduler-executor --dataset transactions --entity transactions --auto-offset-reset=latest --no-strict-offset-reset --consumer-group=snuba-transactions-subscriptions-consumers --followed-consumer-group=transactions_group --schedule-ttl=60 --stale-threshold-seconds=900 --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-subscription-consumer-metrics:
<<: *snuba_defaults
command: subscriptions-scheduler-executor --dataset metrics --entity metrics_sets --entity metrics_counters --auto-offset-reset=latest --no-strict-offset-reset --consumer-group=snuba-metrics-subscriptions-consumers --followed-consumer-group=snuba-metrics-consumers --schedule-ttl=60 --stale-threshold-seconds=900 --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-subscription-consumer-generic-metrics-distributions:
<<: *snuba_defaults
command: subscriptions-scheduler-executor --dataset generic_metrics --entity=generic_metrics_distributions --auto-offset-reset=latest --no-strict-offset-reset --consumer-group=snuba-generic-metrics-distributions-subscriptions-schedulers --followed-consumer-group=snuba-gen-metrics-distributions-consumers --schedule-ttl=60 --stale-threshold-seconds=900 --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-subscription-consumer-generic-metrics-sets:
<<: *snuba_defaults
command: subscriptions-scheduler-executor --dataset generic_metrics --entity=generic_metrics_sets --auto-offset-reset=latest --no-strict-offset-reset --consumer-group=snuba-generic-metrics-sets-subscriptions-schedulers --followed-consumer-group=snuba-gen-metrics-sets-consumers --schedule-ttl=60 --stale-threshold-seconds=900 --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-subscription-consumer-generic-metrics-counters:
<<: *snuba_defaults
command: subscriptions-scheduler-executor --dataset generic_metrics --entity=generic_metrics_counters --auto-offset-reset=latest --no-strict-offset-reset --consumer-group=snuba-generic-metrics-counters-subscriptions-schedulers --followed-consumer-group=snuba-gen-metrics-counters-consumers --schedule-ttl=60 --stale-threshold-seconds=900 --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-subscription-consumer-generic-metrics-gauges:
<<: *snuba_defaults
command: subscriptions-scheduler-executor --dataset generic_metrics --entity=generic_metrics_gauges --auto-offset-reset=latest --no-strict-offset-reset --consumer-group=snuba-generic-metrics-gauges-subscriptions-schedulers --followed-consumer-group=snuba-gen-metrics-gauges-consumers --schedule-ttl=60 --stale-threshold-seconds=900 --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-generic-metrics-distributions-consumer:
<<: *snuba_defaults
command: rust-consumer --storage generic_metrics_distributions_raw --consumer-group snuba-gen-metrics-distributions-consumers --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-generic-metrics-sets-consumer:
<<: *snuba_defaults
command: rust-consumer --storage generic_metrics_sets_raw --consumer-group snuba-gen-metrics-sets-consumers --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-generic-metrics-counters-consumer:
<<: *snuba_defaults
command: rust-consumer --storage generic_metrics_counters_raw --consumer-group snuba-gen-metrics-counters-consumers --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-generic-metrics-gauges-consumer:
<<: *snuba_defaults
command: rust-consumer --storage generic_metrics_gauges_raw --consumer-group snuba-gen-metrics-gauges-consumers --auto-offset-reset=latest --max-batch-time-ms 750 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-profiling-profiles-consumer:
<<: *snuba_defaults
command: rust-consumer --storage profiles --consumer-group snuba-consumers --auto-offset-reset=latest --max-batch-time-ms 1000 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-profiling-functions-consumer:
<<: *snuba_defaults
command: rust-consumer --storage functions_raw --consumer-group snuba-consumers --auto-offset-reset=latest --max-batch-time-ms 1000 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-profiling-profile-chunks-consumer:
<<: *snuba_defaults
command: rust-consumer --storage profile_chunks --consumer-group snuba-consumers --auto-offset-reset=latest --max-batch-time-ms 1000 --no-strict-offset-reset --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-eap-items-consumer:
<<: *snuba_defaults
command: rust-consumer --storage eap_items --consumer-group eap_items_group --auto-offset-reset=latest --max-batch-time-ms 1000 --no-strict-offset-reset --use-rust-processor --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
snuba-subscription-consumer-eap-items:
<<: *snuba_defaults
command: subscriptions-scheduler-executor --dataset events_analytics_platform --entity eap_items --auto-offset-reset=latest --no-strict-offset-reset --consumer-group=snuba-eap-items-subscriptions-consumers --followed-consumer-group=eap_items_group --schedule-ttl=60 --stale-threshold-seconds=900 --health-check-file /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
symbolicator:
<<: *restart_policy
image: "$SYMBOLICATOR_IMAGE"
environment:
SYMBOLICATOR_STATSD_ADDR: ${STATSD_ADDR:-127.0.0.1:8125}
command: run -c /etc/symbolicator/config.yml
volumes:
- "sentry-symbolicator:/data"
- type: bind
read_only: true
source: ./symbolicator
target: /etc/symbolicator
healthcheck:
<<: *healthcheck_defaults
test: ["CMD", "/bin/symbolicator", "healthcheck", "-c", "/etc/symbolicator/config.yml"]
symbolicator-cleanup:
<<: *restart_policy
image: "$SYMBOLICATOR_IMAGE"
environment:
SYMBOLICATOR_STATSD_ADDR: ${STATSD_ADDR:-127.0.0.1:8125}
command: "cleanup -c /etc/symbolicator/config.yml --repeat 1h"
volumes:
- "sentry-symbolicator:/data"
- type: bind
read_only: true
source: ./symbolicator
target: /etc/symbolicator
web:
<<: *sentry_defaults
ulimits:
nofile:
soft: 4096
hard: 4096
healthcheck:
<<: *healthcheck_defaults
test:
- "CMD"
- "/bin/bash"
- "-c"
# Courtesy of https://unix.stackexchange.com/a/234089/108960
- 'exec 3<>/dev/tcp/127.0.0.1/9000 && echo -e "GET /_health/ HTTP/1.1\r\nhost: 127.0.0.1\r\n\r\n" >&3 && grep ok -s -m 1 <&3'
events-consumer:
<<: *sentry_defaults
command: run consumer ingest-events --consumer-group ingest-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
attachments-consumer:
<<: *sentry_defaults
command: run consumer ingest-attachments --consumer-group ingest-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
post-process-forwarder-errors:
<<: *sentry_defaults
command: run consumer --no-strict-offset-reset post-process-forwarder-errors --consumer-group post-process-forwarder --synchronize-commit-log-topic=snuba-commit-log --synchronize-commit-group=snuba-consumers --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
subscription-consumer-events:
<<: *sentry_defaults
command: run consumer events-subscription-results --consumer-group query-subscription-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
##############################################
## Feature Complete Sentry Ingest Consumers ##
##############################################
transactions-consumer:
<<: *sentry_defaults
command: run consumer ingest-transactions --consumer-group ingest-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
metrics-consumer:
<<: *sentry_defaults
command: run consumer ingest-metrics --consumer-group metrics-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
generic-metrics-consumer:
<<: *sentry_defaults
command: run consumer ingest-generic-metrics --consumer-group generic-metrics-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
billing-metrics-consumer:
<<: *sentry_defaults
command: run consumer billing-metrics-consumer --consumer-group billing-metrics-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
ingest-replay-recordings:
<<: *sentry_defaults
command: run consumer ingest-replay-recordings --consumer-group ingest-replay-recordings --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
ingest-occurrences:
<<: *sentry_defaults
command: run consumer ingest-occurrences --consumer-group ingest-occurrences --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
ingest-profiles:
<<: *sentry_defaults
command: run consumer ingest-profiles --consumer-group ingest-profiles --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
ingest-monitors:
<<: *sentry_defaults
command: run consumer ingest-monitors --consumer-group ingest-monitors --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
ingest-feedback-events:
<<: *sentry_defaults
command: run consumer ingest-feedback-events --consumer-group ingest-feedback --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
process-spans:
<<: *sentry_defaults
command: run consumer --no-strict-offset-reset process-spans --consumer-group process-spans --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
process-segments:
<<: *sentry_defaults
command: run consumer --no-strict-offset-reset process-segments --consumer-group process-segments --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
monitors-clock-tick:
<<: *sentry_defaults
command: run consumer monitors-clock-tick --consumer-group monitors-clock-tick --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
monitors-clock-tasks:
<<: *sentry_defaults
command: run consumer monitors-clock-tasks --consumer-group monitors-clock-tasks --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
uptime-results:
<<: *sentry_defaults
command: run consumer uptime-results --consumer-group uptime-results --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
post-process-forwarder-transactions:
<<: *sentry_defaults
command: run consumer --no-strict-offset-reset post-process-forwarder-transactions --consumer-group post-process-forwarder --synchronize-commit-log-topic=snuba-transactions-commit-log --synchronize-commit-group transactions_group --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
post-process-forwarder-issue-platform:
<<: *sentry_defaults
command: run consumer --no-strict-offset-reset post-process-forwarder-issue-platform --consumer-group post-process-forwarder --synchronize-commit-log-topic=snuba-generic-events-commit-log --synchronize-commit-group generic_events_group --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
subscription-consumer-transactions:
<<: *sentry_defaults
command: run consumer transactions-subscription-results --consumer-group query-subscription-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
subscription-consumer-eap-items:
<<: *sentry_defaults
command: run consumer subscription-results-eap-items --consumer-group subscription-results-eap-items --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
subscription-consumer-metrics:
<<: *sentry_defaults
command: run consumer metrics-subscription-results --consumer-group query-subscription-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
subscription-consumer-generic-metrics:
<<: *sentry_defaults
command: run consumer generic-metrics-subscription-results --consumer-group query-subscription-consumer --healthcheck-file-path /tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
profiles:
- feature-complete
sentry-cleanup:
<<: *sentry_defaults
image: sentry-cleanup-self-hosted-local
build:
context: ./cron
args:
BASE_IMAGE: sentry-self-hosted-local
entrypoint: "/entrypoint.sh"
command: '"0 0 * * * gosu sentry sentry cleanup --days $SENTRY_EVENT_RETENTION_DAYS"'
nginx:
<<: *restart_policy
ports:
- "$SENTRY_BIND:80/tcp"
image: "nginx:1.29.5-alpine"
volumes:
- type: bind
read_only: true
source: ./nginx.conf
target: /etc/nginx/nginx.conf
- sentry-nginx-cache:/var/cache/nginx
- sentry-nginx-www:/var/www
healthcheck:
<<: *healthcheck_defaults
test:
- "CMD"
- "/usr/bin/curl"
- http://localhost
depends_on:
web:
<<: *depends_on-healthy
restart: true
relay:
<<: *depends_on-healthy
restart: true
relay:
<<: *restart_policy
image: "$RELAY_IMAGE"
environment:
RELAY_STATSD_ADDR: ${STATSD_ADDR:-127.0.0.1:8125}
volumes:
- type: bind
read_only: true
source: ./relay
target: /work/.relay
- type: bind
read_only: true
source: ./geoip
target: /geoip
depends_on:
kafka:
<<: *depends_on-healthy
redis:
<<: *depends_on-healthy
web:
<<: *depends_on-healthy
healthcheck:
<<: *healthcheck_defaults
test: ["CMD", "/bin/relay", "healthcheck"]
taskbroker:
<<: *restart_policy
image: "$TASKBROKER_IMAGE"
environment:
TASKBROKER_KAFKA_CLUSTER: "kafka:9092"
TASKBROKER_KAFKA_DEADLETTER_CLUSTER: "kafka:9092"
TASKBROKER_DB_PATH: "/opt/sqlite/taskbroker-activations.sqlite"
TASKBROKER_STATSD_ADDR: ${STATSD_ADDR:-127.0.0.1:8125}
volumes:
- sentry-taskbroker:/opt/sqlite
depends_on:
kafka:
<<: *depends_on-healthy
taskscheduler:
<<: *sentry_defaults
command: run taskworker-scheduler
taskworker:
<<: *sentry_defaults
command: run taskworker --concurrency=$SENTRY_TASKWORKER_CONCURRENCY --rpc-host=taskbroker:50051 --health-check-file-path=/tmp/health.txt
healthcheck:
<<: *file_healthcheck_defaults
vroom:
<<: *restart_policy
image: "$VROOM_IMAGE"
environment:
SENTRY_KAFKA_BROKERS_PROFILING: "kafka:9092"
SENTRY_KAFKA_BROKERS_OCCURRENCES: "kafka:9092"
SENTRY_BUCKET_PROFILES: "s3://profiles?region=us-east-1&endpoint=seaweedfs:8333&s3ForcePathStyle=true&disableSSL=true"
AWS_ACCESS_KEY: "sentry"
AWS_SECRET_KEY: "sentry"
SENTRY_SNUBA_HOST: "http://snuba-api:1218"
volumes:
- sentry-vroom:/var/vroom/sentry-profiles
healthcheck:
<<: *healthcheck_defaults
test:
- "CMD"
- "/bin/bash"
- "-c"
# Courtesy of https://unix.stackexchange.com/a/234089/108960
- 'exec 3<>/dev/tcp/127.0.0.1/8085 && echo -e "GET /health HTTP/1.1\r\nhost: 127.0.0.1\r\n\r\n" >&3 && grep OK -s -m 1 <&3'
depends_on:
kafka:
<<: *depends_on-healthy
profiles:
- feature-complete
vroom-cleanup:
<<: [*restart_policy, *pull_policy]
image: vroom-cleanup-self-hosted-local
build:
context: ./cron
args:
BASE_IMAGE: "$VROOM_IMAGE"
entrypoint: "/entrypoint.sh"
environment:
# Leaving the value empty to just pass whatever is set
# on the host system (or in the .env file)
SENTRY_EVENT_RETENTION_DAYS:
command: '"0 0 * * * find /var/vroom/sentry-profiles -type f -mtime +$SENTRY_EVENT_RETENTION_DAYS -delete"'
volumes:
- sentry-vroom:/var/vroom/sentry-profiles
profiles:
- feature-complete
uptime-checker:
<<: *restart_policy
image: "$UPTIME_CHECKER_IMAGE"
command: run
environment:
UPTIME_CHECKER_RESULTS_KAFKA_CLUSTER: kafka:9092
UPTIME_CHECKER_REDIS_HOST: redis://redis:6379
# Set to `true` will allow uptime checks against private IP addresses
UPTIME_CHECKER_ALLOW_INTERNAL_IPS: "false"
# The number of times to retry failed checks before reporting them as failed
UPTIME_CHECKER_FAILURE_RETRIES: "1"
# DNS name servers to use when making checks in the http checker.
# Separated by commas. Leaving this unset will default to the systems dns
# resolver.
#UPTIME_CHECKER_HTTP_CHECKER_DNS_NAMESERVERS: "8.8.8.8,8.8.4.4"
UPTIME_CHECKER_STATSD_ADDR: ${STATSD_ADDR:-127.0.0.1:8125}
depends_on:
kafka:
<<: *depends_on-healthy
redis:
<<: *depends_on-healthy
profiles:
- feature-complete
volumes:
# These store application data that should persist across restarts.
sentry-data:
external: true
sentry-postgres:
external: true
sentry-redis:
external: true
sentry-kafka:
external: true
sentry-clickhouse:
external: true
sentry-seaweedfs:
external: true
# The volume stores cached version of debug symbols, source maps etc. Upon
# removal symbolicator will re-download them.
sentry-symbolicator:
# This volume stores JS SDK assets and the data inside this volume should
# be cleaned periodically on upgrades.
sentry-nginx-www:
# This volume stores profiles and should be persisted.
# Not being external will still persist data across restarts.
# It won't persist if someone does a docker compose down -v.
sentry-vroom:
# This volume stores task data that is inflight
# It should persist across restarts. If this volume is
# deleted, up to ~2048 tasks will be lost.
sentry-taskbroker:
# These store ephemeral data that needn't persist across restarts.
# That said, volumes will be persisted across restarts until they are deleted.
sentry-secrets:
sentry-smtp:
sentry-nginx-cache:
sentry-kafka-log:
sentry-smtp-log:
sentry-clickhouse-log:
================================================
FILE: get-compose-action/action.yaml
================================================
name: "Get Docker Compose"
inputs:
version:
required: false
default: 2.33.1
description: "Docker Compose version"
runs:
using: "composite"
steps:
- name: Get Compose
shell: bash
env:
COMPOSE_VERSION: ${{ inputs.version }}
run: |
# Docker Compose v1 is installed here, remove it
sudo rm -f "/usr/local/bin/docker-compose"
sudo rm -f "/usr/local/lib/docker/cli-plugins/docker-compose"
sudo mkdir -p "/usr/local/lib/docker/cli-plugins"
sudo curl -L https://github.com/docker/compose/releases/download/v"$COMPOSE_VERSION"/docker-compose-`uname -s`-`uname -m` -o "/usr/local/lib/docker/cli-plugins/docker-compose"
sudo chmod +x "/usr/local/lib/docker/cli-plugins/docker-compose"
================================================
FILE: install/_detect-container-engine.sh
================================================
echo "${_group}Detecting container engine ..."
if [[ "${CONTAINER_ENGINE_PODMAN:-0}" -eq 1 ]] && command -v podman &>/dev/null; then
export CONTAINER_ENGINE="podman"
elif command -v docker &>/dev/null; then
export CONTAINER_ENGINE="docker"
else
echo "FAIL: Neither podman nor docker is installed on the system."
exit 1
fi
echo "Detected container engine: $CONTAINER_ENGINE"
echo "${_endgroup}"
================================================
FILE: install/_lib.sh
================================================
# Allow `.env` overrides using the `.env.custom` file.
# We pass this to docker compose in a couple places.
if [[ -f .env.custom ]]; then
_ENV=.env.custom
else
_ENV=.env
fi
# Reading .env.custom has to come first. The value won't be overriden, instead
# it would persist because of `export -p> >"$t"` later, which exports current
# environment variables to a temporary file with a `declare -x KEY=value` format.
# The new values on `.env` would be set only if they are not already set.
if [[ "$_ENV" == ".env.custom" ]]; then
q=$(mktemp) && export -p >"$q" && set -a && . ".env.custom" && set +a && . "$q" && rm "$q" && unset q
fi
# Read .env for default values with a tip o' the hat to https://stackoverflow.com/a/59831605/90297
t=$(mktemp) && export -p >"$t" && set -a && . ".env" && set +a && . "$t" && rm "$t" && unset t
if [ "${GITHUB_ACTIONS:-}" = "true" ]; then
_group="::group::"
_endgroup="::endgroup::"
else
_group="▶ "
_endgroup=""
fi
# A couple of the config files are referenced from other subscripts, so they
# get vars, while multiple subscripts call ensure_file_from_example.
function ensure_file_from_example {
target="$1"
if [[ -f "$target" ]]; then
echo "$target already exists, skipped creation."
else
# sed from https://stackoverflow.com/a/25123013/90297
# shellcheck disable=SC2001
example="$(echo "$target" | sed 's/\.[^.]*$/.example&/')"
if [[ ! -f "$example" ]]; then
echo "Oops! Where did $example go? 🤨 We need it in order to create $target."
exit
fi
echo "Creating $target ..."
cp -n "$example" "$target"
fi
}
# Check the version of $1 is greater than or equal to $2 using sort. Note: versions must be stripped of "v"
function vergte() {
printf "%s\n%s" "$1" "$2" | sort --version-sort --check=quiet --reverse
}
export SENTRY_CONFIG_PY=sentry/sentry.conf.py
export SENTRY_CONFIG_YML=sentry/config.yml
# Increase the default 10 second SIGTERM timeout
# to ensure task queues are properly drained
# between upgrades as task signatures may change across
# versions
export STOP_TIMEOUT=60 # seconds
================================================
FILE: install/_logging.sh
================================================
# Thanks to https://unix.stackexchange.com/a/145654/108960
log_file=sentry_install_log-$(date +'%Y-%m-%d_%H-%M-%S').txt
exec &> >(tee -a "$log_file")
================================================
FILE: install/_min-requirements.sh
================================================
# Don't forget to update the README and other docs when you change these!
MIN_DOCKER_VERSION='19.03.6'
MIN_COMPOSE_VERSION='2.32.2'
MIN_PODMAN_VERSION='4.9.3'
MIN_PODMAN_COMPOSE_VERSION='1.3.0'
MIN_BASH_VERSION='4.4.0'
# 16 GB minimum host RAM, but there'll be some overhead outside of what
# can be allotted to docker
if [[ "$COMPOSE_PROFILES" == "errors-only" ]]; then
MIN_RAM_HARD=7000 # MB
MIN_CPU_HARD=2
else
MIN_RAM_HARD=14000 # MB
MIN_CPU_HARD=4
fi
================================================
FILE: install/bootstrap-s3-nodestore.sh
================================================
echo "${_group}Bootstrapping seaweedfs (node store)..."
start_service_and_wait_ready seaweedfs postgres
$dcx seaweedfs apk add --no-cache s3cmd
$dc exec seaweedfs mkdir -p /data/idx/
s3cmd="$dc exec seaweedfs s3cmd"
bucket_list=$($s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' ls)
if ! echo "$bucket_list" | grep -q "s3://nodestore"; then
apply_config_changes_nodestore=0
# Only touch if no existing nodestore config is found
if ! grep -q "SENTRY_NODESTORE" $SENTRY_CONFIG_PY; then
if [[ -z "${APPLY_AUTOMATIC_CONFIG_UPDATES:-}" ]]; then
echo
echo "We want to migrate Nodestore backend from Postgres to S3 which will"
echo "help reducing Postgres storage issues. In order to do that, we need"
echo "to modify your sentry.conf.py file contents."
echo "Do you want us to do it automatically for you?"
echo
yn=""
until [ ! -z "$yn" ]; do
read -p "y or n? " yn
case $yn in
y | yes | 1)
export apply_config_changes_nodestore=1
echo
echo -n "Thank you."
;;
n | no | 0)
export apply_config_changes_nodestore=0
echo
echo -n "Alright, you will need to update your sentry.conf.py file manually before running 'docker compose up'."
;;
*) yn="" ;;
esac
done
echo
echo "To avoid this prompt in the future, use one of these flags:"
echo
echo " --apply-automatic-config-updates"
echo " --no-apply-automatic-config-updates"
echo
echo "or set the APPLY_AUTOMATIC_CONFIG_UPDATES environment variable:"
echo
echo " APPLY_AUTOMATIC_CONFIG_UPDATES=1 to apply automatic updates"
echo " APPLY_AUTOMATIC_CONFIG_UPDATES=0 to not apply automatic updates"
echo
sleep 5
fi
if [[ "$APPLY_AUTOMATIC_CONFIG_UPDATES" == 1 || "$apply_config_changes_nodestore" == 1 ]]; then
nodestore_config=$(sed -n '/SENTRY_NODESTORE/,/[}]/{p}' sentry/sentry.conf.example.py)
if [[ $($dc exec postgres psql -qAt -U postgres -c "select exists (select * from nodestore_node limit 1)") = "t" ]]; then
nodestore_config=$(echo -e "$nodestore_config" | sed '$s/\}/ "read_through": True,\n "delete_through": True,\n\}/')
fi
echo "$nodestore_config" >>$SENTRY_CONFIG_PY
fi
fi
$dc exec seaweedfs mkdir -p /data/idx/
$s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' mb s3://nodestore
else
echo "Node store already exists, skipping creation..."
fi
if [[ -z "${APPLY_AUTOMATIC_CONFIG_UPDATES:-}" || "$APPLY_AUTOMATIC_CONFIG_UPDATES" == 1 ]]; then
# XXX(aldy505): Should we refactor this?
lifecycle_policy=$(
cat <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<LifecycleConfiguration>
<Rule>
<ID>Sentry-Nodestore-Rule</ID>
<Status>Enabled</Status>
<Filter></Filter>
<Expiration>
<Days>$SENTRY_EVENT_RETENTION_DAYS</Days>
</Expiration>
</Rule>
</LifecycleConfiguration>
EOF
)
echo "Making sure the bucket lifecycle policy is all set up correctly..."
$dc exec seaweedfs sh -c "printf '%s' '$lifecycle_policy' > /tmp/nodestore-lifecycle-policy.xml"
$s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' setlifecycle /tmp/nodestore-lifecycle-policy.xml s3://nodestore
$s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' getlifecycle s3://nodestore
fi
echo "${_endgroup}"
================================================
FILE: install/bootstrap-s3-profiles.sh
================================================
# The purpose of this file is to have both `sentry`-based containers and `vroom` use the same bucket for profiling.
# On pre-25.10.0, we have a `sentry-vroom` volume which stores the profiling data however, since this version,
# the behavior changed, and `vroomrs` now ingests profiles directly. Both services must share the same bucket,
# but at the time of this writing, it's not possible because the `sentry-vroom` volume has ownership set to `vroom:vroom`.
# This prevents the `sentry`-based containers from performing read/write operations on that volume.
#
# Therefore, this script should do the following:
# 1. Check if there are any files inside the `sentry-vroom` volume.
# 2. If (1) finds files, copy those files into a "profiles" bucket on SeaweedFS.
# 3. Point `filestore-profiles` and vroom to the SeaweedFS "profiles" bucket.
# Should only run when `$COMPOSE_PROFILES` is set to `feature-complete`
if [[ "$COMPOSE_PROFILES" == "feature-complete" ]]; then
echo "${_group}Bootstrapping seaweedfs (profiles)..."
start_service_and_wait_ready seaweedfs
$dcx seaweedfs apk add --no-cache s3cmd
s3cmd="$dc exec seaweedfs s3cmd"
bucket_list=$($s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' ls)
if ! echo "$bucket_list" | grep -q "s3://profiles"; then
apply_config_changes_profiles=0
# Only touch if no existing profiles config is found
if ! grep -q "filestore.profiles-backend" $SENTRY_CONFIG_YML; then
if [[ -z "${APPLY_AUTOMATIC_CONFIG_UPDATES:-}" ]]; then
echo
echo "We are migrating the Profiles data directory from the 'sentry-vroom' volume to SeaweedFS."
echo "This migration will ensure profiles ingestion works correctly with the new 'vroomrs'"
echo "and allows both 'sentry' and 'vroom' to transition smoothly."
echo "To complete this, your sentry/config.yml file needs to be modified."
echo "Would you like us to perform this modification automatically?"
echo
yn=""
until [ ! -z "$yn" ]; do
read -p "y or n? " yn
case $yn in
y | yes | 1)
export apply_config_changes_profiles=1
echo
echo -n "Thank you."
;;
n | no | 0)
export apply_config_changes_profiles=0
echo
echo -n "Alright, you will need to update your sentry/config.yml file manually before running 'docker compose up'."
;;
*) yn="" ;;
esac
done
echo
echo "To avoid this prompt in the future, use one of these flags:"
echo
echo " --apply-automatic-config-updates"
echo " --no-apply-automatic-config-updates"
echo
echo "or set the APPLY_AUTOMATIC_CONFIG_UPDATES environment variable:"
echo
echo " APPLY_AUTOMATIC_CONFIG_UPDATES=1 to apply automatic updates"
echo " APPLY_AUTOMATIC_CONFIG_UPDATES=0 to not apply automatic updates"
echo
sleep 5
fi
if [[ "$APPLY_AUTOMATIC_CONFIG_UPDATES" == 1 || "$apply_config_changes_profiles" == 1 ]]; then
profiles_config=$(sed -n '/filestore.profiles-backend/,/s3v4"/{p}' sentry/config.example.yml)
echo "$profiles_config" >>$SENTRY_CONFIG_YML
fi
fi
$s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' mb s3://profiles
# Check if there are files in the sentry-vroom volume
start_service_and_wait_ready vroom
vroom_files_count=$($dc exec vroom sh -c "find /var/vroom/sentry-profiles -type f | wc -l")
if [[ "$vroom_files_count" -gt 0 ]]; then
echo "Migrating $vroom_files_count files from 'sentry-vroom' volume to 'profiles' bucket on SeaweedFS..."
# Use a temporary container to copy files from the volume to SeaweedFS
$dcx -u root vroom sh -c 'mkdir -p /var/lib/apt/lists/partial && apt-get update && apt-get install -y --no-install-recommends s3cmd'
$dc exec vroom sh -c 's3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=seaweedfs:8333 --host-bucket="seaweedfs:8333/%(bucket)" sync /var/vroom/sentry-profiles/ s3://profiles/'
echo "Migration completed."
else
echo "No files found in 'sentry-vroom' volume. Skipping files migration."
fi
else
echo "'profiles' bucket already exists on SeaweedFS. Skipping creation."
fi
if [[ -z "${APPLY_AUTOMATIC_CONFIG_UPDATES:-}" || "$APPLY_AUTOMATIC_CONFIG_UPDATES" == 1 ]]; then
lifecycle_policy=$(
cat <<EOF
<?xml version="1.0" encoding="UTF-8"?>
<LifecycleConfiguration>
<Rule>
<ID>Sentry-Profiles-Rule</ID>
<Status>Enabled</Status>
<Filter></Filter>
<Expiration>
<Days>$SENTRY_EVENT_RETENTION_DAYS</Days>
</Expiration>
</Rule>
</LifecycleConfiguration>
EOF
)
$dc exec seaweedfs sh -c "printf '%s' '$lifecycle_policy' > /tmp/profiles-lifecycle-policy.xml"
$s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' setlifecycle /tmp/profiles-lifecycle-policy.xml s3://profiles
echo "Making sure the bucket lifecycle policy is all set up correctly..."
$s3cmd --access_key=sentry --secret_key=sentry --no-ssl --region=us-east-1 --host=localhost:8333 --host-bucket='localhost:8333/%(bucket)' getlifecycle s3://profiles
fi
echo "${_endgroup}"
fi
================================================
FILE: install/bootstrap-snuba.sh
================================================
echo "${_group}Bootstrapping and migrating Snuba ..."
if [[ -z "${SKIP_SNUBA_MIGRATIONS:-}" ]]; then
$dcr snuba-api bootstrap --force
else
echo "Skipped DB migrations due to SKIP_SNUBA_MIGRATIONS=$SKIP_SNUBA_MIGRATIONS"
fi
echo "${_endgroup}"
================================================
FILE: install/build-docker-images.sh
================================================
echo "${_group}Building and tagging Docker images ..."
echo ""
# Build any service that provides the image sentry-self-hosted-local first,
# as it is used as the base image for sentry-cleanup-self-hosted-local.
$dcb web
# Build each other service individually to localize potential failures better.
for service in $($dc config --services); do
$dcb "$service"
done
echo ""
echo "Docker images built."
echo "${_endgroup}"
================================================
FILE: install/check-latest-commit.sh
================================================
echo "${_group}Checking for latest commit ... "
# Checks if we are on latest commit from github if it is running from master branch
if [[ -d "../.git" && "${SKIP_COMMIT_CHECK:-0}" != 1 ]]; then
if [[ $(git branch --show-current) == "master" ]]; then
if [[ $(git rev-parse HEAD) != $(git ls-remote $(git rev-parse --abbrev-ref @{u} | sed 's/\// /g') | cut -f1) ]]; then
echo "Seems like you are not using the latest commit from the self-hosted repository. Please pull the latest changes and try again, or suppress this check with --skip-commit-check."
exit 1
fi
fi
else
echo "skipped"
fi
echo "${_endgroup}"
================================================
FILE: install/check-memcached-backend.sh
================================================
echo "${_group}Checking memcached backend ..."
if grep -q "\.PyMemcacheCache" "$SENTRY_CONFIG_PY"; then
echo "PyMemcacheCache found in $SENTRY_CONFIG_PY, gonna assume you're good."
else
if grep -q "\.MemcachedCache" "$SENTRY_CONFIG_PY"; then
echo "MemcachedCache found in $SENTRY_CONFIG_PY, you should switch to PyMemcacheCache."
echo "See:"
echo " https://develop.sentry.dev/self-hosted/releases/#breaking-changes"
exit 1
else
echo 'Your setup looks weird. Good luck.'
fi
fi
echo "${_endgroup}"
================================================
FILE: install/check-minimum-requirements.sh
================================================
echo "${_group}Checking minimum requirements ..."
source install/_min-requirements.sh
DOCKER_VERSION=$($CONTAINER_ENGINE version --format '{{.Server.Version}}' || echo '')
if [[ -z "$DOCKER_VERSION" ]]; then
echo "FAIL: Unable to get $CONTAINER_ENGINE version, is the $CONTAINER_ENGINE daemon running?"
exit 1
fi
if [[ "$CONTAINER_ENGINE" == "docker" ]]; then
if ! vergte ${DOCKER_VERSION//v/} $MIN_DOCKER_VERSION; then
echo "FAIL: Expected minimum docker version to be $MIN_DOCKER_VERSION but found $DOCKER_VERSION"
exit 1
fi
if ! vergte ${COMPOSE_VERSION//v/} $MIN_COMPOSE_VERSION; then
echo "FAIL: Expected minimum $dc_base version to be $MIN_COMPOSE_VERSION but found $COMPOSE_VERSION"
exit 1
fi
elif [[ "$CONTAINER_ENGINE" == "podman" ]]; then
if ! vergte ${DOCKER_VERSION//v/} $MIN_PODMAN_VERSION; then
echo "FAIL: Expected minimum podman version to be $MIN_PODMAN_VERSION but found $DOCKER_VERSION"
exit 1
fi
if ! vergte ${COMPOSE_VERSION//v/} $MIN_PODMAN_COMPOSE_VERSION; then
echo "FAIL: Expected minimum $dc_base version to be $MIN_PODMAN_COMPOSE_VERSION but found $COMPOSE_VERSION"
exit 1
fi
fi
echo "Found $CONTAINER_ENGINE version $DOCKER_VERSION"
echo "Found $CONTAINER_ENGINE Compose version $COMPOSE_VERSION"
CPU_AVAILABLE_IN_DOCKER=$($CONTAINER_ENGINE run --rm busybox nproc --all)
if [[ "$CPU_AVAILABLE_IN_DOCKER" -lt "$MIN_CPU_HARD" ]]; then
echo "FAIL: Required minimum CPU cores available to Docker is $MIN_CPU_HARD, found $CPU_AVAILABLE_IN_DOCKER"
exit 1
fi
RAM_AVAILABLE_IN_DOCKER=$($CONTAINER_ENGINE run --rm busybox free -m 2>/dev/null | awk '/Mem/ {print $2}')
if [[ "$RAM_AVAILABLE_IN_DOCKER" -lt "$MIN_RAM_HARD" ]]; then
echo "FAIL: Required minimum RAM available to Docker is $MIN_RAM_HARD MB, found $RAM_AVAILABLE_IN_DOCKER MB"
exit 1
fi
#SSE4.2 required by Clickhouse (https://clickhouse.yandex/docs/en/operations/requirements/)
# On KVM, cpuinfo could falsely not report SSE 4.2 support, so skip the check. https://github.com/ClickHouse/ClickHouse/issues/20#issuecomment-226849297
# This may also happen on other virtualization software such as on VMWare ESXi hosts.
IS_KVM=$($CONTAINER_ENGINE run --rm busybox grep -c 'Common KVM processor' /proc/cpuinfo || :)
if [[ ! "$SKIP_SSE42_REQUIREMENTS" -eq 1 && "$IS_KVM" -eq 0 && "$DOCKER_ARCH" = "x86_64" ]]; then
SUPPORTS_SSE42=$($CONTAINER_ENGINE run --rm busybox grep -c sse4_2 /proc/cpuinfo || :)
if [[ "$SUPPORTS_SSE42" -eq 0 ]]; then
echo "FAIL: The CPU your machine is running on does not support the SSE 4.2 instruction set, which is required for one of the services Sentry uses (Clickhouse). See https://github.com/getsentry/self-hosted/issues/340 for more info."
exit 1
fi
fi
if ! vergte "${BASH_VERSION}" "${MIN_BASH_VERSION}"; then
echo "FAIL: Expected minimum bash version to be ${MIN_BASH_VERSION} but found ${BASH_VERSION}"
exit 1
fi
echo "${_endgroup}"
================================================
FILE: install/create-docker-volumes.sh
================================================
echo "${_group}Creating volumes for persistent storage ..."
create_volume() {
create_command="$CONTAINER_ENGINE volume create"
if [ "$CONTAINER_ENGINE" = "podman" ]; then
create_command="$create_command --ignore $1"
else
create_command="$create_command --name=$1"
fi
$create_command
}
echo "Created $(create_volume sentry-clickhouse)."
echo "Created $(create_volume sentry-data)."
echo "Created $(create_volume sentry-kafka)."
echo "Created $(create_volume sentry-postgres)."
echo "Created $(create_volume sentry-redis)."
echo "Created $(create_volume sentry-seaweedfs)."
echo "${_endgroup}"
================================================
FILE: install/dc-detect-version.sh
================================================
if [ "${GITHUB_ACTIONS:-}" = "true" ]; then
_group="::group::"
_endgroup="::endgroup::"
else
_group="▶ "
_endgroup=""
fi
echo "${_group}Initializing Docker|Podman Compose ..."
export CONTAINER_ENGINE="docker"
if [[ "${CONTAINER_ENGINE_PODMAN:-0}" -eq 1 ]]; then
if command -v podman &>/dev/null; then
export CONTAINER_ENGINE="podman"
else
echo "FAIL: Podman is not installed on the system."
exit 1
fi
fi
# To support users that are symlinking to docker-compose
dc_base="$(${CONTAINER_ENGINE} compose version --short &>/dev/null && echo "$CONTAINER_ENGINE compose" || echo '')"
dc_base_standalone="$(${CONTAINER_ENGINE}-compose version --short &>/dev/null && echo "$CONTAINER_ENGINE-compose" || echo '')"
COMPOSE_VERSION=$([ -n "$dc_base" ] && $dc_base version --short || echo '')
STANDALONE_COMPOSE_VERSION=$([ -n "$dc_base_standalone" ] && $dc_base_standalone version --short || echo '')
if [[ -z "$COMPOSE_VERSION" && -z "$STANDALONE_COMPOSE_VERSION" ]]; then
echo "FAIL: Docker|Podman Compose is required to run self-hosted"
exit 1
fi
if [[ -z "$COMPOSE_VERSION" ]] || [[ -n "$STANDALONE_COMPOSE_VERSION" ]] && ! vergte ${COMPOSE_VERSION//v/} ${STANDALONE_COMPOSE_VERSION//v/}; then
COMPOSE_VERSION="${STANDALONE_COMPOSE_VERSION}"
dc_base="$dc_base_standalone"
fi
if [[ "$CONTAINER_ENGINE" == "podman" ]]; then
NO_ANSI="--no-ansi"
else
NO_ANSI="--ansi never"
fi
if [[ "$(basename $0)" = "install.sh" ]]; then
dc="$dc_base $NO_ANSI --env-file ${_ENV}"
else
dc="$dc_base $NO_ANSI"
fi
proxy_args="--build-arg HTTP_PROXY=${HTTP_PROXY:-} --build-arg HTTPS_PROXY=${HTTPS_PROXY:-} --build-arg NO_PROXY=${NO_PROXY:-} --build-arg http_proxy=${http_proxy:-} --build-arg https_proxy=${https_proxy:-} --build-arg no_proxy=${no_proxy:-}"
exec_proxy_args="-e HTTP_PROXY=${HTTP_PROXY:-} -e HTTPS_PROXY=${HTTPS_PROXY:-} -e NO_PROXY=${NO_PROXY:-} -e http_proxy=${http_proxy:-} -e https_proxy=${https_proxy:-} -e no_proxy=${no_proxy:-}"
if [[ "$CONTAINER_ENGINE" == "podman" ]]; then
proxy_args_dc="--podman-build-args HTTP_PROXY=${HTTP_PROXY:-},HTTPS_PROXY=${HTTPS_PROXY:-},NO_PROXY=${NO_PROXY:-},http_proxy=${http_proxy:-},https_proxy=${https_proxy:-},no_proxy=${no_proxy:-}"
# Disable pod creation as these are one-off commands and creating a pod
# prints its pod id to stdout which is messing with the output that we
# rely on various places such as configuration generation
dcr="$dc --profile=feature-complete --in-pod=false run --rm"
else
proxy_args_dc=$proxy_args
dcr="$dc run --pull=never --rm"
fi
dcb="$dc build $proxy_args"
dbuild="$CONTAINER_ENGINE build $proxy_args"
dcx="$dc exec $exec_proxy_args"
echo "$dcr"
# Utility function to handle --wait with docker and podman
function start_service_and_wait_ready() {
local options=()
local services=()
local found_service=0
for arg in "$@"; do
if [[ $found_service -eq 0 && "$arg" == -* ]]; then
options+=("$arg")
else
found_service=1
services+=("$arg")
fi
done
if [ "$CONTAINER_ENGINE" = "podman" ]; then
$dc up --force-recreate -d "${options[@]}" "${services[@]}"
for service in "${services[@]}"; do
while ! $CONTAINER_ENGINE ps --filter "health=healthy" | grep "$service"; do
sleep 2
done
done
else
$dc up --wait "${options[@]}" "${services[@]}"
fi
}
echo "${_endgroup}"
================================================
FILE: install/detect-platform.sh
================================================
source install/_detect-container-engine.sh
echo "${_group}Detecting Docker platform"
# Sentry SaaS uses stock Yandex ClickHouse, but they don't provide images that
# support ARM, which is relevant especially for Apple M1 laptops, Sentry's
# standard developer environment. As a workaround, we use an altinity image
# targeting ARM.
#
# See https://github.com/getsentry/self-hosted/issues/1385#issuecomment-1101824274
#
# Images built on ARM also need to be tagged to use linux/arm64 on Apple
# silicon Macs to work around an issue where they are built for
# linux/amd64 by default due to virtualization.
# See https://github.com/docker/cli/issues/3286 for the Docker bug.
FORMAT="{{.Architecture}}"
if [[ $CONTAINER_ENGINE == "podman" ]]; then
FORMAT="{{.Host.Arch}}"
fi
DOCKER_ARCH_OUTPUT=$($CONTAINER_ENGINE info --format "$FORMAT" 2>&1)
DOCKER_INFO_EXIT_CODE=$?
if [[ $DOCKER_INFO_EXIT_CODE -ne 0 ]]; then
echo "FAIL: Unable to get $CONTAINER_ENGINE architecture information."
echo "$DOCKER_ARCH_OUTPUT"
if [[ "$DOCKER_ARCH_OUTPUT" == *"permission denied"* ]]; then
echo ""
echo "You may need to add your user to the docker group:"
echo " sudo usermod -aG docker \$USER"
echo "Then log out and log back in, or run: newgrp docker"
fi
exit 1
fi
export DOCKER_ARCH="$DOCKER_ARCH_OUTPUT"
if [[ "$DOCKER_ARCH" = "x86_64" || "$DOCKER_ARCH" = "amd64" ]]; then
export DOCKER_PLATFORM="linux/amd64"
elif [[ "$DOCKER_ARCH" = "aarch64" ]]; then
export DOCKER_PLATFORM="linux/arm64"
else
echo "FAIL: Unsupported docker architecture $DOCKER_ARCH."
exit 1
fi
echo "Detected Docker platform is $DOCKER_PLATFORM"
echo "${_endgroup}"
================================================
FILE: install/ensure-correct-permissions-profiles-dir.sh
================================================
#!/usr/bin/env bash
# TODO: Remove this after the next hard-stop
# Should only run when `$COMPOSE_PROFILES` is set to `feature-complete`
if [[ "$COMPOSE_PROFILES" == "feature-complete" ]]; then
echo "${_group}Ensuring correct permissions on profiles directory ..."
# Check if the parent directory of /var/vroom/sentry-profiles is already owned by vroom:vroom
if [ "$($dcr --no-deps --entrypoint /bin/bash --user root vroom -c "stat -c '%U:%G' /var/vroom/sentry-profiles" 2>/dev/null)" = "vroom:vroom" ]; then
echo "Ownership of /var/vroom/sentry-profiles is already set to vroom:vroom. Skipping chown."
else
$dcr --no-deps --entrypoint /bin/bash --user root vroom -c 'chown -R vroom:vroom /var/vroom/sentry-profiles && chmod -R o+rwx /var/vroom/sentry-profiles'
fi
echo "${_endgroup}"
fi
================================================
FILE: install/ensure-files-from-examples.sh
================================================
echo "${_group}Ensuring files from examples ..."
ensure_file_from_example "$SENTRY_CONFIG_PY"
ensure_file_from_example "$SENTRY_CONFIG_YML"
ensure_file_from_example symbolicator/config.yml
echo "${_endgroup}"
================================================
FILE: install/ensure-relay-credentials.sh
================================================
echo "${_group}Ensuring Relay credentials ..."
RELAY_CONFIG_YML=relay/config.yml
RELAY_CREDENTIALS_JSON=relay/credentials.json
ensure_file_from_example $RELAY_CONFIG_YML
if [[ -f "$RELAY_CREDENTIALS_JSON" ]]; then
echo "$RELAY_CREDENTIALS_JSON already exists, skipped creation."
else
# There are a couple gotchas here:
#
# 1. We need to use a tmp file because if we redirect output directly to
# credentials.json, then the shell will create an empty file that relay
# will then try to read from (regardless of options such as --stdout or
# --overwrite) and fail because it is empty.
#
# 2. We pull relay:nightly before invoking `run relay credentials generate`
# because an implicit pull under the run causes extra stdout that results
# in a garbage credentials.json.
#
# 3. We need to use -T to ensure that we receive output on Docker Compose
# 1.x and 2.2.3+ (funny story about that ... ;). Note that the long opt
# --no-tty doesn't exist in Docker Compose 1.
$dc pull relay
creds="$dcr --no-deps -T relay credentials"
$creds generate --stdout >"$RELAY_CREDENTIALS_JSON".tmp
mv "$RELAY_CREDENTIALS_JSON".tmp "$RELAY_CREDENTIALS_JSON"
if ! grep -q Credentials <($creds show); then
# Let's fail early if creds failed, to make debugging easier.
echo "Failed to create relay credentials in $RELAY_CREDENTIALS_JSON."
echo "--- credentials.json v ---------------------------------------"
cat -v "$RELAY_CREDENTIALS_JSON" || true
echo "--- credentials.json ^ ---------------------------------------"
exit 1
fi
echo "Relay credentials written to $RELAY_CREDENTIALS_JSON."
fi
echo "${_endgroup}"
================================================
FILE: install/error-handling.sh
================================================
echo "${_group}Setting up error handling ..."
if [ -z "${SENTRY_DSN:-}" ]; then
export SENTRY_DSN='https://19555c489ded4769978daae92f2346ca@self-hosted.getsentry.net/3'
fi
$dbuild -t sentry-self-hosted-jq-local --platform="$DOCKER_PLATFORM" jq
jq="$CONTAINER_ENGINE run --rm -i sentry-self-hosted-jq-local"
sentry_cli="$CONTAINER_ENGINE run --rm -v /tmp:/work -e SENTRY_DSN=$SENTRY_DSN getsentry/sentry-cli"
send_envelope() {
# Send envelope
$sentry_cli send-envelope "$envelope_file"
}
generate_breadcrumb_json() {
cat $log_file | $jq -R -c 'split("\n") | {"message": (.[0]//""), "category": "log", "level": "info"}'
}
send_event() {
# Use traceback hash as the UUID since it is 32 characters long
local cmd_exit=$1
local error_msg=$2
local traceback=$3
local traceback_json=$4
local breadcrumbs=$5
local fingerprint_value=$(
echo -n "$cmd_exit $error_msg $traceback" |
$CONTAINER_ENGINE run -i --rm busybox md5sum |
cut -d' ' -f1
)
local envelope_file="sentry-envelope-${fingerprint_value}"
local envelope_file_path="/tmp/$envelope_file"
# If the envelope file exists, we've already sent it
if [[ -f $envelope_file_path ]]; then
echo "Looks like you've already sent this error to us, we're on it :)"
return
fi
# If we haven't sent the envelope file, make it and send to Sentry
# The format is documented at https://develop.sentry.dev/sdk/envelopes/
# Grab length of log file, needed for the envelope header to send an attachment
local file_length=$(wc -c <$log_file | awk '{print $1}')
# Add header for initial envelope information
$jq -n -c --arg event_id "$fingerprint_value" \
--arg dsn "$SENTRY_DSN" \
'$ARGS.named' >"$envelope_file_path"
# Add
gitextract_uhvx5ma5/
├── .craft.yml
├── .editorconfig
├── .gitattributes
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── config.yml
│ │ ├── feature-request.yml
│ │ ├── problem-report.yml
│ │ └── release.yml
│ ├── PULL_REQUEST_TEMPLATE.md
│ ├── dependabot.yml
│ └── workflows/
│ ├── changelog-preview.yml
│ ├── enforce-license-compliance.yml
│ ├── fast-revert.yml
│ ├── lock.yml
│ ├── pre-commit.yml
│ ├── release.yml
│ ├── shellcheck.yml
│ └── test.yml
├── .gitignore
├── .pre-commit-config.yaml
├── .python-version
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE.md
├── README.md
├── _integration-test/
│ ├── conftest.py
│ ├── custom-ca-roots/
│ │ ├── custom-ca-roots-test.py
│ │ └── docker-compose.test.yml
│ ├── fixtures/
│ │ ├── envelope-with-profile
│ │ └── envelope-with-transaction
│ ├── nodejs/
│ │ ├── .gitignore
│ │ ├── instrument.js
│ │ ├── package.json
│ │ └── user-feedback.js
│ ├── test_01_basics.py
│ └── test_02_backup.py
├── _unit-test/
│ ├── _test_setup.sh
│ ├── bootstrap-s3-nodestore-test.sh
│ ├── bootstrap-s3-profiles-test.sh
│ ├── create-docker-volumes-test.sh
│ ├── ensure-relay-credentials-test.sh
│ ├── error-handling-test.sh
│ ├── geoip-test.sh
│ ├── js-sdk-assets-test.sh
│ ├── merge-env-file-test.sh
│ ├── migrate-pgbouncer-test.sh
│ ├── multiple-seaweedfs-bucket-test.sh
│ └── snapshots/
│ └── sentry-envelope-f73e4da437c42a1d28b86a81ebcff35d
├── action.yaml
├── certificates/
│ └── .gitignore
├── clickhouse/
│ ├── Dockerfile
│ ├── config.xml
│ └── default-password.xml
├── codecov.yml
├── cron/
│ ├── Dockerfile
│ └── entrypoint.sh
├── docker-compose.yml
├── geoip/
│ └── GeoLite2-City.mmdb.empty
├── get-compose-action/
│ └── action.yaml
├── install/
│ ├── _detect-container-engine.sh
│ ├── _lib.sh
│ ├── _logging.sh
│ ├── _min-requirements.sh
│ ├── bootstrap-s3-nodestore.sh
│ ├── bootstrap-s3-profiles.sh
│ ├── bootstrap-snuba.sh
│ ├── build-docker-images.sh
│ ├── check-latest-commit.sh
│ ├── check-memcached-backend.sh
│ ├── check-minimum-requirements.sh
│ ├── create-docker-volumes.sh
│ ├── dc-detect-version.sh
│ ├── detect-platform.sh
│ ├── ensure-correct-permissions-profiles-dir.sh
│ ├── ensure-files-from-examples.sh
│ ├── ensure-relay-credentials.sh
│ ├── error-handling.sh
│ ├── generate-secret-key.sh
│ ├── geoip.sh
│ ├── migrate-pgbouncer.sh
│ ├── parse-cli.sh
│ ├── set-up-and-migrate-database.sh
│ ├── setup-js-sdk-assets.sh
│ ├── turn-things-off.sh
│ ├── update-docker-images.sh
│ ├── upgrade-clickhouse.sh
│ ├── upgrade-postgres.sh
│ └── wrap-up.sh
├── install.sh
├── jq/
│ └── Dockerfile
├── nginx.conf
├── optional-modifications/
│ ├── README.md
│ └── patches/
│ └── external-kafka/
│ ├── config.example.yml.patch
│ ├── docker-compose.yml.patch
│ └── sentry.conf.example.py.patch
├── pyproject.toml
├── redis.conf
├── relay/
│ └── config.example.yml
├── scripts/
│ ├── _lib.sh
│ ├── backup.sh
│ ├── bump-version.sh
│ ├── post-release.sh
│ ├── reset.sh
│ └── restore.sh
├── sentry/
│ ├── Dockerfile
│ ├── config.example.yml
│ ├── enhance-image.example.sh
│ ├── entrypoint.sh
│ ├── requirements.example.txt
│ └── sentry.conf.example.py
├── sentry-admin.sh
├── symbolicator/
│ └── config.example.yml
├── unit-test.sh
└── workstation/
├── 200_download-self-hosted.sh
├── 201_install-self-hosted.sh
├── 299_setup-completed.sh
├── README.md
├── commands.sh
├── postinstall/
│ └── Dockerfile
└── preinstall/
└── Dockerfile
SYMBOL INDEX (23 symbols across 5 files)
FILE: _integration-test/conftest.py
function configure_self_hosted_environment (line 14) | def configure_self_hosted_environment(request):
function setup_backup_restore_env_variables (line 44) | def setup_backup_restore_env_variables():
FILE: _integration-test/custom-ca-roots/custom-ca-roots-test.py
class CustomCATests (line 6) | class CustomCATests(unittest.TestCase):
method test_valid_self_signed (line 7) | def test_valid_self_signed(self):
method test_invalid_self_signed (line 10) | def test_invalid_self_signed(self):
FILE: _integration-test/test_01_basics.py
function poll_for_response (line 30) | def poll_for_response(
function get_sentry_dsn (line 49) | def get_sentry_dsn(client: httpx.Client) -> str:
function client_login (line 60) | def client_login():
function test_initial_redirect (line 80) | def test_initial_redirect():
function test_asset_internal_rewrite (line 85) | def test_asset_internal_rewrite():
function test_login (line 93) | def test_login(client_login):
function test_receive_event (line 125) | def test_receive_event(client_login):
function test_cleanup_crons_running (line 141) | def test_cleanup_crons_running():
function test_custom_certificate_authorities (line 160) | def test_custom_certificate_authorities():
function test_receive_transaction_events (line 407) | def test_receive_transaction_events(client_login):
function test_receive_user_feedback_events (line 434) | def test_receive_user_feedback_events(client_login):
function test_receive_logs_events (line 465) | def test_receive_logs_events(client_login):
function test_customizations (line 493) | def test_customizations():
FILE: _integration-test/test_02_backup.py
function test_sentry_admin (line 6) | def test_sentry_admin(setup_backup_restore_env_variables):
function test_01_backup (line 23) | def test_01_backup(setup_backup_restore_env_variables):
function test_02_import (line 44) | def test_02_import(setup_backup_restore_env_variables):
FILE: sentry/sentry.conf.example.py
function unit_text_to_bytes (line 10) | def unit_text_to_bytes(text):
function get_internal_network (line 17) | def get_internal_network():
Condensed preview — 119 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (304K chars).
[
{
"path": ".craft.yml",
"chars": 173,
"preview": "minVersion: 2.21.6\nchangelogPolicy: auto\npreReleaseCommand: bash scripts/bump-version.sh\nartifactProvider:\n name: none\n"
},
{
"path": ".editorconfig",
"chars": 183,
"preview": "root = true\n\n[*]\ncharset = utf-8\nend_of_line = lf\nindent_style = space\ninsert_final_newline = true\n\n[*.sh]\nindent_size ="
},
{
"path": ".gitattributes",
"chars": 134,
"preview": "/.gitattributes export-ignore\n/.gitignore export-ignore\n/.github export-ignore\n/.editorconfig export-ignore\n/.craft.yml "
},
{
"path": ".github/ISSUE_TEMPLATE/config.yml",
"chars": 207,
"preview": "blank_issues_enabled: false\ncontact_links:\n - name: Report a security vulnerability\n url: https://sentry.io/security"
},
{
"path": ".github/ISSUE_TEMPLATE/feature-request.yml",
"chars": 857,
"preview": "name: 💡 Feature Request\ndescription: Tell us about a problem our software could solve but doesn't.\nbody:\n - type: texta"
},
{
"path": ".github/ISSUE_TEMPLATE/problem-report.yml",
"chars": 3473,
"preview": "name: 🐞 Problem Report\ndescription: Tell us about something that's not working the way you expect.\nbody:\n - type: input"
},
{
"path": ".github/ISSUE_TEMPLATE/release.yml",
"chars": 2250,
"preview": "name: 📦 Release Issue\ndescription: Start a new self-hosted Sentry release\ntitle: Release YY.M.N\nbody:\n - type: textarea"
},
{
"path": ".github/PULL_REQUEST_TEMPLATE.md",
"chars": 608,
"preview": "\n\n\n<!-- Describe your PR here. -->\n\n\n\n<!--\n\n Sentry employees and contractors can delete or ignore the following.\n\n-->\n"
},
{
"path": ".github/dependabot.yml",
"chars": 467,
"preview": "version: 2\nupdates:\n - package-ecosystem: docker\n directory: \"/\"\n schedule:\n interval: daily\n open-pull-r"
},
{
"path": ".github/workflows/changelog-preview.yml",
"chars": 341,
"preview": "name: Changelog Preview\non:\n pull_request_target:\n types:\n - opened\n - synchronize\n - reopened\n - edited"
},
{
"path": ".github/workflows/enforce-license-compliance.yml",
"chars": 435,
"preview": "name: Enforce License Compliance\n\non:\n push:\n branches: [master]\n pull_request:\n branches: [master]\n\npermissions"
},
{
"path": ".github/workflows/fast-revert.yml",
"chars": 1651,
"preview": "on:\n pull_request_target:\n types: [labeled]\n workflow_dispatch:\n inputs:\n pr:\n required: true\n "
},
{
"path": ".github/workflows/lock.yml",
"chars": 444,
"preview": "name: 'Lock closed issues/PRs'\non:\n schedule:\n - cron: '11 3 * * *'\n workflow_dispatch:\njobs:\n lock:\n if: githu"
},
{
"path": ".github/workflows/pre-commit.yml",
"chars": 441,
"preview": "name: pre-commit\n\non:\n pull_request:\n push:\n branches: [master]\n\npermissions:\n contents: read\n\njobs:\n pre-commit:"
},
{
"path": ".github/workflows/release.yml",
"chars": 2267,
"preview": "name: Release\non:\n workflow_dispatch:\n inputs:\n version:\n description: Version to release (or \"auto\")\n "
},
{
"path": ".github/workflows/shellcheck.yml",
"chars": 1253,
"preview": "name: \"ShellCheck\"\non:\n push:\n paths:\n - \"**.sh\"\n branches: [master]\n pull_request:\n paths:\n - \"**."
},
{
"path": ".github/workflows/test.yml",
"chars": 2584,
"preview": "\nname: Test\non:\n # Run CI on all pushes to the master and release/** branches, and on all new\n # pull requests, and on"
},
{
"path": ".gitignore",
"chars": 1524,
"preview": "# Error reporting choice cache\n.reporterrors\n\n# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n"
},
{
"path": ".pre-commit-config.yaml",
"chars": 723,
"preview": "exclude: '\\.patch$'\nrepos:\n- repo: local\n hooks:\n # Based on https://github.com/scop/pre-commit-shfmt/blob/main/.pre"
},
{
"path": ".python-version",
"chars": 5,
"preview": "3.12\n"
},
{
"path": "CHANGELOG.md",
"chars": 45117,
"preview": "# Changelog\n\n## 26.3.1\n\n- No documented changes.\n\n## 26.3.0\n\n### New Features ✨\n\n- Reorder pull images by @aldy505 in [#"
},
{
"path": "CONTRIBUTING.md",
"chars": 278,
"preview": "## Testing\n\n### Running Tests with Pytest\n\nWe use pytest for running tests. To run the tests:\n\n1) Ensure that you are in"
},
{
"path": "LICENSE.md",
"chars": 3785,
"preview": "# Functional Source License, Version 1.1, Apache 2.0 Future License\n\n## Abbreviation\n\nFSL-1.1-Apache-2.0\n\n## Notice\n\nCop"
},
{
"path": "README.md",
"chars": 206,
"preview": "# Self-Hosted Sentry 26.3.1\n\n[Sentry](https://sentry.io/), feature-complete and packaged up for low-volume deployments a"
},
{
"path": "_integration-test/conftest.py",
"chars": 1100,
"preview": "import os\nfrom os.path import join\nimport subprocess\n\nimport pytest\n\nSENTRY_CONFIG_PY = \"sentry/sentry.conf.py\"\nSENTRY_T"
},
{
"path": "_integration-test/custom-ca-roots/custom-ca-roots-test.py",
"chars": 382,
"preview": "import unittest\n\nimport requests\n\n\nclass CustomCATests(unittest.TestCase):\n def test_valid_self_signed(self):\n "
},
{
"path": "_integration-test/custom-ca-roots/docker-compose.test.yml",
"chars": 278,
"preview": "version: '3.4'\nservices:\n fixture-custom-ca-roots:\n image: nginx:1.21.0-alpine\n restart: unless-stopped\n volum"
},
{
"path": "_integration-test/fixtures/envelope-with-profile",
"chars": 2683,
"preview": "{\"event_id\":\"66578634d48d433db0ad52882d1efe5b\",\"sent_at\":\"2023-05-17T14:54:31.057Z\",\"sdk\":{\"name\":\"sentry.javascript.nod"
},
{
"path": "_integration-test/fixtures/envelope-with-transaction",
"chars": 1601,
"preview": "{\"event_id\":\"66578634d48d433db0ad52882d1efe5b\",\"sent_at\":\"2023-05-17T14:54:31.057Z\",\"sdk\":{\"name\":\"sentry.javascript.nod"
},
{
"path": "_integration-test/nodejs/.gitignore",
"chars": 13,
"preview": "node_modules\n"
},
{
"path": "_integration-test/nodejs/instrument.js",
"chars": 266,
"preview": "import * as Sentry from \"@sentry/node\";\n\nSentry.init({\n dsn: process.env.SENTRY_DSN,\n sampleRate: 1.0,\n tracesS"
},
{
"path": "_integration-test/nodejs/package.json",
"chars": 345,
"preview": "{\n \"name\": \"sentry-self-hosted-integration-test-nodejs\",\n \"version\": \"0.0.0\",\n \"description\": \"Scripts to run for fea"
},
{
"path": "_integration-test/nodejs/user-feedback.js",
"chars": 214,
"preview": "import * as Sentry from \"@sentry/node\";\n\nSentry.captureFeedback({\n message: \"I love your startup!\",\n name: \"John D"
},
{
"path": "_integration-test/test_01_basics.py",
"chars": 18470,
"preview": "import datetime\nimport json\nimport os\nimport re\nimport shutil\nimport subprocess\nimport sys\nimport time\nfrom functools im"
},
{
"path": "_integration-test/test_02_backup.py",
"chars": 2997,
"preview": "import os\nfrom os.path import join\nimport subprocess\n\n\ndef test_sentry_admin(setup_backup_restore_env_variables):\n se"
},
{
"path": "_unit-test/_test_setup.sh",
"chars": 1474,
"preview": "set -euo pipefail\n\nsource install/_lib.sh\n\n_ORIGIN=$(pwd)\n\nrm -rf /tmp/sentry-self-hosted-test-sandbox.*\n_SANDBOX=\"$(mkt"
},
{
"path": "_unit-test/bootstrap-s3-nodestore-test.sh",
"chars": 393,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\nsource install/dc-detect-version.sh\nsource install/create-docker-v"
},
{
"path": "_unit-test/bootstrap-s3-profiles-test.sh",
"chars": 1330,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\nsource install/dc-detect-version.sh\nsource install/create-docker-v"
},
{
"path": "_unit-test/create-docker-volumes-test.sh",
"chars": 588,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\n\nget_volumes() {\n # If grep returns no strings, we still want to "
},
{
"path": "_unit-test/ensure-relay-credentials-test.sh",
"chars": 745,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\nsource install/dc-detect-version.sh\n\n# using _file format for thes"
},
{
"path": "_unit-test/error-handling-test.sh",
"chars": 2400,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\n\nexport REPORT_SELF_HOSTED_ISSUES=1\n\n# This is set up in dc-detect"
},
{
"path": "_unit-test/geoip-test.sh",
"chars": 326,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\n\nmmdb=geoip/GeoLite2-City.mmdb\n\n# Starts with no mmdb, ends up wit"
},
{
"path": "_unit-test/js-sdk-assets-test.sh",
"chars": 1173,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\nsource install/dc-detect-version.sh\n$dcb --force-rm web\n$dc pull n"
},
{
"path": "_unit-test/merge-env-file-test.sh",
"chars": 933,
"preview": "#!/usr/bin/env bash\n\n# This is a test file for a part of `_lib.sh`, where we read `.env.custom` file if there is one.\n# "
},
{
"path": "_unit-test/migrate-pgbouncer-test.sh",
"chars": 4407,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\nsource install/dc-detect-version.sh\n\nsource install/ensure-files-f"
},
{
"path": "_unit-test/multiple-seaweedfs-bucket-test.sh",
"chars": 1221,
"preview": "#!/usr/bin/env bash\n\nsource _unit-test/_test_setup.sh\nsource install/dc-detect-version.sh\nsource install/create-docker-v"
},
{
"path": "_unit-test/snapshots/sentry-envelope-f73e4da437c42a1d28b86a81ebcff35d",
"chars": 525,
"preview": "{\"event_id\":\"f73e4da437c42a1d28b86a81ebcff35d\",\"dsn\":\"https://19555c489ded4769978daae92f2346ca@self-hosted.getsentry.net"
},
{
"path": "action.yaml",
"chars": 10256,
"preview": "name: \"Sentry self-hosted end-to-end tests\"\ninputs:\n project_name:\n required: false\n description: \"e.g. snuba, se"
},
{
"path": "certificates/.gitignore",
"chars": 50,
"preview": "# Add all custom CAs in this folder\n*\n!.gitignore\n"
},
{
"path": "clickhouse/Dockerfile",
"chars": 34,
"preview": "ARG BASE_IMAGE\nFROM ${BASE_IMAGE}\n"
},
{
"path": "clickhouse/config.xml",
"chars": 1325,
"preview": "<yandex>\n <!-- This include is important! It is required for the version of Clickhouse used on ARM to read the environme"
},
{
"path": "clickhouse/default-password.xml",
"chars": 162,
"preview": "<clickhouse>\n <users>\n <default>\n <password></password>\n <networks>\n <ip>::/0</ip>\n </networks"
},
{
"path": "codecov.yml",
"chars": 124,
"preview": "coverage:\n status:\n project:\n default:\n only_pulls: true\n patch:\n default:\n only_pulls: t"
},
{
"path": "cron/Dockerfile",
"chars": 649,
"preview": "ARG BASE_IMAGE\nFROM ${BASE_IMAGE}\nUSER 0\nRUN if [ -n \"${HTTP_PROXY}\" ]; then echo \"Acquire::http::proxy \\\"${HTTP_PROXY}\\"
},
{
"path": "cron/entrypoint.sh",
"chars": 616,
"preview": "#!/usr/bin/env bash\n\nif [ \"$(ls -A /usr/local/share/ca-certificates/)\" ]; then\n update-ca-certificates\nfi\n\n# Prior art:"
},
{
"path": "docker-compose.yml",
"chars": 35137,
"preview": "x-restart-policy: &restart_policy\n restart: unless-stopped\nx-pull-policy: &pull_policy\n pull_policy: never\nx-depends_o"
},
{
"path": "get-compose-action/action.yaml",
"chars": 771,
"preview": "name: \"Get Docker Compose\"\ninputs:\n version:\n required: false\n default: 2.33.1\n description: \"Docker Compose v"
},
{
"path": "install/_detect-container-engine.sh",
"chars": 403,
"preview": "echo \"${_group}Detecting container engine ...\"\n\nif [[ \"${CONTAINER_ENGINE_PODMAN:-0}\" -eq 1 ]] && command -v podman &>/d"
},
{
"path": "install/_lib.sh",
"chars": 2098,
"preview": "# Allow `.env` overrides using the `.env.custom` file.\n# We pass this to docker compose in a couple places.\nif [[ -f .en"
},
{
"path": "install/_logging.sh",
"chars": 150,
"preview": "# Thanks to https://unix.stackexchange.com/a/145654/108960\nlog_file=sentry_install_log-$(date +'%Y-%m-%d_%H-%M-%S').txt\n"
},
{
"path": "install/_min-requirements.sh",
"chars": 467,
"preview": "# Don't forget to update the README and other docs when you change these!\nMIN_DOCKER_VERSION='19.03.6'\nMIN_COMPOSE_VERSI"
},
{
"path": "install/bootstrap-s3-nodestore.sh",
"chars": 3768,
"preview": "echo \"${_group}Bootstrapping seaweedfs (node store)...\"\n\nstart_service_and_wait_ready seaweedfs postgres\n$dcx seaweedfs "
},
{
"path": "install/bootstrap-s3-profiles.sh",
"chars": 5565,
"preview": "# The purpose of this file is to have both `sentry`-based containers and `vroom` use the same bucket for profiling.\n# On"
},
{
"path": "install/bootstrap-snuba.sh",
"chars": 249,
"preview": "echo \"${_group}Bootstrapping and migrating Snuba ...\"\n\nif [[ -z \"${SKIP_SNUBA_MIGRATIONS:-}\" ]]; then\n $dcr snuba-api b"
},
{
"path": "install/build-docker-images.sh",
"chars": 424,
"preview": "echo \"${_group}Building and tagging Docker images ...\"\n\necho \"\"\n# Build any service that provides the image sentry-self-"
},
{
"path": "install/check-latest-commit.sh",
"chars": 635,
"preview": "echo \"${_group}Checking for latest commit ... \"\n\n# Checks if we are on latest commit from github if it is running from m"
},
{
"path": "install/check-memcached-backend.sh",
"chars": 527,
"preview": "echo \"${_group}Checking memcached backend ...\"\n\nif grep -q \"\\.PyMemcacheCache\" \"$SENTRY_CONFIG_PY\"; then\n echo \"PyMemca"
},
{
"path": "install/check-minimum-requirements.sh",
"chars": 2933,
"preview": "echo \"${_group}Checking minimum requirements ...\"\n\nsource install/_min-requirements.sh\n\nDOCKER_VERSION=$($CONTAINER_ENGI"
},
{
"path": "install/create-docker-volumes.sh",
"chars": 614,
"preview": "echo \"${_group}Creating volumes for persistent storage ...\"\n\ncreate_volume() {\n create_command=\"$CONTAINER_ENGINE volum"
},
{
"path": "install/dc-detect-version.sh",
"chars": 3366,
"preview": "if [ \"${GITHUB_ACTIONS:-}\" = \"true\" ]; then\n _group=\"::group::\"\n _endgroup=\"::endgroup::\"\nelse\n _group=\"▶ \"\n _endgro"
},
{
"path": "install/detect-platform.sh",
"chars": 1664,
"preview": "source install/_detect-container-engine.sh\n\necho \"${_group}Detecting Docker platform\"\n\n# Sentry SaaS uses stock Yandex C"
},
{
"path": "install/ensure-correct-permissions-profiles-dir.sh",
"chars": 812,
"preview": "#!/usr/bin/env bash\n\n# TODO: Remove this after the next hard-stop\n\n# Should only run when `$COMPOSE_PROFILES` is set to "
},
{
"path": "install/ensure-files-from-examples.sh",
"chars": 211,
"preview": "echo \"${_group}Ensuring files from examples ...\"\n\nensure_file_from_example \"$SENTRY_CONFIG_PY\"\nensure_file_from_example "
},
{
"path": "install/ensure-relay-credentials.sh",
"chars": 1684,
"preview": "echo \"${_group}Ensuring Relay credentials ...\"\n\nRELAY_CONFIG_YML=relay/config.yml\nRELAY_CREDENTIALS_JSON=relay/credentia"
},
{
"path": "install/error-handling.sh",
"chars": 8122,
"preview": "echo \"${_group}Setting up error handling ...\"\n\nif [ -z \"${SENTRY_DSN:-}\" ]; then\n export SENTRY_DSN='https://19555c489d"
},
{
"path": "install/generate-secret-key.sh",
"chars": 641,
"preview": "echo \"${_group}Generating secret key ...\"\n\nif grep -xq \"system.secret-key: '!!changeme!!'\" $SENTRY_CONFIG_YML; then\n # "
},
{
"path": "install/geoip.sh",
"chars": 1715,
"preview": "echo \"${_group}Setting up GeoIP integration ...\"\n\n# If `$CONTAINER_ENGINE` is not set, we assume that we are running thi"
},
{
"path": "install/migrate-pgbouncer.sh",
"chars": 2821,
"preview": "echo \"${_group}Migrating Postgres config to PGBouncer...\"\n# If users has this EXACT configuration on their `sentry.conf."
},
{
"path": "install/parse-cli.sh",
"chars": 3397,
"preview": "echo \"${_group}Parsing command line ...\"\n\nshow_help() {\n cat <<EOF\nUsage: $0 [options]\n\nInstall Sentry with \\`docker|po"
},
{
"path": "install/set-up-and-migrate-database.sh",
"chars": 945,
"preview": "echo \"${_group}Setting up / migrating database ...\"\n\nif [[ -z \"${SKIP_SENTRY_MIGRATIONS:-}\" ]]; then\n # Fixes https://g"
},
{
"path": "install/setup-js-sdk-assets.sh",
"chars": 3032,
"preview": "# This will only run if the SETUP_JS_SDK_ASSETS environment variable is set to 1.\n# Think of this as some kind of a feat"
},
{
"path": "install/turn-things-off.sh",
"chars": 928,
"preview": "echo \"${_group}Turning things off ...\"\n\nif [[ -n \"$MINIMIZE_DOWNTIME\" ]]; then\n # Stop everything but relay and nginx\n "
},
{
"path": "install/update-docker-images.sh",
"chars": 845,
"preview": "echo \"${_group}Fetching and updating $CONTAINER_ENGINE images ...\"\n\nif [ \"$CONTAINER_ENGINE\" = \"podman\" ]; then\n # podm"
},
{
"path": "install/upgrade-clickhouse.sh",
"chars": 2479,
"preview": "echo \"${_group}Upgrading Clickhouse ...\"\n\n# First check to see if user is upgrading by checking for existing clickhouse "
},
{
"path": "install/upgrade-postgres.sh",
"chars": 2135,
"preview": "echo \"${_group}Ensuring proper PostgreSQL version ...\"\n\nif [[ -n \"$($CONTAINER_ENGINE volume ls -q --filter name=sentry-"
},
{
"path": "install/wrap-up.sh",
"chars": 1347,
"preview": "if [[ \"$MINIMIZE_DOWNTIME\" ]]; then\n echo \"${_group}Waiting for Sentry to start ...\"\n\n # Start the whole setup, except"
},
{
"path": "install.sh",
"chars": 1707,
"preview": "#!/usr/bin/env bash\nset -eEuo pipefail\ntest \"${DEBUG:-}\" && set -x\n\n# Override any user-supplied umask that could cause "
},
{
"path": "jq/Dockerfile",
"chars": 253,
"preview": "FROM debian:bookworm-slim\n\nLABEL MAINTAINER=\"oss@sentry.io\"\n\nRUN set -x \\\n && apt-get update \\\n && DEBIAN_FRONTEND=non"
},
{
"path": "nginx.conf",
"chars": 2572,
"preview": "user nginx;\nworker_processes auto;\n\nerror_log /var/log/nginx/error.log warn;\npid /var/run/nginx.pid;\n\n\nevents {\n\tworker_"
},
{
"path": "optional-modifications/README.md",
"chars": 3449,
"preview": "# Optional Modifications\n\nWhile the default self-hosted Sentry installation is often sufficient, there are instances whe"
},
{
"path": "optional-modifications/patches/external-kafka/config.example.yml.patch",
"chars": 1228,
"preview": "--- relay/config.example.yml\t2025-05-15 08:27:40.426876887 +0700\n+++ relay/config.example.external-kafka.yml\t2025-05-15 "
},
{
"path": "optional-modifications/patches/external-kafka/docker-compose.yml.patch",
"chars": 8260,
"preview": "--- docker-compose.yml\t2025-07-08 10:22:36.600616503 +0700\n+++ docker-compose.external-kafka.yml\t2025-07-08 10:36:44.069"
},
{
"path": "optional-modifications/patches/external-kafka/sentry.conf.example.py.patch",
"chars": 1313,
"preview": "--- sentry/sentry.conf.example.py\t2025-05-15 08:27:40.427876868 +0700\n+++ sentry/sentry.conf.example.external-kafka.py\t2"
},
{
"path": "pyproject.toml",
"chars": 479,
"preview": "[project]\nname = \"sentry-self-hosted\"\nversion = \"0.1.0\"\ndescription = \"Sentry, feature-complete and packaged up for low-"
},
{
"path": "redis.conf",
"chars": 1252,
"preview": "# redis.conf\n\n# The 'maxmemory' directive controls the maximum amount of memory Redis is allowed to use.\n# Setting 'maxm"
},
{
"path": "relay/config.example.yml",
"chars": 1277,
"preview": "relay:\n upstream: \"http://web:9000/\"\n host: 0.0.0.0\n port: 3000\nlogging:\n level: WARN\nprocessing:\n enabled: true\n "
},
{
"path": "scripts/_lib.sh",
"chars": 2626,
"preview": "#!/usr/bin/env bash\n\nset -eEuo pipefail\n\nif [ -n \"${DEBUG:-}\" ]; then\n set -x\nfi\n\nfunction confirm() {\n read -r -p \"$1"
},
{
"path": "scripts/backup.sh",
"chars": 412,
"preview": "#!/usr/bin/env bash\nMINIMIZE_DOWNTIME=\"${MINIMIZE_DOWNTIME:-}\"\nREPORT_SELF_HOSTED_ISSUES=\"${REPORT_SELF_HOSTED_ISSUES:-}"
},
{
"path": "scripts/bump-version.sh",
"chars": 601,
"preview": "#!/usr/bin/env bash\nset -eu\n\n# Craft passes versions via env vars (preferred) with positional args as fallback\n# for man"
},
{
"path": "scripts/post-release.sh",
"chars": 302,
"preview": "#!/usr/bin/env bash\nset -eu\n\n# Bring master back to nightlies after merge from release branch\ngit checkout master && git"
},
{
"path": "scripts/reset.sh",
"chars": 406,
"preview": "#!/usr/bin/env bash\nMINIMIZE_DOWNTIME=\"${MINIMIZE_DOWNTIME:-}\"\nREPORT_SELF_HOSTED_ISSUES=\"${REPORT_SELF_HOSTED_ISSUES:-}"
},
{
"path": "scripts/restore.sh",
"chars": 414,
"preview": "#!/usr/bin/env bash\nMINIMIZE_DOWNTIME=\"${MINIMIZE_DOWNTIME:-}\"\nREPORT_SELF_HOSTED_ISSUES=\"${REPORT_SELF_HOSTED_ISSUES:-}"
},
{
"path": "sentry/Dockerfile",
"chars": 514,
"preview": "ARG SENTRY_IMAGE\nFROM ${SENTRY_IMAGE}\n\nRUN pip install https://github.com/getsentry/sentry-nodestore-s3/archive/main.zip"
},
{
"path": "sentry/config.example.yml",
"chars": 5779,
"preview": "# While a lot of configuration in Sentry can be changed via the UI, for all\n# new-style config (as of 8.0) you can also "
},
{
"path": "sentry/enhance-image.example.sh",
"chars": 289,
"preview": "#!/bin/bash\nset -euo pipefail\n\n# Enhance the base $SENTRY_IMAGE with additional dependencies, plugins - see https://deve"
},
{
"path": "sentry/entrypoint.sh",
"chars": 328,
"preview": "#!/bin/bash\nset -e\n\nif [ \"$(ls -A /usr/local/share/ca-certificates/)\" ]; then\n update-ca-certificates\nfi\n\nif [ -e /etc/"
},
{
"path": "sentry/requirements.example.txt",
"chars": 136,
"preview": "# sentry/requirements.txt is deprecated, use sentry/enhance-image.sh - see https://develop.sentry.dev/self-hosted/#enhan"
},
{
"path": "sentry/sentry.conf.example.py",
"chars": 18496,
"preview": "# This file is just Python, with a touch of Django which means\n# you can inherit and tweak settings to your hearts conte"
},
{
"path": "sentry-admin.sh",
"chars": 2418,
"preview": "#!/usr/bin/env bash\n\n# Set the script directory as working directory.\ncd $(dirname $0)\n\n# Detect docker and platform sta"
},
{
"path": "symbolicator/config.example.yml",
"chars": 493,
"preview": "# See: https://getsentry.github.io/symbolicator/#configuration\ncache_dir: \"/data\"\nbind: \"0.0.0.0:3021\"\nlogging:\n level:"
},
{
"path": "unit-test.sh",
"chars": 669,
"preview": "#!/usr/bin/env bash\n\n# The test should not be run by regular users. This should only be run in CI or by developers.\nif ["
},
{
"path": "workstation/200_download-self-hosted.sh",
"chars": 275,
"preview": "#!/bin/bash\nset -eo pipefail\n\n# Create getsentry folder and enter.\nmkdir /home/user/getsentry\ncd /home/user/getsentry\n\n#"
},
{
"path": "workstation/201_install-self-hosted.sh",
"chars": 383,
"preview": "#!/bin/bash\n#\n\n# Install self-hosted. Assumed `200_download-self-hosted.sh` has already run.\n./install.sh --skip-commit-"
},
{
"path": "workstation/299_setup-completed.sh",
"chars": 427,
"preview": "#!/bin/bash\n#\n\n# Add a dot file to the home directory indicating that the setup has been completed successfully.\n# The h"
},
{
"path": "workstation/README.md",
"chars": 7969,
"preview": "# Remote Self-Hosted Development on Google Cloud Workstation\n\nThis document specifies how to set up remote workstation d"
},
{
"path": "workstation/commands.sh",
"chars": 434,
"preview": "sudo wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor >packages.microsoft.gpg\nsudo install -D"
},
{
"path": "workstation/postinstall/Dockerfile",
"chars": 1675,
"preview": "# Use a predefined image with no out-of-the-box IDE set up.\nFROM us-west1-docker.pkg.dev/cloud-workstations-images/prede"
},
{
"path": "workstation/preinstall/Dockerfile",
"chars": 1615,
"preview": "# Use a predefined image with no out-of-the-box IDE set up.\nFROM us-west1-docker.pkg.dev/cloud-workstations-images/prede"
}
]
// ... and 1 more files (download for full content)
About this extraction
This page contains the full source code of the getsentry/self-hosted GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 119 files (278.1 KB), approximately 81.4k tokens, and a symbol index with 23 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.