Showing preview only (370K chars total). Download the full file or copy to clipboard to get everything.
Repository: magic-wormhole/magic-wormhole-mailbox-server
Branch: master
Commit: d4e42ade2e8e
Files: 71
Total size: 348.8 KB
Directory structure:
gitextract_ojb3q6ak/
├── .appveyor.yml
├── .coveragerc
├── .gitattributes
├── .github/
│ └── workflows/
│ └── test.yml
├── .gitignore
├── .travis.yml
├── LICENSE
├── MANIFEST.in
├── Makefile
├── NEWS.md
├── README.md
├── docs/
│ ├── Makefile
│ ├── conf.py
│ ├── happy-plant.seq
│ ├── happy.seq
│ ├── index.rst
│ ├── server-protocol.md
│ ├── states.dot
│ └── welcome.md
├── misc/
│ ├── migrate_channel_db.py
│ ├── migrate_usage_db.py
│ ├── munin/
│ │ ├── wormhole_active
│ │ ├── wormhole_errors
│ │ ├── wormhole_event_rate
│ │ ├── wormhole_events
│ │ ├── wormhole_events_alltime
│ │ └── wormhole_version_uptake
│ └── windows-build.cmd
├── newest-version.py
├── setup.cfg
├── setup.py
├── signatures/
│ ├── magic-wormhole-mailbox-server-0.5.0.tar.gz.asc
│ ├── magic-wormhole-mailbox-server-0.5.1.tar.gz.asc
│ ├── magic_wormhole_mailbox_server-0.5.0-py3-none-any.whl.asc
│ ├── magic_wormhole_mailbox_server-0.5.1-py3-none-any.whl.asc
│ ├── magic_wormhole_mailbox_server-0.6.0-py3-none-any.whl.asc
│ ├── magic_wormhole_mailbox_server-0.6.0.tar.gz.asc
│ ├── magic_wormhole_mailbox_server-0.7.0-py3-none-any.whl.asc
│ ├── magic_wormhole_mailbox_server-0.7.0.tar.gz.asc
│ ├── magic_wormhole_mailbox_server-0.8.0-py3-none-any.whl.asc
│ └── magic_wormhole_mailbox_server-0.8.0.tar.gz.asc
├── src/
│ ├── twisted/
│ │ └── plugins/
│ │ └── magic_wormhole_mailbox.py
│ └── wormhole_mailbox_server/
│ ├── __init__.py
│ ├── _version.py
│ ├── database.py
│ ├── db-schemas/
│ │ ├── channel-v1.sql
│ │ ├── upgrade-usage-to-v2.sql
│ │ ├── usage-v1.sql
│ │ └── usage-v2.sql
│ ├── increase_rlimits.py
│ ├── server.py
│ ├── server_tap.py
│ ├── server_websocket.py
│ ├── test/
│ │ ├── __init__.py
│ │ ├── common.py
│ │ ├── test_config.py
│ │ ├── test_database.py
│ │ ├── test_rlimits.py
│ │ ├── test_server.py
│ │ ├── test_service.py
│ │ ├── test_stats.py
│ │ ├── test_util.py
│ │ ├── test_web.py
│ │ ├── test_websocket.py
│ │ ├── test_ws_client.py
│ │ └── ws_client.py
│ ├── util.py
│ └── web.py
├── tox.ini
├── update-version.py
└── versioneer.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .appveyor.yml
================================================
# adapted from https://packaging.python.org/en/latest/appveyor/
environment:
# we tell Tox to use "twisted[windows]", to get pypiwin32 installed
#TWISTED_EXTRAS: "[windows]"
# that didn't work (it seems to work when I run it locally, but on appveyor
# it fails to install the pypiwin32 package). So don't bother telling
# Twisted to support windows: just install it ourselves.
# EXTRA_DEPENDENCY: "pypiwin32"
matrix:
# For Python versions available on Appveyor, see
# http://www.appveyor.com/docs/installed-software#python
- PYTHON: "C:\\Python27"
- PYTHON: "C:\\Python27-x64"
DISTUTILS_USE_SDK: "1"
- PYTHON: "C:\\Python35"
- PYTHON: "C:\\Python36"
- PYTHON: "C:\\Python36-x64"
install:
- |
%PYTHON%\python.exe -m pip install wheel tox
# note:
# %PYTHON% has: python.exe
# %PYTHON%\Scripts has: pip.exe, tox.exe (and others installed by bare pip)
build: off
test_script:
# Put your test command here.
# Note that you must use the environment variable %PYTHON% to refer to
# the interpreter you're using - Appveyor does not do anything special
# to put the Python evrsion you want to use on PATH.
- |
misc\windows-build.cmd %PYTHON%\Scripts\tox.exe -e py
after_test:
# This step builds your wheels.
# Again, you only need build.cmd if you're building C extensions for
# 64-bit Python 3.3/3.4. And you need to use %PYTHON% to get the correct
# interpreter
- |
misc\windows-build.cmd %PYTHON%\python.exe setup.py bdist_wheel
artifacts:
# bdist_wheel puts your built wheel in the dist directory
- path: dist\*
#on_success:
# You can use this step to upload your artifacts to a public website.
# See Appveyor's documentation for more details. Or you can simply
# access your wheels from the Appveyor "artifacts" tab for your build.
================================================
FILE: .coveragerc
================================================
# -*- mode: conf -*-
[run]
# only record trace data for wormhole_mailbox_server.*
source =
wormhole_mailbox_server
# and don't trace the test files themselves, or Versioneer's stuff
omit =
src/wormhole_mailbox_server/test/*
src/wormhole_mailbox_server/_version.py
# This allows 'coverage combine' to correlate the tracing data built while
# running tests in multiple tox virtualenvs. To take advantage of this
# properly, use "coverage erase" before tox, "coverage run --parallel-mode"
# inside tox to avoid overwriting the output data (by writing it into
# .coverage-XYZ instead of just .coverage), and run "coverage combine"
# afterwards.
[paths]
source =
src/
.tox/*/lib/python*/site-packages/
.tox/pypy*/site-packages/
================================================
FILE: .gitattributes
================================================
src/wormhole_mailbox_server/_version.py export-subst
================================================
FILE: .github/workflows/test.yml
================================================
name: Tests
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
testing:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip tox codecov
tox --notest -e coverage
- name: Test
run: |
python --version
tox -e coverage
- name: Upload Coverage
run: codecov
================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
.eggs
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.cache
nosetests.xml
coverage.xml
_trial_temp/
# Translations
*.mo
*.pot
# Django stuff:
*.log
# Sphinx documentation
docs/_build/
# PyBuilder
target/
/twistd.pid
/relay.sqlite
/usage.sqlite
# Virtual environment stuff
venv/
/src/twisted/plugins/dropin.cache
================================================
FILE: .travis.yml
================================================
arch:
- amd64
- ppc64le
language: python
# defaults: the py3.7 environment overrides these
dist: trusty
sudo: false
cache: pip
before_cache:
- rm -f $HOME/.cache/pip/log/debug.log
branches:
except:
- /^WIP-.*$/
install:
- pip install -U pip tox virtualenv codecov
script:
- if [[ $TRAVIS_PYTHON_VERSION == 3.6 ]]; then
tox -e flake8 ;
fi
- tox -e coverage
after_success:
- codecov
matrix:
include:
- python: 2.7
- python: 3.5
- python: 3.6
- python: 3.7
# we don't actually need sudo, but that kicks us onto GCE, which lets
# us get xenial
sudo: true
dist: xenial
- python: nightly
allow_failures:
- python: nightly
================================================
FILE: LICENSE
================================================
The MIT License (MIT)
Copyright (c) 2015 Brian Warner
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: MANIFEST.in
================================================
include versioneer.py
include src/wormhole_mailbox_server/_version.py
include LICENSE README.md NEWS.md
recursive-include docs *.md *.rst *.dot
include docs/conf.py docs/Makefile
include .coveragerc tox.ini snapcraft.yaml
include misc/windows-build.cmd
include misc/*.py
include misc/munin/wormhole_active
include misc/munin/wormhole_errors
include misc/munin/wormhole_event_rate
include misc/munin/wormhole_events
include misc/munin/wormhole_events_alltime
================================================
FILE: Makefile
================================================
# How to Make a Release
# ---------------------
#
# This file answers the question "how to make a release" hopefully
# better than a document does (only meejah and warner may currently do
# the "upload to PyPI" part anyway)
#
default:
echo "see Makefile"
release-clean:
@echo "Cleanup stale release: " `python newest-version.py`
-rm NEWS.md.asc
-rm dist/magic_wormhole_mailbox_server-`python newest-version.py`.tar.gz*
-rm dist/magic_wormhole_mailbox_server-`python newest-version.py`-py3-none-any.whl*
git tag -d `python newest-version.py`
# create a branch, like: git checkout -b prepare-release-0.16.0
# then run these, so CI can run on the release
release:
@echo "Is checkout clean?"
git diff-files --quiet
git diff-index --quiet --cached HEAD --
@echo "Install required build software"
python -m pip install --editable .[dev,release]
@echo "Test README"
python setup.py check -s
@echo "Is GPG Agent rubnning, and has key?"
gpg --pinentry=loopback -u meejah@meejah.ca --armor --clear-sign NEWS.md
@echo "Bump version and create tag"
python update-version.py
# python update-version.py --patch # for bugfix release
@echo "Build and sign wheel"
python setup.py bdist_wheel
gpg --pinentry=loopback -u meejah@meejah.ca --armor --detach-sign dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`-py3-none-any.whl
ls dist/*`git describe --abbrev=0`*
@echo "Build and sign source-dist"
python setup.py sdist
gpg --pinentry=loopback -u meejah@meejah.ca --armor --detach-sign dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`.tar.gz
ls dist/*`git describe --abbrev=0`*
release-test:
gpg --verify dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`.tar.gz.asc
gpg --verify dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`-py3-none-any.whl.asc
python -m venv testmf_venv
testmf_venv/bin/pip install --upgrade pip
testmf_venv/bin/pip install dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`-py3-none-any.whl
testmf_venv/bin/twistd wormhole-mailbox --version
testmf_venv/bin/pip uninstall -y magic_wormhole_mailbox_server
testmf_venv/bin/pip install dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`.tar.gz
testmf_venv/bin/twistd wormhole-mailbox --version
rm -rf testmf_venv
release-upload:
twine upload --username __token__ --password `cat PRIVATE-release-token` dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`-py3-none-any.whl dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`-py3-none-any.whl.asc dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`.tar.gz dist/magic_wormhole_mailbox_server-`git describe --abbrev=0`.tar.gz.asc
mv dist/*-`git describe --abbrev=0`.tar.gz.asc signatures/
mv dist/*-`git describe --abbrev=0`-py3-none-any.whl.asc signatures/
git add signatures/magic_wormhole_mailbox_server-`git describe --abbrev=0`.tar.gz.asc
git add signatures/magic_wormhole_mailbox_server-`git describe --abbrev=0`-py3-none-any.whl.asc
git commit -m "signatures for release"
git push origin-push `git describe --abbrev=0`
dilation.png: dilation.seqdiag
seqdiag --no-transparency -T png --size 1000x800 -o dilation.png
================================================
FILE: NEWS.md
================================================
User-visible changes in "magic-wormhole-mailbox-server":
## Upcoming
* (put release-notes here when merging / proposing a PR)
## Release 0.8.0 (15-May-2026)
* Server header properly reports version (#27)
* introduce a ``"your-address"`` key to the ``"welcome"`` message (to reflect the IP address and port back, #63)
## Release 0.7.0 (14-May-2026)
* CI no longer tests Python 3.9 @meejah
* CI now tests 3.14 @meejah
* non-numeric nameplates rejected with error (@meejah)
* more-complete sequence and state diagrams (@meejah)
* update Munin plugin shebang (@warner)
* Munin plugins open db read-only (@warner)
## Release 0.6.0 (13-Feb-2026)
* CI no longer tests Python 3.8 (it is EOL)
* add Python 3.14
* fix link to transit-relay (@nirit100)
* fix stdout test error (@sblondon)
* remove depracated pkg_resources use (@sblondon)
* syntax modernization (@sblondon)
* use f-strings everywhere (@sblondon)
* replace returnValue() with return (@p12tic)
* make README match tested versions (@p12tic)
* no need to install mock (@bkmgit)
## Release 0.5.1 (9-Nov-2024)
* properly require "setuptools" for install (#47, jameshilliard)
## Release 0.5.0 (7-Nov-2024)
* correctly close a mailbox which still has a nameplate (#28)
* remove python2 support
* test on python 3.8, 3.9, 3.10, 3.11 and 3.12 series
* drop "six" (#35)
* upgrade "versioneer"
## Release 0.4.1 (11-Sep-2019)
* listen on IPv4+IPv6 properly (#16)
## Release 0.4.0 (10-Sep-2019)
* listen on IPv4+IPv6 socket by default (#16)
* deallocate AppNamespace objects when empty (#12)
* add client-version-uptake munin plugin
* drop support for py3.3 and py3.4
## Release 0.3.1 (23-Jun-2018)
Record 'None' for when client doesn't supply a version, to make the math
easier.
## Release 0.3.0 (23-Jun-2018)
Fix munin plugins, record client versions in usageDB.
## Release 0.2.0 (16-Jun-2018)
Improve install docs, clean up Munin plugins, add DB migration tool.
## Release 0.1.0 (19-Feb-2018)
Initial release: Forked from magic-wormhole-0.10.5 (14-Feb-2018)
================================================
FILE: README.md
================================================
# Magic Wormhole Mailbox Server
[](https://pypi.python.org/pypi/magic-wormhole-mailbox-server)
[](https://codecov.io/github/magic-wormhole/magic-wormhole-transit-relay?branch=master)
This repository holds the code for the main server that
[Magic-Wormhole](http://magic-wormhole.io) clients connect to. The server
performs store-and-forward delivery for small key-exchange and control
messages. Bulk data is sent over a direct TCP connection, or through a
[transit-relay](https://github.com/magic-wormhole/magic-wormhole-transit-relay).
Clients connect with WebSockets, for low-latency delivery in the happy case
where both clients are attached at the same time. Message are stored to
enable non-simultaneous clients to make forward progress. The server uses a
small SQLite database for persistence (and clients will reconnect
automatically, allowing the server to be rebooted without losing state). An
optional "usage DB" tracks historical activity for status monitoring and
operational maintenance.
## Installation
```
pip install magic-wormhole-mailbox-server
```
You either want to do this into a "user" environment (putting the ``twist``
and ``twistd`` executables in ``~/.local/bin/``) like this:
```
pip install --user magic-wormhole-mailbox-server
```
or put it into a virtualenv, to avoid modifying the system python's
libraries, like this:
```
virtualenv venv
source venv/bin/activate
pip install magic-wormhole-mailbox-server
```
You probably *don't* want to use ``sudo`` when you run ``pip``, since the
dependencies that get installed may conflict with other python programs on
your computer. ``pipsi`` is usually a good way to install into isolated
environments, but unfortunately it doesn't work for
magic-wormhole-mailbox-server, because we don't have a dedicated command to
start the server (``twist``, described below, comes from the ``twisted``
package, and pipsi doesn't expose executables from dependencies).
For the installation from source, ``clone`` this repo, ``cd`` into the folder,
create and activate a virtualenv, and run ``pip install .``.
## Running A Server
Note that the standard [Magic-Wormhole](http://magic-wormhole.io)
command-line tool is preconfigured to use a mailbox server hosted by the
project, so running your own server is only necessary for custom applications
that use magic-wormhole as a library.
The mailbox server is deployed as a twist/twistd plugin. Running a basic
server looks like this:
```
twist wormhole-mailbox --usage-db=usage.sqlite
```
Use ``twist wormhole-mailbox --help`` for more details.
If you use the default ``--port=tcp:4000``, on a machine named
``example.com``, then clients can reach your server with the following
option:
```
wormhole --relay-url=ws://example.com:4000/v1 send FILENAME
```
## Using Docker
Dockerfile content:
```dockerfile
FROM python:3.11
RUN pip install magic-wormhole-mailbox-server
CMD [ "twist", "wormhole-mailbox","--usage-db=usage.sqlite" ]
```
> Note: This will be running as root, you should adjust it to be in user space for production.
Build and run:
```shell
docker build -t magicwormhole Dockerfile
docker run -p 4000:4000 -d magicwormhole
```
Connect:
```shell
wormhole --relay-url=ws://localhost:4000/v1 send FILENAME
```
## License, Compatibility
This library is released under the MIT license, see LICENSE for details.
This library is compatible with python3 (3.10 and higher).
================================================
FILE: docs/Makefile
================================================
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = Magic-Wormhole-Mailbox-Server
SOURCEDIR = .
BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile diagrams
diagrams:
plantuml -Tpng happy-plant.seq
seqdiag --no-transparency --antialias -T png happy.seq
dot -Tpng states.dot > states.png
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
================================================
FILE: docs/conf.py
================================================
# Magic-Wormhole documentation build configuration file, created by
# sphinx-quickstart on Sun Nov 12 10:24:09 2017.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
from recommonmark.parser import CommonMarkParser
source_parsers = {
".md": CommonMarkParser,
}
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = []
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
source_suffix = ['.rst', '.md']
#source_suffix = '.md'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = 'Magic-Wormhole-Mailbox-Server'
copyright = '2018, Brian Warner'
author = 'Brian Warner'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
def _get_versions():
import os.path, sys, subprocess
here = os.path.dirname(os.path.abspath(__file__))
parent = os.path.dirname(here)
v = subprocess.check_output([sys.executable, "setup.py", "--version"],
cwd=parent)
v = v.decode("ascii")
short = ".".join(v.split(".")[:2])
long = v
return short, long
version, release = _get_versions()
# The short X.Y version.
#version = u'0.10'
# The full version, including alpha/beta/rc tags.
#release = u'0.10.3'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
#
# This is required for the alabaster theme
# refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars
html_sidebars = {
'**': [
'relations.html', # needs 'show_related': True theme option to display
'searchbox.html',
]
}
# -- Options for HTMLHelp output ------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'Magic-Wormhole-Mailbox-Serverdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'Magic-Wormhole-Mailbox-Server.tex', 'Magic-Wormhole-Mailbox-Server Documentation',
'Brian Warner', 'manual'),
]
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'magic-wormhole-mailbox-server', 'Magic-Wormhole-Mailbox-Server Documentation',
[author], 1)
]
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'Magic-Wormhole-Mailbox-Server',
'Magic-Wormhole-Mailbox-Server Documentation',
author, 'Magic-Wormhole-Mailbox-Server',
'One line description of project.',
'Miscellaneous'),
]
================================================
FILE: docs/happy-plant.seq
================================================
@startuml
skinparam defaultFontName "Source Sans Pro"
skinparam defaultFontSize 18
skinparam participantFontSize 22
skinparam arrowFontSize 18
skinparam noteFontSize 22
title Magic Folder Server
alice <- mailbox: welcome
alice -> mailbox: bind appid, side
alice -> mailbox: allocate
alice <- mailbox: allocated: nameplate_id
activate alice #aaddff
alice -> mailbox: claim nameplate_id
alice <- mailbox: claimed: mailbox_id
alice -> mailbox: open mailbox_id
alice -> mailbox: add phase=pake body
alice <- mailbox: message phase=pake side=a body
... code communicated out-of-band ...
bob <- mailbox: welcome
bob -> mailbox: bind appid, side
bob -> mailbox: claim nameplate_id
bob <- mailbox: claimed: mailbox_id
bob -> mailbox: open mailbox_id
bob <- mailbox: message phase=pake side=a body
bob -> mailbox: add phase=pake body
bob <- mailbox: mesage phase=pake side=b body
alice <- mailbox: mesage phase=pake side=b body
alice -> mailbox: release nameplate_id
alice <- mailbox: released
deactivate alice
note over mailbox #eeeeee: Key-Verification messages double as version exchange
alice -> mailbox: add phase=version side=a body
bob <- mailbox: message phase=version side=a body
alice <- mailbox: message phase=version side=a body
bob -> mailbox: add phase=version side=b body
bob <- mailbox: message phase=version side=b body
activate bob #lightgreen
alice <- mailbox: message phase=version side=b body
activate alice #lightgreen
alice -> mailbox: add phase=0 side=a body
alice <- mailbox: message phase=0 side=a body
bob <- mailbox: message phase=0 side=a body
' XXX does this help or hinter understanding?
alice -->> bob: (effectively msg 0 from alice->bob)
alice -> mailbox: add phase=1 side=a body
alice <- mailbox: message phase=1 side=a body
bob <- mailbox: message phase=1 side=a body
' XXX does this help or hinter understanding?
alice -->> bob: (effectively msg 1 from alice->bob)
bob -> mailbox: add phase=0 side=b body
bob <- mailbox: message phase=0 side=b body
alice <- mailbox: message phase=0 side=b body
' XXX does this help or hinter understanding?
alice <<-- bob: (effectively msg 0 from bob->alice)
deactivate alice
deactivate bob
' group #Pink Closing
group Closing
bob -> mailbox: close mailbox_id mood
bob <- mailbox: closed
alice -> mailbox: close mailbox_id mood
alice <- mailbox: closed
end
@enduml
================================================
FILE: docs/happy.seq
================================================
seqdiag {
alice <- mailbox [label = "welcome"]
alice -> mailbox [label = "bind appid, side"]
alice -> mailbox [label = "allocate", note = "side assigned"]
alice <- mailbox [label = "allocated: nameplate_id"]
alice -> mailbox [label = "claim nameplate_id"]
alice <- mailbox [label = "claimed: mailbox_id"]
alice -> mailbox [label = "open mailbox_id"]
alice -> mailbox [label = "add phase=pake body"]
alice <- mailbox [label = "message phase=pake side=a body"]
bob <- mailbox [label = "welcome"]
bob -> mailbox [label = "bind appid, side"]
// note: no allocate / allocated
bob -> mailbox [label = "claim nameplate_id"]
bob <- mailbox [label = "claimed: mailbox_id"]
bob -> mailbox [label = "open mailbox_id"]
bob <- mailbox [label = "message phase=pake side=a body"]
bob -> mailbox [label = "add phase=pake body"]
bob <- mailbox [label = "mesage phase=pake side=b body"]
alice <- mailbox [label = "mesage phase=pake side=b body"]
alice -> mailbox [label = "release nameplate_id"]
alice <- mailbox [label = "released"]
alice -> mailbox [label = "add phase=version side=a body"]
bob <- mailbox [label = "message phase=version side=a body"]
alice <- mailbox [label = "message phase=version side=a body"]
bob -> mailbox [label = "add phase=version side=b body"]
bob <- mailbox [label = "message phase=version side=b body"]
alice <- mailbox [label = "message phase=version side=b body"]
=== application messages ===
alice -> mailbox [label = "add phase=0 side=a body"]
alice <- mailbox [label = "message phase=0 side=a body"]
bob <- mailbox [label = "message phase=0 side=a body"]
alice -> mailbox [label = "add phase=1 side=a body"]
alice <- mailbox [label = "message phase=1 side=a body"]
bob <- mailbox [label = "message phase=1 side=a body"]
bob -> mailbox [label = "add phase=0 side=b body"]
bob <- mailbox [label = "message phase=0 side=b body"]
alice <- mailbox [label = "message phase=0 side=b body"]
=== closing ===
bob -> mailbox [label = "close mailbox_id mood"]
bob <- mailbox [label = "closed"]
alice -> mailbox [label = "close mailbox_id mood"]
alice <- mailbox [label = "closed"]
}
================================================
FILE: docs/index.rst
================================================
.. Magic-Wormhole-Mailbox-Server documentation master file, created by
sphinx-quickstart on Sun Nov 12 10:24:09 2017.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Magic-Wormhole-Mailbox-Server: backend server for magic-wormhole
================================================================
.. toctree::
:maxdepth: 2
:caption: Contents:
welcome
server-protocol
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
================================================
FILE: docs/server-protocol.md
================================================
# Rendezvous Server Protocol
## Concepts
The Rendezvous Server provides queued delivery of binary messages from one
client to a second, and vice versa. Each message contains a "phase" (a
string) and a body (bytestring). These messages are queued in a "Mailbox"
until the other side connects and retrieves them, but are delivered
immediately if both sides are connected to the server at the same time.
Mailboxes are identified by a large random string. "Nameplates", in contrast,
have short numeric identities: in a wormhole code like "4-purple-sausages",
the "4" is the nameplate.
Each client has a randomly-generated "side", a short hex string, used to
differentiate between echoes of a client's own message, and real messages
from the other client.
## Application IDs
The server isolates each application from the others. Each client provides an
"App Id" when it first connects (via the "BIND" message), and all subsequent
commands are scoped to this application. This means that nameplates
(described below) and mailboxes can be re-used between different apps. The
AppID is a unicode string. Both sides of the wormhole must use the same
AppID, of course, or they'll never see each other. The server keeps track of
which applications are in use for maintenance purposes.
Each application should use a unique AppID. Developers are encouraged to use
"DNSNAME/APPNAME" to obtain a unique one: e.g. the `bin/wormhole`
file-transfer tool uses `lothar.com/wormhole/text-or-file-xfer`.
## WebSocket Transport
At the lowest level, each client establishes (and maintains) a WebSocket
connection to the Rendezvous Server. If the connection is lost (which could
happen because the server was rebooted for maintenance, or because the
client's network connection migrated from one network to another, or because
the resident network gremlins decided to mess with you today), clients should
reconnect after waiting a random (and exponentially-growing) delay. The
Python implementation waits about 1 second after the first connection loss,
growing by 50% each time, capped at 1 minute.
Each message to the server is a dictionary, with at least a `type` key, and
other keys that depend upon the particular message type. Messages from server
to client follow the same format.
`misc/dump-timing.py` is a debug tool which renders timing data gathered from
the server and both clients, to identify protocol slowdowns and guide
optimization efforts. To support this, the client/server messages include
additional keys. Client->Server messages include a random `id` key, which is
copied into the `ack` that is immediately sent back to the client for all
commands (logged for the timing tool but otherwise ignored). Some
client->server messages (`list`, `allocate`, `claim`, `release`, `close`,
`ping`) provoke a direct response by the server: for these, `id` is copied
into the response. This helps the tool correlate the command and response.
All server->client messages have a `server_tx` timestamp (seconds since
epoch, as a float), which records when the message left the server. Direct
responses include a `server_rx` timestamp, to record when the client's
command was received. The tool combines these with local timestamps (recorded
by the client and not shared with the server) to build a full picture of
network delays and round-trip times.
All messages are serialized as JSON, encoded to UTF-8, and the resulting
bytes sent as a single "binary-mode" WebSocket payload.
Servers can signal `error` for any message type it does not recognize.
Clients and Servers must ignore unrecognized keys in otherwise-recognized
messages. Clients must ignore unrecognized message types from the Server.
## Connection-Specific (Client-to-Server) Messages
The first thing each client sends to the server, immediately after the
WebSocket connection is established, is a `bind` message. This specifies the
AppID and side (in keys `appid` and `side`, respectively) that all subsequent
messages will be scoped to. While technically each message could be
independent (with its own `appid` and `side`), I thought it would be less
confusing to use exactly one WebSocket per logical wormhole connection.
The first thing the server sends to each client is the `welcome` message.
This is intended to deliver important status information to the client that
might influence its operation. The Python client currently reacts to the
following keys (and ignores all others):
* `current_cli_version`: prompts the user to upgrade if the server's
advertised version is greater than the client's version (as derived from
the git tag)
* `motd`: prints this message, if present; intended to inform users about
performance problems, scheduled downtime, or to beg for donations to keep
the server running
* `error`: causes the client to print the message and then terminate. If a
future version of the protocol requires a rate-limiting CAPTCHA ticket or
other authorization record, the server can send `error` (explaining the
requirement) if it does not see this ticket arrive before the `bind`.
A `ping` will provoke a `pong`: these are only used by unit tests for
synchronization purposes (to detect when a batch of messages have been fully
processed by the server). NAT-binding refresh messages are handled by the
WebSocket layer (by asking Autobahn to send a keepalive messages every 60
seconds), and do not use `ping`.
If any client->server command is invalid (e.g. it lacks a necessary key, or
was sent in the wrong order), an `error` response will be sent, This response
will include the error string in the `error` key, and a full copy of the
original message dictionary in `orig`.
## Nameplates
Wormhole codes look like `4-purple-sausages`, consisting of a number followed
by some random words. This number is called a "Nameplate".
On the Rendezvous Server, the Nameplate contains a pointer to a Mailbox.
Clients can "claim" a nameplate, and then later "release" it. Each claim is
for a specific side (so one client claiming the same nameplate multiple times
only counts as one claim). Nameplates are deleted once the last client has
released it, or after some period of inactivity.
Clients can either make up nameplates themselves, or (more commonly) ask the
server to allocate one for them. Allocating a nameplate automatically claims
it (to avoid a race condition), but for simplicity, clients send a claim for
all nameplates, even ones which they've allocated themselves.
Nameplates (on the server) must live until the second client has learned
about the associated mailbox, after which point they can be reused by other
clients. So if two clients connect quickly, but then maintain a long-lived
wormhole connection, the do not need to consume the limited space of short
nameplates for that whole time.
The `allocate` command allocates a nameplate (the server returns one that is
as short as possible), and the `allocated` response provides the answer.
Clients can also send a `list` command to get back a `nameplates` response
with all allocated nameplates for the bound AppID: this helps the code-input
tab-completion feature know which prefixes to offer. The `nameplates`
response returns a list of dictionaries, one per claimed nameplate, with at
least an `id` key in each one (with the nameplate string). Future versions
may record additional attributes in the nameplate records, specifically a
wordlist identifier and a code length (again to help with code-completion on
the receiver).
## Mailboxes
The server provides a single "Mailbox" to each pair of connecting Wormhole
clients. This holds an unordered set of messages, delivered immediately to
connected clients, and queued for delivery to clients which connect later.
Messages from both clients are merged together: clients use the included
`side` identifier to distinguish echoes of their own messages from those
coming from the other client.
Each mailbox is "opened" by some number of clients at a time, until all
clients have closed it. Mailboxes are kept alive by either an open client, or
a Nameplate which points to the mailbox (so when a Nameplate is deleted from
inactivity, the corresponding Mailbox will be too).
The `open` command both marks the mailbox as being opened by the bound side,
and also adds the WebSocket as subscribed to that mailbox, so new messages
are delivered immediately to the connected client. There is no explicit ack
to the `open` command, but since all clients add a message to the mailbox as
soon as they connect, there will always be a `message` reponse shortly after
the `open` goes through. The `close` command provokes a `closed` response.
The `close` command accepts an optional "mood" string: this allows clients to
tell the server (in general terms) about their experiences with the wormhole
interaction. The server records the mood in its "usage" record, so the server
operator can get a sense of how many connections are succeeding and failing.
The moods currently recognized by the Rendezvous Server are:
* `happy` (default): the PAKE key-establishment worked, and the client saw at
least one valid encrypted message from its peer
* `lonely`: the client gave up without hearing anything from its peer
* `scary`: the client saw an invalid encrypted message from its peer,
indicating that either the wormhole code was typed in wrong, or an attacker
tried (and failed) to guess the code
* `errory`: the client encountered some other error: protocol problem or
internal error
The server will also record `pruney` if it deleted the mailbox due to
inactivity, or `crowded` if more than two sides tried to access the mailbox.
When clients use the `add` command to add a client-to-client message, they
will put the body (a bytestring) into the command as a hex-encoded string in
the `body` key. They will also put the message's "phase", as a string, into
the `phase` key. See client-protocol.md for details about how different
phases are used.
When a client sends `open`, it will get back a `message` response for every
message in the mailbox. It will also get a real-time `message` for every
`add` performed by clients later. These `message` responses include "side"
and "phase" from the sending client, and "body" (as a hex string, encoding
the binary message body). The decoded "body" will either by a random-looking
cryptographic value (for the PAKE message), or a random-looking encrypted
blob (for the VERSION message, as well as all application-provided payloads).
The `message` response will also include `id`, copied from the `id` of the
`add` message (and used only by the timing-diagram tool).
The Rendezvous Server does not de-duplicate messages, nor does it retain
ordering: clients must do both if they need to.
## All Message Types
This lists all message types, along with the type-specific keys for each (if
any), and which ones provoke direct responses:
* S->C welcome {welcome:}
* (C->S) bind {appid:, side:}
* (C->S) list {} -> nameplates
* S->C nameplates {nameplates: [{id: str},..]}
* (C->S) allocate {} -> allocated
* S->C allocated {nameplate:}
* (C->S) claim {nameplate:} -> claimed
* S->C claimed {mailbox:}
* (C->S) release {nameplate:?} -> released
* S->C released
* (C->S) open {mailbox:}
* (C->S) add {phase: str, body: hex} -> message (to all connected clients)
* S->C message {side:, phase:, body:, id:}
* (C->S) close {mailbox:?, mood:?} -> closed
* S->C closed
* S->C ack
* (C->S) ping {ping: int} -> ping
* S->C pong {pong: int}
* S->C error {error: str, orig:}
## Persistence
The server stores all messages in a database, so it should not lose any
information when it is restarted. The server will not send a direct
response until any side-effects (such as the message being added to the
mailbox) have been safely committed to the database.
The client library knows how to resume the protocol after a reconnection
event, assuming the client process itself continues to run.
Clients which terminate entirely between messages (e.g. a secure chat
application, which requires multiple wormhole messages to exchange
address-book entries, and which must function even if the two apps are never
both running at the same time) can use "Journal Mode" to ensure forward
progress is made: see "journal.md" for details.
================================================
FILE: docs/states.dot
================================================
/*digraph {
title [label="Mailbox\lServer Machine" style="dotted"]
start -> opened [label="open(side)"];
opened -> opened [label="open(side)"];
opened -> opened [label="add_message(sided_message)"];
opened -> closing [label="close(side, mood)"];
closing -> closing [label="close(side, mood)"];
}
*/
// note: all messages have an "id" and a "type"
// and the server sends back an "ack" for every one
// but that ack etc isn't covered in these diagrams
digraph {
node [fontname = "Source Sans Pro" fontsize = 22];
edge [fontname = "Source Code Pro" fontsize = 18 fontcolor=blue];
graph [fontname = "Source Sans Pro" fontsize = 22];
title [label="Mailbox Server" style="dotted" fontsize=32];
ranksep = 1;
start [label="START\n(perm=none)"];
start_permissions [label="START\n(perm=hashcash)"];
start_reconn [label="RECONNECT\n(mailbox, perm=none)"];
start_reconn_perm [label="RECONNECT\n(mailbox, perm=hashcash)"];
done [label="DONE\nmood=" shape=box style=filled];
{rank=same; start start_permissions start_reconn start_reconn_perm}
start [shape=box, style=bold];
start -> bound [label="bind(appid, side)"];
# blue, to match seqdiag section on "nameplate allocated"
bound [fillcolor=cadetblue1, style=filled];
have_nameplate [fillcolor=cadetblue1, style=filled];
claimed [fillcolor=cadetblue1, style=filled];
start_permissions [shape=box, style=bold];
start_permissions -> granted [label="submit_permissions()" fontcolor=red];
granted -> bound [label="bind(appid, side)"];
start_reconn [shape=box, style=bold];
start_reconn -> open [label="open(mailbox_id)\l-> messages: send(side, phase, body)\l"];
start_reconn_perm [shape=box, style=bold];
start_reconn_perm -> reconn_granted [label="submit_permissions()" fontcolor=red];
open [fillcolor=green style=filled];
reconn_granted -> open [label="open(mailbox_id)\l-> messages: send(side, phase, body)\l"];
bound -> have_nameplate [label="allocate()\l-> nameplate_id\l"]
# allocate() really does do a claim() .. but you have to call it explicitly too
have_nameplate -> claimed [label="claim(nameplate, side)\l-> mailbox_id\l" fontcolor=darkgreen]
have_nameplate -> done [label="release(nameplate)" fontcolor=red]
# ths is on the "join" side; they are told the nameplate number
bound -> claimed [label="claim(nameplate, side)\l-> mailbox_id\l" fontcolor=darkgreen]
claimed -> unclaimed [label="release(nameplate)" fontcolor=red]
# note: allowing two different paths to 'unclaimed' is I think
# _allowed_ currently by the server, but better to define it with
# juts one way probably.
unclaimed -> open [label="open(mailbox_id)\l-> messages: send(side, phase, body)\l"]
#claimed -> open [label="open(mailbox_id)\l-> send(all_messages)\l"]
#open -> open [label="release(nameplate)"]
open -> open [label="add_message(msg)\l-> send(side, phase, body)\l"]
open -> done [label="close(mailbox_id)" fontcolor=red]
# XXX will get all message already in the box, how to represent?
}
================================================
FILE: docs/welcome.md
================================================
# Magic Wormhole Mailbox Server
[](https://travis-ci.org/warner/magic-wormhole-mailbox-server)
[](https://ci.appveyor.com/project/warner/magic-wormhole-mailbox-server)
[](https://codecov.io/github/warner/magic-wormhole-mailbox-server?branch=master)
This repository holds the code for the main server that
[Magic-Wormhole](http://magic-wormhole.io) clients connect to. The server
performs store-and-forward delivery for small key-exchange and control
messages. Bulk data is sent over a direct TCP connection, or through a
[transit-relay](https://github.com/warner/magic-wormhole-transit-relay).
Clients connect with WebSockets, for low-latency delivery in the happy case
where both clients are attached at the same time. Message are stored in to
enable non-simultaneous clients to make forward progress. The server uses a
small SQLite database for persistence (and clients will reconnect
automatically, allowing the server to be rebooted without losing state). An
optional "usage DB" tracks historical activity for status monitoring and
operational maintenance.
## Running A Server
Note that the standard [Magic-Wormhole](http://magic-wormhole.io)
command-line tool is preconfigured to use a mailbox server hosted by the
project, so running your own server is only necessary for custom applications
that use magic-wormhole as a library.
The mailbox server is deployed as a twist/twistd plugin. Running a basic
server looks like this:
```
twist wormhole-mailbox --usage-db=usage.sqlite
```
Use ``twist wormhole-mailbox --help`` for more details.
If you use the default ``--port=tcp:4000``, on a machine named
``example.com``, then clients can reach your server with the following
option:
```
wormhole --relay-url=ws://example.com:4000/v1 send FILENAME
```
## License, Compatibility
This library is released under the MIT license, see LICENSE for details.
This library is compatible with python2.7, 3.4, 3.5, and 3.6 .
================================================
FILE: misc/migrate_channel_db.py
================================================
"""Migrate the channel data from the old bundled Mailbox Server database.
The magic-wormhole package used to include both servers (Rendezvous and
Transit). "wormhole server" started both of these, and used the
"relay.sqlite" database to store both immediate server state and long-term
usage data.
These were split out to their own packages: version 0.11 omitted the Transit
Relay, and 0.12 removed the Mailbox Server in favor of the new
"magic-wormhole-mailbox-server" distribution.
This script reads the short-term channel data from the pre-0.12
wormhole-server relay.sqlite, and copies it into a new "relay.sqlite"
database in the current directory.
It will refuse to touch an existing "relay.sqlite" file.
The resuting "relay.sqlite" should be passed into --channel-db=, e.g. "twist
wormhole-mailbox --channel-db=.../PATH/TO/relay.sqlite". However in most
cases you can just store it in the default location of "./relay.sqlite" and
omit the --channel-db= argument.
Note that an idle server will have no channel data, so you could instead just
wait for the server to be empty (sqlite3 relay.sqlite message |grep INSERT).
"""
import sys
from wormhole_mailbox_server.database import (open_existing_db,
create_channel_db)
source_fn = sys.argv[1]
source_db = open_existing_db(source_fn)
target_db = create_channel_db("relay.sqlite")
num_rows = 0
for row in source_db.execute("SELECT * FROM `mailboxes`").fetchall():
target_db.execute("INSERT INTO `mailboxes`"
" (`app_id`, `id`, `updated`, `for_nameplate`)"
" VALUES(?,?,?,?)",
(row["app_id"], row["id"], row["updated"],
row["for_nameplate"]))
num_rows += 1
for row in source_db.execute("SELECT * FROM `mailbox_sides`").fetchall():
target_db.execute("INSERT INTO `mailbox_sides`"
" (`mailbox_id`, `opened`, `side`, `added`, `mood`)"
" VALUES(?,?,?,?,?)",
(row["mailbox_id"], row["opened"], row["side"],
row["added"], row["mood"]))
num_rows += 1
for row in source_db.execute("SELECT * FROM `nameplates`").fetchall():
target_db.execute("INSERT INTO `nameplates`"
" (`id`, `app_id`, `name`, `mailbox_id`, `request_id`)"
" VALUES(?,?,?,?,?)",
(row["id"], row["app_id"], row["name"],
row["mailbox_id"], row["request_id"]))
num_rows += 1
for row in source_db.execute("SELECT * FROM `nameplate_sides`").fetchall():
target_db.execute("INSERT INTO `nameplate_sides`"
" (`nameplates_id`, `claimed`, `side`, `added`)"
" VALUES(?,?,?,?)",
(row["nameplates_id"], row["claimed"], row["side"],
row["added"]))
num_rows += 1
for row in source_db.execute("SELECT * FROM `messages`").fetchall():
target_db.execute("INSERT INTO `messages`"
" (`app_id`, `mailbox_id`, `side`, `phase`, `body`, "
" `server_rx`, `msg_id`)"
" VALUES(?,?,?,?,?,?,?)",
(row["app_id"], row["mailbox_id"], row["side"],
row["phase"], row["body"],
row["server_rx"], row["msg_id"]))
num_rows += 1
target_db.commit()
print("channel database migrated (%d rows) into 'relay.sqlite'" % num_rows)
sys.exit(0)
================================================
FILE: misc/migrate_usage_db.py
================================================
"""Migrate the usage data from the old bundled Mailbox Server database.
The magic-wormhole package used to include both servers (Rendezvous and
Transit). "wormhole server" started both of these, and used the
"relay.sqlite" database to store both immediate server state and long-term
usage data.
These were split out to their own packages: version 0.11 omitted the Transit
Relay, and 0.12 removed the Mailbox Server in favor of the new
"magic-wormhole-mailbox-server" distribution.
This script reads the long-term usage data from the pre-0.12 wormhole-server
relay.sqlite, and copies it into a new "usage.sqlite" database in the current
directory.
It will refuse to touch an existing "usage.sqlite" file.
The resuting "usage.sqlite" should be passed into --usage-db=, e.g. "twist
wormhole-mailbox --usage-db=.../PATH/TO/usage.sqlite".
"""
import sys
from wormhole_mailbox_server.database import open_existing_db, create_usage_db
source_fn = sys.argv[1]
source_db = open_existing_db(source_fn)
target_db = create_usage_db("usage.sqlite")
num_nameplate_rows = 0
for row in source_db.execute("SELECT * FROM `nameplate_usage`"
" ORDER BY `started`").fetchall():
target_db.execute("INSERT INTO `nameplates`"
" (`app_id`, `started`, `waiting_time`,"
" `total_time`, `result`)"
" VALUES(?,?,?,?,?)",
(row["app_id"], row["started"], row["waiting_time"],
row["total_time"], row["result"]))
num_nameplate_rows += 1
num_mailbox_rows = 0
for row in source_db.execute("SELECT * FROM `mailbox_usage`"
" ORDER BY `started`").fetchall():
target_db.execute("INSERT INTO `mailboxes`"
" (`app_id`, `for_nameplate`,"
" `started`, `total_time`, `waiting_time`,"
" `result`)"
" VALUES(?,?,?,?,?,?)",
(row["app_id"], row["for_nameplate"],
row["started"], row["total_time"], row["waiting_time"],
row["result"]))
num_mailbox_rows += 1
target_db.execute("INSERT INTO `current`"
" (`rebooted`, `updated`, `blur_time`,"
" `connections_websocket`)"
" VALUES(?,?,?,?)",
(0, 0, 0, 0))
target_db.commit()
print("usage database migrated (%d+%d rows) into 'usage.sqlite'" % (num_nameplate_rows, num_mailbox_rows))
sys.exit(0)
================================================
FILE: misc/munin/wormhole_active
================================================
#! /usr/bin/env python3
"""
Use the following in /etc/munin/plugin-conf.d/wormhole :
[wormhole_*]
env.channeldb /path/to/your/wormhole/server/channel.sqlite
env.usagedb /path/to/your/wormhole/server/usage.sqlite
"""
from __future__ import print_function
import os, sys, time, sqlite3
CONFIG = """\
graph_title Magic-Wormhole Active Channels
graph_vlabel Channels
graph_category wormhole
nameplates.label Nameplates
nameplates.draw LINE2
nameplates.type GAUGE
mailboxes.label Mailboxes
mailboxes.draw LINE2
mailboxes.type GAUGE
messages.label Messages
messages.draw LINE1
messages.type GAUGE
"""
if len(sys.argv) > 1 and sys.argv[1] == "config":
print(CONFIG.rstrip())
sys.exit(0)
usagedbfile = os.environ["usagedb"]
assert os.path.exists(usagedbfile)
usage_db = sqlite3.connect("file:%s?mode=ro" % usagedbfile, uri=True)
channeldbfile = os.environ["channeldb"]
assert os.path.exists(channeldbfile)
channel_db = sqlite3.connect("file:%s?mode=ro" % channeldbfile, uri=True)
MINUTE = 60.0
updated,rebooted = usage_db.execute("SELECT `updated`,`rebooted` FROM `current`").fetchone()
if time.time() > updated + 6*MINUTE:
sys.exit(1) # expired
nameplates = channel_db.execute("SELECT COUNT() FROM `nameplates`").fetchone()[0]
mailboxes = channel_db.execute("SELECT COUNT() FROM `mailboxes`").fetchone()[0]
messages = channel_db.execute("SELECT COUNT() FROM `messages`").fetchone()[0]
print("nameplates.value", nameplates)
print("mailboxes.value", mailboxes)
print("messages.value", messages)
================================================
FILE: misc/munin/wormhole_errors
================================================
#! /usr/bin/env python3
"""
Use the following in /etc/munin/plugin-conf.d/wormhole :
[wormhole_*]
env.usagedb /path/to/your/wormhole/server/usage.sqlite
"""
from __future__ import print_function
import os, sys, time, sqlite3
CONFIG = """\
graph_title Magic-Wormhole Server Errors
graph_vlabel Events Since Reboot
graph_category wormhole
nameplates.label Nameplate Errors (total)
nameplates.draw LINE1
nameplates.type GAUGE
mailboxes.label Mailboxes (total)
mailboxes.draw LINE1
mailboxes.type GAUGE
mailboxes_scary.label Mailboxes (scary)
mailboxes_scary.draw LINE1
mailboxes_scary.type GAUGE
"""
if len(sys.argv) > 1 and sys.argv[1] == "config":
print(CONFIG.rstrip())
sys.exit(0)
usagedbfile = os.environ["usagedb"]
assert os.path.exists(usagedbfile)
usage_db = sqlite3.connect("file:%s?mode=ro" % usagedbfile, uri=True)
MINUTE = 60.0
updated,rebooted = usage_db.execute("SELECT `updated`,`rebooted` FROM `current`").fetchone()
if time.time() > updated + 6*MINUTE:
sys.exit(1) # expired
r1 = usage_db.execute("SELECT COUNT() FROM `nameplates` WHERE `started` >= ?",
(rebooted,)).fetchone()[0]
r2 = usage_db.execute("SELECT COUNT() FROM `nameplates`"
" WHERE `started` >= ?"
" AND `result` = 'happy'",
(rebooted,)).fetchone()[0]
print("nameplates.value", (r1 - r2))
r1 = usage_db.execute("SELECT COUNT() FROM `mailboxes` WHERE `started` >= ?",
(rebooted,)).fetchone()[0]
r2 = usage_db.execute("SELECT COUNT() FROM `mailboxes` WHERE `started` >= ?"
" AND `result` = 'happy'",
(rebooted,)).fetchone()[0]
print("mailboxes.value", (r1 - r2))
r = usage_db.execute("SELECT COUNT() FROM `mailboxes` WHERE `started` >= ?"
" AND `result` = 'scary'",
(rebooted,)).fetchone()[0]
print("mailboxes_scary.value", r)
================================================
FILE: misc/munin/wormhole_event_rate
================================================
#! /usr/bin/env python3
"""
Use the following in /etc/munin/plugin-conf.d/wormhole :
[wormhole_*]
env.usagedb /path/to/your/wormhole/server/usage.sqlite
"""
from __future__ import print_function
import os, sys, time, sqlite3
from collections import defaultdict
CONFIG = """\
graph_title Magic-Wormhole Server Events
graph_vlabel Events per Hour
graph_category wormhole
happy.label Happy
happy.draw LINE
happy.type DERIVE
happy.min 0
happy.max 60
happy.cdef happy,3600,*
incomplete.label Incomplete
incomplete.draw LINE
incomplete.type DERIVE
incomplete.min 0
incomplete.max 60
incomplete.cdef incomplete,3600,*
scary.label Scary
scary.draw LINE
scary.type DERIVE
scary.min 0
scary.max 60
scary.cdef scary,3600,*
"""
if len(sys.argv) > 1 and sys.argv[1] == "config":
print(CONFIG.rstrip())
sys.exit(0)
usagedbfile = os.environ["usagedb"]
assert os.path.exists(usagedbfile)
usage_db = sqlite3.connect("file:%s?mode=ro" % usagedbfile, uri=True)
MINUTE = 60.0
updated,rebooted = usage_db.execute("SELECT `updated`,`rebooted` FROM `current`").fetchone()
if time.time() > updated + 6*MINUTE:
sys.exit(1) # expired
atm = defaultdict(int)
for mood in ["happy", "scary", "lonely", "errory", "pruney", "crowded"]:
atm[mood] = usage_db.execute("SELECT COUNT() FROM `mailboxes`"
" WHERE `result` = ?", (mood,)).fetchone()[0]
print("happy.value", atm["happy"])
print("incomplete.value", (atm["pruney"] + atm["lonely"]))
print("scary.value", atm["scary"])
================================================
FILE: misc/munin/wormhole_events
================================================
#! /usr/bin/env python3
"""
Use the following in /etc/munin/plugin-conf.d/wormhole :
[wormhole_*]
env.usagedb /path/to/your/wormhole/server/usage.sqlite
"""
from __future__ import print_function
import os, sys, time, sqlite3
CONFIG = """\
graph_title Magic-Wormhole Mailbox Events (since reboot)
graph_vlabel Events Since Reboot
graph_category wormhole
happy.label Happy
happy.draw LINE2
happy.type GAUGE
total.label Total
total.draw LINE1
total.type GAUGE
scary.label Scary
scary.draw LINE2
scary.type GAUGE
pruney.label Pruney
pruney.draw LINE1
pruney.type GAUGE
lonely.label Lonely
lonely.draw LINE2
lonely.type GAUGE
errory.label Errory
errory.draw LINE1
errory.type GAUGE
"""
if len(sys.argv) > 1 and sys.argv[1] == "config":
print(CONFIG.rstrip())
sys.exit(0)
usagedbfile = os.environ["usagedb"]
assert os.path.exists(usagedbfile)
usage_db = sqlite3.connect("file:%s?mode=ro" % usagedbfile, uri=True)
MINUTE = 60.0
updated,rebooted,blur = usage_db.execute(
"SELECT `updated`,`rebooted`,`blur_time` FROM `current`").fetchone()
if time.time() > updated + 6*MINUTE:
sys.exit(1) # expired
if blur is not None:
rebooted = blur * (rebooted // blur)
# After a reboot, the operator will get to see events that happen during
# the first blur window (without this adjustment, those events would be
# hidden since they'd appear to start before the reboot). The downside is
# that the counter won't drop down to zero at a reboot (if there are recent
# events).
#r = usage_db.execute("SELECT COUNT(`mood`) FROM `mailboxes` WHERE `started` > ?",
# (rebooted,)).fetchone()
for mood in ["happy", "scary", "lonely", "errory", "pruney", "crowded"]:
r = usage_db.execute("SELECT COUNT() FROM `mailboxes` WHERE `started` >= ?"
" AND `result` = ?",
(rebooted, mood)).fetchone()[0]
print("%s.value" % mood, r)
r = usage_db.execute("SELECT COUNT() FROM `mailboxes` WHERE `started` >= ?",
(rebooted,)).fetchone()[0]
print("total.value", r)
================================================
FILE: misc/munin/wormhole_events_alltime
================================================
#! /usr/bin/env python3
"""
Use the following in /etc/munin/plugin-conf.d/wormhole :
[wormhole_*]
env.usagedb /path/to/your/wormhole/server/usage.sqlite
"""
from __future__ import print_function
import os, sys, time, sqlite3
CONFIG = """\
graph_title Magic-Wormhole Mailbox Events (all time)
graph_vlabel Events Since DB Creation
graph_category wormhole
happy.label Happy
happy.draw LINE2
happy.type GAUGE
total.label Total
total.draw LINE1
total.type GAUGE
scary.label Scary
scary.draw LINE2
scary.type GAUGE
pruney.label Pruney
pruney.draw LINE1
pruney.type GAUGE
lonely.label Lonely
lonely.draw LINE2
lonely.type GAUGE
errory.label Errory
errory.draw LINE1
errory.type GAUGE
"""
if len(sys.argv) > 1 and sys.argv[1] == "config":
print(CONFIG.rstrip())
sys.exit(0)
usagedbfile = os.environ["usagedb"]
assert os.path.exists(usagedbfile)
usage_db = sqlite3.connect("file:%s?mode=ro" % usagedbfile, uri=True)
MINUTE = 60.0
updated,rebooted = usage_db.execute("SELECT `updated`,`rebooted` FROM `current`").fetchone()
if time.time() > updated + 6*MINUTE:
sys.exit(1) # expired
for mood in ["happy", "scary", "lonely", "errory", "pruney", "crowded"]:
r = usage_db.execute("SELECT COUNT() FROM `mailboxes` WHERE `result` = ?",
(mood,)).fetchone()[0]
print("%s.value" % mood, r)
r = usage_db.execute("SELECT COUNT() FROM `mailboxes`").fetchone()[0]
print("total.value", r)
================================================
FILE: misc/munin/wormhole_version_uptake
================================================
#! /usr/bin/env python3
"""
Use the following in /etc/munin/plugin-conf.d/wormhole :
[wormhole_*]
env.usagedb /path/to/your/wormhole/server/usage.sqlite
env.python_client_versions = 0.11.0
The python_client_versions list will be used to choose what to graph: any
python client which reports an application version not on the list will be
listed as 'other', and all non-python clients will be listed as 'non-python',
and clients which don't report a version at all will be listed as 'unknown'.
This list should grow over time just before new versions are released, so the
graph will remain sorted and stable.
"""
from __future__ import print_function
import os, sys, time, sqlite3, collections
CONFIG = """\
graph_title Magic-Wormhole Version Uptake
graph_vlabel Clients
graph_category wormhole
"""
versions = ["unknown", "non-python", "other"]
if "python_client_versions" in os.environ:
versions.extend(os.environ["python_client_versions"].split(","))
names = dict([(v, ("v_" + v).replace(".", "_").replace("-", "_"))
for v in versions])
if len(sys.argv) > 1 and sys.argv[1] == "config":
print(CONFIG.rstrip())
first = True
for v in versions:
name = names[v]
print("%s.label %s" % (name, v))
if first:
print("%s.draw AREA" % name)
first = False
else:
print("%s.draw STACK" % name)
print("%s.type GAUGE" % name)
sys.exit(0)
usagedbfile = os.environ["usagedb"]
assert os.path.exists(usagedbfile)
usage_db = sqlite3.connect("file:%s?mode=ro" % usagedbfile, uri=True)
now = time.time()
MINUTE = 60.0
updated,rebooted = usage_db.execute("SELECT `updated`,`rebooted` FROM `current`").fetchone()
if now > updated + 6*MINUTE:
sys.exit(1) # expired
def dict_factory(cursor, row):
d = {}
for idx, col in enumerate(cursor.description):
d[col[0]] = row[idx]
return d
usage_db.row_factory = dict_factory
seen_sides = set()
counts = collections.defaultdict(int)
for row in usage_db.execute("SELECT * FROM `client_versions`"
" WHERE (`connect_time` > ? AND `connect_time` < ?)",
(now - 60*MINUTE, now)).fetchall():
if row["side"] in seen_sides:
continue
seen_sides.add(row["side"])
if row["implementation"] is None and row["version"] is None:
version = "unknown"
elif row["implementation"] != "python":
version = "non-python"
elif row["version"] in versions:
version = row["version"]
else:
version = "other"
counts[version] += 1
for version in versions:
print("%s.value" % names[version], counts[version])
================================================
FILE: misc/windows-build.cmd
================================================
@echo off
:: To build extensions for 64 bit Python 3, we need to configure environment
:: variables to use the MSVC 2010 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 4
::
:: More details at:
:: https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
IF "%DISTUTILS_USE_SDK%"=="1" (
ECHO Configuring environment to build with MSVC on a 64bit architecture
ECHO Using Windows SDK 7.1
"C:\Program Files\Microsoft SDKs\Windows\v7.1\Setup\WindowsSdkVer.exe" -q -version:v7.1
CALL "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /x64 /release
SET MSSdk=1
REM Need the following to allow tox to see the SDK compiler
SET TOX_TESTENV_PASSENV=DISTUTILS_USE_SDK MSSdk INCLUDE LIB
) ELSE (
ECHO Using default MSVC build environment
)
CALL %*
================================================
FILE: newest-version.py
================================================
#
# print out the most-recent version
#
from dulwich.repo import Repo
from dulwich.porcelain import tag_list
def existing_tags(git):
versions = [
tuple(map(int, v.decode("utf8").split(".")))
for v in tag_list(git)
]
return versions
def main():
git = Repo(".")
print("{}.{}.{}".format(*sorted(existing_tags(git))[-1]))
if __name__ == "__main__":
main()
================================================
FILE: setup.cfg
================================================
[versioneer]
VCS = git
versionfile_source = src/wormhole_mailbox_server/_version.py
versionfile_build = wormhole_mailbox_server/_version.py
tag_prefix =
parentdir_prefix = magic-wormhole-mailbox-server
================================================
FILE: setup.py
================================================
from setuptools import setup
import versioneer
commands = versioneer.get_cmdclass()
trove_classifiers = [
"Development Status :: 4 - Beta",
"Environment :: Console",
"License :: OSI Approved :: MIT License",
"Topic :: Security :: Cryptography",
"Topic :: System :: Networking",
"Topic :: System :: Systems Administration",
"Topic :: Utilities",
]
setup(name="magic-wormhole-mailbox-server",
version=versioneer.get_version(),
description="Securely transfer data between computers",
long_description=open('README.md').read(),
long_description_content_type='text/markdown',
author="Brian Warner",
author_email="warner-magic-wormhole@lothar.com",
license="MIT",
url="https://github.com/warner/magic-wormhole-mailbox-server",
classifiers=trove_classifiers,
package_dir={"": "src"},
packages=["wormhole_mailbox_server",
"wormhole_mailbox_server.test",
"twisted.plugins",
],
package_data={"wormhole_mailbox_server": ["db-schemas/*.sql"]},
install_requires=[
"attrs >= 16.3.0", # 16.3.0 adds __attrs_post_init__
"twisted[tls] >= 17.5.0",
"autobahn[twisted] >= 0.14.1",
"setuptools", # pkg_resources
],
extras_require={
':sys_platform=="win32"': ["pywin32"],
"dev": ["treq", "tox", "pyflakes"],
"release": ["dulwich", "docutils", "wheel"],
},
test_suite="wormhole_mailbox_server.test",
cmdclass=commands,
)
================================================
FILE: signatures/magic-wormhole-mailbox-server-0.5.0.tar.gz.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmcsEawRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaafF2ggAsgMKP6ZyJ1sqJA58trJaufuV7ypqDJyV
UvPcIMHjF55YIJ2CRXt3fO6QFxiG/WHTWswENKxvFEp2F5ZCe13XLZwugX62/6Hc
T1jCIwvjU93yiEdqPvtMcAX5FWJUMKdOmlCqm/sfP5gF7D34O3vsM6wxbF8YlNFo
No0zZvMDxAlPmNER7iTujnckw5jyHqHSFn5AhWqigJTQlB3Mac7eqXuMIuCCOdy+
8PBpv0+jpdvzuq9hTFNvErKvg/Sy37nC1PJkteIXbneQiJjbVcvK4qniROzDbrnp
zbI+WEtCsm7o3ieLxt5P11fPjO+4/Tf9LwyjmkGnn265fUwrHFMx7g==
=5dEl
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic-wormhole-mailbox-server-0.5.1.tar.gz.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmcwGhcRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaaeh4wgAulK5GcWLnePJwUrmKp+2J57zAUOLOONY
218k8Tw6a7DfnbNCmx0+HnSu1zrpty8aGVuNynR3fMeW9pl140ZFYQ+aU96dqaR5
VHt4zbX4o4ZtBb/qxOsKpGljpv3a+47RSrdcpF6zPCVnqz219OykIKgSyURU2DMi
3tqGWGyoES97Rau6P1B9TrNCRdoC9+ajrk74gggkcnXnIPqyLl0KH2CvP+DNzsBA
YEECui2Rqu2WXSYIUY0HuWYP+fCuurqGLhlY3JYVDzUwrO2bK7brOW0lG4B/gZYy
mxBFUh/OVOx3UdBpMJrYFOrH1JVZIFBmu3sF1EUUffM1VndPErIn7w==
=kkQZ
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic_wormhole_mailbox_server-0.5.0-py3-none-any.whl.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmcsEasRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaacL7QgAnFo9ilYlzWRlaXBPu/nx86lWzMP8zOKM
LVD94B6wj3vXqosKpe7I3qmGWZrfo0vjHojzzh7GlUMIyapAB4dQ9jJknKOv7cA1
inrCLzObEVJ2JbnZexR7GIwMNhqIWn/PLd1YNygjn9u/sdLTvheGwmZ4vDBIfuTe
w17QRt5Tne2RjBgpNuJBmCz84AQ0TuwW+9ABU3DO6pcFdBQrNYvgmJkUcj8tYfdr
q+FiN5UIEZT0eBPftawY6LA8J7IYJOZk2035KHfxbejA1z2FmogszP1cGhefj/5G
AARkxkoo83Dx18sqlU1h/2vkH8LNwm8U789qm6divYAjrA+hc/J8zw==
=CC6t
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic_wormhole_mailbox_server-0.5.1-py3-none-any.whl.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmcwGhcRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaaeUfQf/fiRlkCg7buxgODywLgdFOIyPaHNLrCGY
ERmK5UsSSL5CT38e3I3EuNkmUmdQwEUC0DXU0y81QOoPVUC8U1mSqy50C/NnAWdG
Ij1MGYN5Zlp/8Ydt9oeW4CXYA4WkCfAxzD/rYkRwjHDgBW13gAgklIyK0E8Ssg5m
i/32c21JGMnBt36o9yO3VBgqw9Vw6Cr9hoHOHt1xRYEZ4PPVNi/L0WKr9MjSPE+X
twxnKgbWfRVdvC2Lt0hUsFDqy24lGAjZ/n4hrmj+HOJ72D0oczJpFa+7gjr8j7tO
nITTW0/5Tqm+405jSYybtmm8R033ZvMI+okNQ4luShs41kHzo4Odpw==
=MiBZ
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic_wormhole_mailbox_server-0.6.0-py3-none-any.whl.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmmO09cRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaacP8wgArStS7wGiSCiBuEIfue5IStfPmIJOCpGA
ZVkVd3bNscioNv6Xwt7MQItbuKO544VR72uyJre4o/M8OqHjiNEct+0iEoZDIGhH
PfLQ2qduTbZgcNe5NpwdAWcW7v1DHcnY8cSo1UHsXJZg5kZBv1EDVYlwaQi+OLjr
nyD2AeAWf3HfohMdGxgqpcBCTXamCxDSHLjqPS0njgRK9HHpGnLEoxp8wiYPMah3
0eCuJn/SxT2vsO7u5eQYrJ3bfqsUlu4wgelZE88LwLVmD3Pjj9DxMvFP+QPZ0ivv
v3aYft23wET+4gZJP2/tb2zJBern18qohTD3vIakEjUtRDPyiQBXOg==
=nicE
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic_wormhole_mailbox_server-0.6.0.tar.gz.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmmO09cRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaaeYjQf+P3CoVhbOz3V87wAglhn8RWIpRkFmr0oK
4nuWxpVZ9SyIn1OAXto2jjoWQ+ABoN8goUuR233le+zNJ2i+td25gcBMekA0PwDT
cwS53sgWhkrvhsEgYCSCv1Wnbgn3Y0Bo43fJWNEG3itEZgBVgoa6065s3/FF4BKE
OOO3MjKaadBvPUM10MmM/TGvCEJOmQQ+yP0YjMY4+bHOS0Vv/QFVz5/asL0Nla7r
dxBenpLHa46Y5J8vpYU0I7T6esWvZqc13Zvc41i3iTiPhFXnk+U9j9PrXfB6csbT
29RJub8RcfwgegKCNBlKBbTb4FetEQp22sFptQs/5fQXH4AjIY8RyQ==
=eRHe
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic_wormhole_mailbox_server-0.7.0-py3-none-any.whl.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmoGEmkRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaafI3QgAurb2bNlFd8MB1EOVJ8qYH0Ea06Mynuif
UpowOziYAz28fcXYL/gErDFoJNP4yafe0N7NNQwd2QeFcL+suiIucR4EE71c0PKi
JG99oJ3wpEwn/ADuShc+vM+kMGrPCUAHtEBAmUNRHgnAouVdjZYpz7joJcr/uy7R
ZCm6rYqoTqCexXPyax1iWUp5ZMK9fHSOFyoNKXbFDKLekVcyLI9XDInv92x2PCBx
5gna/htnxtUpZGrtSTwe1Yx/hYLb57E7BBnTuWDFMhvH7nSTR/pvGc8hvkzB/hxB
31hnRRrdY7snma2CqK07d2x8K1hORdih+ptHKPSI5kua3UJAnyJqTA==
=Av5c
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic_wormhole_mailbox_server-0.7.0.tar.gz.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmoGEmoRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaacnRAf/dxdrzwCGqTyUU11lf0KwVYcLLgIAKFPJ
PR3vvz66X5dUshKOGoE1IyL1xg0BCOFVnPzKl8ptaDXNPLKXreSRg0ALP6YHXNqt
hhEg4P+GC2E9TkYe4VajQ0Bfa9nN9sylhwEXmur7RGKvtGr9tMbGdP9ROOIsHwNd
ngKA34EpjuqsP7DKChlkfuMS9fq+0wzbfsm+ZItpHrxPXWQ5lbPHfxtVcWj1QjW4
Nj2o1p0lTx74IHGRAjN2ABVe5GV/CZ7cPhHI3KGS7MNYPbqs4HzmeSQR54IKEyK7
Z/xK9S4oIOgifL8tw/2QqnRs/mmMdFN7gYnH4ZONtbBJok5OMd4j+Q==
=a6B/
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic_wormhole_mailbox_server-0.8.0-py3-none-any.whl.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmoHVeoRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaaeeJgf/RfM1riKqo284TZ3wJJip/74CfGCARX9J
JRabVxmnwF6ZLEPNCa2QVCdt3QeLNjcFDG8MHKefI+T4Ii0ljwU2V0zE8AKHlsEI
knH7D8IevrQ3QGFjW5O8N9Ye4OIVIRYnf1JZxgY5Do+2N2hr7jvUrmEHzOXIWtcq
03LcUeGFOdghhxNpRF7JE6yEJ36n4LSddh6tHTYxCppWxOfLJzWQ2XPReL63b50i
+wbYCoN1Ihvk3Y+4nZWb5WrBcL4DHGj7CGP5tYQii7jNKVr/g0s6Hv35Gyx6zPsv
1DE8ZpfaqNonUhXwh1/n0tE++KMrE6sO14434hXiE+KUo9/IHNWPGQ==
=XsJp
-----END PGP SIGNATURE-----
================================================
FILE: signatures/magic_wormhole_mailbox_server-0.8.0.tar.gz.asc
================================================
-----BEGIN PGP SIGNATURE-----
iQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmoHVeoRHG1lZWphaEBt
ZWVqYWguY2EACgkQwmAoAxKAaaexbQf/dXmzsN34HFn+j5gM+RLp/MavbhzRTspQ
wYsSHsqfrcSyFREEHNqZGANRVhkmkyprb3oDFYRMzbsBWZwW8am1EsIrkiJAIkog
ebGLQ9Qmihm9bh+7sDnHC1dyXPP1Jgica9zgFc41GlBpa/mLvx5JjmUBK3zypFXi
rOJMSH7tWiov+sHXNCPcX6xhktAFZR3lWwdVNFfYhqI4g+W2yQd9tC5qKEHTpUJd
hjcMjLkpvpbwpGUjdY6z6oAUpDoy3S94AAtWbqpwKbOqJr9KBgOltQDKgJBJZ1OY
trQY5co4yvsEH3AH92PeZ+X6UnKHRl7kYl97qaak98pZuKlkQKfMYA==
=lk5u
-----END PGP SIGNATURE-----
================================================
FILE: src/twisted/plugins/magic_wormhole_mailbox.py
================================================
from twisted.application.service import ServiceMaker
Mailbox = ServiceMaker(
"Magic-Wormhole Mailbox Server", # name
"wormhole_mailbox_server.server_tap", # module
"Provide the Mailbox server for Magic-Wormhole clients.", # desc
"wormhole-mailbox", # tapname
)
================================================
FILE: src/wormhole_mailbox_server/__init__.py
================================================
from . import _version
__version__ = _version.get_versions()['version']
================================================
FILE: src/wormhole_mailbox_server/_version.py
================================================
# This file helps to compute a version number in source trees obtained from
# git-archive tarball (such as those provided by githubs download-from-tag
# feature). Distribution tarballs (built by setup.py sdist) and build
# directories (produced by setup.py build) will contain a much shorter file
# that just contains the computed version number.
# This file is released into the public domain.
# Generated by versioneer-0.29
# https://github.com/python-versioneer/python-versioneer
"""Git implementation of _version.py."""
import errno
import os
import re
import subprocess
import sys
from typing import Any, Callable, Optional
import functools
def get_keywords() -> dict[str, str]:
"""Get the keywords needed to look up the version information."""
# these strings will be replaced by git during git-archive.
# setup.py/versioneer.py will grep for the variable names, so they must
# each be defined on a line of their own. _version.py will just call
# get_keywords().
git_refnames = "$Format:%d$"
git_full = "$Format:%H$"
git_date = "$Format:%ci$"
keywords = {"refnames": git_refnames, "full": git_full, "date": git_date}
return keywords
class VersioneerConfig:
"""Container for Versioneer configuration parameters."""
VCS: str
style: str
tag_prefix: str
parentdir_prefix: str
versionfile_source: str
verbose: bool
def get_config() -> VersioneerConfig:
"""Create, populate and return the VersioneerConfig() object."""
# these strings are filled in when 'setup.py versioneer' creates
# _version.py
cfg = VersioneerConfig()
cfg.VCS = "git"
cfg.style = ""
cfg.tag_prefix = ""
cfg.parentdir_prefix = "magic-wormhole-mailbox-server"
cfg.versionfile_source = "src/wormhole_mailbox_server/_version.py"
cfg.verbose = False
return cfg
class NotThisMethod(Exception):
"""Exception raised if a method is not valid for the current scenario."""
LONG_VERSION_PY: dict[str, str] = {}
HANDLERS: dict[str, dict[str, Callable]] = {}
def register_vcs_handler(vcs: str, method: str) -> Callable: # decorator
"""Create decorator to mark a method as the handler of a VCS."""
def decorate(f: Callable) -> Callable:
"""Store f in HANDLERS[vcs][method]."""
if vcs not in HANDLERS:
HANDLERS[vcs] = {}
HANDLERS[vcs][method] = f
return f
return decorate
def run_command(
commands: list[str],
args: list[str],
cwd: Optional[str] = None,
verbose: bool = False,
hide_stderr: bool = False,
env: Optional[dict[str, str]] = None,
) -> tuple[Optional[str], Optional[int]]:
"""Call the given command(s)."""
assert isinstance(commands, list)
process = None
popen_kwargs: dict[str, Any] = {}
if sys.platform == "win32":
# This hides the console window if pythonw.exe is used
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
popen_kwargs["startupinfo"] = startupinfo
for command in commands:
try:
dispcmd = str([command] + args)
# remember shell=False, so use git.cmd on windows, not just git
process = subprocess.Popen([command] + args, cwd=cwd, env=env,
stdout=subprocess.PIPE,
stderr=(subprocess.PIPE if hide_stderr
else None), **popen_kwargs)
break
except OSError as e:
if e.errno == errno.ENOENT:
continue
if verbose:
print(f"unable to run {dispcmd}")
print(e)
return None, None
else:
if verbose:
print(f"unable to find command, tried {commands}")
return None, None
stdout = process.communicate()[0].strip().decode()
if process.returncode != 0:
if verbose:
print(f"unable to run {dispcmd} (error)")
print(f"stdout was {stdout}")
return None, process.returncode
return stdout, process.returncode
def versions_from_parentdir(
parentdir_prefix: str,
root: str,
verbose: bool,
) -> dict[str, Any]:
"""Try to determine the version from the parent directory name.
Source tarballs conventionally unpack into a directory that includes both
the project name and a version string. We will also support searching up
two directory levels for an appropriately named parent directory
"""
rootdirs = []
for _ in range(3):
dirname = os.path.basename(root)
if dirname.startswith(parentdir_prefix):
return {"version": dirname[len(parentdir_prefix):],
"full-revisionid": None,
"dirty": False, "error": None, "date": None}
rootdirs.append(root)
root = os.path.dirname(root) # up a level
if verbose:
print("Tried directories %s but none started with prefix %s" %
(str(rootdirs), parentdir_prefix))
raise NotThisMethod("rootdir doesn't start with parentdir_prefix")
@register_vcs_handler("git", "get_keywords")
def git_get_keywords(versionfile_abs: str) -> dict[str, str]:
"""Extract version information from the given file."""
# the code embedded in _version.py can just fetch the value of these
# keywords. When used from setup.py, we don't want to import _version.py,
# so we do it with a regexp instead. This function is not used from
# _version.py.
keywords: dict[str, str] = {}
try:
with open(versionfile_abs) as fobj:
for line in fobj:
if line.strip().startswith("git_refnames ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["refnames"] = mo.group(1)
if line.strip().startswith("git_full ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["full"] = mo.group(1)
if line.strip().startswith("git_date ="):
mo = re.search(r'=\s*"(.*)"', line)
if mo:
keywords["date"] = mo.group(1)
except OSError:
pass
return keywords
@register_vcs_handler("git", "keywords")
def git_versions_from_keywords(
keywords: dict[str, str],
tag_prefix: str,
verbose: bool,
) -> dict[str, Any]:
"""Get version information from git keywords."""
if "refnames" not in keywords:
raise NotThisMethod("Short version file found")
date = keywords.get("date")
if date is not None:
# Use only the last line. Previous lines may contain GPG signature
# information.
date = date.splitlines()[-1]
# git-2.2.0 added "%cI", which expands to an ISO-8601 -compliant
# datestamp. However we prefer "%ci" (which expands to an "ISO-8601
# -like" string, which we must then edit to make compliant), because
# it's been around since git-1.5.3, and it's too difficult to
# discover which version we're using, or to work around using an
# older one.
date = date.strip().replace(" ", "T", 1).replace(" ", "", 1)
refnames = keywords["refnames"].strip()
if refnames.startswith("$Format"):
if verbose:
print("keywords are unexpanded, not using")
raise NotThisMethod("unexpanded keywords, not a git-archive tarball")
refs = {r.strip() for r in refnames.strip("()").split(",")}
# starting in git-1.8.3, tags are listed as "tag: foo-1.0" instead of
# just "foo-1.0". If we see a "tag: " prefix, prefer those.
TAG = "tag: "
tags = {r[len(TAG):] for r in refs if r.startswith(TAG)}
if not tags:
# Either we're using git < 1.8.3, or there really are no tags. We use
# a heuristic: assume all version tags have a digit. The old git %d
# expansion behaves like git log --decorate=short and strips out the
# refs/heads/ and refs/tags/ prefixes that would let us distinguish
# between branches and tags. By ignoring refnames without digits, we
# filter out many common branch names like "release" and
# "stabilization", as well as "HEAD" and "master".
tags = {r for r in refs if re.search(r'\d', r)}
if verbose:
print(f"discarding '{','.join(refs - tags)}', no digits")
if verbose:
print(f"likely tags: {','.join(sorted(tags))}")
for ref in sorted(tags):
# sorting will prefer e.g. "2.0" over "2.0rc1"
if ref.startswith(tag_prefix):
r = ref[len(tag_prefix):]
# Filter out refs that exactly match prefix or that don't start
# with a number once the prefix is stripped (mostly a concern
# when prefix is '')
if not re.match(r'\d', r):
continue
if verbose:
print(f"picking {r}")
return {"version": r,
"full-revisionid": keywords["full"].strip(),
"dirty": False, "error": None,
"date": date}
# no suitable tags, so version is "0+unknown", but full hex is still there
if verbose:
print("no suitable tags, using unknown + full revision id")
return {"version": "0+unknown",
"full-revisionid": keywords["full"].strip(),
"dirty": False, "error": "no suitable tags", "date": None}
@register_vcs_handler("git", "pieces_from_vcs")
def git_pieces_from_vcs(
tag_prefix: str,
root: str,
verbose: bool,
runner: Callable = run_command
) -> dict[str, Any]:
"""Get version from 'git describe' in the root of the source tree.
This only gets called if the git-archive 'subst' keywords were *not*
expanded, and _version.py hasn't already been rewritten with a short
version string, meaning we're inside a checked out source tree.
"""
GITS = ["git"]
if sys.platform == "win32":
GITS = ["git.cmd", "git.exe"]
# GIT_DIR can interfere with correct operation of Versioneer.
# It may be intended to be passed to the Versioneer-versioned project,
# but that should not change where we get our version from.
env = os.environ.copy()
env.pop("GIT_DIR", None)
runner = functools.partial(runner, env=env)
_, rc = runner(GITS, ["rev-parse", "--git-dir"], cwd=root,
hide_stderr=not verbose)
if rc != 0:
if verbose:
print(f"Directory {root} not under git control")
raise NotThisMethod("'git rev-parse --git-dir' returned error")
# if there is a tag matching tag_prefix, this yields TAG-NUM-gHEX[-dirty]
# if there isn't one, this yields HEX[-dirty] (no NUM)
describe_out, rc = runner(GITS, [
"describe", "--tags", "--dirty", "--always", "--long",
"--match", f"{tag_prefix}[[:digit:]]*"
], cwd=root)
# --long was added in git-1.5.5
if describe_out is None:
raise NotThisMethod("'git describe' failed")
describe_out = describe_out.strip()
full_out, rc = runner(GITS, ["rev-parse", "HEAD"], cwd=root)
if full_out is None:
raise NotThisMethod("'git rev-parse' failed")
full_out = full_out.strip()
pieces: dict[str, Any] = {}
pieces["long"] = full_out
pieces["short"] = full_out[:7] # maybe improved later
pieces["error"] = None
branch_name, rc = runner(GITS, ["rev-parse", "--abbrev-ref", "HEAD"],
cwd=root)
# --abbrev-ref was added in git-1.6.3
if rc != 0 or branch_name is None:
raise NotThisMethod("'git rev-parse --abbrev-ref' returned error")
branch_name = branch_name.strip()
if branch_name == "HEAD":
# If we aren't exactly on a branch, pick a branch which represents
# the current commit. If all else fails, we are on a branchless
# commit.
branches, rc = runner(GITS, ["branch", "--contains"], cwd=root)
# --contains was added in git-1.5.4
if rc != 0 or branches is None:
raise NotThisMethod("'git branch --contains' returned error")
branches = branches.split("\n")
# Remove the first line if we're running detached
if "(" in branches[0]:
branches.pop(0)
# Strip off the leading "* " from the list of branches.
branches = [branch[2:] for branch in branches]
if "master" in branches:
branch_name = "master"
elif not branches:
branch_name = None
else:
# Pick the first branch that is returned. Good or bad.
branch_name = branches[0]
pieces["branch"] = branch_name
# parse describe_out. It will be like TAG-NUM-gHEX[-dirty] or HEX[-dirty]
# TAG might have hyphens.
git_describe = describe_out
# look for -dirty suffix
dirty = git_describe.endswith("-dirty")
pieces["dirty"] = dirty
if dirty:
git_describe = git_describe[:git_describe.rindex("-dirty")]
# now we have TAG-NUM-gHEX or HEX
if "-" in git_describe:
# TAG-NUM-gHEX
mo = re.search(r'^(.+)-(\d+)-g([0-9a-f]+)$', git_describe)
if not mo:
# unparsable. Maybe git-describe is misbehaving?
pieces["error"] = f"unable to parse git-describe output: '{describe_out}'"
return pieces
# tag
full_tag = mo.group(1)
if not full_tag.startswith(tag_prefix):
if verbose:
fmt = "tag '%s' doesn't start with prefix '%s'"
print(fmt % (full_tag, tag_prefix))
pieces["error"] = ("tag '%s' doesn't start with prefix '%s'"
% (full_tag, tag_prefix))
return pieces
pieces["closest-tag"] = full_tag[len(tag_prefix):]
# distance: number of commits since tag
pieces["distance"] = int(mo.group(2))
# commit: short hex revision ID
pieces["short"] = mo.group(3)
else:
# HEX: no tags
pieces["closest-tag"] = None
out, rc = runner(GITS, ["rev-list", "HEAD", "--left-right"], cwd=root)
pieces["distance"] = len(out.split()) # total number of commits
# commit date: see ISO-8601 comment in git_versions_from_keywords()
date = runner(GITS, ["show", "-s", "--format=%ci", "HEAD"], cwd=root)[0].strip()
# Use only the last line. Previous lines may contain GPG signature
# information.
date = date.splitlines()[-1]
pieces["date"] = date.strip().replace(" ", "T", 1).replace(" ", "", 1)
return pieces
def plus_or_dot(pieces: dict[str, Any]) -> str:
"""Return a + if we don't already have one, else return a ."""
if "+" in pieces.get("closest-tag", ""):
return "."
return "+"
def render_pep440(pieces: dict[str, Any]) -> str:
"""Build up version string, with post-release "local version identifier".
Our goal: TAG[+DISTANCE.gHEX[.dirty]] . Note that if you
get a tagged build and then dirty it, you'll get TAG+0.gHEX.dirty
Exceptions:
1: no tags. git_describe was just HEX. 0+untagged.DISTANCE.gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
rendered += plus_or_dot(pieces)
rendered += "%d.g%s" % (pieces["distance"], pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0+untagged.%d.g%s" % (pieces["distance"],
pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def render_pep440_branch(pieces: dict[str, Any]) -> str:
"""TAG[[.dev0]+DISTANCE.gHEX[.dirty]] .
The ".dev0" means not master branch. Note that .dev0 sorts backwards
(a feature branch will appear "older" than the master branch).
Exceptions:
1: no tags. 0[.dev0]+untagged.DISTANCE.gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += "%d.g%s" % (pieces["distance"], pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0"
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += "+untagged.%d.g%s" % (pieces["distance"],
pieces["short"])
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def pep440_split_post(ver: str) -> tuple[str, Optional[int]]:
"""Split pep440 version string at the post-release segment.
Returns the release segments before the post-release and the
post-release version number (or -1 if no post-release segment is present).
"""
vc = str.split(ver, ".post")
return vc[0], int(vc[1] or 0) if len(vc) == 2 else None
def render_pep440_pre(pieces: dict[str, Any]) -> str:
"""TAG[.postN.devDISTANCE] -- No -dirty.
Exceptions:
1: no tags. 0.post0.devDISTANCE
"""
if pieces["closest-tag"]:
if pieces["distance"]:
# update the post release segment
tag_version, post_version = pep440_split_post(pieces["closest-tag"])
rendered = tag_version
if post_version is not None:
rendered += ".post%d.dev%d" % (post_version + 1, pieces["distance"])
else:
rendered += ".post0.dev%d" % (pieces["distance"])
else:
# no commits, use the tag as the version
rendered = pieces["closest-tag"]
else:
# exception #1
rendered = "0.post0.dev%d" % pieces["distance"]
return rendered
def render_pep440_post(pieces: dict[str, Any]) -> str:
"""TAG[.postDISTANCE[.dev0]+gHEX] .
The ".dev0" means dirty. Note that .dev0 sorts backwards
(a dirty tree will appear "older" than the corresponding clean one),
but you shouldn't be releasing software with -dirty anyways.
Exceptions:
1: no tags. 0.postDISTANCE[.dev0]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
rendered += ".post%d" % pieces["distance"]
if pieces["dirty"]:
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += f"g{pieces['short']}"
else:
# exception #1
rendered = "0.post%d" % pieces["distance"]
if pieces["dirty"]:
rendered += ".dev0"
rendered += f"+g{pieces['short']}"
return rendered
def render_pep440_post_branch(pieces: dict[str, Any]) -> str:
"""TAG[.postDISTANCE[.dev0]+gHEX[.dirty]] .
The ".dev0" means not master branch.
Exceptions:
1: no tags. 0.postDISTANCE[.dev0]+gHEX[.dirty]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
rendered += ".post%d" % pieces["distance"]
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += plus_or_dot(pieces)
rendered += f"g{pieces['short']}"
if pieces["dirty"]:
rendered += ".dirty"
else:
# exception #1
rendered = "0.post%d" % pieces["distance"]
if pieces["branch"] != "master":
rendered += ".dev0"
rendered += f"+g{pieces['short']}"
if pieces["dirty"]:
rendered += ".dirty"
return rendered
def render_pep440_old(pieces: dict[str, Any]) -> str:
"""TAG[.postDISTANCE[.dev0]] .
The ".dev0" means dirty.
Exceptions:
1: no tags. 0.postDISTANCE[.dev0]
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"] or pieces["dirty"]:
rendered += ".post%d" % pieces["distance"]
if pieces["dirty"]:
rendered += ".dev0"
else:
# exception #1
rendered = "0.post%d" % pieces["distance"]
if pieces["dirty"]:
rendered += ".dev0"
return rendered
def render_git_describe(pieces: dict[str, Any]) -> str:
"""TAG[-DISTANCE-gHEX][-dirty].
Like 'git describe --tags --dirty --always'.
Exceptions:
1: no tags. HEX[-dirty] (note: no 'g' prefix)
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
if pieces["distance"]:
rendered += "-%d-g%s" % (pieces["distance"], pieces["short"])
else:
# exception #1
rendered = pieces["short"]
if pieces["dirty"]:
rendered += "-dirty"
return rendered
def render_git_describe_long(pieces: dict[str, Any]) -> str:
"""TAG-DISTANCE-gHEX[-dirty].
Like 'git describe --tags --dirty --always -long'.
The distance/hash is unconditional.
Exceptions:
1: no tags. HEX[-dirty] (note: no 'g' prefix)
"""
if pieces["closest-tag"]:
rendered = pieces["closest-tag"]
rendered += "-%d-g%s" % (pieces["distance"], pieces["short"])
else:
# exception #1
rendered = pieces["short"]
if pieces["dirty"]:
rendered += "-dirty"
return rendered
def render(pieces: dict[str, Any], style: str) -> dict[str, Any]:
"""Render the given version pieces into the requested style."""
if pieces["error"]:
return {"version": "unknown",
"full-revisionid": pieces.get("long"),
"dirty": None,
"error": pieces["error"],
"date": None}
if not style or style == "default":
style = "pep440" # the default
if style == "pep440":
rendered = render_pep440(pieces)
elif style == "pep440-branch":
rendered = render_pep440_branch(pieces)
elif style == "pep440-pre":
rendered = render_pep440_pre(pieces)
elif style == "pep440-post":
rendered = render_pep440_post(pieces)
elif style == "pep440-post-branch":
rendered = render_pep440_post_branch(pieces)
elif style == "pep440-old":
rendered = render_pep440_old(pieces)
elif style == "git-describe":
rendered = render_git_describe(pieces)
elif style == "git-describe-long":
rendered = render_git_describe_long(pieces)
else:
raise ValueError(f"unknown style '{style}'")
return {"version": rendered, "full-revisionid": pieces["long"],
"dirty": pieces["dirty"], "error": None,
"date": pieces.get("date")}
def get_versions() -> dict[str, Any]:
"""Get version information or return default if unable to do so."""
# I am in _version.py, which lives at ROOT/VERSIONFILE_SOURCE. If we have
# __file__, we can work backwards from there to the root. Some
# py2exe/bbfreeze/non-CPython implementations don't do __file__, in which
# case we can only use expanded keywords.
cfg = get_config()
verbose = cfg.verbose
try:
return git_versions_from_keywords(get_keywords(), cfg.tag_prefix,
verbose)
except NotThisMethod:
pass
try:
root = os.path.realpath(__file__)
# versionfile_source is the relative path from the top of the source
# tree (where the .git directory might live) to this file. Invert
# this to find the root from __file__.
for _ in cfg.versionfile_source.split('/'):
root = os.path.dirname(root)
except NameError:
return {"version": "0+unknown", "full-revisionid": None,
"dirty": None,
"error": "unable to find root of source tree",
"date": None}
try:
pieces = git_pieces_from_vcs(cfg.tag_prefix, root, verbose)
return render(pieces, cfg.style)
except NotThisMethod:
pass
try:
if cfg.parentdir_prefix:
return versions_from_parentdir(cfg.parentdir_prefix, root, verbose)
except NotThisMethod:
pass
return {"version": "0+unknown", "full-revisionid": None,
"dirty": None,
"error": "unable to compute version", "date": None}
================================================
FILE: src/wormhole_mailbox_server/database.py
================================================
import importlib.resources
import os, shutil
import sqlite3
import tempfile
from twisted.python import log
class DBError(Exception):
pass
def get_schema(name, version):
sql_filepath = f"db-schemas/{name}-v{version}.sql"
path = importlib.resources.files("wormhole_mailbox_server").joinpath(sql_filepath)
return path.read_text(encoding="utf-8")
def get_upgrader(name, new_version):
sql_filepath = f"db-schemas/upgrade-{name}-to-v{new_version}.sql"
path = importlib.resources.files("wormhole_mailbox_server").joinpath(sql_filepath)
try:
return path.read_text(encoding="utf-8")
except OSError: # includes FileNotFoundError
raise ValueError("no upgrader for %d" % new_version)
CHANNELDB_TARGET_VERSION = 1
USAGEDB_TARGET_VERSION = 2
def dict_factory(cursor, row):
d = {}
for idx, col in enumerate(cursor.description):
d[col[0]] = row[idx]
return d
def _initialize_db_schema(db, name, target_version):
"""Creates the application schema in the given database.
"""
log.msg(f"populating new database with schema {name} v{target_version}")
schema = get_schema(name, target_version)
db.executescript(schema)
db.execute("INSERT INTO version (version) VALUES (?)",
(target_version,))
db.commit()
def _initialize_db_connection(db):
"""Sets up the db connection object with a row factory and with necessary
foreign key settings.
"""
db.row_factory = dict_factory
db.execute("PRAGMA foreign_keys = ON")
problems = db.execute("PRAGMA foreign_key_check").fetchall()
if problems:
raise DBError(f"failed foreign key check: {problems}")
def _open_db_connection(dbfile):
"""Open a new connection to the SQLite3 database at the given path.
"""
try:
db = sqlite3.connect(dbfile)
_initialize_db_connection(db)
except (OSError, sqlite3.OperationalError, sqlite3.DatabaseError) as e:
# this indicates that the file is not a compatible database format.
# Perhaps it was created with an old version, or it might be junk.
raise DBError(f"Unable to create/open db file {dbfile}: {e}")
return db
def _get_temporary_dbfile(dbfile):
"""Get a temporary filename near the given path.
"""
fd, name = tempfile.mkstemp(
prefix=os.path.basename(dbfile) + ".",
dir=os.path.dirname(dbfile)
)
os.close(fd)
return name
def _atomic_create_and_initialize_db(dbfile, name, target_version):
"""Create and return a new database, initialized with the application
schema.
If anything goes wrong, nothing is left at the ``dbfile`` path.
"""
temp_dbfile = _get_temporary_dbfile(dbfile)
db = _open_db_connection(temp_dbfile)
_initialize_db_schema(db, name, target_version)
db.close()
os.rename(temp_dbfile, dbfile)
return _open_db_connection(dbfile)
def _get_db(dbfile, name, target_version):
"""Open or create the given db file. The parent directory must exist.
Returns the db connection object, or raises DBError.
"""
if dbfile == ":memory:":
db = _open_db_connection(dbfile)
_initialize_db_schema(db, name, target_version)
elif os.path.exists(dbfile):
db = _open_db_connection(dbfile)
else:
db = _atomic_create_and_initialize_db(dbfile, name, target_version)
version = db.execute("SELECT version FROM version").fetchone()["version"]
if version < target_version and dbfile != ":memory:":
backup_fn = "%s-backup-v%d" % (dbfile, version)
log.msg(" storing backup of v%d db in %s" % (version, backup_fn))
shutil.copy(dbfile, backup_fn)
while version < target_version:
log.msg(f" need to upgrade from {version} to {target_version}")
try:
upgrader = get_upgrader(name, version+1)
except ValueError:
log.msg(f" unable to upgrade {version} to {version + 1}")
raise DBError("Unable to upgrade %s to version %s, left at %s"
% (dbfile, version+1, version))
log.msg(f" executing upgrader v{version}->v{version + 1}")
db.executescript(upgrader)
db.commit()
version = version+1
if version != target_version:
raise DBError(f"Unable to handle db version {version}")
return db
def create_or_upgrade_channel_db(dbfile):
return _get_db(dbfile, "channel", CHANNELDB_TARGET_VERSION)
def create_or_upgrade_usage_db(dbfile):
if dbfile is None:
return None
return _get_db(dbfile, "usage", USAGEDB_TARGET_VERSION)
class DBDoesntExist(Exception):
pass
def open_existing_db(dbfile):
assert dbfile != ":memory:"
if not os.path.exists(dbfile):
raise DBDoesntExist()
return _open_db_connection(dbfile)
class DBAlreadyExists(Exception):
pass
def create_channel_db(dbfile):
"""Create the given db file. Refuse to touch a pre-existing file.
This is meant for use by migration tools, to create the output target"""
if dbfile == ":memory:":
db = _open_db_connection(dbfile)
_initialize_db_schema(db, "channel", CHANNELDB_TARGET_VERSION)
elif os.path.exists(dbfile):
raise DBAlreadyExists()
else:
db = _atomic_create_and_initialize_db(dbfile, "channel",
CHANNELDB_TARGET_VERSION)
return db
def create_usage_db(dbfile):
if dbfile == ":memory:":
db = _open_db_connection(dbfile)
_initialize_db_schema(db, "usage", USAGEDB_TARGET_VERSION)
elif os.path.exists(dbfile):
raise DBAlreadyExists()
else:
db = _atomic_create_and_initialize_db(dbfile, "usage",
USAGEDB_TARGET_VERSION)
return db
def dump_db(db):
# to let _iterdump work, we need to restore the original row factory
orig = db.row_factory
try:
db.row_factory = sqlite3.Row
return "".join(db.iterdump())
finally:
db.row_factory = orig
================================================
FILE: src/wormhole_mailbox_server/db-schemas/channel-v1.sql
================================================
-- note: anything which isn't an boolean, integer, or human-readable unicode
-- string, (i.e. binary strings) will be stored as hex
CREATE TABLE `version`
(
`version` INTEGER -- contains one row, set to 1
);
-- Wormhole codes use a "nameplate": a short name which is only used to
-- reference a specific (long-named) mailbox. The codes only use numeric
-- nameplates, but the protocol and server allow can use arbitrary strings.
CREATE TABLE `nameplates`
(
`id` INTEGER PRIMARY KEY AUTOINCREMENT,
`app_id` VARCHAR,
`name` VARCHAR,
`mailbox_id` VARCHAR REFERENCES `mailboxes`(`id`),
`request_id` VARCHAR -- from 'allocate' message, for future deduplication
);
CREATE INDEX `nameplates_idx` ON `nameplates` (`app_id`, `name`);
CREATE INDEX `nameplates_mailbox_idx` ON `nameplates` (`app_id`, `mailbox_id`);
CREATE INDEX `nameplates_request_idx` ON `nameplates` (`app_id`, `request_id`);
CREATE TABLE `nameplate_sides`
(
`nameplates_id` REFERENCES `nameplates`(`id`),
`claimed` BOOLEAN, -- True after claim(), False after release()
`side` VARCHAR,
`added` INTEGER -- time when this side first claimed the nameplate
);
-- Clients exchange messages through a "mailbox", which has a long (randomly
-- unique) identifier and a queue of messages.
-- `id` is randomly-generated and unique across all apps.
CREATE TABLE `mailboxes`
(
`app_id` VARCHAR,
`id` VARCHAR PRIMARY KEY,
`updated` INTEGER, -- time of last activity, used for pruning
`for_nameplate` BOOLEAN -- allocated for a nameplate, not standalone
);
CREATE INDEX `mailboxes_idx` ON `mailboxes` (`app_id`, `id`);
CREATE TABLE `mailbox_sides`
(
`mailbox_id` REFERENCES `mailboxes`(`id`),
`opened` BOOLEAN, -- True after open(), False after close()
`side` VARCHAR,
`added` INTEGER, -- time when this side first opened the mailbox
`mood` VARCHAR
);
CREATE TABLE `messages`
(
`app_id` VARCHAR,
`mailbox_id` VARCHAR,
`side` VARCHAR,
`phase` VARCHAR, -- numeric or string
`body` VARCHAR,
`server_rx` INTEGER,
`msg_id` VARCHAR
);
CREATE INDEX `messages_idx` ON `messages` (`app_id`, `mailbox_id`);
================================================
FILE: src/wormhole_mailbox_server/db-schemas/upgrade-usage-to-v2.sql
================================================
CREATE TABLE `client_versions`
(
`app_id` VARCHAR,
`side` VARCHAR, -- for deduplication of reconnects
`connect_time` INTEGER, -- seconds since epoch, rounded to "blur time"
-- the client sends us a 'client_version' tuple of (implementation, version)
-- the Python client sends e.g. ("python", "0.11.0")
`implementation` VARCHAR,
`version` VARCHAR
);
CREATE INDEX `client_versions_time_idx` on `client_versions` (`connect_time`);
CREATE INDEX `client_versions_appid_time_idx` on `client_versions` (`app_id`, `connect_time`);
DELETE FROM `version`;
INSERT INTO `version` (`version`) VALUES (2);
================================================
FILE: src/wormhole_mailbox_server/db-schemas/usage-v1.sql
================================================
CREATE TABLE `version`
(
`version` INTEGER -- contains one row
);
CREATE TABLE `current`
(
`rebooted` INTEGER, -- seconds since epoch of most recent reboot
`updated` INTEGER, -- when `current` was last updated
`blur_time` INTEGER, -- `started` is rounded to this, or None
`connections_websocket` INTEGER -- number of live clients via websocket
);
-- one row is created each time a nameplate is retired
CREATE TABLE `nameplates`
(
`app_id` VARCHAR,
`started` INTEGER, -- seconds since epoch, rounded to "blur time"
`waiting_time` INTEGER, -- seconds from start to 2nd side appearing, or None
`total_time` INTEGER, -- seconds from open to last close/prune
`result` VARCHAR -- happy, lonely, pruney, crowded
-- nameplate moods:
-- "happy": two sides open and close
-- "lonely": one side opens and closes (no response from 2nd side)
-- "pruney": channels which get pruned for inactivity
-- "crowded": three or more sides were involved
);
CREATE INDEX `nameplates_idx` ON `nameplates` (`app_id`, `started`);
-- one row is created each time a mailbox is retired
CREATE TABLE `mailboxes`
(
`app_id` VARCHAR,
`for_nameplate` BOOLEAN, -- allocated for a nameplate, not standalone
`started` INTEGER, -- seconds since epoch, rounded to "blur time"
`total_time` INTEGER, -- seconds from open to last close
`waiting_time` INTEGER, -- seconds from start to 2nd side appearing, or None
`result` VARCHAR -- happy, scary, lonely, errory, pruney
-- rendezvous moods:
-- "happy": both sides close with mood=happy
-- "scary": any side closes with mood=scary (bad MAC, probably wrong pw)
-- "lonely": any side closes with mood=lonely (no response from 2nd side)
-- "errory": any side closes with mood=errory (other errors)
-- "pruney": channels which get pruned for inactivity
-- "crowded": three or more sides were involved
);
CREATE INDEX `mailboxes_idx` ON `mailboxes` (`app_id`, `started`);
CREATE INDEX `mailboxes_result_idx` ON `mailboxes` (`result`);
================================================
FILE: src/wormhole_mailbox_server/db-schemas/usage-v2.sql
================================================
CREATE TABLE `version`
(
`version` INTEGER -- contains one row
);
CREATE TABLE `current`
(
`rebooted` INTEGER, -- seconds since epoch of most recent reboot
`updated` INTEGER, -- when `current` was last updated
`blur_time` INTEGER, -- `started` is rounded to this, or None
`connections_websocket` INTEGER -- number of live clients via websocket
);
-- one row is created each time a nameplate is retired
CREATE TABLE `nameplates`
(
`app_id` VARCHAR,
`started` INTEGER, -- seconds since epoch, rounded to "blur time"
`waiting_time` INTEGER, -- seconds from start to 2nd side appearing, or None
`total_time` INTEGER, -- seconds from open to last close/prune
`result` VARCHAR -- happy, lonely, pruney, crowded
-- nameplate moods:
-- "happy": two sides open and close
-- "lonely": one side opens and closes (no response from 2nd side)
-- "pruney": channels which get pruned for inactivity
-- "crowded": three or more sides were involved
);
CREATE INDEX `nameplates_idx` ON `nameplates` (`app_id`, `started`);
-- one row is created each time a mailbox is retired
CREATE TABLE `mailboxes`
(
`app_id` VARCHAR,
`for_nameplate` BOOLEAN, -- allocated for a nameplate, not standalone
`started` INTEGER, -- seconds since epoch, rounded to "blur time"
`total_time` INTEGER, -- seconds from open to last close
`waiting_time` INTEGER, -- seconds from start to 2nd side appearing, or None
`result` VARCHAR -- happy, scary, lonely, errory, pruney
-- rendezvous moods:
-- "happy": both sides close with mood=happy
-- "scary": any side closes with mood=scary (bad MAC, probably wrong pw)
-- "lonely": any side closes with mood=lonely (no response from 2nd side)
-- "errory": any side closes with mood=errory (other errors)
-- "pruney": channels which get pruned for inactivity
-- "crowded": three or more sides were involved
);
CREATE INDEX `mailboxes_idx` ON `mailboxes` (`app_id`, `started`);
CREATE INDEX `mailboxes_result_idx` ON `mailboxes` (`result`);
CREATE TABLE `client_versions`
(
`app_id` VARCHAR,
`side` VARCHAR, -- for deduplication of reconnects
`connect_time` INTEGER, -- seconds since epoch, rounded to "blur time"
-- the client sends us a 'client_version' tuple of (implementation, version)
-- the Python client sends e.g. ("python", "0.11.0")
`implementation` VARCHAR,
`version` VARCHAR
);
CREATE INDEX `client_versions_time_idx` on `client_versions` (`connect_time`);
CREATE INDEX `client_versions_appid_time_idx` on `client_versions` (`app_id`, `connect_time`);
================================================
FILE: src/wormhole_mailbox_server/increase_rlimits.py
================================================
try:
# 'resource' is unix-only
from resource import getrlimit, setrlimit, RLIMIT_NOFILE
except ImportError: # pragma: nocover
getrlimit, setrlimit, RLIMIT_NOFILE = None, None, None # pragma: nocover
from twisted.python import log
def increase_rlimits():
if getrlimit is None:
log.msg("unable to import 'resource', leaving rlimit alone")
return
soft, hard = getrlimit(RLIMIT_NOFILE)
if soft >= 10000:
log.msg("RLIMIT_NOFILE.soft was %d, leaving it alone" % soft)
return
# OS-X defaults to soft=7168, and reports a huge number for 'hard',
# but won't accept anything more than soft=10240, so we can't just
# set soft=hard. Linux returns (1024, 1048576) and is fine with
# soft=hard. Cygwin is reported to return (256,-1) and accepts up to
# soft=3200. So we try multiple values until something works.
for newlimit in [hard, 10000, 3200, 1024]:
log.msg(f"changing RLIMIT_NOFILE from ({soft},{hard}) to ({newlimit},{hard})")
try:
setrlimit(RLIMIT_NOFILE, (newlimit, hard))
log.msg("setrlimit successful")
return
except ValueError as e:
log.msg(f"error during setrlimit: {e}")
continue
except:
log.msg("other error during setrlimit, leaving it alone")
log.err()
return
log.msg("unable to change rlimit, leaving it alone")
================================================
FILE: src/wormhole_mailbox_server/server.py
================================================
import os, random, base64, re
from collections import namedtuple
from twisted.python import log
from twisted.application import service
def generate_mailbox_id():
return base64.b32encode(os.urandom(8)).lower().strip(b"=").decode("ascii")
NAMEPLATE_RE = re.compile(r'^\d+$')
def check_valid_nameplate(n):
if not isinstance(n, str):
raise ValueError("nameplate %r is %s not str" % (n, type(n)))
if len(n) > 40:
raise ValueError("nameplate %s .. is too long, %d>40" % (repr(n)[:50], len(n)))
if not NAMEPLATE_RE.search(n):
raise ValueError("nameplate %s has non-digits" % (n,))
class CrowdedError(Exception):
pass
class ReclaimedError(Exception):
pass
Usage = namedtuple("Usage", ["started", "waiting_time", "total_time", "result"])
TransitUsage = namedtuple("TransitUsage",
["started", "waiting_time", "total_time",
"total_bytes", "result"])
SidedMessage = namedtuple("SidedMessage", ["side", "phase", "body",
"server_rx", "msg_id"])
class Mailbox:
def __init__(self, app, db, usage_db, app_id, mailbox_id):
self._app = app
self._db = db
self._usage_db = usage_db
self._app_id = app_id
self._mailbox_id = mailbox_id
self._listeners = {} # handle -> (send_f, stop_f)
# "handle" is a hashable object, for deregistration
# send_f() takes a JSONable object, stop_f() has no args
def open(self, side, when):
# requires caller to db.commit()
assert isinstance(side, str), type(side)
db = self._db
already = db.execute("SELECT * FROM `mailbox_sides`"
" WHERE `mailbox_id`=? AND `side`=?",
(self._mailbox_id, side)).fetchone()
if not already:
db.execute("INSERT INTO `mailbox_sides`"
" (`mailbox_id`, `opened`, `side`, `added`)"
" VALUES(?,?,?,?)",
(self._mailbox_id, True, side, when))
# We accept re-opening a mailbox which a side previously closed,
# unlike claim_nameplate(), which forbids any side from re-claiming a
# nameplate which they previously released. (Nameplates forbid this
# because the act of claiming a nameplate for the first time causes a
# new mailbox to be created, which should only happen once).
# Mailboxes have their own distinct objects (to manage
# subscriptions), so closing one which was already closed requires
# making a new object, which works by calling open() just before
# close(). We really do want to support re-closing closed mailboxes,
# because this enables intermittently-connected clients, who remember
# sending a 'close' but aren't sure whether it was received or not,
# then get shut down. Those clients will wake up and re-send the
# 'close', until they receive the 'closed' ack message.
self._touch(when)
db.commit() # XXX: reconcile the need for this with the comment above
def _touch(self, when):
self._db.execute("UPDATE `mailboxes` SET `updated`=? WHERE `id`=?",
(when, self._mailbox_id))
def get_messages(self):
messages = []
db = self._db
for row in db.execute("SELECT * FROM `messages`"
" WHERE `app_id`=? AND `mailbox_id`=?"
" ORDER BY `server_rx` ASC",
(self._app_id, self._mailbox_id)).fetchall():
sm = SidedMessage(side=row["side"], phase=row["phase"],
body=row["body"], server_rx=row["server_rx"],
msg_id=row["msg_id"])
messages.append(sm)
return messages
def add_listener(self, handle, send_f, stop_f):
#log.msg("add_listener", self._mailbox_id, handle)
self._listeners[handle] = (send_f, stop_f)
#log.msg(" added", len(self._listeners))
return self.get_messages()
def remove_listener(self, handle):
#log.msg("remove_listener", self._mailbox_id, handle)
self._listeners.pop(handle, None)
#log.msg(" removed", len(self._listeners))
def has_listeners(self):
return bool(self._listeners)
def count_listeners(self):
return len(self._listeners)
def broadcast_message(self, sm):
for (send_f, stop_f) in self._listeners.values():
send_f(sm)
def _add_message(self, sm):
self._db.execute("INSERT INTO `messages`"
" (`app_id`, `mailbox_id`, `side`, `phase`, `body`,"
" `server_rx`, `msg_id`)"
" VALUES (?,?,?,?,?, ?,?)",
(self._app_id, self._mailbox_id, sm.side,
sm.phase, sm.body, sm.server_rx, sm.msg_id))
self._touch(sm.server_rx)
self._db.commit()
def add_message(self, sm):
assert isinstance(sm, SidedMessage)
self._add_message(sm)
self.broadcast_message(sm)
def close(self, side, mood, when):
assert isinstance(side, str), type(side)
db = self._db
row = db.execute("SELECT * FROM `mailboxes`"
" WHERE `app_id`=? AND `id`=?",
(self._app_id, self._mailbox_id)).fetchone()
if not row:
return
for_nameplate = row["for_nameplate"]
row = db.execute("SELECT * FROM `mailbox_sides`"
" WHERE `mailbox_id`=? AND `side`=?",
(self._mailbox_id, side)).fetchone()
if not row:
return
db.execute("UPDATE `mailbox_sides` SET `opened`=?, `mood`=?"
" WHERE `mailbox_id`=? AND `side`=?",
(False, mood, self._mailbox_id, side))
db.commit()
# are any sides still open?
side_rows = db.execute("SELECT * FROM `mailbox_sides`"
" WHERE `mailbox_id`=?",
(self._mailbox_id,)).fetchall()
if any([sr["opened"] for sr in side_rows]):
return
# nope. delete and summarize
# if the nameplate is still allocated we'll get a foreign-key
# failure when trying to delete the mailbox, so get rid of
# those first
db.execute("DELETE FROM `nameplate_sides` WHERE `side`=?",
(side,))
db.execute("DELETE FROM `nameplates` WHERE `mailbox_id`=?",
(self._mailbox_id,))
# remove mailbox content
db.execute("DELETE FROM `messages` WHERE `mailbox_id`=?",
(self._mailbox_id,))
db.execute("DELETE FROM `mailbox_sides` WHERE `mailbox_id`=?",
(self._mailbox_id,))
db.execute("DELETE FROM `mailboxes` WHERE `id`=?", (self._mailbox_id,))
if self._usage_db:
self._app._summarize_mailbox_and_store(for_nameplate, side_rows,
when, pruned=False)
self._usage_db.commit()
db.commit()
# Shut down any listeners, just in case they're still lingering
# around.
for (send_f, stop_f) in self._listeners.values():
stop_f()
self._listeners = {}
self._app.free_mailbox(self._mailbox_id)
def _shutdown(self):
# used at test shutdown to accelerate client disconnects
for (send_f, stop_f) in self._listeners.values():
stop_f()
self._listeners = {}
class AppNamespace:
def __init__(self, db, usage_db, blur_usage, log_requests, app_id,
allow_list):
self._db = db
self._usage_db = usage_db
self._blur_usage = blur_usage
self._log_requests = log_requests
self._app_id = app_id
self._mailboxes = {}
self._allow_list = allow_list
def log_client_version(self, server_rx, side, client_version):
if self._blur_usage:
server_rx = self._blur_usage * (server_rx // self._blur_usage)
implementation = client_version[0]
version = client_version[1]
if self._usage_db:
self._usage_db.execute("INSERT INTO `client_versions`"
" (`app_id`, `side`, `connect_time`,"
" `implementation`, `version`)"
" VALUES(?,?,?,?,?)",
(self._app_id, side, server_rx,
implementation, version))
self._usage_db.commit()
def get_nameplate_ids(self):
if not self._allow_list:
return []
return self._get_nameplate_ids()
def _get_nameplate_ids(self):
db = self._db
# TODO: filter this to numeric ids?
c = db.execute("SELECT DISTINCT `name` FROM `nameplates`"
" WHERE `app_id`=?", (self._app_id,))
return {row["name"] for row in c.fetchall()}
def _find_available_nameplate_id(self):
claimed = self._get_nameplate_ids()
for size in range(1,4): # stick to 1-999 for now
available = set()
for id_int in range(10**(size-1), 10**size):
id = "%d" % id_int
if id not in claimed:
available.add(id)
if available:
return random.choice(list(available))
# ouch, 999 currently claimed. Try random ones for a while.
for tries in range(1000):
id_int = random.randrange(1000, 1000*1000)
id = "%d" % id_int
if id not in claimed:
return id
raise ValueError("unable to find a free nameplate-id")
def allocate_nameplate(self, side, when):
nameplate_id = self._find_available_nameplate_id()
mailbox_id = self.claim_nameplate(nameplate_id, side, when)
del mailbox_id # ignored, they'll learn it from claim()
return nameplate_id
def claim_nameplate(self, name, side, when):
# when we're done:
# * there will be one row for the nameplate
# * there will be one 'side' attached to it, with claimed=True
# * a mailbox id and mailbox row will be created
# * a mailbox 'side' will be attached, with opened=True
assert isinstance(name, str), type(name)
assert isinstance(side, str), type(side)
check_valid_nameplate(name)
db = self._db
row = db.execute("SELECT * FROM `nameplates`"
" WHERE `app_id`=? AND `name`=?",
(self._app_id, name)).fetchone()
if not row:
if self._log_requests:
log.msg(f"creating nameplate#{name} for app_id {self._app_id}")
mailbox_id = generate_mailbox_id()
self._add_mailbox(mailbox_id, True, side, when) # ensure row exists
sql = ("INSERT INTO `nameplates`"
" (`app_id`, `name`, `mailbox_id`)"
" VALUES(?,?,?)")
npid = db.execute(sql, (self._app_id, name, mailbox_id)
).lastrowid
else:
npid = row["id"]
mailbox_id = row["mailbox_id"]
row = db.execute("SELECT * FROM `nameplate_sides`"
" WHERE `nameplates_id`=? AND `side`=?",
(npid, side)).fetchone()
if not row:
db.execute("INSERT INTO `nameplate_sides`"
" (`nameplates_id`, `claimed`, `side`, `added`)"
" VALUES(?,?,?,?)",
(npid, True, side, when))
else:
if not row["claimed"]:
raise ReclaimedError("you cannot re-claim a nameplate that your side previously released")
# since that might cause a new mailbox to be allocated
db.commit()
self.open_mailbox(mailbox_id, side, when) # may raise CrowdedError
rows = db.execute("SELECT * FROM `nameplate_sides`"
" WHERE `nameplates_id`=?", (npid,)).fetchall()
if len(rows) > 2:
# this line will probably never get hit: any crowding is noticed
# on mailbox_sides first, inside open_mailbox()
raise CrowdedError("too many sides have claimed this nameplate")
return mailbox_id
def release_nameplate(self, name, side, when):
# when we're done:
# * the 'claimed' flag will be cleared on the nameplate_sides row
# * if the nameplate is now unused (no claimed sides):
# * a usage record will be added
# * the nameplate row will be removed
# * the nameplate sides will be removed
assert isinstance(name, str), type(name)
assert isinstance(side, str), type(side)
db = self._db
np_row = db.execute("SELECT * FROM `nameplates`"
" WHERE `app_id`=? AND `name`=?",
(self._app_id, name)).fetchone()
if not np_row:
return
npid = np_row["id"]
row = db.execute("SELECT * FROM `nameplate_sides`"
" WHERE `nameplates_id`=? AND `side`=?",
(npid, side)).fetchone()
if not row:
return
db.execute("UPDATE `nameplate_sides` SET `claimed`=?"
" WHERE `nameplates_id`=? AND `side`=?",
(False, npid, side))
db.commit()
# now, are there any remaining claims?
side_rows = db.execute("SELECT * FROM `nameplate_sides`"
" WHERE `nameplates_id`=?",
(npid,)).fetchall()
claims = [1 for sr in side_rows if sr["claimed"]]
if claims:
return
# delete and summarize
db.execute("DELETE FROM `nameplate_sides` WHERE `nameplates_id`=?",
(npid,))
db.execute("DELETE FROM `nameplates` WHERE `id`=?", (npid,))
if self._usage_db:
self._summarize_nameplate_and_store(side_rows, when, pruned=False)
self._usage_db.commit()
db.commit()
def _summarize_nameplate_and_store(self, side_rows, delete_time, pruned):
# requires caller to self._usage_db.commit()
u = self._summarize_nameplate_usage(side_rows, delete_time, pruned)
self._usage_db.execute("INSERT INTO `nameplates`"
" (`app_id`,"
" `started`, `total_time`, `waiting_time`, `result`)"
" VALUES (?, ?,?,?,?)",
(self._app_id,
u.started, u.total_time, u.waiting_time, u.result))
def _summarize_nameplate_usage(self, side_rows, delete_time, pruned):
times = sorted([row["added"] for row in side_rows])
started = times[0]
if self._blur_usage:
started = self._blur_usage * (started // self._blur_usage)
waiting_time = None
if len(times) > 1:
waiting_time = times[1] - times[0]
total_time = delete_time - times[0]
result = "lonely"
if len(times) == 2:
result = "happy"
if pruned:
result = "pruney"
if len(times) > 2:
result = "crowded"
return Usage(started=started, waiting_time=waiting_time,
total_time=total_time, result=result)
def _add_mailbox(self, mailbox_id, for_nameplate, side, when):
assert isinstance(mailbox_id, str), type(mailbox_id)
db = self._db
row = db.execute("SELECT * FROM `mailboxes`"
" WHERE `app_id`=? AND `id`=?",
(self._app_id, mailbox_id)).fetchone()
if not row:
self._db.execute("INSERT INTO `mailboxes`"
" (`app_id`, `id`, `for_nameplate`, `updated`)"
" VALUES(?,?,?,?)",
(self._app_id, mailbox_id, for_nameplate, when))
# we don't need a commit here, because mailbox.open() only
# does SELECT FROM `mailbox_sides`, not from `mailboxes`
def open_mailbox(self, mailbox_id, side, when):
assert isinstance(mailbox_id, str), type(mailbox_id)
self._add_mailbox(mailbox_id, False, side, when) # ensure row exists
db = self._db
if not mailbox_id in self._mailboxes: # ensure Mailbox object exists
if self._log_requests:
log.msg(f"spawning #{mailbox_id} for app_id {self._app_id}")
self._mailboxes[mailbox_id] = Mailbox(self,
self._db, self._usage_db,
self._app_id, mailbox_id)
mailbox = self._mailboxes[mailbox_id]
# delegate to mailbox.open() to add a row to mailbox_sides, and
# update the mailbox.updated timestamp
mailbox.open(side, when)
db.commit()
rows = db.execute("SELECT * FROM `mailbox_sides`"
" WHERE `mailbox_id`=?",
(mailbox_id,)).fetchall()
if len(rows) > 2:
raise CrowdedError("too many sides have opened this mailbox")
return mailbox
def free_mailbox(self, mailbox_id):
# called from Mailbox.delete_and_summarize(), which deletes any
# messages
if mailbox_id in self._mailboxes:
self._mailboxes.pop(mailbox_id)
#if self._log_requests:
# log.msg("freed+killed #%s, now have %d DB mailboxes, %d live" %
# (mailbox_id, len(self.get_claimed()), len(self._mailboxes)))
def _summarize_mailbox_and_store(self, for_nameplate, side_rows,
delete_time, pruned):
db = self._usage_db
u = self._summarize_mailbox(side_rows, delete_time, pruned)
db.execute("INSERT INTO `mailboxes`"
" (`app_id`, `for_nameplate`,"
" `started`, `total_time`, `waiting_time`, `result`)"
" VALUES (?,?, ?,?,?,?)",
(self._app_id, for_nameplate,
u.started, u.total_time, u.waiting_time, u.result))
def _summarize_mailbox(self, side_rows, delete_time, pruned):
times = sorted([row["added"] for row in side_rows])
started = times[0]
if self._blur_usage:
started = self._blur_usage * (started // self._blur_usage)
waiting_time = None
if len(times) > 1:
waiting_time = times[1] - times[0]
total_time = delete_time - times[0]
num_sides = len(times)
if num_sides == 0:
result = "quiet"
elif num_sides == 1:
result = "lonely"
else:
result = "happy"
# "mood" is only recorded at close()
moods = [row["mood"] for row in side_rows if row.get("mood")]
if "lonely" in moods:
result = "lonely"
if "errory" in moods:
result = "errory"
if "scary" in moods:
result = "scary"
if pruned:
result = "pruney"
if num_sides > 2:
result = "crowded"
return Usage(started=started, waiting_time=waiting_time,
total_time=total_time, result=result)
def prune(self, now, old):
# The pruning check runs every 10 minutes, and "old" is defined to be
# 11 minutes ago (unit tests can use different values). The client is
# allowed to disconnect for up to 9 minutes without losing the
# channel (nameplate, mailbox, and messages).
# Each time a client does something, the mailbox.updated field is
# updated with the current timestamp. If a client is subscribed to
# the mailbox when pruning check runs, the "updated" field is also
# updated. After that check, if the "updated" field is "old", the
# channel is deleted.
# For now, pruning is logged even if log_requests is False, to debug
# the pruning process, and since pruning is triggered by a timer
# instead of by user action. It does reveal which mailboxes were
# present when the pruning process began, though, so in the log run
# it should do less logging.
log.msg(f" prune begins ({self._app_id})")
db = self._db
modified = False
for mailbox in self._mailboxes.values():
if mailbox.has_listeners():
log.msg(f"touch {mailbox._mailbox_id} because listeners")
mailbox._touch(now)
db.commit() # make sure the updates are visible below
new_mailboxes = set()
old_mailboxes = set()
for row in db.execute("SELECT * FROM `mailboxes` WHERE `app_id`=?",
(self._app_id,)).fetchall():
mailbox_id = row["id"]
log.msg(f" 1: age={now - row['updated']}, old={now - old}, {mailbox_id}")
if row["updated"] > old:
new_mailboxes.add(mailbox_id)
else:
old_mailboxes.add(mailbox_id)
log.msg(" 2: mailboxes:", new_mailboxes, old_mailboxes)
old_nameplates = set()
for row in db.execute("SELECT * FROM `nameplates` WHERE `app_id`=?",
(self._app_id,)).fetchall():
npid = row["id"]
mailbox_id = row["mailbox_id"]
if mailbox_id in old_mailboxes:
old_nameplates.add(npid)
log.msg(" 3: old_nameplates dbids", old_nameplates)
for npid in old_nameplates:
log.msg(" deleting nameplate with dbid", npid)
side_rows = db.execute("SELECT * FROM `nameplate_sides`"
" WHERE `nameplates_id`=?",
(npid,)).fetchall()
db.execute("DELETE FROM `nameplate_sides` WHERE `nameplates_id`=?",
(npid,))
db.execute("DELETE FROM `nameplates` WHERE `id`=?", (npid,))
if self._usage_db:
self._summarize_nameplate_and_store(side_rows, now, pruned=True)
modified = True
# delete all messages for old mailboxes
# delete all old mailboxes
for mailbox_id in old_mailboxes:
log.msg(" deleting mailbox", mailbox_id)
row = db.execute("SELECT * FROM `mailboxes`"
" WHERE `id`=?", (mailbox_id,)).fetchone()
for_nameplate = row["for_nameplate"]
side_rows = db.execute("SELECT * FROM `mailbox_sides`"
" WHERE `mailbox_id`=?",
(mailbox_id,)).fetchall()
db.execute("DELETE FROM `messages` WHERE `mailbox_id`=?",
(mailbox_id,))
db.execute("DELETE FROM `mailbox_sides` WHERE `mailbox_id`=?",
(mailbox_id,))
db.execute("DELETE FROM `mailboxes` WHERE `id`=?",
(mailbox_id,))
if self._usage_db:
self._summarize_mailbox_and_store(for_nameplate, side_rows,
now, pruned=True)
modified = True
if modified:
db.commit()
if self._usage_db:
self._usage_db.commit()
in_use = bool(self._mailboxes)
log.msg(f" prune complete, modified={modified}, in_use={in_use}")
return in_use
def count_listeners(self):
return sum(mailbox.count_listeners()
for mailbox in self._mailboxes.values())
def _shutdown(self):
for channel in self._mailboxes.values():
channel._shutdown()
class Server(service.MultiService):
def __init__(self, db, allow_list, welcome,
blur_usage, usage_db=None, log_file=None):
service.MultiService.__init__(self)
self._db = db
self._allow_list = allow_list
self._welcome = welcome
self._blur_usage = blur_usage
self._log_requests = blur_usage is None
self._usage_db = usage_db
self._log_file = log_file
self._apps = {}
def get_welcome(self):
return self._welcome
def get_log_requests(self):
return self._log_requests
def get_app(self, app_id):
assert isinstance(app_id, str)
if not app_id in self._apps:
if self._log_requests:
log.msg(f"spawning app_id {app_id}")
self._apps[app_id] = AppNamespace(
self._db,
self._usage_db,
self._blur_usage,
self._log_requests,
app_id,
self._allow_list,
)
return self._apps[app_id]
def get_all_apps(self):
apps = set()
for row in self._db.execute("SELECT DISTINCT `app_id`"
" FROM `nameplates`").fetchall():
apps.add(row["app_id"])
for row in self._db.execute("SELECT DISTINCT `app_id`"
" FROM `mailboxes`").fetchall():
apps.add(row["app_id"])
for row in self._db.execute("SELECT DISTINCT `app_id`"
" FROM `messages`").fetchall():
apps.add(row["app_id"])
return apps
def prune_all_apps(self, now, old):
# As with AppNamespace.prune_old_mailboxes, we log for now.
log.msg("beginning app prune")
for app_id in sorted(self.get_all_apps()):
log.msg(f" app prune checking {app_id!r}")
app = self.get_app(app_id)
in_use = app.prune(now, old)
if not in_use:
del self._apps[app_id]
log.msg(f"app prune ends, {len(self._apps)} apps")
def dump_stats(self, now, rebooted):
if not self._usage_db:
return
# write everything to self._usage_db
# Most of our current-status state is recorded in the channel_db, and
# our historical state goes into the usage_db. Both are updated each
# time something changes, so stats monitors can just read things out
# from there. The one bit of runtime state that isn't recorded each
# time is the number of connected clients, which will differ from the
# number of live "sides" briefly after they disconnect but before the
# mailbox is closed.
connections = sum(app.count_listeners()
for app in self._apps.values())
# TODO: this is all connections, not just the websocket ones. We don't
# have non-websocket connections yet, but when we add them, this needs
# to be updated. Probably by asking the WebSocketServerFactory to count
# them.
self._usage_db.execute("DELETE FROM `current`")
self._usage_db.execute("INSERT INTO `current`"
" (`rebooted`, `updated`, `blur_time`,"
" `connections_websocket`)"
" VALUES(?,?,?,?)",
(rebooted, now, self._blur_usage, connections))
self._usage_db.commit()
# current status: expected to be zero most of the time
#c["nameplates_total"] = q("SELECT COUNT() FROM `nameplates`")
# TODO: nameplates with only one side (most of them)
# TODO: nameplates with two sides (very fleeting)
# TODO: nameplates with three or more sides (crowded, unlikely)
#c["mailboxes_total"] = q("SELECT COUNT() FROM `mailboxes`")
# TODO: mailboxes with only one side (most of them)
# TODO: mailboxes with two sides (somewhat fleeting, in-transit)
# TODO: mailboxes with three or more sides (unlikely)
#c["messages_total"] = q("SELECT COUNT() FROM `messages`")
# recent timings (last 100 operations)
# TODO: median/etc of nameplate.total_time
# TODO: median/etc of mailbox.waiting_time (should be the same)
# TODO: median/etc of mailbox.total_time
# other
# TODO: mailboxes without nameplates (needs new DB schema)
def startService(self):
service.MultiService.startService(self)
log.msg("Wormhole relay server running")
if self._blur_usage:
log.msg("blurring access times to %d seconds" % self._blur_usage)
#log.msg("not logging HTTP requests")
else:
log.msg("not blurring access times")
if not self._allow_list:
log.msg("listing of allocated nameplates disallowed")
def stopService(self):
# This forcibly boots any clients that are still connected, which
# helps with unit tests that use threads for both clients. One client
# hits an exception, which terminates the test (and .tearDown calls
# stopService on the relay), but the other client (in its thread) is
# still waiting for a message. By killing off all connections, that
# other client gets an error, and exits promptly.
for app in self._apps.values():
app._shutdown()
return service.MultiService.stopService(self)
def make_server(db, allow_list=True,
advertise_version=None,
signal_error=None,
blur_usage=None,
usage_db=None,
log_file=None,
welcome_motd=None,
):
if blur_usage:
log.msg("blurring access times to %d seconds" % blur_usage)
else:
log.msg("not blurring access times")
welcome = dict()
if welcome_motd is not None:
# adding .motd will cause all clients to display the message,
# then keep running normally
welcome["motd"] = str(welcome_motd)
if advertise_version:
# The primary (python CLI) implementation will emit a message if
# its version does not match this key. If/when we have
# distributions which include older version, but we still expect
# them to be compatible, stop sending this key.
welcome["current_cli_version"] = advertise_version
if signal_error:
welcome["error"] = signal_error
return Server(db, allow_list=allow_list, welcome=welcome,
blur_usage=blur_usage, usage_db=usage_db, log_file=log_file)
================================================
FILE: src/wormhole_mailbox_server/server_tap.py
================================================
import os, json, time
from twisted.internet import reactor
from twisted.python import usage, log
from twisted.application.service import MultiService
from twisted.application.internet import (TimerService,
StreamServerEndpointService)
from twisted.internet import endpoints
from .increase_rlimits import increase_rlimits
from .server import make_server
from .web import make_web_server
from .database import create_or_upgrade_channel_db, create_or_upgrade_usage_db
LONGDESC = """This plugin sets up a 'Mailbox' server for magic-wormhole.
This service forwards short messages between clients, to perform key exchange
and connection setup."""
class Options(usage.Options):
synopsis = "[--port=] [--log-fd] [--blur-usage=] [--usage-db=]"
longdesc = LONGDESC
optParameters = [
("port", "p", r"tcp:4000:interface=\:\:", "endpoint to listen on"),
("blur-usage", None, None, "round logged access times to improve privacy"),
("log-fd", None, None, "write JSON usage logs to this file descriptor"),
("channel-db", None, "relay.sqlite", "location for the state database"),
("usage-db", None, None, "record usage data (SQLite)"),
("advertise-version", None, None, "version to recommend to clients"),
("signal-error", None, None, "force all clients to fail with a message"),
("motd", None, None, "Send a Message of the Day in the welcome"),
]
optFlags = [
("disallow-list", None, "refuse to send list of allocated nameplates"),
]
def __init__(self):
super().__init__()
self["websocket-protocol-options"] = []
self["allow-list"] = True
def opt_disallow_list(self):
self["allow-list"] = False
def opt_log_fd(self, arg):
self["log-fd"] = int(arg)
def opt_blur_usage(self, arg):
# --blur-usage= is in seconds. If the option isn't provided, we'll keep
# the default of None
self["blur-usage"] = int(arg)
def opt_websocket_protocol_option(self, arg):
"""A websocket server protocol option to configure: OPTION=VALUE. This option can be provided multiple times."""
try:
key, value = arg.split("=", 1)
except ValueError:
raise usage.UsageError("format options as OPTION=VALUE")
try:
value = json.loads(value)
except:
raise usage.UsageError(f"could not parse JSON value for {key}")
self["websocket-protocol-options"].append((key, value))
SECONDS = 1.0
MINUTE = 60*SECONDS
# CHANNEL_EXPIRATION_TIME should be longer than EXPIRATION_CHECK_PERIOD
CHANNEL_EXPIRATION_TIME = 11*MINUTE
EXPIRATION_CHECK_PERIOD = 5*MINUTE
def makeService(config, channel_db="relay.sqlite", reactor=reactor):
increase_rlimits()
parent = MultiService()
channel_db = create_or_upgrade_channel_db(config["channel-db"])
usage_db = create_or_upgrade_usage_db(config["usage-db"])
log_file = (os.fdopen(int(config["log-fd"]), "w")
if config["log-fd"] is not None
else None)
server = make_server(channel_db,
allow_list=config["allow-list"],
advertise_version=config["advertise-version"],
signal_error=config["signal-error"],
blur_usage=config["blur-usage"],
usage_db=usage_db,
log_file=log_file,
welcome_motd=config["motd"],
)
server.setServiceParent(parent)
rebooted = time.time()
def expire():
now = time.time()
old = now - CHANNEL_EXPIRATION_TIME
try:
server.prune_all_apps(now, old)
except Exception as e:
# catch-and-log exceptions during prune, so a single error won't
# kill the loop. See #13 for details.
log.msg("error during prune_all_apps")
log.err(e)
server.dump_stats(now, rebooted=rebooted)
TimerService(EXPIRATION_CHECK_PERIOD, expire).setServiceParent(parent)
log_requests = config["blur-usage"] is None
site = make_web_server(server, log_requests,
config["websocket-protocol-options"])
ep = endpoints.serverFromString(reactor, config["port"]) # to listen
StreamServerEndpointService(ep, site).setServiceParent(parent)
log.msg("websocket listening on ws://HOSTNAME:PORT/v1")
return parent
================================================
FILE: src/wormhole_mailbox_server/server_websocket.py
================================================
import time
from twisted.internet import reactor
from twisted.python import log
from autobahn.twisted import websocket
from .server import CrowdedError, ReclaimedError, SidedMessage, check_valid_nameplate
from .util import dict_to_bytes, bytes_to_dict
# The WebSocket allows the client to send "commands" to the server, and the
# server to send "responses" to the client. Note that commands and responses
# are not necessarily one-to-one. All commands provoke an "ack" response
# (with a copy of the original message) for timing, testing, and
# synchronization purposes. All commands and responses are JSON-encoded.
# Each WebSocket connection is bound to one "appid" and one "side", which are
# set by the "bind" command (which must be the first command on the
# connection), and must be set before any other command will be accepted.
# Each connection can be bound to a single "mailbox" (a two-sided
# store-and-forward queue, identified by the "mailbox id": a long, randomly
# unique string identifier) by using the "open" command. This protects the
# mailbox from idle closure, enables the "add" command (to put new messages
# in the queue), and triggers delivery of past and future messages via the
# "message" response. The "close" command removes the binding (but note that
# it does not enable the subsequent binding of a second mailbox). When the
# last side closes a mailbox, its contents are deleted.
# Additionally, the connection can be bound a single "nameplate", which is
# short identifier that makes up the first component of a wormhole code. Each
# nameplate points to a single long-id "mailbox". The "allocate" message
# determines the shortest available numeric nameplate, reserves it, and
# returns the nameplate id. "list" returns a list of all numeric nameplates
# which currently have only one side active (i.e. they are waiting for a
# partner). The "claim" message reserves an arbitrary nameplate id (perhaps
# the receiver of a wormhole connection typed in a code they got from the
# sender, or perhaps the two sides agreed upon a code offline and are both
# typing it in), and the "release" message releases it. When every side that
# has claimed the nameplate has also released it, the nameplate is
# deallocated (but they will probably keep the underlying mailbox open).
# "claim" and "release" may only be called once per connection, however calls
# across connections (assuming a consistent "side") are idempotent. [connect,
# claim, disconnect, connect, claim] is legal, but not useful, as is a
# "release" for a nameplate that nobody is currently claiming.
# "open" and "close" may only be called once per connection. They are
# basically idempotent, however "open" doubles as a subscribe action. So
# [connect, open, disconnect, connect, open] is legal *and* useful (without
# the second "open", the second connection would not be subscribed to hear
# about new messages).
# Inbound (client to server) commands are marked as "->" below. Unrecognized
# inbound keys will be ignored. Outbound (server to client) responses use
# "<-". There is no guaranteed correlation between requests and responses. In
# this list, "A -> B" means that some time after A is received, at least one
# message of type B will be sent out (probably).
# All responses include a "server_tx" key, which is a float (seconds since
# epoch) with the server clock just before the outbound response was written
# to the socket.
# connection -> welcome
# <- {type: "welcome", welcome: {}} # .welcome keys are all optional:
# current_cli_version: out-of-date clients display a warning
# motd: all clients display message, then continue normally
# error: all clients display mesage, then terminate with error
# -> {type: "bind", appid:, side:}
#
# -> {type: "list"} -> nameplates
# <- {type: "nameplates", nameplates: [{id: str,..},..]}
# -> {type: "allocate"} -> nameplate, mailbox
# <- {type: "allocated", nameplate: str}
# -> {type: "claim", nameplate: str} -> mailbox
# <- {type: "claimed", mailbox: str}
# -> {type: "release"}
# .nameplate is optional, but must match previous claim()
# <- {type: "released"}
#
# -> {type: "open", mailbox: str} -> message
# sends old messages now, and subscribes to deliver future messages
# <- {type: "message", side:, phase:, body:, msg_id:}} # body is hex
# -> {type: "add", phase: str, body: hex} # will send echo in a "message"
#
# -> {type: "close", mood: str} -> closed
# .mailbox is optional, but must match previous open()
# <- {type: "closed"}
#
# <- {type: "error", error: str, orig: {}} # in response to malformed msgs
# for tests that need to know when a message has been processed:
# -> {type: "ping", ping: int} -> pong (does not require bind/claim)
# <- {type: "pong", pong: int}
class Error(Exception):
def __init__(self, explain):
self._explain = explain
class WebSocketServer(websocket.WebSocketServerProtocol):
def __init__(self):
websocket.WebSocketServerProtocol.__init__(self)
self._app = None
self._side = None
self._did_allocate = False # only one allocate() per websocket
self._listening = False
self._did_claim = False
self._nameplate_id = None
self._did_release = False
self._did_open = False
self._mailbox = None
self._mailbox_id = None
self._did_close = False
self._peer_addr_port = None
def onConnect(self, request):
rv = self.factory._server
# Caddy uses capitalized headers like X-Real-IP and X-Real-Port, which
# you see if you forward Caddy to netcat. But the twisted/autobahn
# Request object lowercases everything.
#
# We only use this for assistance in NAT hole-punching, so it
# doesn't matter if the client is able to inject their own
# headers and spoof somebody else's IP address, they're only
# hurting themselves
if "x-real-ip" in request.headers:
peer_host = request.headers.get("x-real-ip") # either 1.2.3.4 or 2600:..:1234
peer_port = request.headers.get("x-real-port")
# assume frontends like Caddy don't give us v4-in-v6 addrs
peer_type = "ipv6" if ":" in peer_host else "ipv4"
else:
peer = request.peer
peer_type = peer.split(":", maxsplit=1)[0]
peer_port = peer.rsplit(":", maxsplit=1)[-1]
peer_host = peer[len(peer_type)+1:(-len(peer_port)-1)]
# this gets me [tcp6, ::1, 53276] for a client using ws://localhost:4000/v1
# or ws://[::1]:4000/v1
# and [tcp6, ::ffff:127.0.0.1, 53279] when using ws://127.0.0.1:4000/v1
if peer_type == "tcp6":
peer_type = "ipv6"
if peer_host.startswith("::ffff:"):
peer_host = peer_host.rsplit(":", maxsplit=1)[-1]
peer_type = "ipv4"
else:
peer_type = "ipv4"
self._peer_addr_port = (peer_type, peer_host, int(peer_port))
if rv.get_log_requests():
v = 4 if peer_type == "ipv4" else 6
log.msg(f"ws client connecting: tcp{v}:{peer_host}:{peer_port}")
self._reactor = self.factory.reactor
# can return (name, dict) or name here, where name is
# WebSocket subprotocol name and dict is extra headers (if
# provided) to send
def get_your_address(self):
(peer_type, peer_host, peer_port) = self._peer_addr_port
you = { "port": peer_port }
if peer_type == "ipv4":
you["ipv4"] = peer_host
elif peer_type == "ipv6":
you["ipv6"] = peer_host
return you
def onOpen(self):
rv = self.factory._server
welcome = rv.get_welcome().copy()
welcome["your-address"] = self.get_your_address()
self.send("welcome", welcome=welcome)
def onMessage(self, payload, isBinary):
server_rx = time.time()
msg = bytes_to_dict(payload)
try:
if "type" not in msg:
raise Error("missing 'type'")
self.send("ack", id=msg.get("id"))
mtype = msg["type"]
if mtype == "ping":
return self.handle_ping(msg)
if mtype == "bind":
return self.handle_bind(msg, server_rx)
if not self._app:
raise Error("must bind first")
if mtype == "list":
return self.handle_list()
if mtype == "allocate":
return self.handle_allocate(server_rx)
if mtype == "claim":
return self.handle_claim(msg, server_rx)
if mtype == "release":
return self.handle_release(msg, server_rx)
if mtype == "open":
return self.handle_open(msg, server_rx)
if mtype == "add":
return self.handle_add(msg, server_rx)
if mtype == "close":
return self.handle_close(msg, server_rx)
raise Error("unknown type")
except Error as e:
self.send("error", error=e._explain, orig=msg)
def handle_ping(self, msg):
if "ping" not in msg:
raise Error("ping requires 'ping'")
self.send("pong", pong=msg["ping"])
def handle_bind(self, msg, server_rx):
if self._app or self._side:
raise Error("already bound")
if "appid" not in msg:
raise Error("bind requires 'appid'")
if "side" not in msg:
raise Error("bind requires 'side'")
self._app = self.factory._server.get_app(msg["appid"])
self._side = msg["side"]
client_version = msg.get("client_version", (None, None))
# e.g. ("python", "0.xyz") . <=0.10.5 did not send client_version
self._app.log_client_version(server_rx, self._side, client_version)
def handle_list(self):
nameplate_ids = sorted(self._app.get_nameplate_ids())
# provide room to add nameplate attributes later (like which wordlist
# is used for each, maybe how many words)
nameplates = [{"id": nid} for nid in nameplate_ids]
self.send("nameplates", nameplates=nameplates)
def handle_allocate(self, server_rx):
if self._did_allocate:
raise Error("you already allocated one, don't be greedy")
nameplate_id = self._app.allocate_nameplate(self._side, server_rx)
assert isinstance(nameplate_id, str)
self._did_allocate = True
self.send("allocated", nameplate=nameplate_id)
def handle_claim(self, msg, server_rx):
if "nameplate" not in msg:
raise Error("claim requires 'nameplate'")
if self._did_claim:
raise Error("only one claim per connection")
self._did_claim = True
nameplate_id = msg["nameplate"]
check_valid_nameplate(nameplate_id)
self._nameplate_id = nameplate_id
try:
mailbox_id = self._app.claim_nameplate(nameplate_id, self._side,
server_rx)
except CrowdedError:
raise Error("crowded")
except ReclaimedError:
raise Error("reclaimed")
self.send("claimed", mailbox=mailbox_id)
def handle_release(self, msg, server_rx):
if self._did_release:
raise Error("only one release per connection")
if "nameplate" in msg:
if self._nameplate_id is not None:
# we only care about equality, don't bother with
# check_valid_nameplate()
if msg["nameplate"] != self._nameplate_id:
raise Error("release and claim must use same nameplate")
nameplate_id = msg["nameplate"]
else:
if self._nameplate_id is None:
raise Error("release without nameplate must follow claim")
nameplate_id = self._nameplate_id
assert nameplate_id is not None
self._did_release = True
self._app.release_nameplate(nameplate_id, self._side, server_rx)
self.send("released")
def handle_open(self, msg, server_rx):
if self._mailbox:
raise Error("only one open per connection")
if "mailbox" not in msg:
raise Error("open requires 'mailbox'")
mailbox_id = msg["mailbox"]
assert isinstance(mailbox_id, str)
self._mailbox_id = mailbox_id
try:
self._mailbox = self._app.open_mailbox(mailbox_id, self._side,
server_rx)
except CrowdedError:
raise Error("crowded")
def _send(sm):
self.send("message", side=sm.side, phase=sm.phase,
body=sm.body, server_rx=sm.server_rx, id=sm.msg_id)
def _stop():
pass
self._listening = True
for old_sm in self._mailbox.add_listener(self, _send, _stop):
_send(old_sm)
def handle_add(self, msg, server_rx):
if not self._mailbox:
raise Error("must open mailbox before adding")
if "phase" not in msg:
raise Error("missing 'phase'")
if "body" not in msg:
raise Error("missing 'body'")
msg_id = msg.get("id") # optional
sm = SidedMessage(side=self._side, phase=msg["phase"],
body=msg["body"], server_rx=server_rx,
msg_id=msg_id)
self._mailbox.add_message(sm)
def handle_close(self, msg, server_rx):
if self._did_close:
raise Error("only one close per connection")
if "mailbox" in msg:
if self._mailbox_id is not None:
if msg["mailbox"] != self._mailbox_id:
raise Error("open and close must use same mailbox")
mailbox_id = msg["mailbox"]
else:
if self._mailbox_id is None:
raise Error("close without mailbox must follow open")
mailbox_id = self._mailbox_id
if not self._mailbox:
try:
self._mailbox = self._app.open_mailbox(mailbox_id, self._side,
server_rx)
except CrowdedError:
raise Error("crowded")
if self._listening:
self._mailbox.remove_listener(self)
self._listening = False
self._did_close = True
self._mailbox.close(self._side, msg.get("mood"), server_rx)
self._mailbox = None
self.send("closed")
def send(self, mtype, **kwargs):
kwargs["type"] = mtype
kwargs["server_tx"] = time.time()
payload = dict_to_bytes(kwargs)
self.sendMessage(payload, False)
def onClose(self, wasClean, code, reason):
#log.msg("onClose", self, self._mailbox, self._listening)
if self._mailbox and self._listening:
self._mailbox.remove_listener(self)
class WebSocketServerFactory(websocket.WebSocketServerFactory):
protocol = WebSocketServer
def __init__(self, url, server):
websocket.WebSocketServerFactory.__init__(self, url)
self.setProtocolOptions(autoPingInterval=60, autoPingTimeout=600)
# note: Autobahn uses "self.factory.server" for the Server
# version string, so we musn't use that as well.
self._server = server
from . import __version__
self.server = f"Magic Wormhole Mailbox {__version__}"
self.reactor = reactor # for tests to control
================================================
FILE: src/wormhole_mailbox_server/test/__init__.py
================================================
================================================
FILE: src/wormhole_mailbox_server/test/common.py
================================================
#from __future__ import unicode_literals
from twisted.internet import reactor, endpoints
from twisted.internet.defer import inlineCallbacks
from ..database import create_or_upgrade_channel_db, create_or_upgrade_usage_db
from ..server import make_server
from ..web import make_web_server
class ServerBase:
log_requests = False
@inlineCallbacks
def setUp(self):
self._lp = None
if self.log_requests:
blur_usage = None
else:
blur_usage = 60.0
usage_db = create_or_upgrade_usage_db(":memory:")
yield self._setup_relay(blur_usage=blur_usage, usage_db=usage_db)
@inlineCallbacks
def _setup_relay(self, do_listen=False, web_log_requests=False, **kwargs):
channel_db = create_or_upgrade_channel_db(":memory:")
self._server = make_server(channel_db, **kwargs)
if do_listen:
ep = endpoints.TCP4ServerEndpoint(reactor, 0, interface="127.0.0.1")
self._site = make_web_server(self._server,
log_requests=web_log_requests)
self._lp = yield ep.listen(self._site)
addr = self._lp.getHost()
self.relayurl = "ws://127.0.0.1:%d/v1" % addr.port
self.rdv_ws_port = addr.port
def tearDown(self):
if self._lp:
return self._lp.stopListening()
class _Util:
def _nameplate(self, app, name):
np_row = app._db.execute("SELECT * FROM `nameplates`"
" WHERE `app_id`='appid' AND `name`=?",
(name,)).fetchone()
if not np_row:
return None, None
npid = np_row["id"]
side_rows = app._db.execute("SELECT * FROM `nameplate_sides`"
" WHERE `nameplates_id`=?",
(npid,)).fetchall()
return np_row, side_rows
def _mailbox(self, app, mailbox_id):
mb_row = app._db.execute("SELECT * FROM `mailboxes`"
" WHERE `app_id`='appid' AND `id`=?",
(mailbox_id,)).fetchone()
if not mb_row:
return None, None
side_rows = app._db.execute("SELECT * FROM `mailbox_sides`"
" WHERE `mailbox_id`=?",
(mailbox_id,)).fetchall()
return mb_row, side_rows
def _messages(self, app):
c = app._db.execute("SELECT * FROM `messages`"
" WHERE `app_id`='appid' AND `mailbox_id`='mid'")
return c.fetchall()
================================================
FILE: src/wormhole_mailbox_server/test/test_config.py
================================================
from twisted.python.usage import UsageError
from twisted.trial import unittest
from .. import server_tap
PORT = r"tcp:4000:interface=\:\:"
class Config(unittest.TestCase):
def test_defaults(self):
o = server_tap.Options()
o.parseOptions([])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
def test_advertise_version(self):
o = server_tap.Options()
o.parseOptions(["--advertise-version=1.0"])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": "1.0",
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
def test_blur(self):
o = server_tap.Options()
o.parseOptions(["--blur-usage=60"])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": 60,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
def test_channel_db(self):
o = server_tap.Options()
o.parseOptions(["--channel-db=other.sqlite"])
self.assertEqual(o, {"port": PORT,
"channel-db": "other.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
def test_disallow_list(self):
o = server_tap.Options()
o.parseOptions(["--disallow-list"])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": False,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
def test_log_fd(self):
o = server_tap.Options()
o.parseOptions(["--log-fd=5"])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": 5,
"websocket-protocol-options": [],
})
def test_port(self):
o = server_tap.Options()
o.parseOptions(["-p", "tcp:5555"])
self.assertEqual(o, {"port": "tcp:5555",
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
o = server_tap.Options()
o.parseOptions(["--port=tcp:5555"])
self.assertEqual(o, {"port": "tcp:5555",
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
def test_signal_error(self):
o = server_tap.Options()
o.parseOptions(["--signal-error=ohnoes"])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": "ohnoes",
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
def test_usage_db(self):
o = server_tap.Options()
o.parseOptions(["--usage-db=usage.sqlite"])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": "usage.sqlite",
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [],
})
def test_websocket_protocol_option_1(self):
o = server_tap.Options()
o.parseOptions(["--websocket-protocol-option", 'foo="bar"'])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [("foo", "bar")],
})
def test_websocket_protocol_option_2(self):
o = server_tap.Options()
o.parseOptions(["--websocket-protocol-option", 'foo="bar"',
"--websocket-protocol-option", 'baz=[1,"buz"]',
])
self.assertEqual(o, {"port": PORT,
"channel-db": "relay.sqlite",
"disallow-list": 0,
"allow-list": True,
"advertise-version": None,
"signal-error": None,
"usage-db": None,
"blur-usage": None,
"motd": None,
"log-fd": None,
"websocket-protocol-options": [("foo", "bar"),
("baz", [1, "buz"]),
],
})
def test_websocket_protocol_option_errors(self):
o = server_tap.Options()
with self.assertRaises(UsageError):
o.parseOptions(["--websocket-protocol-option", 'foo'])
with self.assertRaises(UsageError):
# I would be nice if this worked, but the 'bar' isn't JSON. To
# enable passing lists and more complicated things as values,
# simple string values must be passed with additional quotes
# (e.g. '"bar"')
o.parseOptions(["--websocket-protocol-option", 'foo=bar'])
def test_string(self):
o = server_tap.Options()
s = str(o)
self.assertIn("This plugin sets up a 'Mailbox' server", s)
self.assertIn("--blur-usage=", s)
self.assertIn("round logged access times to improve privacy", s)
================================================
FILE: src/wormhole_mailbox_server/test/test_database.py
================================================
import os
from twisted.python import filepath
from twisted.trial import unittest
from .. import database
from ..database import (CHANNELDB_TARGET_VERSION, USAGEDB_TARGET_VERSION,
_get_db, dump_db, DBError)
class Get(unittest.TestCase):
def test_create_default(self):
db_url = ":memory:"
db = _get_db(db_url, "channel", CHANNELDB_TARGET_VERSION)
rows = db.execute("SELECT * FROM version").fetchall()
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["version"], CHANNELDB_TARGET_VERSION)
def test_open_existing_file(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "normal.db")
db = _get_db(fn, "channel", CHANNELDB_TARGET_VERSION)
rows = db.execute("SELECT * FROM version").fetchall()
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["version"], CHANNELDB_TARGET_VERSION)
db2 = _get_db(fn, "channel", CHANNELDB_TARGET_VERSION)
rows = db2.execute("SELECT * FROM version").fetchall()
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["version"], CHANNELDB_TARGET_VERSION)
def test_open_bad_version(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "old.db")
db = _get_db(fn, "channel", CHANNELDB_TARGET_VERSION)
db.execute("UPDATE version SET version=999")
db.commit()
with self.assertRaises(DBError) as e:
_get_db(fn, "channel", CHANNELDB_TARGET_VERSION)
self.assertIn("Unable to handle db version 999", str(e.exception))
def test_open_corrupt(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "corrupt.db")
with open(fn, "wb") as f:
f.write(b"I am not a database")
with self.assertRaises(DBError) as e:
_get_db(fn, "channel", CHANNELDB_TARGET_VERSION)
self.assertIn("not a database", str(e.exception))
def test_failed_create_allows_subsequent_create(self):
patch = self.patch(database, "get_schema", lambda version: b"this is a broken schema")
dbfile = filepath.FilePath(self.mktemp())
self.assertRaises(Exception, lambda: _get_db(dbfile.path))
patch.restore()
_get_db(dbfile.path, "channel", CHANNELDB_TARGET_VERSION)
def test_upgrade(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "upgrade.db")
self.assertNotEqual(USAGEDB_TARGET_VERSION, 1)
# create an old-version DB in a file
db = _get_db(fn, "usage", 1)
rows = db.execute("SELECT * FROM version").fetchall()
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["version"], 1)
del db
# then upgrade the file to the latest version
dbA = _get_db(fn, "usage", USAGEDB_TARGET_VERSION)
rows = dbA.execute("SELECT * FROM version").fetchall()
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["version"], USAGEDB_TARGET_VERSION)
dbA_text = dump_db(dbA)
del dbA
# make sure the upgrades got committed to disk
dbB = _get_db(fn, "usage", USAGEDB_TARGET_VERSION)
dbB_text = dump_db(dbB)
del dbB
self.assertEqual(dbA_text, dbB_text)
# The upgraded schema should be equivalent to that of a new DB.
latest_db = _get_db(":memory:", "usage", USAGEDB_TARGET_VERSION)
latest_text = dump_db(latest_db)
with open("up.sql","w") as f: f.write(dbA_text)
with open("new.sql","w") as f: f.write(latest_text)
# debug with "diff -u _trial_temp/up.sql _trial_temp/new.sql"
self.assertEqual(dbA_text, latest_text)
def test_upgrade_fails(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "upgrade.db")
self.assertNotEqual(USAGEDB_TARGET_VERSION, 1)
# create an old-version DB in a file
db = _get_db(fn, "usage", 1)
rows = db.execute("SELECT * FROM version").fetchall()
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["version"], 1)
del db
# then upgrade the file to a too-new version, for which we have no
# upgrader
with self.assertRaises(DBError):
_get_db(fn, "usage", USAGEDB_TARGET_VERSION+1)
class CreateChannel(unittest.TestCase):
def test_memory(self):
db = database.create_channel_db(":memory:")
latest_text = dump_db(db)
self.assertIn("CREATE TABLE", latest_text)
def test_preexisting(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "preexisting.db")
with open(fn, "w"):
pass
with self.assertRaises(database.DBAlreadyExists):
database.create_channel_db(fn)
def test_create(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "created.db")
db = database.create_channel_db(fn)
latest_text = dump_db(db)
self.assertIn("CREATE TABLE", latest_text)
def test_create_or_upgrade(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "created.db")
db = database.create_or_upgrade_channel_db(fn)
latest_text = dump_db(db)
self.assertIn("CREATE TABLE", latest_text)
class CreateUsage(unittest.TestCase):
def test_memory(self):
db = database.create_usage_db(":memory:")
latest_text = dump_db(db)
self.assertIn("CREATE TABLE", latest_text)
def test_preexisting(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "preexisting.db")
with open(fn, "w"):
pass
with self.assertRaises(database.DBAlreadyExists):
database.create_usage_db(fn)
def test_create(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "created.db")
db = database.create_usage_db(fn)
latest_text = dump_db(db)
self.assertIn("CREATE TABLE", latest_text)
def test_create_or_upgrade(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "created.db")
db = database.create_or_upgrade_usage_db(fn)
latest_text = dump_db(db)
self.assertIn("CREATE TABLE", latest_text)
def test_create_or_upgrade_disabled(self):
db = database.create_or_upgrade_usage_db(None)
self.assertIs(db, None)
class OpenChannel(unittest.TestCase):
def test_open(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "created.db")
db1 = database.create_channel_db(fn)
latest_text = dump_db(db1)
self.assertIn("CREATE TABLE", latest_text)
db2 = database.open_existing_db(fn)
self.assertIn("CREATE TABLE", dump_db(db2))
def test_doesnt_exist(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "created.db")
with self.assertRaises(database.DBDoesntExist):
database.open_existing_db(fn)
class OpenUsage(unittest.TestCase):
def test_open(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "created.db")
db1 = database.create_usage_db(fn)
latest_text = dump_db(db1)
self.assertIn("CREATE TABLE", latest_text)
db2 = database.open_existing_db(fn)
self.assertIn("CREATE TABLE", dump_db(db2))
def test_doesnt_exist(self):
basedir = self.mktemp()
os.mkdir(basedir)
fn = os.path.join(basedir, "created.db")
with self.assertRaises(database.DBDoesntExist):
database.open_existing_db(fn)
================================================
FILE: src/wormhole_mailbox_server/test/test_rlimits.py
================================================
from unittest import mock
from twisted.trial import unittest
from ..increase_rlimits import increase_rlimits
class RLimits(unittest.TestCase):
def test_rlimit(self):
def patch_r(name, *args, **kwargs):
return mock.patch("wormhole_mailbox_server.increase_rlimits." + name, *args, **kwargs)
fakelog = []
def checklog(*expected):
self.assertEqual(fakelog, list(expected))
fakelog[:] = []
NF = "NOFILE"
mock_NF = patch_r("RLIMIT_NOFILE", NF)
with patch_r("log.msg", fakelog.append):
with patch_r("getrlimit", None):
increase_rlimits()
checklog("unable to import 'resource', leaving rlimit alone")
with mock_NF:
with patch_r("getrlimit", return_value=(20000, 30000)) as gr:
increase_rlimits()
self.assertEqual(gr.mock_calls, [mock.call(NF)])
checklog("RLIMIT_NOFILE.soft was 20000, leaving it alone")
with patch_r("getrlimit", return_value=(10, 30000)) as gr:
with patch_r("setrlimit", side_effect=TypeError("other")):
with patch_r("log.err") as err:
increase_rlimits()
self.assertEqual(err.mock_calls, [mock.call()])
checklog("changing RLIMIT_NOFILE from (10,30000) to (30000,30000)",
"other error during setrlimit, leaving it alone")
for maxlimit in [40000, 20000, 9000, 2000, 1000]:
def setrlimit(which, newlimit):
if newlimit[0] > maxlimit:
raise ValueError("nope")
return None
calls = []
expected = []
for tries in [30000, 10000, 3200, 1024]:
calls.append(mock.call(NF, (tries, 30000)))
expected.append("changing RLIMIT_NOFILE from (10,30000) to (%d,30000)" % tries)
if tries > maxlimit:
expected.append("error during setrlimit: nope")
else:
expected.append("setrlimit successful")
break
else:
expected.append("unable to change rlimit, leaving it alone")
with patch_r("setrlimit", side_effect=setrlimit) as sr:
increase_rlimits()
self.assertEqual(sr.mock_calls, calls)
checklog(*expected)
================================================
FILE: src/wormhole_mailbox_server/test/test_server.py
================================================
from unittest import mock
from twisted.trial import unittest
from twisted.python import log
from .common import ServerBase, _Util
from ..server import (make_server, Usage,
SidedMessage, CrowdedError, AppNamespace)
from ..database import create_channel_db, create_usage_db
npid = "1"
class Server(_Util, ServerBase, unittest.TestCase):
def test_apps(self):
app1 = self._server.get_app("appid1")
self.assertIdentical(app1, self._server.get_app("appid1"))
app2 = self._server.get_app("appid2")
self.assertNotIdentical(app1, app2)
def test_nameplate_allocation(self):
app = self._server.get_app("appid")
nids = set()
# this takes a second, and claims all the short-numbered nameplates
def add():
nameplate_id = app.allocate_nameplate("side1", 0)
self.assertEqual(type(nameplate_id), str)
nid = int(nameplate_id)
nids.add(nid)
for i in range(9): add()
self.assertNotIn(0, nids)
self.assertEqual(set(range(1,10)), nids)
for i in range(100-10): add()
self.assertEqual(len(nids), 99)
self.assertEqual(set(range(1,100)), nids)
for i in range(1000-100): add()
self.assertEqual(len(nids), 999)
self.assertEqual(set(range(1,1000)), nids)
add()
self.assertEqual(len(nids), 1000)
biggest = max(nids)
self.assertTrue(1000 <= biggest < 1000000, biggest)
def test_nameplate_allocation_failure(self):
app = self._server.get_app("appid")
# pretend to fill all 1M <7-digit nameplates, it should give up
# eventually
def _get_nameplate_ids():
return {"%d" % id_int for id_int in range(1, 1000*1000)}
app._get_nameplate_ids = _get_nameplate_ids
with self.assertRaises(ValueError) as e:
app.allocate_nameplate("side1", 0)
self.assertIn("unable to find a free nameplate-id", str(e.exception))
def test_nameplate(self):
app = self._server.get_app("appid")
name = app.allocate_nameplate("side1", 0)
self.assertEqual(type(name), str)
nid = int(name)
self.assertTrue(0 < nid < 10, nid)
self.assertEqual(app.get_nameplate_ids(), {name})
# allocate also does a claim
np_row, side_rows = self._nameplate(app, name)
self.assertEqual(len(side_rows), 1)
self.assertEqual(side_rows[0]["side"], "side1")
self.assertEqual(side_rows[0]["added"], 0)
# duplicate claims by the same side are combined
mailbox_id = app.claim_nameplate(name, "side1", 1)
self.assertEqual(type(mailbox_id), str)
self.assertEqual(mailbox_id, np_row["mailbox_id"])
np_row, side_rows = self._nameplate(app, name)
self.assertEqual(len(side_rows), 1)
self.assertEqual(side_rows[0]["added"], 0)
self.assertEqual(mailbox_id, np_row["mailbox_id"])
# and they don't updated the 'added' time
mailbox_id2 = app.claim_nameplate(name, "side1", 2)
self.assertEqual(mailbox_id, mailbox_id2)
np_row, side_rows = self._nameplate(app, name)
self.assertEqual(len(side_rows), 1)
self.assertEqual(side_rows[0]["added"], 0)
# claim by the second side is new
mailbox_id3 = app.claim_nameplate(name, "side2", 3)
self.assertEqual(mailbox_id, mailbox_id3)
np_row, side_rows = self._nameplate(app, name)
self.assertEqual(len(side_rows), 2)
self.assertEqual(sorted([row["side"] for row in side_rows]),
sorted(["side1", "side2"]))
self.assertIn(("side2", 3),
[(row["side"], row["added"]) for row in side_rows])
# a third claim marks the nameplate as "crowded", and adds a third
# claim (which must be released later), but leaves the two existing
# claims alone
self.assertRaises(CrowdedError,
app.claim_nameplate, name, "side3", 4)
np_row, side_rows = self._nameplate(app, name)
self.assertEqual(len(side_rows), 3)
# releasing a non-existent nameplate is ignored
app.release_nameplate(name+"not", "side4", 0)
# releasing a side that never claimed the nameplate is ignored
app.release_nameplate(name, "side4", 0)
np_row, side_rows = self._nameplate(app, name)
self.assertEqual(len(side_rows), 3)
# releasing one side leaves the second claim
app.release_nameplate(name, "side1", 5)
np_row, side_rows = self._nameplate(app, name)
claims = [(row["side"], row["claimed"]) for row in side_rows]
self.assertIn(("side1", False), claims)
self.assertIn(("side2", True), claims)
self.assertIn(("side3", True), claims)
# releasing one side multiple times is ignored
app.release_nameplate(name, "side1", 5)
np_row, side_rows = self._nameplate(app, name)
claims = [(row["side"], row["claimed"]) for row in side_rows]
self.assertIn(("side1", False), claims)
self.assertIn(("side2", True), claims)
self.assertIn(("side3", True), claims)
# release the second side
app.release_nameplate(name, "side2", 6)
np_row, side_rows = self._nameplate(app, name)
claims = [(row["side"], row["claimed"]) for row in side_rows]
self.assertIn(("side1", False), claims)
self.assertIn(("side2", False), claims)
self.assertIn(("side3", True), claims)
# releasing the third side frees the nameplate, and adds usage
app.release_nameplate(name, "side3", 7)
np_row, side_rows = self._nameplate(app, name)
self.assertEqual(np_row, None)
usage = app._usage_db.execute("SELECT * FROM `nameplates`").fetchone()
self.assertEqual(usage["app_id"], "appid")
self.assertEqual(usage["started"], 0)
self.assertEqual(usage["waiting_time"], 3)
self.assertEqual(usage["total_time"], 7)
self.assertEqual(usage["result"], "crowded")
def test_mailbox(self):
app = self._server.get_app("appid")
mailbox_id = "mid"
m1 = app.open_mailbox(mailbox_id, "side1", 0)
mb_row, side_rows = self._mailbox(app, mailbox_id)
self.assertEqual(len(side_rows), 1)
self.assertEqual(side_rows[0]["side"], "side1")
self.assertEqual(side_rows[0]["added"], 0)
# opening the same mailbox twice, by the same side, gets the same
# object, and does not update the "added" timestamp
self.assertIdentical(m1, app.open_mailbox(mailbox_id, "side1", 1))
mb_row, side_rows = self._mailbox(app, mailbox_id)
self.assertEqual(len(side_rows), 1)
self.assertEqual(side_rows[0]["side"], "side1")
self.assertEqual(side_rows[0]["added"], 0)
# opening a second side gets the same object, and adds a new claim
self.assertIdentical(m1, app.open_mailbox(mailbox_id, "side2", 2))
mb_row, side_rows = self._mailbox(app, mailbox_id)
self.assertEqual(len(side_rows), 2)
adds = [(row["side"], row["added"]) for row in side_rows]
self.assertIn(("side1", 0), adds)
self.assertIn(("side2", 2), adds)
# a third open marks it as crowded
self.assertRaises(CrowdedError,
app.open_mailbox, mailbox_id, "side3", 3)
mb_row, side_rows = self._mailbox(app, mailbox_id)
self.assertEqual(len(side_rows), 3)
m1.close("side3", "company", 4)
# closing a side that never claimed the mailbox is ignored
m1.close("side4", "mood", 4)
mb_row, side_rows = self._mailbox(app, mailbox_id)
self.assertEqual(len(side_rows), 3)
# closing one side leaves the second claim
m1.close("side1", "mood", 5)
mb_row, side_rows = self._mailbox(app, mailbox_id)
sides = [(row["side"], row["opened"], row["mood"]) for row in side_rows]
self.assertIn(("side1", False, "mood"), sides)
self.assertIn(("side2", True, None), sides)
self.assertIn(("side3", False, "company"), sides)
# closing one side multiple times is ignored
m1.close("side1", "mood", 6)
mb_row, side_rows = self._mailbox(app, mailbox_id)
sides = [(row["side"], row["opened"], row["mood"]) for row in side_rows]
self.assertIn(("side1", False, "mood"), sides)
self.assertIn(("side2", True, None), sides)
self.assertIn(("side3", False, "company"), sides)
l1 = []; stop1 = []; stop1_f = lambda: stop1.append(True)
m1.add_listener("handle1", l1.append, stop1_f)
# closing the seco
gitextract_ojb3q6ak/ ├── .appveyor.yml ├── .coveragerc ├── .gitattributes ├── .github/ │ └── workflows/ │ └── test.yml ├── .gitignore ├── .travis.yml ├── LICENSE ├── MANIFEST.in ├── Makefile ├── NEWS.md ├── README.md ├── docs/ │ ├── Makefile │ ├── conf.py │ ├── happy-plant.seq │ ├── happy.seq │ ├── index.rst │ ├── server-protocol.md │ ├── states.dot │ └── welcome.md ├── misc/ │ ├── migrate_channel_db.py │ ├── migrate_usage_db.py │ ├── munin/ │ │ ├── wormhole_active │ │ ├── wormhole_errors │ │ ├── wormhole_event_rate │ │ ├── wormhole_events │ │ ├── wormhole_events_alltime │ │ └── wormhole_version_uptake │ └── windows-build.cmd ├── newest-version.py ├── setup.cfg ├── setup.py ├── signatures/ │ ├── magic-wormhole-mailbox-server-0.5.0.tar.gz.asc │ ├── magic-wormhole-mailbox-server-0.5.1.tar.gz.asc │ ├── magic_wormhole_mailbox_server-0.5.0-py3-none-any.whl.asc │ ├── magic_wormhole_mailbox_server-0.5.1-py3-none-any.whl.asc │ ├── magic_wormhole_mailbox_server-0.6.0-py3-none-any.whl.asc │ ├── magic_wormhole_mailbox_server-0.6.0.tar.gz.asc │ ├── magic_wormhole_mailbox_server-0.7.0-py3-none-any.whl.asc │ ├── magic_wormhole_mailbox_server-0.7.0.tar.gz.asc │ ├── magic_wormhole_mailbox_server-0.8.0-py3-none-any.whl.asc │ └── magic_wormhole_mailbox_server-0.8.0.tar.gz.asc ├── src/ │ ├── twisted/ │ │ └── plugins/ │ │ └── magic_wormhole_mailbox.py │ └── wormhole_mailbox_server/ │ ├── __init__.py │ ├── _version.py │ ├── database.py │ ├── db-schemas/ │ │ ├── channel-v1.sql │ │ ├── upgrade-usage-to-v2.sql │ │ ├── usage-v1.sql │ │ └── usage-v2.sql │ ├── increase_rlimits.py │ ├── server.py │ ├── server_tap.py │ ├── server_websocket.py │ ├── test/ │ │ ├── __init__.py │ │ ├── common.py │ │ ├── test_config.py │ │ ├── test_database.py │ │ ├── test_rlimits.py │ │ ├── test_server.py │ │ ├── test_service.py │ │ ├── test_stats.py │ │ ├── test_util.py │ │ ├── test_web.py │ │ ├── test_websocket.py │ │ ├── test_ws_client.py │ │ └── ws_client.py │ ├── util.py │ └── web.py ├── tox.ini ├── update-version.py └── versioneer.py
SYMBOL INDEX (381 symbols across 28 files)
FILE: docs/conf.py
function _get_versions (line 59) | def _get_versions():
FILE: newest-version.py
function existing_tags (line 9) | def existing_tags(git):
function main (line 17) | def main():
FILE: src/wormhole_mailbox_server/_version.py
function get_keywords (line 22) | def get_keywords() -> dict[str, str]:
class VersioneerConfig (line 35) | class VersioneerConfig:
function get_config (line 46) | def get_config() -> VersioneerConfig:
class NotThisMethod (line 60) | class NotThisMethod(Exception):
function register_vcs_handler (line 68) | def register_vcs_handler(vcs: str, method: str) -> Callable: # decorator
function run_command (line 79) | def run_command(
function versions_from_parentdir (line 127) | def versions_from_parentdir(
function git_get_keywords (line 156) | def git_get_keywords(versionfile_abs: str) -> dict[str, str]:
function git_versions_from_keywords (line 184) | def git_versions_from_keywords(
function git_pieces_from_vcs (line 252) | def git_pieces_from_vcs(
function plus_or_dot (line 388) | def plus_or_dot(pieces: dict[str, Any]) -> str:
function render_pep440 (line 395) | def render_pep440(pieces: dict[str, Any]) -> str:
function render_pep440_branch (line 420) | def render_pep440_branch(pieces: dict[str, Any]) -> str:
function pep440_split_post (line 450) | def pep440_split_post(ver: str) -> tuple[str, Optional[int]]:
function render_pep440_pre (line 460) | def render_pep440_pre(pieces: dict[str, Any]) -> str:
function render_pep440_post (line 484) | def render_pep440_post(pieces: dict[str, Any]) -> str:
function render_pep440_post_branch (line 511) | def render_pep440_post_branch(pieces: dict[str, Any]) -> str:
function render_pep440_old (line 540) | def render_pep440_old(pieces: dict[str, Any]) -> str:
function render_git_describe (line 562) | def render_git_describe(pieces: dict[str, Any]) -> str:
function render_git_describe_long (line 582) | def render_git_describe_long(pieces: dict[str, Any]) -> str:
function render (line 602) | def render(pieces: dict[str, Any], style: str) -> dict[str, Any]:
function get_versions (line 638) | def get_versions() -> dict[str, Any]:
FILE: src/wormhole_mailbox_server/database.py
class DBError (line 8) | class DBError(Exception):
function get_schema (line 11) | def get_schema(name, version):
function get_upgrader (line 16) | def get_upgrader(name, new_version):
function dict_factory (line 28) | def dict_factory(cursor, row):
function _initialize_db_schema (line 34) | def _initialize_db_schema(db, name, target_version):
function _initialize_db_connection (line 44) | def _initialize_db_connection(db):
function _open_db_connection (line 54) | def _open_db_connection(dbfile):
function _get_temporary_dbfile (line 66) | def _get_temporary_dbfile(dbfile):
function _atomic_create_and_initialize_db (line 76) | def _atomic_create_and_initialize_db(dbfile, name, target_version):
function _get_db (line 89) | def _get_db(dbfile, name, target_version):
function create_or_upgrade_channel_db (line 126) | def create_or_upgrade_channel_db(dbfile):
function create_or_upgrade_usage_db (line 129) | def create_or_upgrade_usage_db(dbfile):
class DBDoesntExist (line 134) | class DBDoesntExist(Exception):
function open_existing_db (line 137) | def open_existing_db(dbfile):
class DBAlreadyExists (line 143) | class DBAlreadyExists(Exception):
function create_channel_db (line 146) | def create_channel_db(dbfile):
function create_usage_db (line 161) | def create_usage_db(dbfile):
function dump_db (line 172) | def dump_db(db):
FILE: src/wormhole_mailbox_server/db-schemas/channel-v1.sql
type `version` (line 5) | CREATE TABLE `version`
type `nameplates` (line 14) | CREATE TABLE `nameplates`
type `nameplates_idx` (line 22) | CREATE INDEX `nameplates_idx` ON `nameplates` (`app_id`, `name`)
type `nameplates_mailbox_idx` (line 23) | CREATE INDEX `nameplates_mailbox_idx` ON `nameplates` (`app_id`, `mailbo...
type `nameplates_request_idx` (line 24) | CREATE INDEX `nameplates_request_idx` ON `nameplates` (`app_id`, `reques...
type `nameplate_sides` (line 26) | CREATE TABLE `nameplate_sides`
type `mailboxes` (line 38) | CREATE TABLE `mailboxes`
type `mailboxes_idx` (line 45) | CREATE INDEX `mailboxes_idx` ON `mailboxes` (`app_id`, `id`)
type `mailbox_sides` (line 47) | CREATE TABLE `mailbox_sides`
type `messages` (line 56) | CREATE TABLE `messages`
type `messages_idx` (line 66) | CREATE INDEX `messages_idx` ON `messages` (`app_id`, `mailbox_id`)
FILE: src/wormhole_mailbox_server/db-schemas/upgrade-usage-to-v2.sql
type `client_versions` (line 1) | CREATE TABLE `client_versions`
type `client_versions_time_idx` (line 11) | CREATE INDEX `client_versions_time_idx` on `client_versions` (`connect_t...
type `client_versions_appid_time_idx` (line 12) | CREATE INDEX `client_versions_appid_time_idx` on `client_versions` (`app...
FILE: src/wormhole_mailbox_server/db-schemas/usage-v1.sql
type `version` (line 1) | CREATE TABLE `version`
type `current` (line 6) | CREATE TABLE `current`
type `nameplates` (line 15) | CREATE TABLE `nameplates`
type `nameplates_idx` (line 28) | CREATE INDEX `nameplates_idx` ON `nameplates` (`app_id`, `started`)
type `mailboxes` (line 31) | CREATE TABLE `mailboxes`
type `mailboxes_idx` (line 47) | CREATE INDEX `mailboxes_idx` ON `mailboxes` (`app_id`, `started`)
type `mailboxes_result_idx` (line 48) | CREATE INDEX `mailboxes_result_idx` ON `mailboxes` (`result`)
FILE: src/wormhole_mailbox_server/db-schemas/usage-v2.sql
type `version` (line 1) | CREATE TABLE `version`
type `current` (line 6) | CREATE TABLE `current`
type `nameplates` (line 15) | CREATE TABLE `nameplates`
type `nameplates_idx` (line 28) | CREATE INDEX `nameplates_idx` ON `nameplates` (`app_id`, `started`)
type `mailboxes` (line 31) | CREATE TABLE `mailboxes`
type `mailboxes_idx` (line 47) | CREATE INDEX `mailboxes_idx` ON `mailboxes` (`app_id`, `started`)
type `mailboxes_result_idx` (line 48) | CREATE INDEX `mailboxes_result_idx` ON `mailboxes` (`result`)
type `client_versions` (line 50) | CREATE TABLE `client_versions`
type `client_versions_time_idx` (line 60) | CREATE INDEX `client_versions_time_idx` on `client_versions` (`connect_t...
type `client_versions_appid_time_idx` (line 61) | CREATE INDEX `client_versions_appid_time_idx` on `client_versions` (`app...
FILE: src/wormhole_mailbox_server/increase_rlimits.py
function increase_rlimits (line 8) | def increase_rlimits():
FILE: src/wormhole_mailbox_server/server.py
function generate_mailbox_id (line 6) | def generate_mailbox_id():
function check_valid_nameplate (line 11) | def check_valid_nameplate(n):
class CrowdedError (line 19) | class CrowdedError(Exception):
class ReclaimedError (line 21) | class ReclaimedError(Exception):
class Mailbox (line 32) | class Mailbox:
method __init__ (line 33) | def __init__(self, app, db, usage_db, app_id, mailbox_id):
method open (line 43) | def open(self, side, when):
method _touch (line 73) | def _touch(self, when):
method get_messages (line 77) | def get_messages(self):
method add_listener (line 90) | def add_listener(self, handle, send_f, stop_f):
method remove_listener (line 96) | def remove_listener(self, handle):
method has_listeners (line 101) | def has_listeners(self):
method count_listeners (line 104) | def count_listeners(self):
method broadcast_message (line 107) | def broadcast_message(self, sm):
method _add_message (line 111) | def _add_message(self, sm):
method add_message (line 121) | def add_message(self, sm):
method close (line 126) | def close(self, side, mood, when):
method _shutdown (line 180) | def _shutdown(self):
class AppNamespace (line 187) | class AppNamespace:
method __init__ (line 189) | def __init__(self, db, usage_db, blur_usage, log_requests, app_id,
method log_client_version (line 199) | def log_client_version(self, server_rx, side, client_version):
method get_nameplate_ids (line 213) | def get_nameplate_ids(self):
method _get_nameplate_ids (line 218) | def _get_nameplate_ids(self):
method _find_available_nameplate_id (line 225) | def _find_available_nameplate_id(self):
method allocate_nameplate (line 243) | def allocate_nameplate(self, side, when):
method claim_nameplate (line 249) | def claim_nameplate(self, name, side, when):
method release_nameplate (line 299) | def release_nameplate(self, name, side, when):
method _summarize_nameplate_and_store (line 341) | def _summarize_nameplate_and_store(self, side_rows, delete_time, pruned):
method _summarize_nameplate_usage (line 351) | def _summarize_nameplate_usage(self, side_rows, delete_time, pruned):
method _add_mailbox (line 370) | def _add_mailbox(self, mailbox_id, for_nameplate, side, when):
method open_mailbox (line 384) | def open_mailbox(self, mailbox_id, side, when):
method free_mailbox (line 407) | def free_mailbox(self, mailbox_id):
method _summarize_mailbox_and_store (line 417) | def _summarize_mailbox_and_store(self, for_nameplate, side_rows,
method _summarize_mailbox (line 428) | def _summarize_mailbox(self, side_rows, delete_time, pruned):
method prune (line 462) | def prune(self, now, old):
method count_listeners (line 552) | def count_listeners(self):
method _shutdown (line 556) | def _shutdown(self):
class Server (line 561) | class Server(service.MultiService):
method __init__ (line 562) | def __init__(self, db, allow_list, welcome,
method get_welcome (line 574) | def get_welcome(self):
method get_log_requests (line 576) | def get_log_requests(self):
method get_app (line 579) | def get_app(self, app_id):
method get_all_apps (line 594) | def get_all_apps(self):
method prune_all_apps (line 607) | def prune_all_apps(self, now, old):
method dump_stats (line 618) | def dump_stats(self, now, rebooted):
method startService (line 664) | def startService(self):
method stopService (line 675) | def stopService(self):
function make_server (line 686) | def make_server(db, allow_list=True,
FILE: src/wormhole_mailbox_server/server_tap.py
class Options (line 17) | class Options(usage.Options):
method __init__ (line 35) | def __init__(self):
method opt_disallow_list (line 40) | def opt_disallow_list(self):
method opt_log_fd (line 43) | def opt_log_fd(self, arg):
method opt_blur_usage (line 46) | def opt_blur_usage(self, arg):
method opt_websocket_protocol_option (line 51) | def opt_websocket_protocol_option(self, arg):
function makeService (line 71) | def makeService(config, channel_db="relay.sqlite", reactor=reactor):
FILE: src/wormhole_mailbox_server/server_websocket.py
class Error (line 93) | class Error(Exception):
method __init__ (line 94) | def __init__(self, explain):
class WebSocketServer (line 97) | class WebSocketServer(websocket.WebSocketServerProtocol):
method __init__ (line 98) | def __init__(self):
method onConnect (line 113) | def onConnect(self, request):
method get_your_address (line 153) | def get_your_address(self):
method onOpen (line 162) | def onOpen(self):
method onMessage (line 168) | def onMessage(self, payload, isBinary):
method handle_ping (line 204) | def handle_ping(self, msg):
method handle_bind (line 209) | def handle_bind(self, msg, server_rx):
method handle_list (line 223) | def handle_list(self):
method handle_allocate (line 230) | def handle_allocate(self, server_rx):
method handle_claim (line 238) | def handle_claim(self, msg, server_rx):
method handle_release (line 256) | def handle_release(self, msg, server_rx):
method handle_open (line 276) | def handle_open(self, msg, server_rx):
method handle_add (line 298) | def handle_add(self, msg, server_rx):
method handle_close (line 311) | def handle_close(self, msg, server_rx):
method send (line 337) | def send(self, mtype, **kwargs):
method onClose (line 343) | def onClose(self, wasClean, code, reason):
class WebSocketServerFactory (line 349) | class WebSocketServerFactory(websocket.WebSocketServerFactory):
method __init__ (line 352) | def __init__(self, url, server):
FILE: src/wormhole_mailbox_server/test/common.py
class ServerBase (line 8) | class ServerBase:
method setUp (line 12) | def setUp(self):
method _setup_relay (line 22) | def _setup_relay(self, do_listen=False, web_log_requests=False, **kwar...
method tearDown (line 34) | def tearDown(self):
class _Util (line 38) | class _Util:
method _nameplate (line 39) | def _nameplate(self, app, name):
method _mailbox (line 51) | def _mailbox(self, app, mailbox_id):
method _messages (line 62) | def _messages(self, app):
FILE: src/wormhole_mailbox_server/test/test_config.py
class Config (line 7) | class Config(unittest.TestCase):
method test_defaults (line 8) | def test_defaults(self):
method test_advertise_version (line 24) | def test_advertise_version(self):
method test_blur (line 40) | def test_blur(self):
method test_channel_db (line 56) | def test_channel_db(self):
method test_disallow_list (line 72) | def test_disallow_list(self):
method test_log_fd (line 88) | def test_log_fd(self):
method test_port (line 104) | def test_port(self):
method test_signal_error (line 135) | def test_signal_error(self):
method test_usage_db (line 151) | def test_usage_db(self):
method test_websocket_protocol_option_1 (line 167) | def test_websocket_protocol_option_1(self):
method test_websocket_protocol_option_2 (line 183) | def test_websocket_protocol_option_2(self):
method test_websocket_protocol_option_errors (line 203) | def test_websocket_protocol_option_errors(self):
method test_string (line 214) | def test_string(self):
FILE: src/wormhole_mailbox_server/test/test_database.py
class Get (line 8) | class Get(unittest.TestCase):
method test_create_default (line 9) | def test_create_default(self):
method test_open_existing_file (line 16) | def test_open_existing_file(self):
method test_open_bad_version (line 29) | def test_open_bad_version(self):
method test_open_corrupt (line 41) | def test_open_corrupt(self):
method test_failed_create_allows_subsequent_create (line 51) | def test_failed_create_allows_subsequent_create(self):
method test_upgrade (line 58) | def test_upgrade(self):
method test_upgrade_fails (line 93) | def test_upgrade_fails(self):
class CreateChannel (line 111) | class CreateChannel(unittest.TestCase):
method test_memory (line 112) | def test_memory(self):
method test_preexisting (line 117) | def test_preexisting(self):
method test_create (line 126) | def test_create(self):
method test_create_or_upgrade (line 134) | def test_create_or_upgrade(self):
class CreateUsage (line 142) | class CreateUsage(unittest.TestCase):
method test_memory (line 143) | def test_memory(self):
method test_preexisting (line 148) | def test_preexisting(self):
method test_create (line 157) | def test_create(self):
method test_create_or_upgrade (line 165) | def test_create_or_upgrade(self):
method test_create_or_upgrade_disabled (line 173) | def test_create_or_upgrade_disabled(self):
class OpenChannel (line 177) | class OpenChannel(unittest.TestCase):
method test_open (line 178) | def test_open(self):
method test_doesnt_exist (line 188) | def test_doesnt_exist(self):
class OpenUsage (line 195) | class OpenUsage(unittest.TestCase):
method test_open (line 196) | def test_open(self):
method test_doesnt_exist (line 206) | def test_doesnt_exist(self):
FILE: src/wormhole_mailbox_server/test/test_rlimits.py
class RLimits (line 5) | class RLimits(unittest.TestCase):
method test_rlimit (line 6) | def test_rlimit(self):
FILE: src/wormhole_mailbox_server/test/test_server.py
class Server (line 11) | class Server(_Util, ServerBase, unittest.TestCase):
method test_apps (line 12) | def test_apps(self):
method test_nameplate_allocation (line 18) | def test_nameplate_allocation(self):
method test_nameplate_allocation_failure (line 44) | def test_nameplate_allocation_failure(self):
method test_nameplate (line 55) | def test_nameplate(self):
method test_mailbox (line 146) | def test_mailbox(self):
method test_messages (line 216) | def test_messages(self):
method test_early_close (line 272) | def test_early_close(self):
class Prune (line 284) | class Prune(unittest.TestCase):
method _get_mailbox_updated (line 286) | def _get_mailbox_updated(self, app, mbox_id):
method test_update (line 292) | def test_update(self):
method test_apps (line 306) | def test_apps(self):
method test_nameplates (line 314) | def test_nameplates(self):
method test_mailboxes (line 356) | def test_mailboxes(self):
method test_lots (line 387) | def test_lots(self):
method test_one (line 394) | def test_one(self):
method one (line 398) | def one(self, nameplate, mailbox, has_listeners):
class Summary (line 452) | class Summary(unittest.TestCase):
method test_mailbox (line 453) | def test_mailbox(self):
method test_nameplate (line 496) | def test_nameplate(self):
method test_nameplate_disallowed (line 513) | def test_nameplate_disallowed(self):
method test_nameplate_allowed (line 519) | def test_nameplate_allowed(self):
method test_blur (line 525) | def test_blur(self):
method test_no_blur (line 543) | def test_no_blur(self):
class Startup (line 583) | class Startup(unittest.TestCase):
method test_empty (line 585) | def test_empty(self, fake_log):
method test_allow_list (line 596) | def test_allow_list(self, fake_log):
method test_blur_usage (line 607) | def test_blur_usage(self, fake_log):
class MakeServer (line 618) | class MakeServer(unittest.TestCase):
method test_welcome_empty (line 619) | def test_welcome_empty(self):
method test_welcome_error (line 624) | def test_welcome_error(self):
method test_welcome_advertise_version (line 634) | def test_welcome_advertise_version(self):
method test_welcome_message_of_the_day (line 644) | def test_welcome_message_of_the_day(self):
FILE: src/wormhole_mailbox_server/test/test_service.py
class Service (line 6) | class Service(unittest.TestCase):
method test_defaults (line 7) | def test_defaults(self):
method test_log_fd (line 32) | def test_log_fd(self):
FILE: src/wormhole_mailbox_server/test/test_stats.py
class _Make (line 8) | class _Make:
method make (line 9) | def make(self, blur_usage=None, with_usage_db=True):
class Current (line 16) | class Current(_Make, unittest.TestCase):
method test_current_no_mailboxes (line 17) | def test_current_no_mailboxes(self):
method test_current_no_listeners (line 25) | def test_current_no_listeners(self):
method test_current_one_listener (line 34) | def test_current_one_listener(self):
class ClientVersion (line 44) | class ClientVersion(_Make, unittest.TestCase):
method test_add_version (line 45) | def test_add_version(self):
method test_add_version_extra_fields (line 52) | def test_add_version_extra_fields(self):
method test_blur (line 59) | def test_blur(self):
method test_no_usage_db (line 66) | def test_no_usage_db(self):
class Nameplate (line 70) | class Nameplate(_Make, unittest.TestCase):
method test_nameplate_happy (line 71) | def test_nameplate_happy(self):
method test_nameplate_lonely (line 83) | def test_nameplate_lonely(self):
method test_nameplate_pruney (line 91) | def test_nameplate_pruney(self):
method test_nameplate_crowded (line 99) | def test_nameplate_crowded(self):
method test_nameplate_crowded_pruned (line 123) | def test_nameplate_crowded_pruned(self):
method test_no_db (line 136) | def test_no_db(self):
method test_nameplate_happy_blur_usage (line 142) | def test_nameplate_happy_blur_usage(self):
class Mailbox (line 154) | class Mailbox(_Make, unittest.TestCase):
method test_mailbox_prune_quiet (line 155) | def test_mailbox_prune_quiet(self):
method test_mailbox_lonely (line 164) | def test_mailbox_lonely(self):
method test_mailbox_happy (line 174) | def test_mailbox_happy(self):
method test_mailbox_happy_blur_usage (line 186) | def test_mailbox_happy_blur_usage(self):
method test_mailbox_lonely_connected (line 198) | def test_mailbox_lonely_connected(self):
method test_mailbox_scary (line 213) | def test_mailbox_scary(self):
method test_mailbox_errory (line 225) | def test_mailbox_errory(self):
method test_mailbox_errory_scary (line 237) | def test_mailbox_errory_scary(self):
method test_mailbox_crowded (line 249) | def test_mailbox_crowded(self):
FILE: src/wormhole_mailbox_server/test/test_util.py
class Utils (line 5) | class Utils(unittest.TestCase):
method test_to_bytes (line 6) | def test_to_bytes(self):
method test_bytes_to_hexstr (line 16) | def test_bytes_to_hexstr(self):
method test_hexstr_to_bytes (line 22) | def test_hexstr_to_bytes(self):
method test_dict_to_bytes (line 29) | def test_dict_to_bytes(self):
method test_bytes_to_dict (line 35) | def test_bytes_to_dict(self):
FILE: src/wormhole_mailbox_server/test/test_web.py
class WebSocketProtocolOptions (line 16) | class WebSocketProtocolOptions(unittest.TestCase):
method test_set (line 18) | def test_set(self, fake_factory):
class LogRequests (line 27) | class LogRequests(ServerBase, unittest.TestCase):
method setUp (line 28) | def setUp(self):
method tearDown (line 31) | def tearDown(self):
method make_client (line 37) | def make_client(self):
method test_log_http (line 46) | def test_log_http(self):
method test_log_websocket (line 57) | def test_log_websocket(self):
method test_no_log_http (line 69) | def test_no_log_http(self):
method test_no_log_websocket (line 80) | def test_no_log_websocket(self):
class WebSocketAPI (line 90) | class WebSocketAPI(_Util, ServerBase, unittest.TestCase):
method setUp (line 92) | def setUp(self):
method tearDown (line 100) | def tearDown(self):
method make_client (line 106) | def make_client(self):
method check_welcome (line 114) | def check_welcome(self, data):
method test_welcome (line 119) | def test_welcome(self):
method test_bind (line 126) | def test_bind(self):
method test_bind_with_client_version (line 165) | def test_bind_with_client_version(self):
method test_bind_without_client_version (line 180) | def test_bind_without_client_version(self):
method test_bind_with_client_version_extra_junk (line 194) | def test_bind_with_client_version_extra_junk(self):
method test_list (line 209) | def test_list(self):
method test_allocate (line 239) | def test_allocate(self):
method test_claim (line 272) | def test_claim(self):
method test_claim_crowded (line 308) | def test_claim_crowded(self):
method test_release (line 324) | def test_release(self):
method test_release_named (line 356) | def test_release_named(self):
method test_release_named_ignored (line 369) | def test_release_named_ignored(self):
method test_release_named_mismatch (line 379) | def test_release_named_mismatch(self):
method test_open (line 394) | def test_open(self):
method test_open_crowded (line 433) | def test_open_crowded(self):
method test_add (line 449) | def test_add(self):
method test_close (line 484) | def test_close(self):
method test_close_named (line 511) | def test_close_named(self):
method test_close_named_ignored (line 524) | def test_close_named_ignored(self):
method test_close_named_mismatch (line 534) | def test_close_named_mismatch(self):
method test_close_crowded (line 548) | def test_close_crowded(self):
method test_disconnect (line 565) | def test_disconnect(self):
method test_interrupted_client_nameplate (line 586) | def test_interrupted_client_nameplate(self):
method test_interrupted_client_nameplate_reclaimed (line 646) | def test_interrupted_client_nameplate_reclaimed(self):
method test_interrupted_client_mailbox (line 686) | def test_interrupted_client_mailbox(self):
FILE: src/wormhole_mailbox_server/test/test_websocket.py
class FakeServer (line 10) | class FakeServer:
method get_welcome (line 16) | def get_welcome(self):
method get_log_requests (line 21) | def get_log_requests(self):
class WebSocket (line 25) | class WebSocket(unittest.TestCase):
method setUp (line 30) | def setUp(self):
method tearDown (line 35) | def tearDown(self):
method create_server_protocol (line 38) | def create_server_protocol(self):
method test_server_version_string (line 52) | def test_server_version_string(self):
method test_reflected_address (line 75) | def test_reflected_address(self):
method test_reflected_caddy (line 105) | def test_reflected_caddy(self):
method test_reflected_caddyv6 (line 145) | def test_reflected_caddyv6(self):
FILE: src/wormhole_mailbox_server/test/test_ws_client.py
class WSClientSync (line 6) | class WSClientSync(unittest.TestCase):
method test_sync (line 10) | def test_sync(self):
FILE: src/wormhole_mailbox_server/test/ws_client.py
class WSClient (line 6) | class WSClient(websocket.WebSocketClientProtocol):
method __init__ (line 7) | def __init__(self):
method onOpen (line 13) | def onOpen(self):
method onMessage (line 15) | def onMessage(self, payload, isBinary):
method close (line 27) | def close(self):
method onClose (line 31) | def onClose(self, wasClean, code, reason):
method next_event (line 35) | def next_event(self):
method next_non_ack (line 44) | def next_non_ack(self):
method strip_acks (line 53) | def strip_acks(self):
method send (line 56) | def send(self, mtype, **kwargs):
method send_notype (line 61) | def send_notype(self, **kwargs):
method sync (line 66) | def sync(self):
class WSFactory (line 78) | class WSFactory(websocket.WebSocketClientFactory):
FILE: src/wormhole_mailbox_server/util.py
function to_bytes (line 5) | def to_bytes(u):
function bytes_to_hexstr (line 7) | def bytes_to_hexstr(b):
function hexstr_to_bytes (line 12) | def hexstr_to_bytes(hexstr):
function dict_to_bytes (line 17) | def dict_to_bytes(d):
function bytes_to_dict (line 22) | def bytes_to_dict(b):
FILE: src/wormhole_mailbox_server/web.py
class Root (line 6) | class Root(Resource):
method __init__ (line 8) | def __init__(self):
class PrivacyEnhancedSite (line 12) | class PrivacyEnhancedSite(server.Site):
method log (line 14) | def log(self, request):
function make_web_server (line 19) | def make_web_server(server, log_requests, websocket_protocol_options=()):
FILE: update-version.py
function existing_tags (line 30) | def existing_tags(git):
function create_new_version (line 38) | def create_new_version(git, only_patch):
function main (line 48) | async def main(reactor):
FILE: versioneer.py
class VersioneerConfig (line 332) | class VersioneerConfig:
function get_root (line 344) | def get_root() -> str:
function get_config_from_root (line 393) | def get_config_from_root(root: str) -> VersioneerConfig:
class NotThisMethod (line 441) | class NotThisMethod(Exception):
function register_vcs_handler (line 450) | def register_vcs_handler(vcs: str, method: str) -> Callable: # decorator
function run_command (line 459) | def run_command(
function git_get_keywords (line 1194) | def git_get_keywords(versionfile_abs: str) -> dict[str, str]:
function git_versions_from_keywords (line 1222) | def git_versions_from_keywords(
function git_pieces_from_vcs (line 1290) | def git_pieces_from_vcs(
function do_vcs_install (line 1427) | def do_vcs_install(versionfile_source: str, ipy: Optional[str]) -> None:
function versions_from_parentdir (line 1465) | def versions_from_parentdir(
function versions_from_file (line 1511) | def versions_from_file(filename: str) -> dict[str, Any]:
function write_to_version_file (line 1528) | def write_to_version_file(filename: str, versions: dict[str, Any]) -> None:
function plus_or_dot (line 1538) | def plus_or_dot(pieces: dict[str, Any]) -> str:
function render_pep440 (line 1545) | def render_pep440(pieces: dict[str, Any]) -> str:
function render_pep440_branch (line 1570) | def render_pep440_branch(pieces: dict[str, Any]) -> str:
function pep440_split_post (line 1600) | def pep440_split_post(ver: str) -> tuple[str, Optional[int]]:
function render_pep440_pre (line 1610) | def render_pep440_pre(pieces: dict[str, Any]) -> str:
function render_pep440_post (line 1634) | def render_pep440_post(pieces: dict[str, Any]) -> str:
function render_pep440_post_branch (line 1661) | def render_pep440_post_branch(pieces: dict[str, Any]) -> str:
function render_pep440_old (line 1690) | def render_pep440_old(pieces: dict[str, Any]) -> str:
function render_git_describe (line 1712) | def render_git_describe(pieces: dict[str, Any]) -> str:
function render_git_describe_long (line 1732) | def render_git_describe_long(pieces: dict[str, Any]) -> str:
function render (line 1752) | def render(pieces: dict[str, Any], style: str) -> dict[str, Any]:
class VersioneerBadRootError (line 1788) | class VersioneerBadRootError(Exception):
function get_versions (line 1792) | def get_versions(verbose: bool = False) -> dict[str, Any]:
function get_version (line 1868) | def get_version() -> str:
function get_cmdclass (line 1873) | def get_cmdclass(cmdclass: Optional[dict[str, Any]] = None):
function do_setup (line 2172) | def do_setup() -> int:
function scan_setup_py (line 2229) | def scan_setup_py() -> int:
function setup_command (line 2266) | def setup_command() -> NoReturn:
Condensed preview — 71 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (379K chars).
[
{
"path": ".appveyor.yml",
"chars": 1827,
"preview": "# adapted from https://packaging.python.org/en/latest/appveyor/\n\n\nenvironment:\n # we tell Tox to use \"twisted[windows]\""
},
{
"path": ".coveragerc",
"chars": 757,
"preview": "# -*- mode: conf -*-\n\n[run]\n# only record trace data for wormhole_mailbox_server.*\nsource =\n wormhole_mailbox_server\n#"
},
{
"path": ".gitattributes",
"chars": 53,
"preview": "src/wormhole_mailbox_server/_version.py export-subst\n"
},
{
"path": ".github/workflows/test.yml",
"chars": 665,
"preview": "name: Tests\n\non:\n push:\n branches: [ master ]\n pull_request:\n branches: [ master ]\n\njobs:\n testing:\n runs-on"
},
{
"path": ".gitignore",
"chars": 812,
"preview": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\n"
},
{
"path": ".travis.yml",
"chars": 699,
"preview": "arch:\n - amd64\n - ppc64le\nlanguage: python\n# defaults: the py3.7 environment overrides these\ndist: trusty\nsudo: false\n"
},
{
"path": "LICENSE",
"chars": 1080,
"preview": "The MIT License (MIT)\n\nCopyright (c) 2015 Brian Warner\n\nPermission is hereby granted, free of charge, to any person obta"
},
{
"path": "MANIFEST.in",
"chars": 458,
"preview": "include versioneer.py\ninclude src/wormhole_mailbox_server/_version.py\ninclude LICENSE README.md NEWS.md\nrecursive-includ"
},
{
"path": "Makefile",
"chars": 3167,
"preview": "# How to Make a Release\n# ---------------------\n#\n# This file answers the question \"how to make a release\" hopefully\n# b"
},
{
"path": "NEWS.md",
"chars": 2040,
"preview": "\n\nUser-visible changes in \"magic-wormhole-mailbox-server\":\n\n## Upcoming\n\n* (put release-notes here when merging / propos"
},
{
"path": "README.md",
"chars": 3688,
"preview": "# Magic Wormhole Mailbox Server\n[](https://pypi.p"
},
{
"path": "docs/Makefile",
"chars": 770,
"preview": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHI"
},
{
"path": "docs/conf.py",
"chars": 5751,
"preview": "# Magic-Wormhole documentation build configuration file, created by\n# sphinx-quickstart on Sun Nov 12 10:24:09 2017.\n#\n#"
},
{
"path": "docs/happy-plant.seq",
"chars": 2443,
"preview": "@startuml\n\nskinparam defaultFontName \"Source Sans Pro\"\nskinparam defaultFontSize 18\nskinparam participantFontSize 22\nski"
},
{
"path": "docs/happy.seq",
"chars": 2187,
"preview": "seqdiag {\n alice <- mailbox [label = \"welcome\"]\n alice -> mailbox [label = \"bind appid, side\"]\n alice -> mailbox [lab"
},
{
"path": "docs/index.rst",
"chars": 548,
"preview": ".. Magic-Wormhole-Mailbox-Server documentation master file, created by\n sphinx-quickstart on Sun Nov 12 10:24:09 2017."
},
{
"path": "docs/server-protocol.md",
"chars": 12271,
"preview": "# Rendezvous Server Protocol\n\n## Concepts\n\nThe Rendezvous Server provides queued delivery of binary messages from one\ncl"
},
{
"path": "docs/states.dot",
"chars": 3128,
"preview": "/*digraph {\n title [label=\"Mailbox\\lServer Machine\" style=\"dotted\"]\n\n start -> opened [label=\"open(side)\"];\n\n o"
},
{
"path": "docs/welcome.md",
"chars": 2234,
"preview": "# Magic Wormhole Mailbox Server\n[![Build Status](https://travis-ci.org/warner/magic-wormhole-mailbox-server.svg?branch=m"
},
{
"path": "misc/migrate_channel_db.py",
"chars": 3496,
"preview": "\"\"\"Migrate the channel data from the old bundled Mailbox Server database.\n\nThe magic-wormhole package used to include bo"
},
{
"path": "misc/migrate_usage_db.py",
"chars": 2519,
"preview": "\"\"\"Migrate the usage data from the old bundled Mailbox Server database.\n\nThe magic-wormhole package used to include both"
},
{
"path": "misc/munin/wormhole_active",
"chars": 1508,
"preview": "#! /usr/bin/env python3\n\n\"\"\"\nUse the following in /etc/munin/plugin-conf.d/wormhole :\n\n[wormhole_*]\nenv.channeldb /path/"
},
{
"path": "misc/munin/wormhole_errors",
"chars": 1920,
"preview": "#! /usr/bin/env python3\n\n\"\"\"\nUse the following in /etc/munin/plugin-conf.d/wormhole :\n\n[wormhole_*]\nenv.usagedb /path/to"
},
{
"path": "misc/munin/wormhole_event_rate",
"chars": 1500,
"preview": "#! /usr/bin/env python3\n\n\"\"\"\nUse the following in /etc/munin/plugin-conf.d/wormhole :\n\n[wormhole_*]\nenv.usagedb /path/to"
},
{
"path": "misc/munin/wormhole_events",
"chars": 2068,
"preview": "#! /usr/bin/env python3\n\n\"\"\"\nUse the following in /etc/munin/plugin-conf.d/wormhole :\n\n[wormhole_*]\nenv.usagedb /path/to"
},
{
"path": "misc/munin/wormhole_events_alltime",
"chars": 1419,
"preview": "#! /usr/bin/env python3\n\n\"\"\"\nUse the following in /etc/munin/plugin-conf.d/wormhole :\n\n[wormhole_*]\nenv.usagedb /path/to"
},
{
"path": "misc/munin/wormhole_version_uptake",
"chars": 2664,
"preview": "#! /usr/bin/env python3\n\n\"\"\"\nUse the following in /etc/munin/plugin-conf.d/wormhole :\n\n[wormhole_*]\nenv.usagedb /path/to"
},
{
"path": "misc/windows-build.cmd",
"chars": 838,
"preview": "@echo off\n:: To build extensions for 64 bit Python 3, we need to configure environment\n:: variables to use the MSVC 2010"
},
{
"path": "newest-version.py",
"chars": 399,
"preview": "#\n# print out the most-recent version\n#\n\nfrom dulwich.repo import Repo\nfrom dulwich.porcelain import tag_list\n\n\ndef exis"
},
{
"path": "setup.cfg",
"chars": 203,
"preview": "[versioneer]\nVCS = git\nversionfile_source = src/wormhole_mailbox_server/_version.py\nversionfile_build = wormhole_mailbox"
},
{
"path": "setup.py",
"chars": 1566,
"preview": "from setuptools import setup\n\nimport versioneer\n\ncommands = versioneer.get_cmdclass()\n\ntrove_classifiers = [\n \"Develo"
},
{
"path": "signatures/magic-wormhole-mailbox-server-0.5.0.tar.gz.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmcsEawRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic-wormhole-mailbox-server-0.5.1.tar.gz.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmcwGhcRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic_wormhole_mailbox_server-0.5.0-py3-none-any.whl.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmcsEasRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic_wormhole_mailbox_server-0.5.1-py3-none-any.whl.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmcwGhcRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic_wormhole_mailbox_server-0.6.0-py3-none-any.whl.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmmO09cRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic_wormhole_mailbox_server-0.6.0.tar.gz.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmmO09cRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic_wormhole_mailbox_server-0.7.0-py3-none-any.whl.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmoGEmkRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic_wormhole_mailbox_server-0.7.0.tar.gz.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmoGEmoRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic_wormhole_mailbox_server-0.8.0-py3-none-any.whl.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmoHVeoRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "signatures/magic_wormhole_mailbox_server-0.8.0.tar.gz.asc",
"chars": 512,
"preview": "-----BEGIN PGP SIGNATURE-----\n\niQFFBAABCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmoHVeoRHG1lZWphaEBt\nZWVqYWguY2EACgkQwmAoAxKA"
},
{
"path": "src/twisted/plugins/magic_wormhole_mailbox.py",
"chars": 282,
"preview": "from twisted.application.service import ServiceMaker\n\nMailbox = ServiceMaker(\n \"Magic-Wormhole Mailbox Server\", # nam"
},
{
"path": "src/wormhole_mailbox_server/__init__.py",
"chars": 72,
"preview": "from . import _version\n__version__ = _version.get_versions()['version']\n"
},
{
"path": "src/wormhole_mailbox_server/_version.py",
"chars": 24447,
"preview": "# This file helps to compute a version number in source trees obtained from\n# git-archive tarball (such as those provide"
},
{
"path": "src/wormhole_mailbox_server/database.py",
"chars": 6026,
"preview": "import importlib.resources\nimport os, shutil\nimport sqlite3\nimport tempfile\n\nfrom twisted.python import log\n\nclass DBErr"
},
{
"path": "src/wormhole_mailbox_server/db-schemas/channel-v1.sql",
"chars": 2077,
"preview": "\n-- note: anything which isn't an boolean, integer, or human-readable unicode\n-- string, (i.e. binary strings) will be s"
},
{
"path": "src/wormhole_mailbox_server/db-schemas/upgrade-usage-to-v2.sql",
"chars": 601,
"preview": "CREATE TABLE `client_versions`\n(\n `app_id` VARCHAR,\n `side` VARCHAR, -- for deduplication of reconnects\n `connect_time` "
},
{
"path": "src/wormhole_mailbox_server/db-schemas/usage-v1.sql",
"chars": 1978,
"preview": "CREATE TABLE `version`\n(\n `version` INTEGER -- contains one row\n);\n\nCREATE TABLE `current`\n(\n `rebooted` INTEGER, -- sec"
},
{
"path": "src/wormhole_mailbox_server/db-schemas/usage-v2.sql",
"chars": 2510,
"preview": "CREATE TABLE `version`\n(\n `version` INTEGER -- contains one row\n);\n\nCREATE TABLE `current`\n(\n `rebooted` INTEGER, -- sec"
},
{
"path": "src/wormhole_mailbox_server/increase_rlimits.py",
"chars": 1430,
"preview": "try:\n # 'resource' is unix-only\n from resource import getrlimit, setrlimit, RLIMIT_NOFILE\nexcept ImportError: # pr"
},
{
"path": "src/wormhole_mailbox_server/server.py",
"chars": 30649,
"preview": "import os, random, base64, re\nfrom collections import namedtuple\nfrom twisted.python import log\nfrom twisted.application"
},
{
"path": "src/wormhole_mailbox_server/server_tap.py",
"chars": 4516,
"preview": "import os, json, time\nfrom twisted.internet import reactor\nfrom twisted.python import usage, log\nfrom twisted.applicatio"
},
{
"path": "src/wormhole_mailbox_server/server_websocket.py",
"chars": 15624,
"preview": "import time\nfrom twisted.internet import reactor\nfrom twisted.python import log\nfrom autobahn.twisted import websocket\nf"
},
{
"path": "src/wormhole_mailbox_server/test/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "src/wormhole_mailbox_server/test/common.py",
"chars": 2613,
"preview": "#from __future__ import unicode_literals\nfrom twisted.internet import reactor, endpoints\nfrom twisted.internet.defer imp"
},
{
"path": "src/wormhole_mailbox_server/test/test_config.py",
"chars": 9686,
"preview": "from twisted.python.usage import UsageError\nfrom twisted.trial import unittest\nfrom .. import server_tap\n\nPORT = r\"tcp:4"
},
{
"path": "src/wormhole_mailbox_server/test/test_database.py",
"chars": 7909,
"preview": "import os\nfrom twisted.python import filepath\nfrom twisted.trial import unittest\nfrom .. import database\nfrom ..database"
},
{
"path": "src/wormhole_mailbox_server/test/test_rlimits.py",
"chars": 2744,
"preview": "from unittest import mock\nfrom twisted.trial import unittest\nfrom ..increase_rlimits import increase_rlimits\n\nclass RLim"
},
{
"path": "src/wormhole_mailbox_server/test/test_server.py",
"chars": 26671,
"preview": "from unittest import mock\nfrom twisted.trial import unittest\nfrom twisted.python import log\nfrom .common import ServerBa"
},
{
"path": "src/wormhole_mailbox_server/test/test_service.py",
"chars": 3072,
"preview": "from twisted.trial import unittest\nfrom unittest import mock\nfrom twisted.application.service import MultiService\nfrom ."
},
{
"path": "src/wormhole_mailbox_server/test/test_stats.py",
"chars": 13856,
"preview": "#import io, json\nfrom twisted.trial import unittest\nfrom ..database import create_channel_db, create_usage_db\nfrom ..ser"
},
{
"path": "src/wormhole_mailbox_server/test/test_util.py",
"chars": 1252,
"preview": "import unicodedata\nfrom twisted.trial import unittest\nfrom .. import util\n\nclass Utils(unittest.TestCase):\n def test_"
},
{
"path": "src/wormhole_mailbox_server/test/test_web.py",
"chars": 27313,
"preview": "import io, time\nfrom unittest import mock\nimport treq\nfrom twisted.trial import unittest\nfrom twisted.internet import de"
},
{
"path": "src/wormhole_mailbox_server/test/test_websocket.py",
"chars": 5562,
"preview": "import json\nfrom twisted.trial import unittest\nfrom twisted.internet.defer import inlineCallbacks\nfrom twisted.internet."
},
{
"path": "src/wormhole_mailbox_server/test/test_ws_client.py",
"chars": 1767,
"preview": "import json\nfrom twisted.trial import unittest\nfrom twisted.internet.defer import inlineCallbacks\nfrom .ws_client import"
},
{
"path": "src/wormhole_mailbox_server/test/ws_client.py",
"chars": 2505,
"preview": "import json, itertools\nfrom twisted.internet import defer\nfrom twisted.internet.defer import inlineCallbacks\nfrom autoba"
},
{
"path": "src/wormhole_mailbox_server/util.py",
"chars": 736,
"preview": "# No unicode_literals\nimport json, unicodedata\nfrom binascii import hexlify, unhexlify\n\ndef to_bytes(u):\n return unic"
},
{
"path": "src/wormhole_mailbox_server/web.py",
"chars": 930,
"preview": "from twisted.web import server, static\nfrom twisted.web.resource import Resource\nfrom .server_websocket import WebSocket"
},
{
"path": "tox.ini",
"chars": 1400,
"preview": "# Tox (http://tox.testrun.org/) is a tool for running tests\n# in multiple virtualenvs. This configuration file will run "
},
{
"path": "update-version.py",
"chars": 2499,
"preview": "#\n# this updates the (tagged) version of the software\n#\n# it will only update the \"minor\" version (e.g. 0.12.* -> 0.13.0"
},
{
"path": "versioneer.py",
"chars": 86667,
"preview": "# Version: 0.29\n\n\"\"\"The Versioneer - like a rocketeer, but for versions.\n\nThe Versioneer\n==============\n\n* like a rocket"
}
]
About this extraction
This page contains the full source code of the magic-wormhole/magic-wormhole-mailbox-server GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 71 files (348.8 KB), approximately 91.1k tokens, and a symbol index with 381 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.