Showing preview only (354K chars total). Download the full file or copy to clipboard to get everything.
Repository: komuw/sewer
Branch: master
Commit: d12d5a29b8db
Files: 90
Total size: 330.9 KB
Directory structure:
gitextract_ki74leed/
├── .circleci/
│ ├── codecov.yml
│ └── config.yml
├── .github/
│ ├── CONTRIBUTING.md
│ ├── ISSUE_TEMPLATE.md
│ ├── PULL_REQUEST_TEMPLATE.md
│ └── workflows/
│ └── build.yml
├── .gitignore
├── CONTRIBUTORS.md
├── LICENSE.txt
├── Makefile
├── README.md
├── docs/
│ ├── ACME.md
│ ├── Aliasing.md
│ ├── CHANGELOG.md
│ ├── DNS-Propagation.md
│ ├── LegacyDNS.md
│ ├── UnifiedProvider.md
│ ├── catalog.md
│ ├── crypto.md
│ ├── dns-01.md
│ ├── drivers/
│ │ ├── route53.md
│ │ └── unbound_ssh.md
│ ├── http-01.md
│ ├── index.md
│ ├── notes/
│ │ ├── 0.8.2-notes.md
│ │ ├── 0.8.3-notes.md
│ │ └── 0.8.4-notes.md
│ ├── preview/
│ │ ├── cloaca.md
│ │ └── cloaca_config.md
│ ├── sewer-as-a-library.md
│ ├── sewer-cli.md
│ ├── unpropagated.md
│ └── wildcards.md
├── mypy.ini
├── pyproject.toml
├── setup.py
├── sewer/
│ ├── __init__.py
│ ├── __main__.py
│ ├── auth.py
│ ├── catalog.json
│ ├── catalog.py
│ ├── cli.py
│ ├── client.py
│ ├── config.py
│ ├── crypto.py
│ ├── dns_providers/
│ │ ├── __init__.py
│ │ ├── acmedns.py
│ │ ├── aliyundns.py
│ │ ├── auroradns.py
│ │ ├── cloudflare.py
│ │ ├── cloudns.py
│ │ ├── common.py
│ │ ├── dnspod.py
│ │ ├── duckdns.py
│ │ ├── gandi.py
│ │ ├── hurricane.py
│ │ ├── powerdns.py
│ │ ├── rackspace.py
│ │ ├── route53.py
│ │ ├── tests/
│ │ │ ├── __init__.py
│ │ │ ├── test_acmedns.py
│ │ │ ├── test_aliyundns.py
│ │ │ ├── test_aurora.py
│ │ │ ├── test_cloudflare.py
│ │ │ ├── test_cloudns.py
│ │ │ ├── test_common.py
│ │ │ ├── test_dnspod.py
│ │ │ ├── test_duckdns.py
│ │ │ ├── test_gandi.py
│ │ │ ├── test_hedns.py
│ │ │ ├── test_powerdns.py
│ │ │ ├── test_rackspace.py
│ │ │ ├── test_route53.py
│ │ │ ├── test_unbound_ssh.py
│ │ │ └── test_utils.py
│ │ └── unbound_ssh.py
│ ├── lib.py
│ ├── meta.json
│ ├── providers/
│ │ ├── __init__.py
│ │ ├── demo.py
│ │ └── tests/
│ │ ├── __init__.py
│ │ └── test_demo.py
│ └── tests/
│ ├── __init__.py
│ ├── test_Client.py
│ ├── test_auth.py
│ ├── test_catalog.py
│ ├── test_lib.py
│ └── test_utils.py
└── tests/
├── crypto_test.py
└── data/
└── README
================================================
FILE CONTENTS
================================================
================================================
FILE: .circleci/codecov.yml
================================================
codecov:
notify:
require_ci_to_pass: yes
coverage:
range: 70..100
round: down
precision: 2
status:
project:
default:
# basic
target: 85
comment:
layout: "reach, diff, flags, files"
behavior: default
require_changes: yes # if true: only post the comment if coverage changes
================================================
FILE: .circleci/config.yml
================================================
# Python CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-python/ for more details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
# use `-browsers` prefix for selenium tests, e.g. `3.6.1-browsers`
- image: circleci/python:3.6
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/repo
steps:
- checkout
- run:
name: install dependencies
command: |
sudo pip3 install -e .[dev,test,alldns]
# run tests! Most config is in pyprojects.toml rather than --options since 0.8.4
- run:
name: run tests
command: |
make testdata
find . -type f -name *.pyc -delete | echo
coverage erase
coverage run && ln -s tests/coverage/.coverage .coverage && bash <(curl -s https://codecov.io/bash)
- run:
name: run tests reports
command: |
coverage report --fail-under=85
- run:
name: run static analyzers
command: |
black --check . || { printf "\\n\\t please use black to format your code."; exit 77; }
pylint --enable=E --disable=W,R,C --unsafe-load-any-extension=y sewer/
- run:
name: run sewer cli
command: |
sewer --version && sewer --help
# run make upload
- store_artifacts:
path: test-reports
destination: test-reports
================================================
FILE: .github/CONTRIBUTING.md
================================================
# Contributing to sewer
Thank you for thinking of contributing to sewer. Every contribution to
sewer is important to us. You may not know it, but you are about to
contribute towards making the world a safer and more secure place.
Contributor offers to license certain software (a “Contribution” or multiple
“Contributions”) to sewer, and sewer agrees to accept said Contributions,
under the terms of the MIT License. Contributor understands and agrees that
sewer shall have the irrevocable and perpetual right to make and distribute
copies of any Contribution, as well as to create and distribute collective
works and derivative works of any Contribution, under the MIT License.
## To contribute:
- fork this repo.
- cd sewer
- open an issue on this repo. In your issue, outline what it is you want to add and why.
- install pre-requiste software:
```shell
pip3 install -e .[dev,test]
```
- python cryptography generally only requires openssl's libraries. To run
the full set of tests (make test), you **also need the openssl program**
- make the changes you want on your fork.
- your changes should have backward compatibility in mind unless it is impossible to do so.
- add your name and contact(optional) to CONTRIBUTORS.md
- add tests
- format your code using [black](https://github.com/ambv/black) *NB:*
requires black 19.3.b0 or newer (19.10b0 is used by the CI):
```shell
black -l 100 -t py35 .
```
- run [pylint](https://pypi.python.org/pypi/pylint) on the code and fix any issues:
```shell
pylint --enable=E --disable=W,R,C sewer/
```
- run tests and make sure everything is passing:
```shell
make test
```
- open a pull request on this repo.
NB: I make no commitment of accepting your pull requests.
##Styles
Martin (@mmaney) has a few things to say about what he's looking for aside
from code that works:
- Python is not Java. There is rarely an excuse for @staticmethod, we can
use first-class non-member _functions_ when _self_ would just be baggage.
- When fixing things, I approve of trying to identify a minimal change that
repairs the bug (or adds a feature, etc.). But be prepared to get
feedback asking (sometimes sketching) a more intrusive refactoring that
perhaps incidentally fixes the bug. It's not that I don't appreciate a
small, focused patch, but there's a lot of houscleaning going on these days!
- And sometimes your PR (or less often a bug report itself) will get me
thinking about a piece of work I hadn't focused on yet (or get me thinking
about it in a more productive way), and all of a sudden I've stolen your
patch and wrapped it in a larger refactoring. Don't doubt that your
contribution was appreciated, you just yodeled and triggered an avalanche
that you weren't expecting!
- black is the current fad, but its indifference to actual readability in
favor of slavish consistency to simplistic rules sometimes makes me ill.
But for now it's a thing.
## Creating a new release:
To create a new release on [https://pypi.org/project/sewer](https://pypi.org/project/sewer);
- Create a new branch
- Update `sewer/meta.json` with the new version number.
The version numbers should follow semver.
- Update `docs/CHANGELOG.md` with information about what is changing in the new release.
- Open a pull request and have it go through the normal review process.
- Upon approval of the pull request, squash and merge it.
Remember that the squash & merge commit message should ideally be the message that was in the pull request template.
- Once succesfully merged, run;
```bash
git checkout master
git pull --tags
# if plain "python" runs Py2, prefix "make ..." with "PYTHON=python3"
# or have it set in your shell environment.
make uploadprod
```
That should upload the new release on pypi. You do need to have
permissions to upload to pypi. Currently only
[@komuw](https://github.com/komuw) and
[@mmaney](https://github.com/mmaney) have pypi permissions, so if you
need to create a new release, do talk to him to do that. In the future,
more contributors may be availed permissions to pypi.
================================================
FILE: .github/ISSUE_TEMPLATE.md
================================================
## Which version of python are you using?
## What operating system and version of operating system are you using?
## What version of sewer are you using?
## What did you do? (be as detailed as you can)
## What did you expect to see/happen/not happen?
## What did you actually see/happen?
## Paste here the log output generated by `sewer`, if any. Please remember to remove any sensitive items from the log before pasting here.
## If you can, run sewer with loglevel set to debug; eg `sewer --loglevel DEBUG`
Alternatively if you want to conribute to this repo, answer this questions instead in your issue:
## What is it that you would like to propose to add/remove/change?
## Why do you want to add/remove/change that?
## How do you want to go about adding/removing/changing that?
================================================
FILE: .github/PULL_REQUEST_TEMPLATE.md
================================================
Thank you for contributing to sewer.
Every contribution to sewer is important to us.
You may not know it, but you have just contributed to making the world a more safer and secure place.
Contributor offers to license certain software (a “Contribution” or multiple
“Contributions”) to sewer, and sewer agrees to accept said Contributions,
under the terms of the MIT License.
Contributor understands and agrees that sewer shall have the irrevocable and perpetual right to make
and distribute copies of any Contribution, as well as to create and distribute collective works and
derivative works of any Contribution, under the MIT License.
Now,
## What(What have you changed?)
## Why(Why did you change it?)
================================================
FILE: .github/workflows/build.yml
================================================
name: Build
on: push
jobs:
linux:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.10.x'
cache: 'pip'
- name: install dependencies
shell: bash
run: |
sudo apt install build-essential
python -m pip install -e .[dev,test,alldns]
- name: run tests
shell: bash
run: |
make testdata
find . -type f -name *.pyc -delete | echo
coverage erase
coverage run
- name: generate test reports
shell: bash
run: |
coverage report --fail-under=85
- name: run static analyzers
shell: bash
run: |
black --check . || { printf "\\n\\t please use black to format your code."; exit 77; }
pylint --enable=E --disable=W,R,C --unsafe-load-any-extension=y sewer/
- name: run sewer cli
shell: bash
run: |
sewer --version && sewer --help
================================================
FILE: .gitignore
================================================
# Compiled Python modules
*.pyo
*.pyc
# SQLite
*.db
*.s3db
*.sqlite3
# Vagrant private files
.vagrant/*
.vagrant*
#logs
*.log
*.log.*
#pids
*.pid
#ppa public keys
*.asc*
# markdown preview
README.html
# vscode editor
.vscode/
.DS_Store
# testing artifacts, temp files, etc.
/tests/coverage/
/tests/tmp/
# Setuptools distribution folder.
/dist/
/build/
# Python egg metadata, regenerated from source files by setuptools.
/*.egg-info
# virtualenv
.venv
# certificates and keys
*.csr
*.crt
*.key
*.pem
.idea
.vscode
.mypy_cache/
local/
================================================
FILE: CONTRIBUTORS.md
================================================
Thank you for contributing to sewer.
Every contribution to sewer is important to us.
> Contributor offers to license certain software (a _Contribution_ or
multiple _Contributions_) to sewer, and sewer agrees to accept said
Contributions, under the terms of the MIT License. Contributor understands
and agrees that sewer shall have the irrevocable and perpetual right to make
and distribute copies of any Contribution, as well as to create and
distribute collective works and derivative works of any Contribution, under
the MIT License.
Contributors
------------
- Author: [komu W](https://www.komu.engineer)
- Current maintainer: [@mmaney](https://github.com/mmaney)
- [Wilfried Jonker](wjonker.nl)
- [András Veres-Szentkirályi, dnet](https://techblog.vsza.hu/)
- [menduo](https://menduo.net)
- [m4ldonado](https://github.com/m4ldonado)
- [luisbarrueco](https://github.com/luisbarrueco)
- [Tungsteno74](https://github.com/Tungsteno74)
- [Harold Bradley III](https://haroldbradleyiii.com/)
- [@etienne-napoleone](https://github.com/etienne-napoleone)
- [@soloradish](https://github.com/soloradish)
- [Moritz Ulmer](https://www.protohaus.org)
- [rozgonik](https://github.com/rozgonik)
- [alec T](https://github.com/AlecTroemel)
- [Don S](https://github.com/donspaulding)
- [Julien Demoor](https://github.com/jdkx)
- [@tkalus](https://github.com/tkalus)
================================================
FILE: LICENSE.txt
================================================
The MIT License (MIT)
Copyright (c) 2017 Komu Wairagu
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: Makefile
================================================
# we may need something other than the system default here
# use $ PYTHON=python3 make ... for example
python = python
ifdef PYTHON
python = ${PYTHON}
endif
# invoke these using ${python} -m to avoid yet more xxxx3 naming issues
twine = ${python} -m twine
pip = ${python} -m pip
coverage = ${python} -m coverage
black = ${python} -m black
pylint = ${python} -m pylint
mypy = ${python} -m mypy
VERSION_STRING=$$(sed -n -e '/"version"/ s/.*version": *"\([^"]*\)".*/\1/p' <sewer/meta.json)
.PHONY: build
build: # build distribution artifacts
rm -rf build
rm -rf dist
rm -rf sewer.egg-info
${python} setup.py sdist
${python} setup.py bdist_wheel
uploadtest: build # build and upload to pypi-test
@${twine} upload dist/* -r testpypi
@${pip} install -U -i https://testpypi.python.org/pypi sewer
release2pypi: build upload2pypi release-tag # build & upload to pypi
@echo "${pip} install -U sewer"
.PHONY: upload2pypi release-tag
upload2pypi:
${twine} upload dist/*
release-tag:
@printf "\n creating git tag: $(VERSION_STRING) \n"
@printf "\n with commit message, see Changelong: https://github.com/komuw/sewer/blob/master/CHANGELOG.md \n" && git tag -a "$(VERSION_STRING)" -m "see Changelong: https://github.com/komuw/sewer/blob/master/CHANGELOG.md"
@printf "\n git push the tag::\n" && git push --all -u --follow-tags
# TESTS - target "test" runs the unit tests under coverage and reports both.
TDATA = tests/data
.PHONY: clean coverage format-check lint mypy
test: testdata coverage mypy lint format-check
testdata: rsatestkeys secptestkeys
-mkdir tests/tmp
coverage: clean
@printf "\n coverage erase::\n" && ${coverage} erase
@printf "\n coverage run::\n" && ${coverage} run
@printf "\n coverage report::\n" && ${coverage} report --show-missing --fail-under=85
clean:
find . -type f -name *.pyc -delete | echo
-rm -r tests/tmp
-mkdir tests/tmp
mypy:
${mypy} sewer/client.py sewer/cli.py
lint:
@printf "\n run pylint::\n" && ${pylint} --enable=E --disable=W,R,C --unsafe-load-any-extension=y ${LINTARGS} sewer/
format-check:
@printf "\n run black::\n" && ${black} --check .
rsatestkeys: ${TDATA}/rsa2048.pem ${TDATA}/rsa3072.pem ${TDATA}/rsa4096.pem
${TDATA}/rsa2048.pem:
openssl genpkey -out ${TDATA}/rsa2048.pem -algorithm RSA -pkeyopt rsa_keygen_bits:2048
${TDATA}/rsa3072.pem:
openssl genpkey -out ${TDATA}/rsa3072.pem -algorithm RSA -pkeyopt rsa_keygen_bits:3072
${TDATA}/rsa4096.pem:
openssl genpkey -out ${TDATA}/rsa4096.pem -algorithm RSA -pkeyopt rsa_keygen_bits:4096
secptestkeys: ${TDATA}/secp256r1.pem ${TDATA}/secp384r1.pem
${TDATA}/secp256r1.pem:
openssl genpkey -out ${TDATA}/secp256r1.pem -algorithm EC -pkeyopt ec_paramgen_curve:P-256
${TDATA}/secp384r1.pem:
openssl genpkey -out ${TDATA}/secp384r1.pem -algorithm EC -pkeyopt ec_paramgen_curve:P-384
# not actually useful with LE at this time
${TDATA}/secp521r1.pem:
openssl genpkey -out ${TDATA}/secp521r1.pem -algorithm EC -pkeyopt ec_paramgen_curve:P-521
================================================
FILE: README.md
================================================
## Sewer
[](https://github.com/komuw/sewer/.github/workflows/build.yml)
[](https://codecov.io/gh/komuW/sewer)
[](https://github.com/komuw/sewer)
Sewer is a Let's Encrypt(ACME) client.
It's name is derived from Kenyan hip hop artiste, Kitu Sewer.
- The stable release is
[0.8.4](https://komuw.github.io/sewer/notes/0.8.4-notes).
- More history (including notes on 0.8.5-to-be) in the
[CHANGELOG](https://komuw.github.io/sewer/CHANGELOG).
PYTHON compatibility: 3.7 and above are tested.
I (maintainer @mmaney) loiter in channel ##sewer (on irc.freenode.net) for
those who remember IRC. Don't ask to ask, but waiting is.
## Features
- Obtain or renew SSL/TLS certificates from [Let's Encrypt](https://letsencrypt.org)
- Supports acme version 2 (current RFC including post-as-get).
- Support for SAN certificates.
- Supports [wildcard certificates](https://komuw.github.io/sewer/wildcards).
- Bundling certificates.
- Support for both RSA and ECDSA for account and certificate keys.
- Supports [DNS and HTTP](https://komuw.github.io/sewer/UnifiedProvider) challenges
- List of currently supported
[DNS services and BYO-service notes](https://komuw.github.io/sewer/dns-01)
- HTTP challenges are a new feature, no operational drivers in the tree
yet. [See usage and BYO-service notes](https://komuw.github.io/sewer/http-01)
- sewer is both a [command-line program](https://komuw.github.io/sewer/sewer-cli)
and a [Python library](https://komuw.github.io/sewer/sewer-as-a-library) for custom use
- Well written(if I have to say so myself):
- [Good test coverage](https://codecov.io/gh/komuW/sewer)
- [Passing continuous integration](https://circleci.com/gh/komuW/sewer)
- [High grade statically analyzed code](https://www.codacy.com/app/komuW/sewer/dashboard)
- type hinting to support mypy verification is a recently begun WIP
## Installation
```shell
pip install sewer
# with All DNS Provider support, include aliyun, Hurricane Electric, Aurora, ACME ...
# pip3 install sewer[alldns]
# with Cloudflare support
# pip3 install sewer[cloudflare]
# with Aliyun support
# pip3 install sewer[aliyun]
# with HE DNS(Hurricane Electric DNS) support
# pip3 install sewer[hurricane]
# with Aurora DNS Support
# pip3 install sewer[aurora]
# with ACME DNS Support
# pip3 install sewer[acmedns]
# with Rackspace DNS Support
# pip3 install sewer[rackspace]
# with DNSPod DNS Support
# pip3 install sewer[dnspod]
# with DuckDNS DNS Support
# pip3 install sewer[duckdns]
# with ClouDNS DNS Support
# pip3 install sewer[cloudns]
# with AWS Route 53 DNS Support
# pip3 install sewer[route53]
# with PowerDNS DNS Support
# pip3 install sewer[powerdns]
```
sewer(since version 0.5.0) is now python3 only. To install the (now
unsupported) python2 version:
```shell
pip install sewer==0.3.0
```
Sewer is in active development and it's API will change in backward incompatible ways.
[https://pypi.python.org/pypi/sewer](https://pypi.python.org/pypi/sewer)
## Development setup
See the how to contribute [documentation](https://github.com/komuw/sewer/blob/master/.github/CONTRIBUTING.md)
## FAQ
- Why another ACME client?
I wanted an ACME client that I could use to programmatically(as a library) acquire/get certificates. However I could not
find anything satisfactory for use in Python code.
- Why is it called Sewer?
I really like the Kenyan hip hop artiste going by the name of Kitu Sewer.
================================================
FILE: docs/ACME.md
================================================
# ACME, RFCs, and confusion, oh my!
ACME grew out of early, ad-hoc procedures designed to let CAs issue large
numbers of certificates with low overhead. As described in RFC855, these
would go something like this:
> * Generate a PKCS#10 [RFC2986] Certificate Signing Request (CSR).
> * Cut and paste the CSR into a CA's web page.
> * Prove ownership of the domain(s) in the CSR by one of the following methods:
> + Put a CA-provided challenge at a specific place on the web server
> + Put a CA-provided challenge in a DNS record corresponding to the target domain
> + Receive a CA-provided challenge at (hopefully) an administrator-controlled email
> address corresponding to the domain, and then respond to it on the CA's web page
> * Download the issued certificate and install it on the user's web server
>
> With the exception of the CSR itself and the certificates that are
> issued, these are all completely ad hoc procedures and are
> accomplished by getting the human user to follow interactive natural
> language instructions from the CA rather than by machine-implemented
> published protocols.
HTTP validation was the first mechanism, matching the first method of
proving ownership in the above. The rest of what
[Let's Encrypt](https://letsencrypt.org)
added was automating the process (and rearranging it a bit, having the proof
of control happen before the CSR, etc.). Years later, the IETF standardized
the ACME protocol, and there are other variants that have been (or will be)
standardized.
## RFC8555
The [IETF](https://www.ietf.org/) has published
[RFC8555](https://tools.ietf.org/html/rfc8555) defining the ACME protocol
for http-01 and dns-01 validations of dns-name authorizations. These are
the sort of ACME authorizations that we usually think of, and which sewer
works with. The RFC was published in the spring of 2019, but it wasn't
until near the end of that year that Let's Encrypt adopted the full v.2 on
only their *staging* server. There's some elaborate and, from what I can
make out, often-shifting schedule for various partial transitions, but I'm
not going to try to make sense of them. As of the beginning of 2020, the
only immediate effect on sewer was that one could no longer run it against
the *staging* server. The next big change is when that same restriction is
rolled out on LE's *production* server later in the year. Since sewer
v0.8.2, which implemented the final RFC8555 protocol at least well enough to
work with LE's server implementation, our tl;dr is just this:
> If you get a failure running an older version of sewer, get v0.8.2 or
later. This is a known problem: v0.8.2 or later is the fix.
### RSA Keys
LE probably accepts RSA keys of a wide range of sizes. Traditionally sewer
has used a 2048 bit RSA key by default, with a _bits_ option available in
the Client() interface, with no support from the command line. As part of
the crypto overhaul, 0.8.4 has introduced support for several sizes of RSA
key by name (matching the elliptical curve keys it added). 2048, 3072 and
4096 bit RSA keys can be called for, and the default (for keys not
explicitly provided and so generated by the cli program) is changed to a
3072 bit RSA key for both the account key and the certificate key. Either
or both can use an externally generated key (of one of these sizes only) or
you can (from the cli) change the generated key type & size for each.
SHA256 is the only hash algorithm for signing with an RSA key that LE
currently accepts, and so there is no option to change it.
### Elliptic Curve Keys and Certificates
RFC8555 specifies _An ACME server MUST implement the "ES256" signature algorithm
[RFC7518] and SHOULD implement the "EdDSA" signature algorithm using
the "Ed25519" variant (indicated by "crv") [RFC8037]. As of 0.8.4, sewer
supports all the EC curves which LE currently accepts. These are
_secp256r1_ (P-256 curve using ES256 for signatures) and _secp384r1_ (P-384
and ES384). So once again there's no option for changing the signature
hashing algorithm.
LE does not currently accept an Ed25519 elliptic key not for any technical
reason but because the CAB (?) hasn't yet added it to their list of approved
keys/algorithms. Apparently they see little benefit, and much potential
confusion, in accepting such as an account key when they are constrained not
to accept one in a csr.
## Other RFCs
You think RFC8555 is a lot to take in? There is, as they say, much more:
this collection of RFCs that describe details referred to but not expounded
in good ol' 8555.
[RFC7515](https://tools.ietf.org/html/rfc7515) specifies the Json Web
Signature and JOSE Header which is used by the ACME protocol.
[RFC7517](https://tools.ietf.org/html/rfc7517) describes the Json Web Key
[RFC7518](https://www.rfc-editor.org/rfc/rfc7518) describes Json Web
Algorithms, a few of which are used in the ACME protocol
[RFC7807](https://tools.ietf.org/html/rfc7807) describes the JSON structure
of an error document such as those used by ACME
[RFC8037](https://tools.ietf.org/html/rfc8037) covers the ED25519 details
(not yet supported in either LE nor sewer).
Oh yes, there's another RFC for the new TLS authorization method which sewer
has (so far) no interest in.
================================================
FILE: docs/Aliasing.md
================================================
# Aliasing for ACME Validation
The idea is presented (for dns-01 authorizations) in [an article at
letsencrypt.org](https://letsencrypt.org/2019/10/09/onboarding-your-customers-with-lets-encrypt-and-acme.html)
which shows an example of DNS aliasing and describes what was likely the
original motivation - a hosting provider running the certificate process on
behalf of his customers. Like all DNS aliasing, it uses a CNAME at the
canonical name `_acme-challenge.domain.tld` to redirect the ACME server to a
different fqdn that is more convenient for provisioning of the validation
responses. I'm not going to try to convince you that you should use
aliasing, because if you need it you probably already know that, or at least
know that the process isn't working smoothly as-is.
The `alias` option in sewer is available to drivers that derive from
`DNSProviderBase`.
>Added in 0.8.3: `--p_opts alias=...`, but legacy drivers don't take
advantage of the aliasing support in their parent classes yet.
## Isn't aliasing just for DNS?
No. HTTP has had, as a side effect of common web server and client behavior,
a kind of aliasing since the very beginning of ACME. Usually it's
convenient enough to provision the validation at the canonical
`/.well-known/acme-challenge/<token>` location. But if it isn't, either
`acme-challenge` or `.well-known` can be mapped (by server configuration or
externally by symlink, usually) to some other location. If it's desired to
serve the validation file from some other domain or server altogether, an
HTTP redirect can often be used (and that's also a third way to place the
file elsewhere within the web server's accesible filesystem).
The RFC says that (for http-01 challenges) the ACME server "SHOULD follow
redirects", which would allow for an analogous aliasing. Lets' Encrypt's
servers [do follow redirects and
CNAMES](https://letsencrypt.org/docs/challenge-types/).
So aliasing can be used with HTTP validations, though it's probably less
often needed since the privilege needed to directly configure the canonical
response file is likely to be the same (or even less) than that needed to
setup the new certificate. And it's possible that you've already used it
without thinking of it as _aliasing_ because it uses such basic HTTP
behavior (and so needs no support from sewer).
## Preparing for DNS aliasing
The first thing which you must have is a way to manage DNS TXT records. In
fact, you need to be able to control both the real domain's records (in
order to setup the CNAME entries, but that's something that needs be done
only once) as well as managing the alias domain records through the
service-specific driver. Generally, expect to need a new-model driver
rather than an existing legacy driver, on the assumption that it's not much
more work to migrate the legacy driver to the new interface while adding the
alias support. I have my fingers crossed, at any rate...
With alias-capable driver in hand, you then setup CNAME records for every
DNS name that you wish to use with the alias domain. In traditional zone
file form that might look something like this [excerpt]:
; existing record for your web or other server
name.example.com. IN A 111.222.333.444
; then add the CNAME at the ACME-prefixed name
_acme-challenge.name.example.com. IN CNAME name.example.com.alias.org
In online domain editors, the names are usually given without the full
domain suffix that's shown here (example.com). The A record (or it could be
a CNAME) for `name.example.com` that directs to your server is shown as an
example here.
The added CNAME record is the redirect from the conventional ACME challenge
DNS name, pointing to the TXT record in the alias domain. When it sees that
CNAME, the ACME server will proceed to look for the challenge's TXT record
at `name.example.com.alias.org`. Since the alias-aware driver will have
setup that TXT record, the server will retrieve it and validate your right
to issue for `name.example.com`.
Note that the alias domain can be ANY valid domain that you can manage. In
particular, it can be in a different tld (as shown here) or a different
domain in the same tld, or even in a sub-domain (eg.
`validation.example.com`) of the target's domain that has been delgated to
that more convenient DNS server. And you can setup aliased TXT challenge
records for names from any number of _real_ domains as long as the CNAME
redirects can be provisioned.
## Using DNS aliases in sewer
This is really pretty short & sweet.
All that's needed, once the setup is done, is to pass `alias=alias.org` to
the alias-supporting driver when it's created.
From the command line, that's `--p_opts alias=alias.org`.
================================================
FILE: docs/CHANGELOG.md
================================================
# `sewer` changelog:
## **pre-release** 0.8.5
- driver for Windows DNS server (local only) [IN PROGRESS]
- cleanup that was deferred from 0.8.4 (affects developers, not cli users)
- crypto.py refactored
- mypy added to tests
- dns_providers have had non-base imports cleaned up: use local `# type:
ignore` annotations
- a few non-service-specific libs marked globally to be ignored
- REMOVED obsolescent dns_provider_name class variables (use the JSON
catalog, added in 0.8.3)
- REMOVED obsolescent guards around service-specific imports and the
corresponding delayed exceptions (the unnecessary imports that used to
require the guards were removed in 0.8.3)
- crypto.py's tests migrated to pytest format as tests/crypto_test.py
- Fixed the alias support code and unbound_ssh, its only in-tree client, to
use correct names for alias option parameters
- Aliasing document updated to current client options
- in-tree tests began migrating to pytest format (and moving to ./tests)
## **version:** 0.8.4
- add support for ECDSA keys
CLI changes:
- `--acct_key` & `--cert_key` should be used to designate the file that
holds the keys to be used (rather than having new ones generated).
`--account_key` & `--certificate_key` are still accepted as synonyms.
- add `--acct_key_type` & `--cert_key_type` to allow choice of RSA or EC
keys and key sizes when sewer is generating them for you.
- changed default for generated keys to 3072 bit RSA (had been 2048 bit)
- add `--is_new_key` to allow for first-time registration of your own
account key (using `--acct_key`) generated outside of sewer.
Internal changes for library clients:
- Client methods cert() and renew() are deprecated; just call
get_certificate() directly instead.
- Client **no longer generates keys**. (see below)
- crytographic refactoring
- AcmeKey, AcmeAccount & AcmeCsr in crypto.py; uses only cryptography library
- Client interface changes due to crypto refactoring
- dropped `account_key` and `certificate_key` optional arguments to Client
- added `acct_key` and `cert_key` REQUIRED arguments to Client taking
AcmeAccount and AcmeKey objects, respectively.
- add `is_new_acct` argument to force registration of the supplied account
key
- dropped `bits` argument because Client no longer generates keys!
- dropped `digest` argument since there are currently no alternate digest
methods for the different key types. (was this ever used?)
## **version:** 0.8.3
Features and Improvements:
- added `--acme-timeout <seconds>` option to adjust timeout on queries to
the ACME server
- `--action {run,renew}` has been doing nothing useful and is now deprecated.
- added `--p_opt <name>=<value>` for passing kwargs to drivers
- Added optional parameters accepted by base class for DNS drivers:
- `alias=<alias_domain>` specifies a separate domain for DNS challenges
(requires driver support, see [Aliasing](Aliasing))
- `prop_delay=<seconds>` gives a fixed delay (sleep) after challenge setup
- gandi (legacy DNS driver) fixed internal bugs that broke common wildcard
use cases (eg., `*.domain.tld`) as well as the "wildcard plus" pattern
- added unbound_ssh legacy-style DNS provider as a working demo of adding
new features to legacy drivers. It does work in the right environment, and
could be useful to someone other than myself (mm).
Internals:
- added [catalog.py](catalog) to manage provider catalogs; includes
get_provider(name) method to replace `import ......{name.}ClassName`
- replace __version__.py with meta.json; setup.py converted; add sewer_meta()
in lib.py; cli.py converted; client.py converted
- added catalog.json defining known drivers and their interfaces; also
information about dependencies for setup.py
- added `**kwargs` to all legacy providers to allow new options that are
handled in a parent class to pass through (for `alias`, `prop_delay`, etc.)
- removed imports that were in `sewer/__init__` and
`sewer/dns_providers/__init__`; fixed all uses in cli.py and tests.
- began cleanup/refactor of cli.py (there will be more to come and/or a new,
more config driven, alternative command (0.9?))
- added `__main__.py` to support `python -m sewer` invocation of `sewer-cli`
- fixed imports in client.py that didn't actually import the parts of
OpenSSL and cryptography that we use (worked because we import requests?)
See also [release notes](notes/0.8.3-notes).
## **version:** 0.8.2
Feature additions:
- support current RFC8555 protocol (LE staging current, production requires in Nov)
- added DNS providers powerdns and gandi
Internals (features and/or annoying changes for sewer-as-a-library users)
- unified dns-01 and http-01 providers; support challenge propagation check
- added support for non-dns (http-01 challenge) provider
- collect shared (internal) functions into lib.py
- use unitest.mock rather than external module
- client no longer prepends`*.` to wildcards; remove spotty code in providers to strip it
- begin addition of annotations, mostly opportunistically
See also [release notes](notes/0.8.2-notes).
## **version:** 0.8.1
- Fix bug where `sewer` was unable to delete wildcard names from clouflare: https://github.com/komuw/sewer/pull/139
- Fix a StopIteration bug: https://github.com/komuw/sewer/pull/148
- Add guide on how to create a new pypi release
## **version:** 0.8.0
- Fix bug where `sewer` would log twice: https://github.com/komuw/sewer/pull/137
Thanks to [@mmaney](https://github.com/mmaney) for this
## **version:** 0.7.9
- Fix bug where Aliyun response is in bytes: https://github.com/komuw/sewer/pull/133
Thanks to [@ButterflyTech](https://github.com/ButterflyTech) for this
## **version:** 0.7.8
- Add support for Cloudflare token auth: https://github.com/komuw/sewer/pull/130
Thanks to [@moritz89](https://github.com/moritz89) for this
## **version:** 0.7.7
- Add support for Support AWS Route53: https://github.com/komuw/sewer/pull/126
Thanks to [@soloradish](https://github.com/soloradish) for this
## **version:** 0.7.6
- Fix logging, sewer was redefining root logger: https://github.com/komuw/sewer/pull/125
Thanks to [@etienne-napoleone](https://github.com/etienne-napoleone) for this
## **version:** 0.7.5
- Fix pypi upload script
## **version:** 0.7.4
- Adds support for [ClouDNS](https://www.cloudns.net/): https://github.com/komuw/sewer/pull/122
Thanks to [@hbradleyiii](https://github.com/hbradleyiii) for this
================================================
FILE: docs/DNS-Propagation.md
================================================
# Waiting for Mr. DNS or Someone Like Him
Q: How long does it take after you've setup the challenge response TXT records
until they're actually accessible to the ACME server?
A: Good Question!
According to [Let's Encrypt](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge)
it can take up to an hour. Depends on the DNS service. Some provide a way
to check that your changes are fully propagated to all their servers. With
many, however, you just have to wait. But be sure you wait long enough,
because Let's Encrypt DOES NOT implement automatic or triggered retry of a
failed authorization - you have to restart the [same] order or else start
all over again.
Sewer provides a flexible _delay until actually published_ mechanism through
three optional driver parameters, `prop_delay`, `prop_timeout`,
`prop_sleep_times`, and the [`unpropagated` method](unpropagated).
Let's see how they're used in various circumstances.
## No API support, no reliable way to check: just delay
If you can't check that the TXT records are fully published, then all you
can do is delay for a while. Perhaps the DNS service will suggest a safe
time. If not, you'll have to start with a guess and adjust from there based
on your experience. Choosing the right number - long enough but not
excessively long - can be hard, but applying it is easy. Just add
`prop_delay=SIMPLE_DELAY_TIME` to the driver's initialization parameters,
and sewer's engine will add that many seconds of delay after the challenge
setup returns before it signals the ACME server to validate those
challenges.
**CLI option --p_opt prop_delay=... is available for all drivers since 0.8.3**
## API support or can check: use a timeout
If the DNS service gives you a way to check that the propagation is
complete, or if there are not too many authoritative servers (viz., not an
anycast system), you can use that actual check (implemented in the driver's
`unpropagated` method) and the engine will run that check until it succeeds
or until a timeout you specify is exceeded. However the check is being
done, you setup the timeout by adding `prop_timeout=MAX_WAIT_TIME` to the
driver parameters. If you know that it takes at least some minimum time to
propagate, you may also pass `prop_delay` to make the engine delay that long
before it starts checking. And there's a delay between checks that has a
hopefully sensible default, but which you can adjust if necessary through
the `prop_sleep_times` parameter.
**no drivers implement `unpropagated` as of 0.8.3**
## You probably don't need to change `prop_sleep_times`
Unless you do, but if it's not obvious, just leave it.
This parameter defines the lengths of sleeps the engine will add following a
call to `unpropagated` that reports not ready. As an optional parameter
passed to the driver, `prop_sleep_times` can be an integer number of seconds
or a list or tuple of such delays which will be used in order. The final
value in the sequence will be reused indefinitely.
Example: the default value is (1, 2, 4, 8) which provides an exponential
backoff up to an 8 second delay, then sticks there. _[the values could
change - it's just what seemed reasonable to me]_ So if there's no delay,
and the check call takes no measurable time (and reports not ready each
time), it will look something like this with `prop_timeout=20`:
| time | action |
| ---: | --- |
| 0 | call unpropagated, sleep(1) |
| 1 | call unproagagted, sleep(2) |
| 2 | call unpropagated, sleep(4) |
| 6 | call unpropagated, sleep(8) |
| 14 | call unpropagated, sleep(8) |
| 22 | call unpropagated, timeout! |
This shows both the last value repeating and the way the timeout and sleeps
interact. The check for timeout is done only AFTER a call to unpropagated
AND the chance to exit with success if it finally reports the challenges are
ready. So the timeout isn't a hard maximum time, but it's bounded to be no
more than one sleep interval (plus actual time to run `unpropagated`, of
course) over `prop_timeout`.
## Other Notes and Advanced Use
These values are setup through the Provider on the reasonable assumption
that they will vary most directly with the choice of service provider, so
the individual drivers are best suited to provide sensible defaults where
appropriate (and possible!). Sewer's engine implements the delay and check
loop (with timeout) because the mechanism is the same for all providers (and
may be useful for other than the DNS-based challenges for which it has been
implemented).
If you are using sewer as a library and find that you can make a better
estimate of the propagation after the driver is setup (perhaps using a
service-specific method to access part of the service's API or run some
tests), you could adjust those parameters through the same-named attributes
on the Provider instance. This is solidly in the categories of don't do it
unless you're sure you need to, and be prepared to own both the pieces if
you break it!
## Could this be used with non-DNS drivers?
Yes! I have no experience with http-01 in any setting where such a delay
might be needed, but the mechanism is implemented in sewer's engine, and all
that needs be done is to setup the parameters (and implement unpropagated in
the driver if using more than just `prop_delay`) as described above and
there you go!
================================================
FILE: docs/LegacyDNS.md
================================================
## Legacy DNS challenge providers
### `BaseDns` shim class
A child of `DNSProviderBase` that acts as an adapter between the new
Provider interface and the legacy DNS provider interface.
#### `__init__(self, **kwargs: Any) -> None`
Accepts no arguments itself; doesn't expect any to be passed by Legacy code.
Injects `chal_types=["dns-01"]`.
#### `setup(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`
Iterates over the challenges, extracting the values needed for the legacy
DNS interface from each challenge in the list, and passing them to
`create_dns_record`. Always returns an empty list since there is no error
return from `create_dns_record` other than raising an exception.
#### `unpropagated(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`
Always returns an empty list, signalling "all ready as far as I know".
A legacy DNS driver wishing to do something useful here MAY implement
`unpropagated` without updating the rest of its interface.
#### `clear(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`
Same as setup except it calls the legacy `delete_dns_record`, of course.
### Legacy DNS class
#### `__init__(self, ..., **kwargs)`
Args handled by the driver should be explicitly named, with defaults where
that makes sense. Starting in 0.8.3, the `**kwargs` bucket has been added
to provide pass-through to the base class.
#### `def create_dns_record(self, domain_name, domain_dns_value)`
Minimum is to add `_acme-challenge` prefix to domain_name and post the
challenge response (domain_dns_value) as that name's TXT value.
All very provider-dependent.
#### `def delete_dns_record(self, domain_name, domain_dns_value)`
In theory it should undo the effects of setup.
In practice, at least one of the services is unable to do that
(according to the author's comment).
### Legacy DNS vs Aliasing
Legacy DNS drivers MAY change to use the [aliasing](ALiasing) methods
inherited from `DNSProviderBase`, though this will require a potentially
fragile faking of the new-model challenge dict in the driver. See the
`unbound_ssh` example driver, and bear in mind that a change to the data
type of the challenge items IS anticipated, perhaps in 0.9.
================================================
FILE: docs/UnifiedProvider.md
================================================
# DNS and HTTP challenges unified
_There's still a draft when the wind is blowing, but it's getting less._
## Dedication
It is indisputable that this is, in the first instance, Alec Troemel's fault,
since he added support for http-01 challenges.
Also indisputable is that the many changes to both code and overall design
made in the process of unifying the two types of challenges,
while influenced by Alec's code and our discussions, are entirely my fault.
Alec cannot be blamed for my choices!
## A few words about words
Because the word "provider" is so overloaded, I'm going to refer to the
service-specific implementations as "drivers", except when I forget, or
missed changing an old use. "Provider" is still used in the class names. And
then there are the "service providers", viz., DNS services or web hosts,
etc.
## Overview (tl;dr)
`ProviderBase` described here defines the interface the ACME engine uses
with new-model drivers (all http-01 drivers, as there are no old ones). New
drivers normally should inherit from the `DNSProviderBase` or
`HTTPProviderBase` classes in auth.py.
`DNSProviderBase` has support for [aliasing](Aliasing), though the
individual drivers need to be created (or modified) to support it at this
time. _unbound_ssh is a quirky but working example that supports aliasing.`
## ProviderBase interface for ACME engine
The interface between the ACME protocol code and any driver implementation
consists of three methods, `setup`, `unpropagated` and `clear`. The first
and last are not unlike methods used by the legacy drivers, but they accept
a list of one or more challenges rather than one challenge at a time.
--- most of this is in [unpropagated], or should be. ToDo: reconcile
The `unpropagated` method was added with DNS propagation delays in mind. It
should be possible for legacy drivers to implement this without a full
conversion to the new-model as a temporary adaptation for those who need
this feature. Just override the null version provided by `BaseDns`.
`unpropagated` checks all the challenges in the list it is passed, and
returns a list containing the ones which are *not* yet ready to be
validated. This should be more reliable than adding an ad-hoc delay before
_responding_ to the ACME server as well as avoiding wasting time.
The errata list returned by by all three methods has tuples for elements,
where each tuple holds three values: the status string, the msg string, and
the original, unmodified challenge item (dictionary). This is defined as
types `ErrataItemType` and `ErrataListType` in auth.py
> LE's ACME server, for one, implements neither automatic nor triggered
retries, so it's important not to _respond_ to a challenge before the
validation response is actually accessible. And yes, the RFC's language
does encourage confusing the respond-to-challenge API request with the
challenge response (TXT) that the server has to find when it probes for it.
> My thinking on this is that, while the ACME engine's code can know what
names to check, in the really interesting case of widely distributed
(anycast?) DNS service, figuring out which DNS server to query must be left
to the service-specific driver. In some cases the service may provide an
API for checking some internal status that might be faster and/or more
reliable than polling DNS servers. For cases where all that can (or needs)
to be done is some DNS lookups, well, that can be packaged as a function.
--- end reconcile block
This is the pattern which all three methods use: accept a list of challenges
(each a dictionary) to process, and return an errata list containing the
subset which have problems or are not ready. So in all cases an empty list
returned means that all went well.
## `ProviderBase`
Abstract base class for driver implementations ultimately inherit from.
### ProviderBase __init__
__init__(self,
*,
chal_types: Sequence[str],
logger: Optional[LoggerType] = None,
LOG_LEVEL: Optional[str] = "INFO",
prop_delay: int = 0
prop_timeout: int = 0,
prop_sleep_times: Union[Sequence[int], int] = (1, 2, 4, 8)
) -> None:
The drivers' `__init__` methods accept only keyword arguments. We can see
that ProviderBase has become the final recipient of quite a few, mostly
optional, arguments. They ended up here because they are not specific to a
subclass; a counterexample is the `alias` parameter, which is handled in
`DNSProviderBase`. The conventions for ProviderBase and its subclasses are:
- keyword only arguments (other than self, of course)
- Required arguments never have a default value
- Optional arguments must have a default, of course
- Everything not explicitly handled is left to kwargs
Conveniently, ProviderBase's __init__ demonstrates all of these aside from
the use of kwargs (because it is the final base class, so any unrecognized
arguments would be in error):
- chal_types is required, so it has no default value and will be diagnosed by
Python's calling mechanism if omitted.
- logger/LOG_LEVEL are... weird. Without the legacy DNS providers I would
be inclined to just require logger, pushing the job of setting up logging
firmly back up the stack. As it is, logger cannot be required (yet), so
both have defaults that work with the __init__ logic to setup logging as
sanely as possible. Eventually LOG_LEVEL should get deprecated and then
dropped, and logger is just required...
- the prop_* arguments are all optional, and receive default values that the
engine code is designed to deal with - by disabling the optional behavior
they control. These are all parameters that were introduced for a
lower-level driver or driverBase class, but which have migrated up to
ProviderBase because they may apply to any sort of Provider.
In all subclasses, kwargs is expected to catch parameters that may need to
pass up the Provider classes, and so it must be passed to super()__init__.
It is allowed to add, change, or even remove items from kwargs if necessary
--- see the intermediate *ProviderBase classes.
--- (see where for args documentation? DNS-Alias and DNS-Propagation & ???)
### `setup(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Tuple[str, str, Dict[str, str]]]`
The `setup` method is called to publish one or more challenges. Each item
in the list describes one challenge.
(_the description of the challenges list is common to all three methods_)
The items are dictionaries with keys and values that come from the ACME
authz query, or are derived from it and the account key [see note].
For dns-01 and http-01 challenges, the required keys are:
* ident_value - the value of the identifier to be validated (1)
* token - the validation nonce
* key_auth - validation value (hash of nonce + secret key's thumbprint)
> The current per-challenge dict holds a subset of the authz values, and
some of the names (and structure) are different. *This is likely to change
in the future!* (0.9?)
The plan is to include other values from the ACME _authorization object_
response, as well as non-authz values, so the driver implementation MUST
accept additional keys in the dictionary. Likewise, the list SHOULD include
only outstanding challenges, and the call(s) to the driver SHOULD be omitted
if there are none. But the latter, especially, is just the plan, so
throwing an exception if the challenges list is empty is JUST NOT ON.
> Allowing an empty challenges list is also convenient for unit tests.
Each of the three methods return an errata list of the challenge items which
encountered an issue - couldn't create, isn't published, removal failed. So
in all cases, an empty list means all is well.
The *errata list* is a list-like containing a tuple for each failed or
unready challenge. The tuples have three elements: a status (str), a msg
(str) intended to enlighten a human observer, and the original challenge
item (the dictionary from the argument list). The status MUST be one of the
defined values:
| status | applies to | meaning |
| --- | --- | --- |
| "failed" | all | challenge for which a failure occurred |
| "skipped" | setup | may skip challenges after one has a hard failure |
| "unready" | unpropagated | soft fail: record not deployed to authoritative server(s). If a non-recoverable error is detected, then use _failed_. |
### `unpropagated(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Tuple[str, str, Dict[str, str]]]`
This method is expected to be needed mostly for DNS challenges, but it
should be used whenever a service provider has a relatively slow or
unpredictable lag between the challenge being posted by `setup` and that
challenge data being visible to the world. When there's no expectation of
such lag, or no way to reliably check that the challenge has propagated,
this may as well just return an empty list, and we'll all hope for the best.
### `clear(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Tuple[str, str, Dict[str, str]]]`
`clear`, unlike `setup`, SHOULD NOT stop processing challenges after hitting
an error. It's possible that any reported errors will be treated as
potential soft errors and the operation retried (with only the unready
challenges).
_? should have a status word for "this one's hard failed, forget about it"?_
## `DNSProviderBase`
The driver *interface* is the same for everything except legacy DNS drivers,
but there are some differences which it makes no sense to push into
`ProviderBase`. `DNSProviderBase` provides a nice example of this:
`__init__(self, *, alias: str = "", **kwargs: Any) -> None`
def cname_domain(self, chal: Dict[str, str]) -> Union[str, None]
def target_domain(self, chal: Dict[str, str]) -> str
The class's `__init__` handles the `alias` argument, and provides chal_types
suitable for a DNS driver if they weren't already present. Its value, if
any, is stored locally for use by the helper methods. `target_domain` is to
be used in the driver to get the actual DNS name for the challenge TXT,
handling both the aliasing and non-aliasing case. `cname_domain` forms the
DNS name for the CNAME that should exist in the aliasing case and returns it
for the use of a hypothetical sanity check, or None when not aliasing.
## `HTTPProviderBase`
This intermediate base class stands ready to handle any HTTP-specific
options or helper methods. No additions are expected until sewer has had
some actual drivers added. It also provides chal_types if needed.
================================================
FILE: docs/catalog.md
================================================
# The Catalog of Drivers
The driver catalog, `sewer/catalog.json`, replaces scattered facilities that
were used to stitch things together. The import farms in `sewer.__init__`
and `sewer.dns_providers.__init__` have already been removed; with the
catalog in place, redundant lists in cli.py and setup.py are also removed,
replaced by use of the catalog's data and a few lines of code. The
`dns_provider_name` is deprecated in favor of the catalog as well.
`catalog.py` wraps the catalog in a class, adding some convenience methods
for listing the known drivers, looking up a driver's data by name, and
loading a driver's implementation class by name. But using the catalog
without `catalog.py` is as easy as loading it using the standard lib's json
facilities - it's all lists & dicts (see eg. setup.py which loads the
catalog this way to avoid potential issues with trying to call into the
package's code before it's installed).
## Catalog structure
The catalog resides in a JSON file that loads as an array of dictionaries,
one element for each registered driver. The per-driver record contains the
following items (some optional):
- **name** The name used to identify this driver, eg., `--provider <name>`
to `sewer-cli`. These names need to be unique, but are not required to
match the module or implementing class names. (legacy DNS drivers usually
matched the module name, but not always)
- **desc** A brief description of the driver, intended for display to humans
to help them understand what each driver is, eg. in --known_providers output
- **chals** list of strings for the type of challenge(s) this driver
handles. (if more than one type, in order of preference?)
- **args** A list of the driver-specific [parameters](#args-parameter-descriptors)
- **path** The path to use to import the driver's Python module.
_Default_ is `sewer.providers.{name}`
- **cls** Name of module attribute which is called with parameters to get a
working instance of the driver. Usually a class, but a factory function
may be used. _Default_ is `Provider`.
- **features** - a list of strings that name the optional features that this
driver supports. _Default_ is an empty list.
- **memo** Additional text/comments about the driver, the descriptor, etc.
- **deps** list of additional projects this driver requires (for setup)
## args - parameter desciptors
This is a bit of a mess due to legacy drivers that ignored the established
conventions. To be fair to them, those conventions weren't clearly
documented (then - see below). This adds some complications to preserve
compatibility, as usual. Let's begin with a minimal descriptor for a driver
that conforms to the new convention (hint: it's imaginary at this time):
{
"name": "well_behaved",
"desc": "made-up example driver that's mostly defaults",
"chals": ["dns-01"],
"args": [
{ "name": "api_id", "req": 1 },
{ "name": "api_key", "req": 1},
],
"features": [ "alias" ],
}
This describes a dns-01 challenge driver that is found in the module
`sewer.providers.well_behaved`, constructed from a class named `Provider`.
The constructor takes two required arguments, `api_id` and `api_key`, which
the program should accept from environment variables `WELL_BEHAVED_API_ID"
and "WELL_BEHAVED_API_KEY". Since it is a `dns-01` challenge provider and
up to date, it adds the claim that it supports the `alias` feature. It
doesn't support the "unpropagated" feature - perhaps the DNS service has no
API to check the propagation of changes.
If this had been a difficult old legacy driver, the descriptor might have
looked more like this:
{
"name": "difficult",
"desc": "made-up example driver that's as non-default as can be",
"chals": ["dns-01"],
"args": [
{ "name": "api_id",
"req": 1,
"param": "difficult_api_id",
"envvar": "DIFFICULT_DNS_API_ID",
},
{ "name": "api_key",
"req": 1,
"param": "DIFFICULT_API_KEY",
"envvar": "DIFFICULT_DNS_API_KEY"},
},
{ "name": "api_base_url",
"param": "API_BASE_URL",
"envvar": "",
},
],
"path": "sewer.dns_providers.difficultdns",
"cls": "DifficultDNSDns",
"features": [],
"memo", "difficult, indeed..."
}
This driver has both parameter names and envvar names that defy convention,
so both the parameter and envvar name must be given explicitly. There is
also an optional parameter that has never had an associated envvar that the
implementation used.
## driver parameter and environment variable names
The convention is that the envvar name (if any) SHOULD be formed from the
driver name and the individual args' names (see the first envvar rule
below). This gives envvar names similar to, sometimes identical to, the
ones already used with legacy DNS drivers. One thing that is changing is
that the parameter names, which in the old convention were
THE_SAME_AS_ENVVAR_NAMES, are changing to be lower case and losing
driver-name prefixes, etc. Where appropriate, the new names will use just a
few shared names, viz., `id`, `key`, `token`.
Obviously the drivers and envvar names are not so consistent among the
legacy DNS drivers. Therefore the descriptor has both `param` and `envvar`
values, along with a set of rules for resolving the names to be used.
### parameter name rules
1. `descriptor.args[n].name` is the "modern" name for the nth parameter
2. if `param` is given, it overrides the "modern" name
### environment name rules
1. f"{descriptor.name}_{descriptor.args[n].name}".upper() is the default
2. if `envvar` is given, it overrides the default
Two guidelines for the use of envvars:
1. If `envvar` is given, is not the empty string, and the so-named envvar is
not found, the invoking code MAY also look for the default-named envvar
before reporting a missing envvar.
2. If `envvar` is set to the empty string, then catalog using code will not
look for a matching envvar at all.
## catalog representation in Python
For now, see the brief implementation in sewer/catalog.py for the way the
JSON structure is mapped into a ProviderDescriptor instance.
================================================
FILE: docs/crypto.md
================================================
# A crypto module for ACME
There were several motivations behind the creation of `crypto.py`:
- a desire to convert the OpenSSL (Python code) usage to the preferred
cryptography package. Note that this is only a change in which Python
wrapper around the openssl binary is used!
- breaking the Client global mess up into more cohesive components
- Oh, and @jfb's PR adding ECDSA certificate keys was the spark that
triggered the whole thing into [code] motion.
## Preliminaries
I use the term `private key` quite a bit, as it is the term widely used for
some representation of both the public and private parts of an asymmetric
key.
## AcmeKey
`AcmeKey` is a parameterized holder for the primitive key classes of the
underlying implementation (the cryptography package).
So now Client and the cli code can stop schlepping around a string that
holds the key in PEM format. That's part of the reason this work
intentionally broke the names in Client (eg., acct_key in place of account
key) when it switched to an AcmeKey object.
#### Factory (class)methods
Two essential factories:
- `create(key_type: str) -> AcmeKeyType` generates a new private key
of the named kind.
- `from_pem(pem_data: bytes) -> AcmeKeyType` returns an AcmeKey object
containing the key serialized in the PEM bytes, assuming it is one of the
types of keys that are known (viz., implemented in a subclass of AcmeKey).
And an inessential convenience:
- `read_pem(filename: str) -> AcmeKeyType` loads the key from a PEM format
file.
### AcmeKey attributes
- pk, the private key in the form of cryptography's key class
Sewer's code never needs to touch the pk directly.
### AcmeKey methods
- `to_pem(self) -> bytes` Returns all the key's info in PEM format
- `write_pem(self, filename:str) -> None` Writes private key to file as PEM
private_bytes is the only method that all ACME clients must use, and that
will often be done indirectly through to_file.
- `sign_message(self, message: bytes) -> bytes` Calculate the signature for
the given message. (uses SHA256 only because that's what ACME specifies)
> You will have noticed the symmetry of the method names: to/from_pem for
in-memory byte strings, read/write_pem for keys in files.
## AcmeAccount
Accounts are keys which are registered with the ACME service and thereafter
used to identify and authenticate the origina of most of the messages sent
to the service. They are a subclass of AcmeKey that extends the interface
to include things only an account key has to do.
### AcmeAccount attributes
- _kid: str — the _key identifier_ that comes from registering pk with ACME.
- _timestamp: float — when the account was [most recently] registered
- _jwk: Dict[str, str] — cached JWK or None
_kid is needed only for constructing the signed ACME requests, and is used
mostly as an in-memory value. There is experimental support for saving the
Key ID and timestamp along with the account key.
### AcmeAccount methods
- `jwk(self) -> Dict[str, str]` Returns the JSON Web Key as a dictionary
(with binary values base64 encoded). (value is cached)
- `set_kid(self, kid: str, timestamp: float = None) -> None` Hook for ACME
register_account or its caller to use to attach the registered key's kid
to the AcmeKey object. If timestamp is not given, the current time will
be used.
set_kid() is pure implementation detail, a stash for the account's registered
URL on a specific ACME server. timestamp is what time.time() returns.
> not yet implemented: write_extended_pem, read_extended_pem (or will read
just be an override that loads the extended values if present?)
## AcmeCsr
This is currently a minimal replacement for the OpenSSL-based create_csr
method. Which might be all an ACME client requires, so perhaps the current
design will be more or less how it comes out? There will be an additional
flag for setting the "must be stapled" certificate extension, but there's
really not much else to add, based on a review of the de-facto standard of
cert-bot's options.
One thing to note: yes, the choice of DER format is intentional and
necessary, as the ACME protocol requires that format, base64-url encoded,
_without_ the starting and ending text lines that PEM adds.
================================================
FILE: docs/dns-01.md
================================================
# DNS service drivers
ACME's dns-01 authorization was sewer's original target. There are a number
of DNS services supported in-tree, and implementations for other services
are difficult to write only if the service's API is difficult.
## DNS services supported
Currently, these are all _legacy_ drivers, built on the original DNS-only
interface. That's okay, there's no plan to drop them (just a hope that
interested users will step up to get them updated), but that does mean that
support for some features varies.
| service | driver name | wc+ | alias | prop | notes |
| --- | --- | :-: | :-: | :-: | --- |
| [acme-dns](https://github.com/joohoi/acme-dns) | acmedns | ? | no | no | |
| [Aliyun](https://help.aliyun.com/document_detail/29739.html) | aliyun | ? | no | no | |
| [Aurora](https://www.pcextreme.com/aurora/dns) | aurora | ? | no | no | |
| [Cloudflare](https://www.cloudflare.com/dns) | cloudflare | ? | no | no | patch in #123, needs confirmation? |
| [ClouDNS](https://www.cloudns.net) | cloudns | ? | no | no | test coverage 75% |
| [DNSPod](https://www.dnspod.cn/) | dnspod | ? | no | no | |
| [DuckDNS](https://www.duckdns.org/) | duckdns | ? | no | no | |
| [Gandi](https://doc.livedns.gandi.net/) | gandi | OK | no | no | wildcard & other fixes in 0.8.3 |
| [Hurricane Electric](https://dns.he.net/) | hurricane | ? | no | no | test coverage 70% |
| [PowerDNS](https://doc.powerdns.com/authoritative/http-api/index.html) | powerdns | NO | no | no | apparently not in 0.8.2; bug #195 |
| [Rackspace](https://www.rackspace.com/cloud/dns) | rackspace | ? | no | no | test coverage 69% |
| [Route 53 (AWS)](https://aws.amazon.com/route53/) | route53 (1) | OK | no | no | wc+ in 0.8.2; not in CLI |
| Unbound | unbound_ssh | OK | yes | no | Working demonstrator model for local unbound server |
- _wc+_ (wilcard plus) is specifically about a single certificate that has
at least two registered names: `domain.tld` and `*.domain.tld`. This
specific combination has issues with some service providers/s.p.'s
API/drivers. So far it's been possible to make it work by changing the
drivers, but it has to be done one by one.
- _alias_ publishing challenges in a different [sub]domain than the
identities being authorized. See [Aliasing](Aliasing).
- _prop_ support for the [unpropagated](unpropagated) interface. Can be
added to any driver but may only be worthwhile with service API support?
## Add a driver for your DNS service
Most (?) of the DNS drivers came about because someone wanted to use sewer
with their DNS service provider, but there wasn't a driver to use with the
SP yet. This involvement of sewer and DNS-service users is a practical
necessity, as there is no substitute for being able to test the driver
against the DNS-service, and many such services are for-pay or bundled with
other for-pay services.
sewer's [legacy DNS driver interface](LegacyDNS) (BaseDns in dns_providers/common.py)
is deprecated, although there is no plan for its removal other than
_after they have all migrated to the new interface_.
New DNS drivers should use DNSProviderBase from the start, of course,
and will be placed in sewer/providers/ if added to the project.
# sketch of simple dns-01 provider, including alias support
from .. import auth
from .. import lib
class Provider(auth.DNSProviderBase):
def __init__(self, *, my_api_url, my_api_id, my_api_key, **kwargs):
super().__init__(self, **kwargs)
self.api_url = my_api_url
self.api_id = my_api_id
self.api_key = my_api_key
def setup(self, challenges):
for challenge in challenges:
fqdn = self.target_domain(challenge)
txt_value = lib.dns_challenge(challenge["key_auth"])
self.my_api_add_txt(fqdn, txt_value)
def unpropagated(self, challenges):
return [] # if service has a propagation check, use it here
def clear(self, challenges):
# like setup, but calling my_api_del_txt; may not need txt_value
def my_api_add_txt(self, fqdn, txt_value):
# this is where you talk to the DNS service to add a TXT
def my_api_del_txt(self, fqdn):
# talk to DNS service to remove TXT
Most of your work is in implementing the two methods (or one method, or
inline code, but inline makes testing without access to the service more
difficult) which actually communicate with the DNS service. This can be
easy or very difficult, depending on the service provider's API (or lack of
designed API if you have to use a mix of web scraping and HTTP request
generation to operate a mechanism that was designed for interactive use).
The above is bare-bones, not taking advantage of the batching of challenges
which the new-model interface provides - that can be a big win for large-SAN
certificates if you have to grovel the service's API (or web pages) to guide
the construction of your commands to them. It does show the use of
target_domain to support [DNS aliasing](Aliasing).
================================================
FILE: docs/drivers/route53.md
================================================
## route53 - driver for AWS DNS service
### Command line use
route53 has never been wired into `sewer-cli`, and that hasn't really
changed in 0.8.3. It does appear in the list of "known providers", but it
isn't usable, and raises an exception if named by `--provider`.
Adding that integration is on the list, but seeing as no one has complained
about this lack up to now it's nowhere near the top. :-(
### Programmatic use
Apparently everyone using sewer's route53 has been rolling their own
wrapper, since it has only been available for such use to date. There is a
patch to extend that Route53Dns.__init__ to allow additional AWS-specific
methods of authentication which I expect will ship in 0.8.3.
================================================
FILE: docs/drivers/unbound_ssh.md
================================================
## unbound_ssh legacy DNS driver
A working, if somewhat quirky, driver to setup challenges in local data of
the [unbound](https://nlnetlabs.nl/projects/unbound/about/) caching
resolver. As the name suggests, it relies upon ssh to provide an
authenticated connection the server; inside that connection the
`unbound-control` program is used to add and remove the records. The driver
does NOT handle the login authorization, assuming that it is running
interactively and ssh will prompt for your input, or that a key agent (eg.,
ssh-agent) is active to supply the cryptographic credentials. That's the
_somewhat quirky_ part!
### `__init__(self, *, ssh_des, **kwargs)`
There is one REQUIRED parameter, `ssh_des`, which is the login target, such
as acme_user@ns1.example.com. This is simply passed to the ssh command,
along with the `unbound-control` commands to be executed on the destination
machine.
### Driver features
unbound_ssh supports the `alias` parameter.
Only `prop_delay` is supported; there is no custom `unpropagated` method.
### Usage
From the command line:
python3 -m sewer ... --provider=unbound_ssh --p_opt ssh_des=acme@ns.example.com ...
From custom code:
from sewer.dns_providers.unbound_ssh import UnboundSSH
provider = UnboundSSH(ssh_des="acme@ns.example.com", alias="validation.example.com")
...
### Bugs
Sadly, This was written using the old paradigm where both the module name
and the class name were more-or-less the same name aside from
capitalization... and often less predictable changes. Should have been
unbound_ssh.Provider ...
The `unbound-control` commands generated could be run locally with not very
much change to the driver. Perhaps that will become part of a demonstration
of some different features in the future.
================================================
FILE: docs/http-01.md
================================================
# HTTP challenge providers
There are no http-01 drivers in sewer yet.
## Bring your own HTTP provider
**To be rewritten.** For now, see `providers/demo.py` for some hints, and
[UnifiedProviders](UnifiedProviders) for doumentation of the interface.
================================================
FILE: docs/index.md
================================================
## sewer, the ACME library and command-line client
This is a quick & dirty directory of the docs directory, which is still a
work in progress - in particular, some of the files are not yet properly
linked from the README or other docs. There's some overlap and redundancy,
and doubtless some out of date bits. The _internals_ documents include some
that were written more as technical essays when I was sorting out some
issue, and those may not have been updated since before the features were
implemented.
### General and "user" docs.
- [README](https://github.com/komuw/sewer) and project page.
- [CHANGELOG](CHANGELOG) with links to per-release notes when they exist
- [sewer-cli](sewer-cli) Documentation for the command line tool
### Internals (docs for direct users of Client, etc.)
- [sewer-as-a-library](sewer-as-a-library) Rewritten _Usage_ section from
README using new interfaces, etc. WIP.
- [Aliasing.md](Aliasing), just renamed from DNS-Aliasing, so the contents must
still be in flux, too. Probably a lumpy blending of implementation and
user notes, still.
- [catalog](catalog) the driver catalog and support in catalog.py
- [dns-01](dns-01) sewer's existing (legacy) DNS providers as well as some
skeleton code for implementing a new-model driver
- [DNS-Propagation.md](DNS-Propagation) has become (is becoming?) the document
about _what_ propagation means to the ACME process and _how_ we might
manage it. From the title you can tell it began when I was still thinking
of propagation as DNS thing.
- [http-01](http-01) guide for writing HTTP driver **TO DO**
- [LegacyDNS.md](LegacyDNS) Documents the new-model shim (still named
BaseDNS) that allows unmigrated Legacy DNS drivers to continue working.
Authors of any DNS module, Legacy or new, should review this... and,
hopefully, migrate to or start anew as new-model Providers.
- [UnifiedProvider.md](UnifiedProvider) was the first intentional thought
doc. It has been written and revised repeatedly since Alec's original
http-01 changes got me thinking about how best to accomodate both, as well
as other validation methods that were already RFCs or on their way.
- [unpropagated.md](unpropagated) was the first separate piece. Begun as I was
figuring out what to do, and changes upon changes here as well. Heading
towards being documentation of the implementation.
- [wildcards.md](wildcards) Notes about wildcard certificates - they should
work for all drivers now! - and a special case where there are probably
still issues.
- [ACME](ACME) A bit of history, a start on a technical essay, or ???
Notes on various things related to ACME and Let's Encrypt's servers.
================================================
FILE: docs/notes/0.8.2-notes.md
================================================
## 0.8.2 release
0.8.2 contains a lot more work - and changes - than recent releases,
hence this verbose guide to what's been going on in sewer this spring.
To my mind, the big change has been landing the revised RFC protocol changes.
This allows sewer to operate against LE's staging server again,
and to continue to work with their production server when they drop compatibility
with the earlier version of the protocol in November.
Other changes that may be equally important to some users have been the addition
of drivers for the powerdns and gandi DNS services,
and changes to accomodate http-01 challenge providers.
The interface for dns-01 and http-01 challenge providers has been unified
from its initial form, and hopefully that interface is general enough
to accomodate not only dns-01 and http-01, but other future challenge types.
### bugs, fixed or known
There are two related issues with wildcard certificates that have turned up
in some providers.
The first of these was fixed in 0.8.1, when we stopped Client from prefixing
wildcard names with "*." when passing them to the providers.
That issue has been known for a long time, and some providers already had a
workaround - but sometimes the workaround wasn't complete (PR #139, eg.).
The second issue arises only when requesting a wildcard certificate (for
*.domain.tld, say) that is to also cover the naked domain (domain.tld).
This arises when the DNS service has issues with setting up two TXT records
for the two separate challenges ACME needs, because they both are on
domain.tld.
There doesn't seem to be any easy global fix for this, as there was for the
first problem, so it's being fixed provider by provider as it arises (and
there's a user of that service to help with the fix).
### other changes
The *cli* program has, I believe, no user-visible incompatibilties.
================================================
FILE: docs/notes/0.8.3-notes.md
================================================
## Sewer 0.8.3 Release Notes
This will attempt to list all the changes that affect users of the
`sewer-cli` program, including even cosmetic changes. If you use sewer as a
library you may find internal changes not called out here.
**New `sewer-cli` features are usually just mentioned, see
[sewer-cli.md](sewer-cli) for more complete documentation.**
### What's New
- added many words in the /docs directory. A lot of it is internals
documentation; a lot of it was written to help me understand exactly how
some things worked and decide how they should be improved - design essays,
as it were. Much of it is sure to still be in some intermediate state.
- new documentation of the [`sewer-cli` user command](sewer-cli). Tries to
be _as-is in 0.8.3_ while warning about things that are sure to change
later.
- `--acme_timeout` has been added (revised from menduo's PR #154)
- `--p_opt name=value ...` allows passing multiple driver options through
the command line. This is preferred over single-purpose cli options that
were briefly present in pre-release work since they eliminate the need to
manually add new driver options to the cli parser.
- `alias` parameter (in DNSProviderBase) added; available through `--p_opt
alias=...` for `sewer-cli`. **NB: legacy drivers do NOT implement
aliasing yet - code changes required** except for the demonstration DNS
driver, unbound_ssh.
- `prop_delay`, `prop_timeout` and `prop_sleep_times` (in DNSProviderBase)
Available through the `--p_opt` option in `sewer-cli`, though no legacy
provider has the driver support necessary to make `prop_timeout` and
`prop_sleep_times` work.
### What's Changed
Mostly I've tried to avoid changes that were likely to break things. More
so for `sewer-cli` than those who use the inner workings of Client, of
course.
- the default log level for providers changed from INFO to WARNING,
so by default they don't natter so much. Makes `sewer-cli` a little quieter
when there are no problems. (there's more to be done - it looks like some
messages that are clearly informational are written at a higher priority.)
- all the legacy DNS providers have been minimally revised to accept some
new options (alias and prop_delay) and pass them up to their parent classes.
- `sewer-cli` interface to providers has been augmented to pass some new options
(--acme_timeout, driver parameters from --p_opts)
- `sewer-cli` has had long option abbreviations disabled in argparse. This
was never documented in sewer, and is an attractive nuisance since the
addition of another option can break an abbreviation. Everyone wants more
options, right? <wink>
- `unbound_ssh` driver, a working demonstration of using aliasing support in
a legacy DNS driver. Needs a rather specific environment to work, but I
just renewed a handful of certificates using it the other day. :-)
- JSON configuration has arrived. meta.json replaces __version__.py.
catalog.json adds a central description of known drivers, replacing the
mess of imports (removed, see below), and some non-DRY lists in setup.py
and cli.py.
- `sewer.catalog` provides loading of the JSON catalog as well as methods to
lookup the driver's descriptor by name and load the module; replaces the
mess of imports
### Breakage
- removed all the imports in __init__.py (both sewer & dns_providers). This
*will* affect you if you've just done `import sewer` and access especially
the provider classes as eg. `sewer.ThatDNSDns`. ~~Using proper imports,
eg., `import sewer.dns_providers.thatdns.ThatDNSDns` is the current
workaround, sorry.~~ Recommended use is now something like
```
from sewer import catalog
pro_cls = catalog.ProviderCatalog().get_provider("route53")
provider = pro_cls(...arguments as needed...)
```
**This does NOT affect `sewer-cli` users.**
### Deprecated
- `--action {run,renew}` option has never actually had any effect and is no
longer required (since 0.8.2?). LOGS A WARNING IN 0.8.3.
================================================
FILE: docs/notes/0.8.4-notes.md
================================================
# Sewer 0.8.4 Release Notes
## What's New
- ECDSA keys supported (account and/or certificate keys)
- optional `client` argument to route53 driver for special use cases
## What's Changed in the sewer Command
Basically nothing for many users, other than the change in default for
sewer-generated RSA key size to follow current security guidance. You will
have to use the new `*_key_type` options if you have a hard requirement for
2048 bit RSA keys (most likely for the certificate key only).
- `--acct_key` & `--cert_key` should be used to designate the file that
holds the keys to be used (rather than having new ones generated).
`--account_key` & `--certificate_key` are still accepted as synonyms but
will be phased out later.
- add `--acct_key_type` & `--cert_key_type` to allow choice of RSA or EC
keys and key sizes when sewer is generating them for you.
- changed default for generated keys to 3072 bit RSA (had been 2048 bit)
- add `--is_new_key` to allow for first-time registration of your own
account key (using `--acct_key`) generated outside of sewer.
`--is-new-key` allows an externally generated account key (specified with
`--acct_key`) to be registered with ACME when first used. Since there's
been basically no way to do this from the CLI, I doubt the addition will
affect anyone who isn't interested in the new capability.
## Internal changes for library clients:
- Client class methods cert() and renew() are deprecated; just call
get_certificate() directly instead.
- Client class **no longer generates keys**. (see below)
- crytographic refactoring
- AcmeKey, AcmeAccount & AcmeCsr in crypto.py; uses only cryptography library
- This is what enables ECDSA keys and provides selection by name
- Client.__init__ interface changes due to crypto refactoring
- dropped `account_key` and `certificate_key` optional arguments to Client
- added `acct_key` and `cert_key` REQUIRED arguments to Client taking
AcmeAccount and AcmeKey objects, respectively.
- add `is_new_acct` argument to force registration of the supplied account
key
- dropped `bits` argument because Client no longer generates keys!
- dropped `digest` argument since there are currently no alternate digest
methods for the different key types. (was this ever used?)
## Breakage
None that I know of (aside from hard changes in Client() interface listed
above), so let me know if you find any!
## Deprecated
- `--action {run,renew}` option has never actually had any effect and is no
longer required (since 0.8.2?). LOGS A WARNING IN 0.8.3. **Will be
removed in 0.8.5.**
- As mentioned above, CLI `*_key` argument names are changing. Expect they
will issue a warning in 0.8.5 and go away in 0.8.6.
================================================
FILE: docs/preview/cloaca.md
================================================
# Cloaca, aka sewer-cli-next?
_This is so preliminary that it hasn't been written yet. A design essay in
a rambling style is all it is._
The design of the sewer-cli program is focused on getting one certificate
with no state (aside from the optionally reused account key) on disk. I can
say from much personal experience that this is wonderful for doing ad-hoc
tests while changing the implementation, and it's workable even for getting
a handful of certificates (ah, shell script, how I both love and hate
thee...), but I've long thought about a better way - better, at least, for
how I'd like to handle certificate renewal.
## Shortcomings of sewer-cli
One thing that I kept tripping on was the apparent lack of a simple way to
setup a new account key for later use. And strictly speaking, sewer-cli
just doesn't do that. Enlightenment comes when you give proper
consideration to sewer-cli's scope - it gets only one certificate per
invocation, so you can harvest the one it created and reuse it later. I
missed that because I was already looking to get several (three, IIRC)
certificates when I first started using sewer.
The other "issues" that come to mind are just limitations of the intended
scope of sewer-cli. As mentioned above, I ended up doing some shell
scripting to work around _all those --options, some the same and some that
change for each certificate_. And the other part I wanted to automate (and
felt less happy doing in a shell script for $REASONS) was getting the new
certificates installed after they were created.
So cloaca addresses these.
## The cloaca command
First change - the cloaca command takes, as its first, required, non-option
argument, the sub-command which selects the operation to carry out. The
short list so far goes like this:
- account - create key, register, deregister, maybe transfer, others
- renew - with no options, renew & install (if configured) all certs in cloaca.ini
And more to come. Plenty of operations are defined in RFC8555 which never
got into sewer-cli. Goal will be to support all the operations in the RFC
that Let's Encrypt has implemented.
Second change - cloaca is more [config-driven](cloaca_config) than
--option-driven. Which doesn't mean options won't be available, but maybe
not to the point of emulating sewer-cli's ability to renew one certificate
without any configuration.
_Okay, that's it for now. Need to get some proof of concept code to see if
any of the untested ideas are more problematic than I believe._
================================================
FILE: docs/preview/cloaca_config.md
================================================
# Configuration file for cloaca
_This is a pre-coding, let alone release, preview of a more config-driven
user command that's just starting to bubble in the crock pot here. It could
end up being cli-next if it's practical to combine the two different
approaches in one. Either way, it needs a less awkward name than cli-next. :-)_
Cloaca's configuration has been driven by a couple use cases. If you have
only a single certificate, for one or a few identities (SANs), the
traditional option-driven CLI program is perhaps easier. Cloaca is designed
for the case where several certificates are being managed together. It also
adds installation support to get those certificates onto the servers.
Somewhat to my surprise, it has ended up using the simple "ini" file format.
## Introductory examples
A minimal configuration for one certificate for test.example.com that just
leaves test.example.com.key and test.example.com.key sitting in the current
working directory:
[cert_test.example.com]
account_email = webmaster@example.com
provider = demo_dns
This demonstrates an important convention: certificate sections are named by
prepending "cert_" to the domain identity. The only other data it requires
to produce a new key and certificate, is the name of the Provider class that
will handle publishing the challenge responses. As always, the driver's
auhtentication parameters are here assumed to be passed in through
environment variables (but they could be additional items in that section if
you like).
Now, let's add the promised installation to this example:
[cert_test.example.com]
account_email = webmaster@example.com
provider = demo_dns
install_transport = ssh
install_ssh_to = root@test.example.com
install_dir = /etc/apache2/ssl
install_post_cmd = system restart apache2
Getting a little longer, and it glosses over how it gets the ssh key it will
need for the installation. So what happens when we have multiple
certificates?
[cert_default]
account_key = account.key
account_email = webmaster@example.com
provider = demo_dns
install_transport = ssh
install_dir = /etc/apache2/ssl
install_post_cmd = systemctl restart apache2
[cert_test.example.com]
install_ssh_to = root@test.example.com
[cert_example.com]
SAN = www.example.com
install_ssh_to = hostmaster@www.example.com
[cert_webmail.example.com]
install_ssh_to = postmaster@webmail.example.com
install_dir = /etc/certificates
install_post_cmd = systemctl restart dovecot
There's a reason this avoids the native [DEFAULT] section, which we'll come
back to later.
## Configuration Reference
All of the actual configuration options can appear in a certificate section.
A certificate section is any section whose name is formed by prepending
"cert_" to the principle domain name (CN in certificate speak). The
available keys:
- account_key = filepath to account key to use for this certificate
- account_email = what it says, obviously
- account_file = filepath to file with account key, email, and other info TBD
(this might replace _key and _email, those coming as args to account creation?)
- provider = name of Provider class
- provider_KW = value, passed to class constructor as KW=value
- SAN = san1, san2, ... comma-separated list of additional identifiers to add to certificate
- install_transport = name of method to use, eg., ssh, cp, ...
- install_ssh_to = user@fqdn for any ssh-based transport (also for post_cmd, etc)
- install_dir = path (should be absolute) to directory where new key & crt are installed
- install_post_cmd = command to be run after key is installed (to make it take effect)
As seen above, a [cert_default] section will be folded in to every [cert_CN]
section. This is convenient for truly universal settings, but the `_method`
mechanism is preferred for settings that apply less universally. In any
event, an explicit setting in [cert_CN] will always take precedence over
those injected from other sources.
### The `_method` Method
This allows a group of settings that are shared among some of the
certificates to be grouped and defined once, then included by name. More or
less:
[cert_default]
account_key = account.key
provider = demo_dns
[install_apache2]
transport = ssh
dir = /etc/apache2/ssl
post_cmd = systemctl restart apache2
[cert_test.example.com]
install_method = apache2
install_ssh_to = root@test.example.com
[cert_example.com]
SAN = www.example.com
install_method = apache2
install_ssh_to = hostmaster@www.example.com
The mechanism finds keys of the form PREFIX_method = NAME and looks for
a section [PREFIX_NAME]. It then replaces the PREFIX_method item with
the items in that section after prepending PREFIX_ to each key. So in the
above, each occurence of
install_method = apache2
is replaced by items from [install_apache2] which become
install_transport = ssh
install_dir = /etc/apache2/ssl
install_post_cmd = systemctl restart apache2
Which is much like what a [cert_default] section would do except it wouldn't
try to add the apache config items into certs that are for an nginx hosted
domain, say. The `_method` method only injects the settings it's explicitly
asked to insert.
================================================
FILE: docs/sewer-as-a-library.md
================================================
# Sewer as a Python Library
>`sewer-the-library` is in a period of heavy change (summer 2020 - ?). I'll
try to keep the examples (below) and other docs up to date, but I'm sure
things will lag sometimes.
This document is neither a "cookbook" nor in any way a substitute for the
documentation of sewer's parts and internals, such as they are. Let's try
to list the existing docs:
- [Cryptographic library](crypto) this is in decent shape because it's been
created and kept up to date in sync with crypto.py - both too new to have
bit rot yet.
- [ACME protocol](ACME) was a piece I started writing while learning the
quirks of the ACMEprotocol. Quite incomplete, the main thing it brings to
the table is the link to RFC8555, which is the protocol's definition. Of
course there are other foundational RFCs to be read...
- The [driver catalog](catalog) is another new part that isn't yet being
used to its fullest. It glues the drivers and the CLI program together,
and stands ready to help your bespoke front end likewise unless your
target is so specific you can just manually import the only driver you
need.
- Drivers! So much of this is about those intermediaries between sewer and
the diverse services that actually publish our challenge responses.
+ [Unified provider](UnifiedProvider) began as a technical essay when I
was starting to sort through the problems and possibilities Alec's
original http-01 driver support introduced. It's an uneven blend of
design philosophy and code documentation, with plenty of ToDo in it.
+ [Wildcard certificates](wildcards) are one of the things the dns-01
challenge type brought to the table. Some notes on how they work and
what issues remain.
+ [Aliasing](Aliasing) can be a handy technique to manage dns-01
challenges without needing to deal with a primary DNS provider whose
support for fast-propagating, short-lived TXT records leaves something
to be desired.
+ Speaking of DNS Propagation, we find [DNS propagation](DNS-Propagation),
which talks about what it is and why you might need it, and offers what
documentation there is on the parameters to pass to the drivers to
control it. And for driver writers, mostly,
[unpropagated](unpropagated) discusses what's needed to add a probe &
wait timeout loop.
_more to do <sigh>_
## Usage examples
Keep in mind that these are untested code intended to demonstrate how the
major features are used. Supporting details may not be repeated for each
similar example, or may not be present in any of them.
```python
import sewer.client
from sewer.crypto import AcmeKey
# [[ change this to load using the catalog! ]]
import sewer.dns_providers.cloudflare
dns_class = sewer.dns_providers.cloudflare.CloudFlareDns(
CLOUDFLARE_EMAIL='example@example.com',
CLOUDFLARE_API_KEY='nsa-grade-api-key'
)
# 1. to create a new certificate (new account and certificate keys)
client = sewer.client.Client(
domain_name='example.com',
dns_class=dns_class,
acct_key=AcmeKey.create("rsa2048"),
cert_key=AcmeKey.create("rsa2048")
)
certificate = client.cert()
# NB: new crypto keeps keys & certs in python objects. They intentionally
# do not convert to printable form automatically (__str__, etc.)
print("your certificate is:", certificate.private_bytes())
print("your certificate's key is:", cert_key.private_bytes())
print("your letsencrypt.org account key is:", acct_key.private_bytes())
# NB: your certificate_key and account_key should be SECRET.
# You can write these out to individual files, eg::
with open('certificate.crt', 'wb') as f:
f.write(certificate.private_bytes())
with open('certificate.key', 'wb') as f:
f.write(certkey.private_bytes())
with open('account.key', 'w') f:
f.write(acctkey.private_bytes())
# the acct_key also contains ACME's "kid" identifier if you're interested
# 2. to renew a certificate:
dns_class = sewer.dns_providers.cloudflare.CloudFlareDns(
CLOUDFLARE_EMAIL='example@example.com',
CLOUDFLARE_API_KEY='nsa-grade-api-key'
)
# load saved keys or create new as you prefer
acct_key = AcmeKey.from_file("account.key")
cert_key = AcmeKey.from_file("certificate_key")
client = sewer.client.Client(
domain_name='example.com',
dns_class=dns_class,
acct_key=acct_key,
cert_key=cert_key,
)
certificate = client.renew()
certificate_key = client.certificate_key
with open('certificate.crt', 'w') as certificate_file:
certificate_file.write(certificate)
with open('certificate.key', 'w') as certificate_key_file:
certificate_key_file.write(certificate_key)
# 3. You can also request/renew wildcard certificates:
dns_class = sewer.dns_providers.cloudflare.CloudFlareDns(
CLOUDFLARE_EMAIL='example@example.com',
CLOUDFLARE_API_KEY='nsa-grade-api-key'
)
client = sewer.client.Client(
domain_name='*.example.com',
dns_class=dns_class,
# load or create keys
)
certificate = client.cert()
cert_key = client.cert_key
acct_key = client.acct_key
```
================================================
FILE: docs/sewer-cli.md
================================================
## Sewer's user command (so many --options!)
Sewer's command line interface, historically named just "sewer" or
"sewer-cli", and implemented in `sewer/cli.py`, is now also available using
the python command line option (eg. `python3 -m sewer`). In these docs
we'll call it `sewer-cli` in order to avoid already overloaded or generic
names.
The command line tool, however invoked, is still a good vehicle for creating
or renewing a single certificate. Simple cases may need only a few
--options, but as time goes by the possibilities keep increasing. The
official doumentation of the options that `sewer-cli` supports remains the
output from running `sewer-cli --help`, but that can be rather terse. Here
we will discuss what the options are and why they are needed, especially
some recently [or soon-to-be!] added options.
### _key_type_ values
This is new with the crypto overhaul (pre-0.8.4). You can now choose the
type and size of both the account and certificate keys to be generated by
sewer (if you don't pass it existing keys for one or both).
| _key_type_ | key & size | notes |
| --- | :-: | --- |
| rsa2048 | RSA 2048 bits | old sewer default |
| rsa3072 | RSA 3072 bits | NEW sewer default |
| rsa4096 | RSA 4096 bits | |
| secp256r1 | ECDSA 256 bits | |
| secp384r1 | ECDSA 384 bits | |
| secp521r1 | ECDSA 521 bits | **not accepted for certificate key** |
> NOTE that the default generated key has changed from 2048 bit RSA to 3072
bit RSA. This is in keeping with current NIST reccomendations. Unless you
have a need to continue to use RSA account keys (existing scripts assume
RSA, perhaps), one of the ECDSA types is suggested.
The choice of key_type would be easy if not for external factors: ECDSA is
widely preferred on most grounds, but RSA may be required for backwards
compatibility with old software or appliances. Some new applications and
devices, OTOH, are dropping RSA due to its resource demands (CPU time and
memory).
The 521 bit EC key is still valid in the specs, but currently most (?) browsers
don't support it, as LE has chosen to reject certificates using that key
type and size.
### sewer-cli General Options
`--version`
`--known_providers`
> These are both immediate action options. They print their information and
exit, ignoring all other arguments (to include argparse errors).
`--log_level`
`--action` "run"|"renew"
> **OBSOLESCENT**. No longer required in 0.8.3! Default is "renew".
Has no effect other than changing one word used in one message text.
Whether to create a server key and certificate de novo or reuse the existing
server key (the only thing that CAN be reused) depends only on whether
`--certificate_key` is given.
`--acme_timeout` _seconds_ {7}
> Used to adjust the timeout applied to all requests to the ACME server.
If you need to increase this timeout you'll know it <wink>.
_added by #188 in pre-0.8.3; reworked from #154 from @menduo_
### ACME options
`--endpoint` "production"|"staging"
> Default is "production", viz., issue a legitimate certificate. Use
"staging" for testing!
_protocol changes enforced since late 2019 for staging are fixed in 0.8.2_
### Account options
To an ACME server, an account is a key pair which has been registered with
that server. Oh, there's other information that MAY be attached when it is
registered - if you pass you email address to LE you can get timely reminders
about certificates that need to be renewed soon. But basically, it's that
key pair.
By default, `sewer-cli` will create a new, unique key pair each time it's
run. And this is okay, because it will also save the key alongside the
certificate and the key that's attested to by the certificate. But if you
don't want every cert to be issued to a new identity, you'll need to use
`--acct_key` to provide the already-registered one. **New in 0.8.4:** you
can use a new, unregistered account key if you also use the --is_new_acct
option (and email, of course).
After the certificate has been created and downloaded, the account key
`sewer-cli` used will be saved alongside the certificate and certificate
key. After this,the new account key is registered with the ACME endpoint
and may be used for future certificate requests using `--acct_key`.
`--acct_key` _filepath_
> Filepath to existing, already registered ACME account key. Default is to
create a new key and register it. Preferred over `--account_key` now.
`acct_key_type` _key_type_
> Type of key to generate if `--acct_key` not given. Default is rsa3087.
`--is_new_acct`
> Used with `--acct_key`, allows a key you created outside of sewer to be
registered as an account key the first time it's used.
`--email` _email_address_
### Challenge publisher options
`--provider`|`--dns` **name**
> Name of the [DNS] provider to use.
`--dns` is OBSOLESCENT, prefer `--provider` which will be required in 0.9.
As of 0.8.3 this still only supports the legacy DNS providers.
> _ALTERNATE FOR 0.9: make `provider` a required positional parameter,
in accordance with argparse's good advice that
"users expect options to be optional"._
#### Driver parameters
During the pre-0.8.3 work, several long options were added to `sewer-cli`
for individual new driver parameters. These single-use options will be
retired with the release of 0.8.3, as they are all redundant since the
introduction of `--p_opts`.
`--p_opts` name=value ...
>Added late in 0.8.3 development, this will be the only way to pass
parameters into the drivers when 0.8.3 releases. Like `--alt_domains`,
there can be any number of named parameters following `--p_opts`.
##### Propagation management parameters
There are two sorts of things for which we have to wait: the ACME server
(see `--acme_timeout`) and the service provider (especially DNS propagation
across a global anycast service). Although these are USED in the core
engine code, they are SET through the driver. The reasoning is that the
driver is the only part of sewer that might sensibly "know" what sensible
defaults might be for its service provider, and what features are available.
So eg., although `--prop_timeout` is available to all drivers, legacy DNS
drivers might want to issue a warning if it is specified since (unless
they're updated) they do not support the `prop_timeout` mechanism.
`--p_opts prop_delay=<seconds>`
> Adds a fixed delay after the challenge response have all been setup to
allow the challenge to propagate before any other processing; the default is
no delay. This is the simplest of the propagation waiting methods, and the
only one available to ~~unmodified~~ minimally modified legacy DNS drivers
such as those in 0.8.3.
_This was `--prop_delay <seconds>` during 0.8.3. development_
`--p_opts prop_timeout=<seconds>`
> Activates the active propagation checks and sets the timeout for that
process; default is to not do these checks. This requires an implementation
of the driver `unpropagated` method to be useful. Legacy DNS drivers
inherit a null implementation which always reports _all are ready_ which
short-circuits this process. _to be added when there's driver support_
`--p_opts prop_sleep_times=<seconds,seconds,...>`
> Comma-separated list of integer number of seconds to sleep after the
first, second, ... _not all ready_ response from the driver's
`unpropagated` method. The last value is re-used after the list has been
used up. Default is "1,2,4,8". _to be added when there's driver support_
##### DNS driver parameters
`--p_opts alias_domain=<alias_domain_name>`
> Configure an alternate DNS domain in which the challenge responses will be
placed. See [Aliasing](Aliasing) for details. **Legacy DNS
providers accept this, but require further modification to actually apply
the aliasing that's supported by their parent classes.**
_This was `--alias_domain <name>` during 0.8.3 development.`
### Certificate info
`--domain` **CN-name**
> The primary identity for the certificate. REQUIRED, no default. CN-name
is also used to form the default names for a number of files.
`--alt_domains` _SAN-name ..._
> List of alternate identities to be included in the certificate. Not quite
what pedants would call "SAN", since this should NOT include the CN-name.
Multiple identities may be given, and sewer-cli will take all parameters
(aka words) as SAN-names until it encounters another option (double-dash).
Default is an empty list.
`--out_dir` _dirpath_
> Set directory where the certificate and key files will be stored. Default
is to use the current working directory where sewer was run.
`--cert_key` _filepath_
> File path to your existing certificate key. Preferred over
`--certificate_key`. As with `acct_key`, if this is not specified, a new
key will be created. Has a similar effect to certbot's `--reuse-key` (sp?)
if it points to the key file from the previous run.
`--cert_key_type` key_type
> Type of key to generate if `--cert_key` not given. Default is rsa3087.
--bundle_name _basename_
> Base name to use for output file, eg., out_dir/basename.{account.key,crt,key}
Default is to use the CN-name
================================================
FILE: docs/unpropagated.md
================================================
# Waiting for the Challenge to Propagate
When you use a service provider's API to setup a challenge response, how
long does it take before the ACME server can reliably get that answer?
Especially with global anycast DNS services, it can take a while! This
delay between _posted to API_ and _actually online everywhere that matters_
is the propagation delay we're talking about here.
## How shall we wait, let me count the ways
sewer provides two kinds of delay that can be used to deal with propagation
within the service provider's systems. Although this was designed mostly
for DNS validation, it may be needed for other types depending on the
service provider. The two kinds of delay are (1) an unconditional sleep and
(2) an iterative probe/sleep delay loop that has a timeout to keep it from
waiting _too long_.
### To sleep, perchance t'will be enough
The unconditional sleep is implemented in the sewer core logic and is
available to any driver which can pass `prop_delay=seconds_to_sleep` along
to its parent, and so upwards to `ProviderBase`. If set to a positive
value, it simply sleeps for that number of seconds after the challenges have
all been setup.
--- Available in drivers in 0.8.3.
### Check twice, respond once
The iterative probe is a more active sort of delay: it repeatedly calls the
driver's `unpropagated` method to test whether the challenges are all in
place. Sadly, the service providers who most need this kind of check are
probably the ones it is most difficult to meaningfully test: by design, you
cannot know which anycast DNS or CDN machine(s) the ACME server will query,
now which ones you'll get in a simple probe by DNS name. Some service
providers may give you a way to query through their API. Do what you can...
There are two parameters that manage this: `prop_timeout` and the optional
`prop_sleep_times`. The first has to be present for the probing to happen
at all; by default this checking is skipped. the sleep times is an array of
integers, with a default value [1, 2, 4, 8] which causes the loop to sleep 1
second after the first probe if the challenges are not all ready, then 2, 4,
8, and ever after 8 seconds, continuing until it's been at least
prop_timeout seconds since the first probe.
This does also require an implementaion of the `unpropagated` method in the
driver. The only sane default, used in the legacy drivers' shim class, is
to always return success without any actual checking.
## Advice to driver authors and users
Authors: If the service gives you a way to do a meaningful check and it's
needed, please implement `unpropagated`, and mention that in the driver's
documentation. Otherwise, just make sure it inherits or implements a null
check. Feel free to set default values for delay and/or timeout if its
predictable enough, but be sure not to overide the values if the user passes
his own into the driver. And document, document, document!
Users: Check those driver docs to see what's supported. Most of the time, a
goodly `prop_delay` will get you past the propagation most of the time, and
is more likely to be available.
_Driver docs are a WIP. Currently not much to see other than the features
table in [dns-01](dns-01), such as it is._
================================================
FILE: docs/wildcards.md
================================================
# Wildcard Certificates
Since 0.8.2, sewer should be able to request and receive simple wildcard
certificates using any of the DNS drivers. In earlier versions there was an
eccentric re-naming of wildcard targets in the core logic which the drivers
would, sometimes unreliably, remove. _tl;dr: before 0.8.2 it depended on the
driver._
## One issue remains in 0.8.3
Certificates with a wildcard CN name, eg., `domain=*.example.com`, are valid
for all and only the immediate sub domains of example.com. They do NOT
validate for example.com itself, which may come as a surprise if you have
used some other (commercial?) providers, as they may silently add the
naked domain as described below.
To create such "wildcard-plus" certificates in sewer, you would still use
`domain=*.example.com`, then add `alt_domains=example.com`. Sewer itself,
both through sewer-cli as well as the library interface (Client), is fully
capable of handling this. The issue arises when publishing the challenge
response. To a DNS ([1](#footnote1)) driver, this will appear as two
different TXT values for the same name (in this case "example.com").
Traditional DNS systems (inevitable eg.: bind) have no problem with having
multiple TXT records like this, but many DNS service providers are using
very different software. To be honest, the problem some of sewer's drivers
have with this may be in the service provider's core system or just in their
API layer. But we have had a problem using those APIs when setting up such
wildcard-plus-naked-domain certificate's validations, and from here on the
outside we can only deal with them one by one.
<span id="footnote1">(1)</span> HTTP challenges don't have this issue
because each challenge uses a unique file name/URL component.
> There is a general fix that OUGHT to be possible: have sewer's core logic
recognize cases where such "duplicate" challenges exist, and if the driver
doesn't announce itself capable of handling it, use a multi-step process to
publish a non-overlapping subset of the challenges, wait for propagation,
respond to the ACME server, and wait for the challenges to be validated,
then remove the subset; then repeat the whole process for the "duplicate"
validation. For now, my (mmaney) "plan" is to continue helping driver
authors fix the drivers they're familiar with as a bug is reported (and try
to talk them into migrating to the new-model interface, of course!).
================================================
FILE: mypy.ini
================================================
# ignoring missing annotations is still a way of life. In general, add
# mypy-package here for general-purpose libraries (cryptography.x509.oid and
# tldextract are good examples); use local "# type: ignore" markup for the
# service-specific libraries used in drivers (eg. boto3 in route53.py)
#
# sadly, Guido resists using pyproject.toml, so this file must add its clutter.
[mypy]
[mypy-cryptography.x509.oid]
ignore_missing_imports = True
[mypy-tldextract]
ignore_missing_imports = True
================================================
FILE: pyproject.toml
================================================
[tool.coverage.run]
command_line = "-m pytest"
source = ["sewer"]
omit = [
"*test*",
"*__init__.py"
]
data_file = "tests/coverage/.coverage"
[tool.coverage.report]
fail_under = 85
show_missing = true
omit = [
"*__main__.py",
"*acme.py",
"*cli.py"
]
[tool.coverage.html]
directory = "tests/data/coverage/html"
[tool.pytest.ini_options]
console_output_style = "classic"
addopts = "--color=no"
# nigri delenda est - getting to be more trouble than help (no, NOT kwargs, )
[tool.black]
line-length = 100
target-version = [
"py37",
"py38",
"py39",
"py310",
"py311",
"py312",
]
================================================
FILE: setup.py
================================================
import codecs, json, os
from setuptools import setup, find_packages
# long description comes from README.md
with codecs.open("README.md", "r", encoding="utf8") as f:
long_description = f.read()
ldct = "text/markdown"
# version and other fields in about, with envvar override
with codecs.open(os.path.join("sewer", "meta.json"), "r", encoding="utf8") as f:
meta = json.load(f)
for k in meta:
if "SETUP_" + k in os.environ:
meta[k] = os.environ["SETUP_" + k]
# provider catalog, used to construct the list of extras and their deps, and all their deps
with codecs.open(os.path.join("sewer", "catalog.json"), "r", encoding="utf8") as f:
catalog = json.load(f)
provider_deps_map = dict((i["name"], i["deps"]) for i in catalog)
all_deps_of_all_providers = list(set(sum((i["deps"] for i in catalog), [])))
setup(
long_description=long_description,
long_description_content_type=ldct,
classifiers=[
"Development Status :: 4 - Beta",
"Environment :: Console",
"Intended Audience :: Developers",
"Topic :: Software Development :: Build Tools",
"Topic :: Internet :: WWW/HTTP",
"Topic :: Security",
"Topic :: System :: Installation/Setup",
"Topic :: System :: Networking",
"Topic :: System :: Systems Administration",
"Topic :: Utilities",
"License :: OSI Approved :: MIT License",
# Specify the Python versions you support here. In particular, ensure
# that you indicate whether you support Python 2, Python 3 or both.
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
],
packages=find_packages(exclude=["docs", "*tests*"]),
install_requires=["requests", "cryptography"],
extras_require=dict(
provider_deps_map,
dev=["twine", "wheel"],
test=["mypy>=0.780", "coverage>=5.0", "pytest>=6.0", "pylint>=2.6.0", "black==19.10b0"],
alldns=all_deps_of_all_providers,
),
# data files to be placed in project directory, not zip safe but zips suck anyway
package_data={"sewer": ["*.json"]},
zip_safe=False,
# To provide executable scripts, use entry points in preference to the
# "scripts" keyword. Entry points provide cross-platform support and allow
# pip to create the appropriate form of executable for the target platform.
# entry_points={
# 'console_scripts': [
# 'sample=sample:main',
# ],
# },
entry_points={"console_scripts": ["sewer=sewer.cli:main", "sewer-cli=sewer.cli:main"]},
### CANNOT FIX ### black sometimes ignores explicit version and adds the invalid comma anyway
# fmt: off
**meta
# fmt: on
)
================================================
FILE: sewer/__init__.py
================================================
================================================
FILE: sewer/__main__.py
================================================
from . import cli
cli.main()
================================================
FILE: sewer/auth.py
================================================
from typing import Any, Dict, Optional, Sequence, Tuple, Union, cast
from .lib import create_logger, LoggerType
ChalItemType = Dict[str, str]
ChalListType = Sequence[ChalItemType]
ErrataItemType = Tuple[str, str, ChalItemType]
ErrataListType = Sequence[ErrataItemType]
class ProviderBase:
"""
New-model driver documentation is in docs/UnifiedProvider.md
"""
def __init__(
self,
*,
chal_types: Sequence[str],
logger: Optional[LoggerType] = None,
LOG_LEVEL: str = "INFO",
prop_delay: int = 0,
prop_timeout: int = 0,
prop_sleep_times: Union[Sequence[int], int] = (1, 2, 4, 8),
) -> None:
# TypeError if missing, still check that it's a sequencey value; non-str vals, meh
if not isinstance(chal_types, (list, tuple)):
raise ValueError("chal_types must be a list or tuple of strings, not: %s" % chal_types)
self.chal_types = chal_types
# setup logging. let it pass if both are given; logger supersedes old LOG_LEVEL
if logger:
self.logger = logger
else:
self.logger = create_logger(__name__, LOG_LEVEL)
# prop_* control delay before and timeout of checking loop as well as internal sleeps
self.prop_delay = int(prop_delay)
self.prop_timeout = int(prop_timeout)
### eratta ### accepts str value(s) that pass int(); low importance
if isinstance(prop_sleep_times, (list, tuple)):
self.prop_sleep_times = tuple(int(v) for v in prop_sleep_times)
else:
self.prop_sleep_times = (int(cast(int, prop_sleep_times)),)
def setup(self, challenges: ChalListType) -> ErrataListType:
raise NotImplementedError("setup method not implemented by %s" % self.__class__)
def unpropagated(self, challenges: ChalListType) -> ErrataListType:
raise NotImplementedError("unpropagated method not implemented by %s" % self.__class__)
def clear(self, challenges: ChalListType) -> ErrataListType:
raise NotImplementedError("clear method not implemented by %s" % self.__class__)
class HTTPProviderBase(ProviderBase):
"""
Base class for new-model HTTP drivers
Currently this is a null adapter, holding a place in line for any future shared
implementation. It may never become non-null aside from the default chal_types
it provides, but it's a small price to pay to avoid having to stuff it in later.
"""
def __init__(self, **kwargs: Any) -> None:
if "chal_types" not in kwargs:
kwargs["chal_types"] = ["http-01"]
super().__init__(**kwargs)
class DNSProviderBase(ProviderBase):
"""
Base class for new-model DNS drivers - legacy drivers use the one in common.py
Accepts the alias optional argument and adds cname_domain and target_domain
to support the implementation of aliasing in drivers that inherit from it.
"""
def __init__(self, *, alias: str = "", **kwargs: Any) -> None:
if "chal_types" not in kwargs:
kwargs["chal_types"] = ["dns-01"]
super().__init__(**kwargs)
self.alias = alias
### support for using a DNS alias
def cname_domain(self, chal: Dict[str, str]) -> Union[str, None]:
"returns fqdn where CNAME should be if aliasing, else None"
return "_acme-challenge." + chal["ident_value"] if self.alias else None
def target_domain(self, chal: Dict[str, str]) -> str:
"returns fqdn where challenge TXT should be placed"
d = chal["ident_value"]
return "_acme-challenge." + d if not self.alias else d + "." + self.alias
================================================
FILE: sewer/catalog.json
================================================
[
{
"name": "acmedns",
"desc": "AcmeDns DNS provider",
"chals": ["dns-01"],
"args": [
{
"name": "api_user",
"req": 1,
"old_param": "ACME_DNS_API_USER"
},
{
"name": "api_key",
"req": 1,
"old_param": "ACME_DNS_API_KEY"
},
{
"name": "api_base_url",
"req": 1,
"old_param": "ACME_DNS_API_BASE_URL"
}
],
"path": "sewer.dns_providers.acmedns",
"cls": "AcmeDnsDns",
"deps": ["dnspython"]
},
{
"name": "aliyun",
"desc": "Alibaba Cloud DNS service",
"chals": ["dns-01"],
"args": [
{
"name": "ak",
"req": 1,
"old_param": "aliyun_ak",
"old_envvar": "ALIYUN_AK_ID"
},
{
"name": "secret",
"req": 1,
"old_param": "aliyun_secret",
"old_envvar": "ALIYUN_AK_SECRET"
},
{
"name": "endpoint",
"old_param": "aliyun_endpoint",
"old_envvar": "ALIYUN_ENDPOINT"
}
],
"path": "sewer.dns_providers.aliyundns",
"cls": "AliyunDns",
"deps": ["aliyun-python-sdk-core-v3", "aliyun-python-sdk-alidns"],
"memo": "default value of endpoint in ad-hoc cli.py code - add default to args?"
},
{
"name": "aurora",
"desc": "Aurora DNS service from pcextreme hosting",
"chals": ["dns-01"],
"args": [
{
"name": "api_key",
"req": 1
},
{
"name": "secret_key",
"req": 1
}
],
"path": "sewer.dns_providers.auroradns",
"cls": "AuroraDns",
"deps": ["tldextract", "apache-libcloud"]
},
{
"name": "cloudflare",
"desc": "Cloudflare DNS using either email & key or just a token",
"chals": ["dns-01"],
"args": [
{
"name": "email"
},
{
"name": "api_key"
},
{
"name": "api_base_url"
},
{
"name": "token"
}
],
"path": "sewer.dns_providers.cloudflare",
"cls": "CloudFlareDns",
"deps": [],
"memo": "accepts EITHER token OR both email & key; driver MUST sanity check"
},
{
"name": "cloudns",
"desc": "ClouDNS service",
"chals": ["dns-01"],
"args": [
],
"path": "sewer.dns_providers.cloudns",
"cls": "ClouDNSDns",
"deps": ["cloudns-api"],
"memo": "API library grovels the environment for its access parameters directly"
},
{
"name": "dnspod",
"desc": "DNSPod DNS provider",
"chals": ["dns-01"],
"args": [
{
"name": "id",
"req": 1
},
{
"name": "api_key",
"req": 1
},
{
"name": "api_base_url"
}
],
"path": "sewer.dns_providers.dnspod",
"cls": "DNSPodDns",
"deps": [],
"memo": "api_base_url not usually used? [VERIFY]"
},
{
"name": "duckdns",
"desc": "DuckDNS DNS provider",
"chals": ["dns-01"],
"args": [
{
"name": "token",
"req": 1,
"old_param": "duckdns_token",
"old_envvar": "DUCKDNS_TOKEN"
},
{
"name": "api_base_url",
"old_param": "DUCKDNS_API_BASE_URL",
"old_envvar": ""
}
],
"path": "sewer.dns_providers.duckdns",
"cls": "DuckDNSDns",
"deps": [],
"memo": "as-is code does not look for envvar for base_url; maybe for testing only?"
},
{
"name": "gandi",
"desc": "Gandi DNS service",
"chals": ["dns-01"],
"args": [
{
"name": "api_key",
"req": 1,
"old_param": "GANDI_API_KEY"
}
],
"path": "sewer.dns_providers.gandi",
"cls": "GandiDns",
"deps": []
},
{
"name": "hurricane",
"desc": "Hurricane Electric DNS service",
"chals": ["dns-01"],
"args": [
{
"name": "username",
"req": 1,
"old_param": "he_username"
},
{
"name": "password",
"req": 1,
"old_param": "he_password"
}
],
"path": "sewer.dns_providers.hurricane",
"cls": "HurricaneDns",
"deps": ["hurricanedns"]
},
{
"name": "powerdns",
"desc": "PowerDNS DNS provider",
"chals": ["dns-01"],
"args": [
{
"name": "api_key",
"req": 1,
"old_param": "powerdns_api_key",
"old_envvar": "POWERDNS_API_KEY"
},
{
"name": "api_url",
"req": 1,
"old_param": "powerdns_api_url",
"old_envvar": "POWERDNS_API_URL"
}
],
"path": "sewer.dns_providers.powerdns",
"cls": "PowerDNSDns",
"deps": [],
"memo": "could drop old_envvar if prediction ignored old_param?"
},
{
"name": "rackspace",
"desc": "Rackspace DNS service",
"chals": ["dns-01"],
"args": [
{
"name": "username",
"req": 1
},
{
"name": "api_key",
"req": 1
}
],
"path": "sewer.dns_providers.rackspace",
"cls": "RackspaceDns",
"deps": ["tldextract"]
},
{
"name": "route53",
"desc": "Amazon cloud DNS service",
"chals": ["dns-01"],
"args": [
{
"name": "id",
"req": 1,
"old_param": "access_key_id"
},
{
"name": "key",
"req": 1,
"old_param": "secret_access_key"
}
],
"path": "sewer.dns_providers.route53",
"cls": "Route53Dns",
"deps": ["boto3"],
"memo": "DUMMY LISTING: route53 has never been integrated into cli.py, so this DOESN'T WORK yet"
},
{
"name": "unbound_ssh",
"desc": "Working demonstrater of legacy DNS adopting new features",
"chals": ["dns-01"],
"args": [
{
"name": "ssh_des",
"req": 1,
"envvar": ""
}
],
"path": "sewer.dns_providers.unbound_ssh",
"cls": "UnboundSsh",
"features": ["alias"],
"deps": []
}
]
================================================
FILE: sewer/catalog.py
================================================
import codecs, importlib, json, os
from typing import Dict, List, Sequence
from .auth import ProviderBase
class ProviderDescriptor:
def __init__(
self,
*,
name: str,
desc: str,
chals: Sequence[str],
args: Sequence[Dict[str, str]],
deps: Sequence[str],
path: str = None,
cls: str = None,
features: Sequence[str] = None,
memo: str = None,
) -> None:
"initialize a driver descriptor from one item in the catalog"
self.name = name
self.desc = desc
self.chals = chals
self.args = args
self.deps = deps
self.path = path
self.cls = cls
self.features = [] if features is None else features
self.memo = memo
def __str__(self) -> str:
return "Descriptor %s" % self.name
def get_provider(self) -> ProviderBase:
"return the class that implements this driver"
module_name = self.path if self.path else ("sewer.providers." + self.name)
module = importlib.import_module(module_name)
return getattr(module, self.cls if self.cls else "Provider")
class ProviderCatalog:
def __init__(self, filepath: str = "") -> None:
"intialize a catalog from either the default catalog.json or one named by filepath"
if not filepath:
here = os.path.abspath(os.path.dirname(__file__))
filepath = os.path.join(here, "catalog.json")
with codecs.open(filepath, "r", encoding="utf8") as f:
raw_catalog = json.load(f)
items = {} # type: Dict[str, ProviderDescriptor]
for item in raw_catalog:
k = item["name"]
if k in items:
print("WARNING: duplicate name %s skipped in catalog %s" % (k, filepath))
else:
items[k] = ProviderDescriptor(**item)
self.items = items
def get_item_list(self) -> List[ProviderDescriptor]:
"return the list of items in the catalog, sorted by name"
res = [i for i in self.items.values()]
res.sort(key=lambda i: i.name)
return res
def get_descriptor(self, name: str) -> ProviderDescriptor:
"return the ProviderDescriptor that matches name"
return self.items[name]
def get_provider(self, name: str) -> ProviderBase:
"return the class that implements the named driver"
return self.get_descriptor(name).get_provider()
================================================
FILE: sewer/cli.py
================================================
import argparse, os
from . import client, config, lib
from .catalog import ProviderCatalog
from .crypto import AcmeKey, AcmeAccount, key_type_choices
DEFAULT_KEY_TYPE = "rsa3072"
def setup_parser(catalog):
"""
return configured ArgumentParser - catalog-driven list of providers
"""
parser = argparse.ArgumentParser(
prog="sewer",
description="Sewer is an ACME client for getting certificates from Let's Encrypt",
allow_abbrev=False,
formatter_class=argparse.RawTextHelpFormatter,
)
### immediate action "options"
parser.add_argument(
"--version",
action="version",
version="%(prog)s {version}".format(version=lib.sewer_meta("version")),
help="The currently installed sewer version.",
)
parser.add_argument(
"--known_providers",
action="version",
version="Known Providers:\n "
+ "\n ".join("%s %s" % (i.name, i.desc) for i in catalog.get_item_list()),
help="Show a list of the known providers and exit.",
)
### ACME account options
parser.add_argument(
"--acct_key",
"--account_key",
dest="acct_key_file",
type=argparse.FileType("rb"),
help="File to load registered ACME account key from. Default is to create one.",
)
parser.add_argument(
"--acct_key_type",
choices=key_type_choices,
default=DEFAULT_KEY_TYPE,
help=(
"Type of acct key to generate if not loaded by --acct_key. Default %s."
% DEFAULT_KEY_TYPE
),
),
parser.add_argument("--email", help="Email to be used for registration of an ACME account.")
parser.add_argument(
"--is_new_acct",
action="store_true",
help="Register the key (from --acct_key) rather than assuming it's already registered.",
),
### certificate options
parser.add_argument(
"--cert_key",
"--certificate_key",
dest="cert_key_file",
type=argparse.FileType("rb"),
help="File to load existing certificate key from. Default is to create key.",
)
parser.add_argument(
"--cert_key_type",
choices=[kt for kt in key_type_choices if kt != "secp521r1"],
default=DEFAULT_KEY_TYPE,
help=(
"Type of cert key to generate if not loaded by --cert_key. Default %s."
% DEFAULT_KEY_TYPE
),
),
parser.add_argument(
"--domain",
required=True,
help="The DNS identity which will be the certificate's Common Name. May be a wildcard.",
)
parser.add_argument(
"--alt_domains",
default=[],
nargs="*",
help="Optional alternate (SAN) identities to be added to the CN on this certificate.",
)
parser.add_argument(
"--bundle_name",
help="The basename for the output files. Default is the CN given by --domain.",
)
parser.add_argument(
"--out_dir",
default=os.getcwd(),
help="Directory that stores certificate and keys files; current dir is default.",
)
### challenge provider options
parser.add_argument(
"--provider",
"--dns",
metavar="<name>",
dest="provider",
required=True,
choices=[i.name for i in catalog.get_item_list()],
help="Name of the challenge provider to use. (--dns is OBSOLESCENT; prefer --provider)",
)
parser.add_argument(
"--p_opts", default=[], nargs="*", help="Option(s) to pass to provider, each is key=value"
)
### protocol options
parser.add_argument(
"--endpoint",
default="production",
choices=["production", "staging"],
help="Select between Let's Encrypt's endpoints. Default is production.",
)
parser.add_argument(
"--acme_timeout",
type=int,
default=7,
help="Timeout (maximum wait) for all requests to the ACME service. Default is 7",
)
### sewer command options
parser.add_argument(
"--loglevel",
default="INFO",
choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
help="The log level to output log messages at. \
eg: --loglevel DEBUG",
)
parser.add_argument(
"--action",
choices=["run", "renew"],
default="none",
help="[DEPRECATED] The action that you want to perform (has never done anything).",
)
return parser
def get_provider(provider_name, provider_kwargs, catalog, logger):
"""
return class (or callable) that will return the Provider instance to use
"""
### TODO ### part of catalog's motivation is to replace all this ad hoc copypasta.
if provider_name == "cloudflare":
from .dns_providers.cloudflare import CloudFlareDns
CLOUDFLARE_EMAIL = os.environ.get("CLOUDFLARE_EMAIL", None)
CLOUDFLARE_API_KEY = os.environ.get("CLOUDFLARE_API_KEY", None)
CLOUDFLARE_TOKEN = os.environ.get("CLOUDFLARE_TOKEN", None)
if CLOUDFLARE_EMAIL and CLOUDFLARE_API_KEY and not CLOUDFLARE_TOKEN:
dns_class = CloudFlareDns(
CLOUDFLARE_EMAIL=CLOUDFLARE_EMAIL,
CLOUDFLARE_API_KEY=CLOUDFLARE_API_KEY,
**provider_kwargs,
)
elif CLOUDFLARE_TOKEN and not CLOUDFLARE_EMAIL and not CLOUDFLARE_API_KEY:
dns_class = CloudFlareDns(CLOUDFLARE_TOKEN=CLOUDFLARE_TOKEN, **provider_kwargs)
else:
err = (
"ERROR:: Please supply either CLOUDFLARE_EMAIL and CLOUDFLARE_API_KEY"
"or CLOUDFLARE_TOKEN as environment variables."
)
logger.error(err)
raise KeyError(err)
elif provider_name == "aurora":
from .dns_providers.auroradns import AuroraDns
try:
AURORA_API_KEY = os.environ["AURORA_API_KEY"]
AURORA_SECRET_KEY = os.environ["AURORA_SECRET_KEY"]
dns_class = AuroraDns(
AURORA_API_KEY=AURORA_API_KEY,
AURORA_SECRET_KEY=AURORA_SECRET_KEY,
**provider_kwargs,
)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "acmedns":
from .dns_providers.acmedns import AcmeDnsDns
try:
ACME_DNS_API_USER = os.environ["ACME_DNS_API_USER"]
ACME_DNS_API_KEY = os.environ["ACME_DNS_API_KEY"]
ACME_DNS_API_BASE_URL = os.environ["ACME_DNS_API_BASE_URL"]
dns_class = AcmeDnsDns(
ACME_DNS_API_USER=ACME_DNS_API_USER,
ACME_DNS_API_KEY=ACME_DNS_API_KEY,
ACME_DNS_API_BASE_URL=ACME_DNS_API_BASE_URL,
**provider_kwargs,
)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "aliyun":
from .dns_providers.aliyundns import AliyunDns
try:
aliyun_ak = os.environ["ALIYUN_AK_ID"]
aliyun_secret = os.environ["ALIYUN_AK_SECRET"]
aliyun_endpoint = os.environ.get("ALIYUN_ENDPOINT", "cn-beijing")
dns_class = AliyunDns(aliyun_ak, aliyun_secret, aliyun_endpoint, **provider_kwargs)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "hurricane":
from .dns_providers.hurricane import HurricaneDns
try:
he_username = os.environ["HURRICANE_USERNAME"]
he_password = os.environ["HURRICANE_PASSWORD"]
dns_class = HurricaneDns(he_username, he_password, **provider_kwargs)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "rackspace":
from .dns_providers.rackspace import RackspaceDns
try:
RACKSPACE_USERNAME = os.environ["RACKSPACE_USERNAME"]
RACKSPACE_API_KEY = os.environ["RACKSPACE_API_KEY"]
dns_class = RackspaceDns(RACKSPACE_USERNAME, RACKSPACE_API_KEY, **provider_kwargs)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "dnspod":
from .dns_providers.dnspod import DNSPodDns
try:
DNSPOD_ID = os.environ["DNSPOD_ID"]
DNSPOD_API_KEY = os.environ["DNSPOD_API_KEY"]
dns_class = DNSPodDns(DNSPOD_ID, DNSPOD_API_KEY, **provider_kwargs)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "duckdns":
from .dns_providers.duckdns import DuckDNSDns
try:
duckdns_token = os.environ["DUCKDNS_TOKEN"]
dns_class = DuckDNSDns(duckdns_token=duckdns_token, **provider_kwargs)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "cloudns":
from .dns_providers.cloudns import ClouDNSDns
try:
dns_class = ClouDNSDns(**provider_kwargs)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "powerdns":
from .dns_providers.powerdns import PowerDNSDns
try:
powerdns_api_key = os.environ["POWERDNS_API_KEY"]
powerdns_api_url = os.environ["POWERDNS_API_URL"]
dns_class = PowerDNSDns(powerdns_api_key, powerdns_api_url, **provider_kwargs)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "gandi":
from .dns_providers.gandi import GandiDns
try:
gandi_api_key = os.environ["GANDI_API_KEY"]
dns_class = GandiDns(GANDI_API_KEY=gandi_api_key, **provider_kwargs)
except KeyError as e:
logger.error("ERROR:: Please supply {0} as an environment variable.".format(str(e)))
raise
elif provider_name == "unbound_ssh":
from .dns_providers.unbound_ssh import UnboundSsh
# check & report, let calling protocol crash it.
if "ssh_des" not in provider_kwargs:
logger.error("ERROR: unbound_ssh REQUIRES ssh_des option.")
dns_class = UnboundSsh(**provider_kwargs) # pylint: disable=E1125
elif provider_name == "route53":
raise ValueError("route53 driver can only be used programmatically at this time, sorry")
else:
raise ValueError("The dns provider {0} is not recognised.".format(provider_name))
logger.info("Using %s as registered provider.", provider_name)
return dns_class
def main():
"See docs/sewer-cli.md for docs & examples"
catalog = ProviderCatalog()
parser = setup_parser(catalog)
args = parser.parse_args()
loglevel = args.loglevel
logger = lib.create_logger(None, loglevel)
provider_name = args.provider
domain = args.domain
alt_domains = args.alt_domains
if args.action != "none":
logger.warning("DEPRECATION WARNING: --action option is obsolete and will be dropped soon")
bundle_name = args.bundle_name
endpoint = args.endpoint
email = args.email
out_dir = args.out_dir
### FIX ME ### to keep special options --domain_alias & --prop-*, or use -p_opts instead?
provider_kwargs = {}
for p in args.p_opts:
parts = p.split("=")
if len(parts) == 2:
provider_kwargs[parts[0]] = parts[1]
# Make sure the output dir user specified is writable
if not os.access(out_dir, os.W_OK):
raise OSError("The dir '{0}' is not writable".format(out_dir))
if args.acct_key_file:
account = AcmeAccount.from_pem(args.acct_key_file.read())
is_new_acct = args.is_new_acct
else:
account = AcmeAccount.create(args.acct_key_type)
is_new_acct = True
if args.cert_key_file:
cert_key = AcmeKey.from_pem(args.cert_key.read())
else:
cert_key = AcmeKey.create(args.cert_key_type)
if bundle_name:
file_name = bundle_name
else:
file_name = "{0}".format(domain)
if endpoint == "staging":
ACME_DIRECTORY_URL = config.ACME_DIRECTORY_URL_STAGING
else:
ACME_DIRECTORY_URL = config.ACME_DIRECTORY_URL_PRODUCTION
dns_class = get_provider(provider_name, provider_kwargs, catalog, logger)
acme_client = client.Client(
provider=dns_class,
domain_name=domain,
domain_alt_names=alt_domains,
contact_email=email,
account=account,
is_new_acct=is_new_acct,
cert_key=cert_key,
ACME_DIRECTORY_URL=ACME_DIRECTORY_URL,
LOG_LEVEL=loglevel,
ACME_REQUEST_TIMEOUT=args.acme_timeout,
)
# prepare file path
account_key_file_path = os.path.join(out_dir, "{0}.account.key".format(file_name))
crt_file_path = os.path.join(out_dir, "{0}.crt".format(file_name))
crt_key_file_path = os.path.join(out_dir, "{0}.key".format(file_name))
# write out account_key in out_dir directory
account.write_pem(account_key_file_path)
logger.info("account key succesfully written to {0}.".format(account_key_file_path))
certificate = acme_client.get_certificate()
# write out certificate and certificate key in out_dir directory
with open(crt_file_path, "w") as certificate_file:
certificate_file.write(certificate)
logger.info("certificate succesfully written to {0}.".format(crt_file_path))
cert_key.write_pem(crt_key_file_path)
logger.info("certificate key succesfully written to {0}.".format(crt_key_file_path))
================================================
FILE: sewer/client.py
================================================
import json, time, platform
from hashlib import sha256
from typing import Dict, Sequence, Tuple, Union
import requests
from .auth import ChalListType, ErrataListType, ProviderBase
from .config import ACME_DIRECTORY_URL_PRODUCTION
from .crypto import AcmeCsr, AcmeKey, AcmeAccount
from .lib import create_logger, log_response, safe_base64, sewer_meta, AcmeRegistrationError
class Client:
"""
refer to docs/sewer-as-a-library for usage, etc.
"""
def __init__(
self,
*,
domain_name: str,
account: AcmeAccount,
cert_key: AcmeKey,
is_new_acct=False,
dns_class: ProviderBase = None,
domain_alt_names: Sequence[str] = None,
contact_email: str = None,
provider: ProviderBase = None,
ACME_REQUEST_TIMEOUT: int = 7,
ACME_AUTH_STATUS_WAIT_PERIOD: int = 8,
ACME_AUTH_STATUS_MAX_CHECKS: int = 3,
ACME_DIRECTORY_URL: str = ACME_DIRECTORY_URL_PRODUCTION,
ACME_VERIFY: bool = True,
LOG_LEVEL: str = "INFO",
):
### do some type checking of some parameters
### FIX ME ### spotty and not always complete; some should raise TypeError, not ValueError
if not isinstance(domain_alt_names, (type(None), list)):
raise ValueError(
"domain_alt_names should be None or a list of strings, not %s" % domain_alt_names
)
if not isinstance(contact_email, (type(None), str)):
raise ValueError("contact_email should be None or a string, not %s" % contact_email)
if LOG_LEVEL.upper() not in ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]:
raise ValueError(
"LOG_LEVEL must be one of 'DEBUG', 'INFO', 'WARNING', 'ERROR' or 'CRITICAL'"
)
if dns_class is not None and provider is not None:
raise ValueError(
"Client was passed both the DEPRECATED dns_class argument and provider."
)
if not isinstance(account, AcmeAccount):
raise TypeError("The account argument must be an AcmeAccount.")
if not isinstance(cert_key, AcmeKey):
raise TypeError("The argument cert_key must be an AcmeKey.")
### setup Client's global variables
self.domain_name = domain_name
# long winded is both stricter check as well as giving mypy a clear enough hint
if isinstance(provider, ProviderBase):
self.provider = provider
elif isinstance(dns_class, ProviderBase):
self.provider = dns_class
if not domain_alt_names:
domain_alt_names = []
self.domain_alt_names = list(set(domain_alt_names))
self.contact_email = contact_email
self.ACME_REQUEST_TIMEOUT = ACME_REQUEST_TIMEOUT
self.ACME_AUTH_STATUS_WAIT_PERIOD = ACME_AUTH_STATUS_WAIT_PERIOD
self.ACME_AUTH_STATUS_MAX_CHECKS = ACME_AUTH_STATUS_MAX_CHECKS
self.ACME_DIRECTORY_URL = ACME_DIRECTORY_URL
self.ACME_VERIFY = ACME_VERIFY
self.LOG_LEVEL = LOG_LEVEL.upper()
self.account = account
self.cert_key = cert_key
self.is_new_acct = is_new_acct
self.logger = create_logger(__name__, LOG_LEVEL)
try:
self.all_domain_names = [self.domain_name] + self.domain_alt_names
self.User_Agent = self.get_user_agent()
acme_endpoints = self.get_acme_endpoints().json()
self.ACME_GET_NONCE_URL = acme_endpoints["newNonce"]
self.ACME_TOS_URL = acme_endpoints["meta"]["termsOfService"]
self.ACME_KEY_CHANGE_URL = acme_endpoints["keyChange"]
self.ACME_NEW_ACCOUNT_URL = acme_endpoints["newAccount"]
self.ACME_NEW_ORDER_URL = acme_endpoints["newOrder"]
self.ACME_REVOKE_CERT_URL = acme_endpoints["revokeCert"]
self.acme_csr = AcmeCsr(cn=domain_name, san=domain_alt_names, key=self.cert_key)
if dns_class is not None:
self.logger.warning(
"DEPRECATED parameter 'dns_class' will be removed in 0.9; use 'provider' instead"
)
self.logger.info(
"intialise_success, sewer_version={0}, domain_names={1}, acme_server={2}".format(
sewer_meta("version"),
self.all_domain_names,
self.ACME_DIRECTORY_URL[:20] + "...",
)
)
### FIX ME ### [:100] is bandaid to reduce spew during tests
except Exception as e:
self.logger.error("Unable to intialise Client. error={0}".format(str(e)[:100]))
raise e
def GET(self, url: str) -> requests.Response:
"""
wrap requests.get (and post and head, below) to allow:
* injection of e.g. UserAgent header in one place rather than all over
* hides requests itself to allow for change (unlikely) or use of Session
* paves the way to inject the verify option, required to use pebble
"""
return self._request("GET", url)
# HEAD is still waiting for the test rewrite to let it be used... very low priority :-(
def HEAD(self, url: str) -> requests.Response:
return self._request("HEAD", url)
def POST(
self, url: str, *, data: bytes = None, headers: Dict[str, str] = None
) -> requests.Response:
return self._request("POST", url, data=data, headers=headers)
def _request(
self, method: str, url: str, *, data: bytes = None, headers: Dict[str, str] = None
) -> requests.Response:
"""
shared implementation for GET, POST and HEAD
* injects standard request options unless they are already given in headers
* header:UserAgent, timeout
* verify - this is a hack to make sewer accept pebble's intentionally bogus cert
"""
if headers is None:
headers = {}
if "UserAgent" not in headers:
headers["UserAgent"] = self.User_Agent
kwargs = {"timeout": self.ACME_REQUEST_TIMEOUT} # type: Dict[str, Union[str, int]]
### FIX ME ### can get current bogus cert from pebble, figure out how to use it here?
# if ACME_VERIFY is false, disable certificate check in request
if not self.ACME_VERIFY:
kwargs["verify"] = False
# this is what we'd do if damn near every test didn't mock requests.{get,post}
# response = requests.request(method, url, headers=headers, **kwargs)
# awkward implementation to maintain compatibility with current mocked tests
if method == "GET":
# mypy seems to be confused if params isn't explicitly passed, wtf?
response = requests.get(url, params=None, headers=headers, **kwargs)
elif method == "HEAD":
response = requests.head(url, headers=headers, **kwargs)
elif method == "POST":
response = requests.post(url, data, headers=headers, **kwargs)
return response
@staticmethod
def get_user_agent():
return "python-requests/{requests_version} ({system}: {machine}) sewer {sewer_version} ({sewer_url})".format(
requests_version=requests.__version__,
system=platform.system(),
machine=platform.machine(),
sewer_version=sewer_meta("version"),
sewer_url=sewer_meta("url"),
)
def get_acme_endpoints(self):
self.logger.debug("get_acme_endpoints")
get_acme_endpoints = self.GET(self.ACME_DIRECTORY_URL)
self.logger.debug(
"get_acme_endpoints_response. status_code={0}".format(get_acme_endpoints.status_code)
)
if get_acme_endpoints.status_code not in [200, 201]:
raise ValueError(
"Error while getting Acme endpoints: status_code={status_code} response={response}".format(
status_code=get_acme_endpoints.status_code,
response=log_response(get_acme_endpoints),
)
)
return get_acme_endpoints
### FIX ME ### this is a kludge to fix Alec's needs until there's time to do the Acme* refactor
def acme_register(self):
self.logger.info("acme_register%s" % " (is new account)" if self.is_new_acct else "")
if self.account.has_kid():
self.logger.info("acme_register: key was already registered")
return None
if not self.is_new_acct:
payload = {"onlyReturnExisting": True}
elif self.contact_email:
payload = {
"termsOfServiceAgreed": True,
"contact": ["mailto:{0}".format(self.contact_email)],
}
else:
payload = {"termsOfServiceAgreed": True}
url = self.ACME_NEW_ACCOUNT_URL
response = self.make_signed_acme_request(
url=url, payload=json.dumps(payload), needs_jwk=True
)
self.logger.debug(
"response. status_code={0}. response={1}".format(
response.status_code, log_response(response)
)
)
if response.status_code not in [201, 200, 409]:
raise AcmeRegistrationError(
"Error while registering: status_code={status_code} response={response}".format(
status_code=response.status_code, response=log_response(response),
)
)
self.account.set_kid(response.headers["Location"])
self.logger.info("acme_register_success")
return response
def apply_for_cert_issuance(self):
"""
https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.4
The order object returned by the server represents a promise that if
the client fulfills the server's requirements before the "expires"
time, then the server will be willing to finalize the order upon
request and issue the requested certificate. In the order object,
any authorization referenced in the "authorizations" array whose
status is "pending" represents an authorization transaction that the
client must complete before the server will issue the certificate.
Once the client believes it has fulfilled the server's requirements,
it should send a POST request to the order resource's finalize URL.
The POST body MUST include a CSR:
The date values seem to be ignored by LetsEncrypt although they are
in the ACME draft spec; https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.4
"""
self.logger.info("apply_for_cert_issuance (newOrder)")
identifiers = []
for domain_name in self.all_domain_names:
identifiers.append({"type": "dns", "value": domain_name})
payload = {"identifiers": identifiers}
url = self.ACME_NEW_ORDER_URL
apply_for_cert_issuance_response = self.make_signed_acme_request(
url=url, payload=json.dumps(payload)
)
self.logger.debug(
"apply_for_cert_issuance_response. status_code={0}. response={1}".format(
apply_for_cert_issuance_response.status_code,
log_response(apply_for_cert_issuance_response),
)
)
if apply_for_cert_issuance_response.status_code != 201:
raise ValueError(
"Error applying for certificate issuance: status_code={status_code} response={response}".format(
status_code=apply_for_cert_issuance_response.status_code,
response=log_response(apply_for_cert_issuance_response),
)
)
apply_for_cert_issuance_response_json = apply_for_cert_issuance_response.json()
finalize_url = apply_for_cert_issuance_response_json["finalize"]
authorizations = apply_for_cert_issuance_response_json["authorizations"]
self.logger.info("apply_for_cert_issuance_success")
return authorizations, finalize_url
def get_identifier_authorization(self, auth_url: str) -> Dict[str, str]:
"""
https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.5
When a client receives an order from the server it downloads the
authorization resources by sending GET requests to the indicated
URLs. If the client initiates authorization using a request to the
new authorization resource, it will have already received the pending
authorization object in the response to that request.
This is also where we get the challenges/tokens.
"""
self.logger.info("get_identifier_authorization for %s" % auth_url)
response = self.make_signed_acme_request(auth_url, payload="")
self.logger.debug(
"get_identifier_authorization_response. status_code={0}. response={1}".format(
response.status_code, log_response(response)
)
)
if response.status_code not in [200, 201]:
raise ValueError(
"Error getting identifier authorization: status_code={status_code} response={response}".format(
status_code=response.status_code, response=log_response(response)
)
)
response_json = response.json()
domain = response_json["identifier"]["value"]
wildcard = response_json.get("wildcard")
for i in response_json["challenges"]:
if i["type"] in self.provider.chal_types:
challenge = i
challenge_token = challenge["token"]
challenge_url = challenge["url"]
identifier_auth = {
"domain": domain,
"url": auth_url,
"wildcard": wildcard,
"token": challenge_token,
"challenge_url": challenge_url,
}
self.logger.debug(
"get_identifier_authorization_success. identifier_auth={0}".format(identifier_auth)
)
self.logger.info(
"get_identifier_authorization got %s, token=%s" % (challenge_url, challenge_token)
)
return identifier_auth
def get_keyauthorization(self, token):
self.logger.debug("get_keyauthorization")
acme_header_jwk_json = json.dumps(self.account.jwk(), sort_keys=True, separators=(",", ":"))
acme_thumbprint = safe_base64(sha256(acme_header_jwk_json.encode("utf8")).digest())
acme_keyauthorization = "{0}.{1}".format(token, acme_thumbprint)
return acme_keyauthorization
def check_authorization_status(self, authorization_url, desired_status=None):
"""
https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.5.1
To check on the status of an authorization, the client sends a GET(polling)
request to the authorization URL, and the server responds with the
current authorization object.
https://tools.ietf.org/html/draft-ietf-acme-acme#section-8.2
Clients SHOULD NOT respond to challenges until they believe that the
server's queries will succeed. If a server's initial validation
query fails, the server SHOULD retry[intended to address things like propagation delays in
HTTP/DNS provisioning] the query after some time.
The server MUST provide information about its retry state to the
client via the "errors" field in the challenge and the Retry-After
"""
self.logger.debug("check_authorization_status")
desired_status = desired_status or ["pending", "valid"]
number_of_checks = 0
while True:
time.sleep(self.ACME_AUTH_STATUS_WAIT_PERIOD)
response = self.make_signed_acme_request(authorization_url, payload="")
authorization_status = response.json()["status"]
number_of_checks = number_of_checks + 1
self.logger.debug(
"response. status_code={0}. response={1}".format(
response.status_code, log_response(response),
)
)
if authorization_status in desired_status:
break
if number_of_checks == self.ACME_AUTH_STATUS_MAX_CHECKS:
raise StopIteration(
"Checks done={0}. Max checks allowed={1}. Interval between checks={2}seconds.".format(
number_of_checks,
self.ACME_AUTH_STATUS_MAX_CHECKS,
self.ACME_AUTH_STATUS_WAIT_PERIOD,
)
)
self.logger.debug("check_authorization_status_success")
return response
def respond_to_challenge(self, acme_keyauthorization, challenge_url):
"""
https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.5.1
To prove control of the identifier and receive authorization, the
client needs to respond with information to complete the challenges.
The server is said to "finalize" the authorization when it has
completed one of the validations, by assigning the authorization a
status of "valid" or "invalid".
Usually, the validation process will take some time, so the client
will need to poll the authorization resource to see when it is finalized.
To check on the status of an authorization, the client sends a GET(polling)
request to the authorization URL, and the server responds with the
current authorization object.
"""
self.logger.info(
"respond_to_challenge for %s at %s" % (acme_keyauthorization, challenge_url)
)
payload = json.dumps({"keyAuthorization": "{0}".format(acme_keyauthorization)})
respond_to_challenge_response = self.make_signed_acme_request(challenge_url, payload)
self.logger.debug(
"respond_to_challenge_response. status_code={0}. response={1}".format(
respond_to_challenge_response.status_code,
log_response(respond_to_challenge_response),
)
)
self.logger.info("respond_to_challenge_success")
return respond_to_challenge_response
def send_csr(self, finalize_url):
"""
https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.4
Once the client believes it has fulfilled the server's requirements,
it should send a POST request(include a CSR) to the order resource's finalize URL.
A request to finalize an order will result in error if the order indicated does not have status "pending",
if the CSR and order identifiers differ, or if the account is not authorized for the identifiers indicated in the CSR.
The CSR is sent in the base64url-encoded version of the DER format(OpenSSL.crypto.FILETYPE_ASN1)
A valid request to finalize an order will return the order to be finalized.
The client should begin polling the order by sending a
GET request to the order resource to obtain its current state.
"""
self.logger.info("send_csr")
payload = {"csr": safe_base64(self.acme_csr.public_bytes())}
send_csr_response = self.make_signed_acme_request(
url=finalize_url, payload=json.dumps(payload)
)
self.logger.debug(
"send_csr_response. status_code={0}. response={1}".format(
send_csr_response.status_code, log_response(send_csr_response)
)
)
if send_csr_response.status_code not in [200, 201]:
raise ValueError(
"Error sending csr: status_code={status_code} response={response}".format(
status_code=send_csr_response.status_code,
response=log_response(send_csr_response),
)
)
send_csr_response_json = send_csr_response.json()
certificate_url = send_csr_response_json["certificate"]
self.logger.info("send_csr_success")
return certificate_url
def download_certificate(self, certificate_url: str) -> str:
self.logger.info("download_certificate")
response = self.make_signed_acme_request(certificate_url, payload="")
self.logger.debug(
"download_certificate_response. status_code={0}. response={1}".format(
response.status_code, log_response(response)
)
)
if response.status_code not in [200, 201]:
raise ValueError(
"Error fetching signed certificate: status_code={status_code} response={response}".format(
status_code=response.status_code, response=log_response(response)
)
)
pem_certificate = response.content.decode("utf-8")
self.logger.info("download_certificate_success")
return pem_certificate
def get_nonce(self):
"""
https://tools.ietf.org/html/draft-ietf-acme-acme#section-6.4
Each request to an ACME server must include a fresh unused nonce
in order to protect against replay attacks.
"""
self.logger.debug("get_nonce")
response = self.GET(self.ACME_GET_NONCE_URL)
nonce = response.headers["Replay-Nonce"]
return nonce
def get_acme_header(self, url, needs_jwk=False):
"""
https://tools.ietf.org/html/draft-ietf-acme-acme#section-6.2
The JWS Protected Header MUST include the following fields:
- "alg" (Algorithm)
- "jwk" (JSON Web Key, only for requests to new-account and revoke-cert resources)
- "kid" (Key ID, for all other requests). gotten from self.ACME_NEW_ACCOUNT_URL
- "nonce". gotten from self.ACME_GET_NONCE_URL
- "url"
"""
self.logger.debug("get_acme_header")
header = {"alg": self.account.key_desc.alg, "nonce": self.get_nonce(), "url": url}
if needs_jwk:
header["jwk"] = self.account.jwk()
else:
header["kid"] = self.account.kid
return header
def make_signed_acme_request(self, url, payload, needs_jwk=False):
self.logger.debug("make_signed_acme_request")
headers = {}
payload64 = safe_base64(payload)
protected = self.get_acme_header(url, needs_jwk)
protected64 = safe_base64(json.dumps(protected))
message = ("%s.%s" % (protected64, payload64)).encode("utf-8")
# signature = self.sign_message(message="{0}.{1}".format(protected64, payload64)) # bytes
# signature64 = safe_base64(signature) # str
signature64 = safe_base64(self.account.sign_message(message))
data = json.dumps(
{"protected": protected64, "payload": payload64, "signature": signature64}
)
headers.update({"Content-Type": "application/jose+json"})
response = self.POST(url, data=data.encode("utf8"), headers=headers)
return response
def get_certificate(self):
self.logger.debug("get_certificate")
challenges = []
try:
self.acme_register()
authorizations, finalize_url = self.apply_for_cert_issuance()
for auth_url in authorizations:
identifier_auth = self.get_identifier_authorization(auth_url)
token = identifier_auth["token"]
challenge = {
"ident_value": identifier_auth["domain"],
"token": token,
"key_auth": self.get_keyauthorization(token), # responder acme_keyauth..
"wildcard": identifier_auth["wildcard"],
"auth_url": auth_url, # responder auth.._url
"chal_url": identifier_auth["challenge_url"], # responder challenge_url
}
challenges.append(challenge)
# any errors in setup are fatal (here - they are all necessary for same cert)
failures = self.provider.setup(challenges)
if failures:
raise RuntimeError("get_certificate: challenge setup failed for %s" % failures)
### FIX ME ### should abort cert and try to clear on error
error, errata_list = self.propagation_delay(challenges)
# for a case where you want certificates for *.example.com and example.com
# you have to create both auth records AND then respond to the challenge.
# see issues/83
for chal in challenges:
# Make sure the authorization is in a status where we can submit a challenge
# response. The authorization can be in the "valid" state before submitting
# a challenge response if there was a previous authorization for these hosts
# that was successfully validated, still cached by the server.
auth_status_response = self.check_authorization_status(chal["auth_url"])
if auth_status_response.json()["status"] == "pending":
self.respond_to_challenge(chal["key_auth"], chal["chal_url"])
### TO DO ### this is the obfuscated timeout loop. Clean this mess up!
### # # # ### it also keeps trying even when the auth is failed :-(
### FIX? ### shouldn't this be checking the ORDER's status for completion?
# that is at least the most frugal of queries approach...
for chal in challenges:
# Before sending a CSR, we need to make sure the server has completed the
# validation for all the authorizations
self.check_authorization_status(chal["auth_url"], ["valid"])
certificate_url = self.send_csr(finalize_url)
certificate = self.download_certificate(certificate_url)
### FIX ME ### [:100] is a bandaid to reduce spew during tests
except Exception as e:
self.logger.error("Error: Unable to issue certificate. error={0}".format(str(e)[:100]))
raise e
finally:
# best-effort attempt to clear challenges
failures = self.provider.clear(challenges)
return certificate
def sleep_iter(self):
"returns values from list, then repeats last value forever"
for cur_time in self.provider.prop_sleep_times:
yield cur_time
while True:
yield cur_time
def propagation_delay(self, challenges: ChalListType) -> Tuple[str, ErrataListType]:
"""
Wait for the challenges to propagate through the service.
Returns (error: str, errata_list)
* ("", []) is complete success
* ("timeout", [...]) list contains challenges that weren't ready
* ("failure", [...]) list contains both failed and not-yet-ready challenges
See docs/unpropagated.md for the details.
"""
if self.provider.prop_delay:
time.sleep(self.provider.prop_delay)
if self.provider.prop_timeout:
unready = challenges
end_time = time.time() + self.provider.prop_timeout
sleep_time = self.sleep_iter()
num_checks = 0
while unready:
errata = self.provider.unpropagated(unready)
num_checks += 1
# right idea, but details aren't yet nailed down?
# failed = [e for e in errata if e['status'].startswith("FAIL")]
if errata:
poll_time = time.time()
# intentional: do an "extra" check rather than running short
if end_time < poll_time:
break
# wait a while to let more propagation happen
time.sleep(next(sleep_time))
unready = [err[2] for err in errata]
if unready:
### FIX ME ### might be good for mock tests, but really should try to clear, eh?
# return ("timeout", unready)
raise RuntimeError(
"propagation_delay: time out after %s probes: %s" % (num_checks, unready)
)
return ("", [])
def cert(self):
self.logger.warning("DEPRECATED: Client.cert is deprecated as of 0.8.4")
return self.get_certificate()
def renew(self):
self.logger.warning("DEPRECATED: Client.renew is deprecated as of 0.8.4")
return self.cert()
================================================
FILE: sewer/config.py
================================================
ACME_DIRECTORY_URL_STAGING = "https://acme-staging-v02.api.letsencrypt.org/directory"
ACME_DIRECTORY_URL_PRODUCTION = "https://acme-v02.api.letsencrypt.org/directory"
================================================
FILE: sewer/crypto.py
================================================
import time
from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import ec, padding, rsa, utils
from cryptography.hazmat.primitives.serialization import (
load_pem_private_key,
Encoding,
NoEncryption,
PrivateFormat,
)
from cryptography.hazmat.backends import default_backend, openssl
from typing import Any, Callable, cast, Dict, List, Optional, Tuple, Type, Union
from .lib import AcmeError, safe_base64
class AcmeAbstractError(AcmeError):
pass
### Exceptions specific to ACME crypto operations
class AcmeKeyError(AcmeError):
pass
class AcmeKeyTypeError(AcmeError):
pass
class AcmeKidError(AcmeKeyError):
pass
### types for things defined here
### FIX ME ### is there any way to eliminate the repetition here? cryptography classes...
private_key_types = (openssl.rsa._RSAPrivateKey, openssl.ec._EllipticCurvePrivateKey)
PrivateKeyType = Union[openssl.rsa._RSAPrivateKey, openssl.ec._EllipticCurvePrivateKey]
### low level key type table
class KeyDesc:
pk_type: PrivateKeyType
def __init__(
self,
key_size: int,
type_name: str,
jwk_const: Dict[str, str],
jwk_attr: Dict[str, str],
alg: str,
key_bytes: int,
) -> None:
self.key_size = key_size
self.type_name = type_name
self.jwk_const = jwk_const
self.jwk_attr = jwk_attr
self.alg = alg
self.key_bytes = key_bytes
def generate(self) -> PrivateKeyType:
raise AcmeAbstractError("KeyDesc.generate")
def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:
raise AcmeAbstractError("KeyDesc.sign")
def match(self, pk: PrivateKeyType) -> bool:
if isinstance(pk, self.pk_type) and pk.key_size == self.key_size:
return True
return False
class RsaKeyDesc(KeyDesc):
pk_type = rsa.RSAPrivateKey
def __init__(self, key_size: int) -> None:
type_name = "rsa%s" % key_size
super().__init__(key_size, type_name, {"kty": "RSA"}, {"e": "e", "n": "n"}, "RS256", 0)
def generate(self) -> PrivateKeyType:
return rsa.generate_private_key(65537, self.key_size, default_backend())
def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:
"Yes, SHA256 is hardwired. As of Sep 2020, LE rejects other hashes for RSA"
return pk.sign(message, padding.PKCS1v15(), hashes.SHA256())
class EcKeyDesc(KeyDesc):
pk_type = ec.EllipticCurvePrivateKey
def __init__(self, key_size: int, hash_type, alg: str, key_bytes: int) -> None:
name = "secp%sr1" % key_size
curve = "P-%s" % key_size
super().__init__(
key_size, name, {"kty": "EC", "crv": curve}, {"x": "x", "y": "y"}, alg, key_bytes
)
self.curve = getattr(ec, name.upper())
self.hash_type = hash_type
def generate(self) -> PrivateKeyType:
return ec.generate_private_key(self.curve, default_backend())
def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:
# EC sign method returns ASN.1 encoded values for some inane reason
r, s = utils.decode_dss_signature(pk.sign(message, ec.ECDSA(self.hash_type())))
return r.to_bytes(self.key_bytes, "big") + s.to_bytes(self.key_bytes, "big")
key_table = [
RsaKeyDesc(2048),
RsaKeyDesc(3072),
RsaKeyDesc(4096),
EcKeyDesc(256, hashes.SHA256, "ES256", 32),
EcKeyDesc(384, hashes.SHA384, "ES384", 48),
# EcKeyDesc(521, hashes.SHA512, 64, "ES512", 66), this is where the key size != hash size?
]
# extract just the names for option choice lists, etc.
key_type_choices = [kd.type_name for kd in key_table]
def resolve_key_desc(key: Union[str, PrivateKeyType]) -> KeyDesc:
"""
Given a private key or a registered key type name, find the unique matching
descriptor and return it.
Raises exceptions if no match is found or if more than one matches (internal
table error!).
"""
if isinstance(key, private_key_types):
kdl = [kd for kd in key_table if kd.match(key)]
kt = str(type(key))
else:
kdl = [kd for kd in key_table if kd.type_name == key]
kt = key
if not kdl:
raise AcmeKeyTypeError("Unknown key type: %s", kt)
if len(kdl) != 1:
raise AcmeKeyError("Internal error: key type %s matches %s entries!" % (kt, len(kdl)))
return kdl[0]
### AcmeKey, finally!
AcmeKeyType = Union["AcmeKey", "AcmeAccount"]
class AcmeKey:
"""
AcmeKey is a parameterized wrapper around the private key type that are
useful with ACME services. Key creation, loading and storing, message
signing, and generating the key's JWK are all provided. Only key creation
needs to be told the kind or size of key, and other differences in these
operations are hidden away.
See the key_table (or key_type_choices if you just want a list of the
valid type names for the create method ( eg., sewer's cli program).
These are based on what Let's Encrypt's servers accept as of Sep 2020:
RSA with SHA256, P-256 with SHA256, and P-384 with SHA384. LE doubtless
accepts many other key sizes than our simple list-based setup provides,
but these are the ones sewer has actually tested (which is how we found
out that RSA was SHA256 only, and P-521 wasn't available at all). Of
course this can change, which is much of the reason for the table-driven
approach used here.
"""
def __init__(self, pk: PrivateKeyType, key_desc: KeyDesc) -> None:
self.pk = pk
self.key_desc = key_desc
#
self._jwk: Optional[Dict[str, str]] = None
### Key Constructors
@classmethod
def create(cls: Type["AcmeKey"], key_type_name: str) -> "AcmeKey":
"""
Factory method to create a new key of key_type, returned as an AcmeKey.
"""
kd = resolve_key_desc(key_type_name)
return cls(kd.generate(), kd)
@classmethod
def from_pem(cls: Type["AcmeKey"], pem_data: bytes) -> "AcmeKey":
"""
load a key from the PEM-format bytes, return an AcmeKey
NB: since it's not stored in the PEM, the kid is empty (None)
"""
pk = load_pem_private_key(pem_data, None, default_backend())
kd = resolve_key_desc(pk)
return cls(pk, kd)
@classmethod
def read_pem(cls: Type["AcmeKey"], filename: str) -> "AcmeKey":
"convenience method to load a PEM-format key; returns the AcmeKey"
with open(filename, "rb") as f:
return cls.from_pem(f.read())
### shared methods
def to_pem(self) -> bytes:
"return private key's serialized (PEM) form"
pem_data = self.pk.private_bytes(
encoding=Encoding.PEM, format=PrivateFormat.PKCS8, encryption_algorithm=NoEncryption()
)
return pem_data
def write_pem(self, filename: str) -> None:
"convenience method to write out the key in PEM form"
with open(filename, "wb") as f:
f.write(self.to_pem())
def sign_message(self, message: bytes) -> bytes:
return self.key_desc.sign(self.pk, message)
### An ACME account is identified by a key. When registered there is a Key ID as well.
class AcmeAccount(AcmeKey):
"""
Only an account key needs (or has) a Key ID associated with it.
"""
def __init__(self, pk: PrivateKeyType, key_desc: KeyDesc = None) -> None:
if key_desc is None:
key_desc = resolve_key_desc(pk)
super().__init__(pk, key_desc)
self.__kid: Optional[str] = None
self._timestamp: Optional[float] = None
self.__jwk: Optional[Dict[str, str]] = None
### kid's descriptor methods
def get_kid(self) -> str:
if self.__kid is None:
raise AcmeKidError("Attempt to access a Key ID that hasn't been set. Register key?")
return self.__kid
def set_kid(self, kid: str, timestamp: float = None) -> None:
"The kid can be set only once, but we overlook exact duplicate set calls"
if self.__kid and self.__kid != kid:
raise AcmeKidError("Cannot alter a key's kid")
self.__kid = kid
self._timestamp = timestamp if timestamp is not None else time.time()
def del_kid(self) -> None:
"Doesn't actually del the hidden attribute, just resets the value to None (empty)"
self.__kid = None
kid = property(get_kid, set_kid, del_kid)
def has_kid(self) -> bool:
"need a non-exploding test for the presence of a Key ID"
return not self.__kid is None
### extend AcmeKey with new methods
def jwk(self) -> Dict[str, str]:
"""
Returns the key's JWK as a dictionary
CACHES result in _jwk
"""
if not self.__jwk:
jwk = {}
pubnums = self.pk.public_key().public_numbers()
jwk.update(self.key_desc.jwk_const)
for name, attr_name in self.key_desc.jwk_attr.items():
val = getattr(pubnums, attr_name)
numbytes = self.key_desc.key_bytes
if numbytes == 0:
numbytes = (val.bit_length() + 7) // 8
jwk[name] = safe_base64(val.to_bytes(numbytes, "big"))
self.__jwk = jwk
return self.__jwk
### TODO ### store & load file format with kid, timestamp and pk.
#
# RFC7568 says that at least most implementations accept text outside the
# BEGIN...END lines, especially in PKIX certificates. So I plan to do
# something like this:
#
# KID: https://acme-v02.api.letsencrypt.org/acme/account/1a2b3c4d5e6f7g8h9i0j
# Timestamp: 1600452956.446775
# -----BEGIN PRIVATE KEY-----
#
# Both openssl pkey and the cryptography library can load a PEM decorated
# like that. Only possible question is whether the '\n' line ending needs
# to be adjusted for non-Unix systems.
def write_key(self, filename: str) -> None:
"Like write_pem but prepends the KID and timestamp if those are present"
with open(filename, "wb") as f:
if self.__kid:
f.write(("KID: %s\n" % self.__kid).encode())
if self._timestamp:
f.write(("Timestamp: %s\n" % self._timestamp).encode())
f.write(self.to_pem())
@classmethod
def read_key(cls: Type["AcmeAccount"], filename: str) -> "AcmeAccount":
with open(filename, "rb") as f:
data = f.read()
prefix = b""
n = data.find(b"-----BEGIN")
if 0 < n:
prefix = data[:n]
data = data[n:]
acct = cast("AcmeAccount", cls.from_pem(data))
if prefix:
parts = prefix.split(b"\n")
for p in parts:
if p.startswith(b"KID: "):
acct.__kid = p[5:].decode()
elif p.startswith(b"Timestamp: "):
acct._timestamp = float(p[11:])
return acct
### We also need to generate Certificate Signing Requests
class AcmeCsr:
def __init__(self, *, cn: str, san: List[str], key: AcmeKey) -> None:
"""
temporary "just like Client.create_csr", more or less
TODO: "must staple" extension; NOT elaborating subject name, since LE
suggests that even CN may be replaced by a meaningless number in some
vague future version of the server. I guess they're right that
browsers ignore the CN already (aside from displaying it if asked).
"""
csrb = x509.CertificateSigningRequestBuilder()
csrb = csrb.subject_name(x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, cn)]))
all_names = list(set([cn] + san))
SAN: List[x509.GeneralName] = [x509.DNSName(name) for name in all_names]
csrb = csrb.add_extension(x509.SubjectAlternativeName(SAN), critical=False)
self.csr = csrb.sign(key.pk, hashes.SHA256(), default_backend())
def public_bytes(self) -> bytes:
return self.csr.public_bytes(Encoding.DER)
================================================
FILE: sewer/dns_providers/__init__.py
================================================
================================================
FILE: sewer/dns_providers/acmedns.py
================================================
import urllib.parse
import requests
from dns.resolver import Resolver
from . import common
from ..lib import log_response
class AcmeDnsDns(common.BaseDns):
def __init__(self, ACME_DNS_API_USER, ACME_DNS_API_KEY, ACME_DNS_API_BASE_URL, **kwargs):
self.ACME_DNS_API_USER = ACME_DNS_API_USER
self.ACME_DNS_API_KEY = ACME_DNS_API_KEY
self.HTTP_TIMEOUT = 65 # seconds
if ACME_DNS_API_BASE_URL[-1] != "/":
self.ACME_DNS_API_BASE_URL = ACME_DNS_API_BASE_URL + "/"
else:
self.ACME_DNS_API_BASE_URL = ACME_DNS_API_BASE_URL
super().__init__(**kwargs)
def create_dns_record(self, domain_name, domain_dns_value):
self.logger.info("create_dns_record")
resolver = Resolver(configure=False)
resolver.nameservers = ["8.8.8.8"]
answer = resolver.query("_acme-challenge.{0}.".format(domain_name), "TXT")
subdomain, _ = str(answer.canonical_name).split(".", 1)
url = urllib.parse.urljoin(self.ACME_DNS_API_BASE_URL, "update")
headers = {"X-Api-User": self.ACME_DNS_API_USER, "X-Api-Key": self.ACME_DNS_API_KEY}
body = {"subdomain": subdomain, "txt": domain_dns_value}
update_acmedns_dns_record_response = requests.post(
url, headers=headers, json=body, timeout=self.HTTP_TIMEOUT
)
self.logger.debug(
"update_acmedns_dns_record_response. status_code={0}. response={1}".format(
update_acmedns_dns_record_response.status_code,
log_response(update_acmedns_dns_record_response),
)
)
if update_acmedns_dns_record_response.status_code != 200:
# raise error so that we do not continue to make calls to ACME
# server
raise ValueError(
"Error creating acme-dns dns record: status_code={status_code} response={response}".format(
status_code=update_acmedns_dns_record_response.status_code,
response=log_response(update_acmedns_dns_record_response),
)
)
self.logger.info("create_dns_record_end")
def delete_dns_record(self, domain_name, domain_dns_value):
self.logger.info("delete_dns_record")
# acme-dns doesn't support this
self.logger.info("delete_dns_record_success")
================================================
FILE: sewer/dns_providers/aliyundns.py
================================================
import json
from aliyunsdkcore import client # type: ignore
import aliyunsdkalidns.request.v20150109 # type:ignore
from aliyunsdkalidns.request.v20150109 import (
DescribeDomainRecordsRequest,
AddDomainRecordRequest,
DeleteDomainRecordRequest,
)
from . import common
class _ResponseForAliyun(object):
"""
wrapper aliyun resp to the format sewer wanted.
"""
def __init__(self, status_code=200, content=None, headers=None):
self.status_code = status_code
self.headers = headers or {}
self.content = content or {}
self.content = json.dumps(content)
super(_ResponseForAliyun, self).__init__()
def json(self):
return json.loads(self.content)
class AliyunDns(common.BaseDns):
def __init__(self, key, secret, endpoint="cn-beijing", debug=False, **kwargs):
"""
aliyun dns client
:param str key: access key
:param str secret: access sceret
:param str endpoint: endpoint
:param bool debug: if debug?
"""
super().__init__(**kwargs)
self._key = key
self._secret = secret
self._endpoint = endpoint
self._debug = debug
self.clt = client.AcsClient(self._key, self._secret, self._endpoint, debug=self._debug)
def _send_reqeust(self, request):
"""
send request to aliyun
"""
request.set_accept_format("json")
try:
status, headers, result = self.clt.implementation_of_do_action(request)
if isinstance(result, bytes):
result = result.decode()
result = json.loads(result)
if "Message" in result or "Code" in result:
result["Success"] = False
self.logger.warning("aliyundns resp error: %s", result)
except Exception as exc:
self.logger.warning("aliyundns failed to send request: %s, %s", str(exc), request)
status, headers, result = 502, {}, b'{"Success": false}'
result = json.loads(result)
if self._debug:
self.logger.info("aliyundns request name: %s", request.__class__.__name__)
self.logger.info("aliyundns request query: %s", request.get_query_params())
return _ResponseForAliyun(status, result, headers)
def query_recored_items(self, host, zone=None, tipe=None, page=1, psize=200):
"""
query recored items.
:param str host: like example.com
:param str zone: like menduo.example.com
:param str tipe: TXT, CNAME, IP or other
:param int page:
:param int psize:
:return dict: res = {
'DomainRecords':
{'Record': [
{
'DomainName': 'menduo.net',
'Line': 'default',
'Locked': False,
'RR': 'zb',
'RecordId': '3989515483698964',
'Status': 'ENABLE',
'TTL': 600,
'Type': 'A',
'Value': '127.0.0.1',
'Weight': 1
},
{
'DomainName': 'menduo.net',
'Line': 'default',
'Locked': False,
'RR': 'a.sub',
'RecordId': '3989515480778964',
'Status': 'ENABLE',
'TTL': 600,
'Type': 'CNAME',
'Value': 'h.p.menduo.net',
'Weight': 1
}
]
},
'PageNumber': 1,
'PageSize': 20,
'RequestId': 'FC4D02CD-EDCC-4EE8-942F-1497CCC3B10E',
'TotalCount': 95
}
"""
request = DescribeDomainRecordsRequest.DescribeDomainRecordsRequest()
request.get_action_name()
request.set_DomainName(host)
request.set_PageNumber(page)
request.set_PageSize(psize)
if zone:
request.set_RRKeyWord(zone)
if tipe:
request.set_TypeKeyWord(tipe)
resp = self._send_reqeust(request)
body = resp.json()
return body
def query_recored_id(self, root, zone, tipe="TXT"):
"""
find recored
:param str root: root host, like example.com
:param str zone: sub zone, like menduo.example.com
:param str tipe: record tipe, TXT, CNAME, IP. we use TXT
:return str:
"""
record_id = None
recoreds = self.query_recored_items(root, zone, tipe=tipe)
recored_list = recoreds.get("DomainRecords", {}).get("Record", [])
recored_item_list = [i for i in recored_list if i["RR"] == zone]
if len(recored_item_list):
record_id = recored_item_list[0]["RecordId"]
return record_id
@staticmethod
def extract_zone(domain_name):
"""
extract domain to root, sub, acme_txt
:param str domain_name: the value sewer client passed in, like *.menduo.example.com
:return tuple: root, zone, acme_txt
"""
if domain_name.count(".") > 1:
zone, middle, last = str(domain_name).rsplit(".", 2)
root = ".".join([middle, last])
acme_txt = "_acme-challenge.%s" % zone
else:
zone = ""
root = domain_name
acme_txt = "_acme-challenge"
return root, zone, acme_txt
def create_dns_record(self, domain_name, domain_dns_value):
"""
create a dns record
:param str domain_name: the value sewer client passed in, like *.menduo.example.com
:param str domain_dns_value: the value sewer client passed in.
:return _ResponseForAliyun:
"""
self.logger.info("create_dns_record start: %s", (domain_name, domain_dns_value))
root, _, acme_txt = self.extract_zone(domain_name)
request = AddDomainRecordRequest.AddDomainRecordRequest()
request.set_DomainName(root)
request.set_TTL(600)
request.set_RR(acme_txt)
request.set_Type("TXT")
request.set_Value(domain_dns_value)
resp = self._send_reqeust(request)
self.logger.info("create_dns_record end: %s", (domain_name, domain_dns_value, resp.json()))
return resp
def delete_dns_record(self, domain_name, domain_dns_value):
"""
delete a txt record we created just now.
:param str domain_name: the value sewer client passed in, like *.menduo.example.com
:param str domain_dns_value: the value sewer client passed in. we do not use this.
:return _ResponseForAliyun:
:return:
"""
self.logger.info("delete_dns_record start: %s", (domain_name, domain_dns_value))
root, _, acme_txt = self.extract_zone(domain_name)
record_id = self.query_recored_id(root, acme_txt)
if not record_id:
msg = "failed to find record_id of domain: %s, value: %s", domain_name, domain_dns_value
self.logger.warning(msg)
return
self.logger.info("start to delete dns record, id: %s", record_id)
request = DeleteDomainRecordRequest.DeleteDomainRecordRequest()
request.set_RecordId(record_id)
resp = self._send_reqeust(request)
self.logger.info("delete_dns_record end: %s", (domain_name, domain_dns_value, resp.json()))
return resp
================================================
FILE: sewer/dns_providers/auroradns.py
================================================
# DNS Provider for AuroRa DNS from the dutch hosting provider pcextreme
# https://www.pcextreme.nl/aurora/dns
# Aurora uses libcloud from apache
# https://libcloud.apache.org/
import tldextract
from libcloud.dns.providers import get_driver # type: ignore
from libcloud.dns.types import Provider, RecordType # type: ignore
from . import common
class AuroraDns(common.BaseDns):
"""
Todo: re-organize this class so that we make it easier to mock things out to
facilitate better tests.
"""
def __init__(self, AURORA_API_KEY, AURORA_SECRET_KEY, **kwargs):
self.AURORA_API_KEY = AURORA_API_KEY
self.AURORA_SECRET_KEY = AURORA_SECRET_KEY
super().__init__(**kwargs)
def create_dns_record(self, domain_name, domain_dns_value):
self.logger.info("create_dns_record")
extractedDomain = tldextract.extract(domain_name)
domainSuffix = extractedDomain.domain + "." + extractedDomain.suffix
if extractedDomain.subdomain == "":
subDomain = "_acme-challenge"
else:
subDomain = "_acme-challenge." + extractedDomain.subdomain
cls = get_driver(Provider.AURORADNS)
driver = cls(key=self.AURORA_API_KEY, secret=self.AURORA_SECRET_KEY)
zone = driver.get_zone(domainSuffix)
zone.create_record(name=subDomain, type=RecordType.TXT, data=domain_dns_value)
self.logger.info("create_dns_record_success")
return
def delete_dns_record(self, domain_name, domain_dns_value):
self.logger.info("delete_dns_record")
extractedDomain = tldextract.extract(domain_name)
domainSuffix = extractedDomain.domain + "." + extractedDomain.suffix
if extractedDomain.subdomain == "":
subDomain = "_acme-challenge"
else:
subDomain = "_acme-challenge." + extractedDomain.subdomain
cls = get_driver(Provider.AURORADNS)
driver = cls(key=self.AURORA_API_KEY, secret=self.AURORA_SECRET_KEY)
zone = driver.get_zone(domainSuffix)
records = driver.list_records(zone)
for x in records:
if x.name == subDomain and x.type == "TXT":
record_id = x.id
self.logger.info(
"Found recor
gitextract_ki74leed/
├── .circleci/
│ ├── codecov.yml
│ └── config.yml
├── .github/
│ ├── CONTRIBUTING.md
│ ├── ISSUE_TEMPLATE.md
│ ├── PULL_REQUEST_TEMPLATE.md
│ └── workflows/
│ └── build.yml
├── .gitignore
├── CONTRIBUTORS.md
├── LICENSE.txt
├── Makefile
├── README.md
├── docs/
│ ├── ACME.md
│ ├── Aliasing.md
│ ├── CHANGELOG.md
│ ├── DNS-Propagation.md
│ ├── LegacyDNS.md
│ ├── UnifiedProvider.md
│ ├── catalog.md
│ ├── crypto.md
│ ├── dns-01.md
│ ├── drivers/
│ │ ├── route53.md
│ │ └── unbound_ssh.md
│ ├── http-01.md
│ ├── index.md
│ ├── notes/
│ │ ├── 0.8.2-notes.md
│ │ ├── 0.8.3-notes.md
│ │ └── 0.8.4-notes.md
│ ├── preview/
│ │ ├── cloaca.md
│ │ └── cloaca_config.md
│ ├── sewer-as-a-library.md
│ ├── sewer-cli.md
│ ├── unpropagated.md
│ └── wildcards.md
├── mypy.ini
├── pyproject.toml
├── setup.py
├── sewer/
│ ├── __init__.py
│ ├── __main__.py
│ ├── auth.py
│ ├── catalog.json
│ ├── catalog.py
│ ├── cli.py
│ ├── client.py
│ ├── config.py
│ ├── crypto.py
│ ├── dns_providers/
│ │ ├── __init__.py
│ │ ├── acmedns.py
│ │ ├── aliyundns.py
│ │ ├── auroradns.py
│ │ ├── cloudflare.py
│ │ ├── cloudns.py
│ │ ├── common.py
│ │ ├── dnspod.py
│ │ ├── duckdns.py
│ │ ├── gandi.py
│ │ ├── hurricane.py
│ │ ├── powerdns.py
│ │ ├── rackspace.py
│ │ ├── route53.py
│ │ ├── tests/
│ │ │ ├── __init__.py
│ │ │ ├── test_acmedns.py
│ │ │ ├── test_aliyundns.py
│ │ │ ├── test_aurora.py
│ │ │ ├── test_cloudflare.py
│ │ │ ├── test_cloudns.py
│ │ │ ├── test_common.py
│ │ │ ├── test_dnspod.py
│ │ │ ├── test_duckdns.py
│ │ │ ├── test_gandi.py
│ │ │ ├── test_hedns.py
│ │ │ ├── test_powerdns.py
│ │ │ ├── test_rackspace.py
│ │ │ ├── test_route53.py
│ │ │ ├── test_unbound_ssh.py
│ │ │ └── test_utils.py
│ │ └── unbound_ssh.py
│ ├── lib.py
│ ├── meta.json
│ ├── providers/
│ │ ├── __init__.py
│ │ ├── demo.py
│ │ └── tests/
│ │ ├── __init__.py
│ │ └── test_demo.py
│ └── tests/
│ ├── __init__.py
│ ├── test_Client.py
│ ├── test_auth.py
│ ├── test_catalog.py
│ ├── test_lib.py
│ └── test_utils.py
└── tests/
├── crypto_test.py
└── data/
└── README
SYMBOL INDEX (428 symbols across 43 files)
FILE: sewer/auth.py
class ProviderBase (line 11) | class ProviderBase:
method __init__ (line 16) | def __init__(
method setup (line 48) | def setup(self, challenges: ChalListType) -> ErrataListType:
method unpropagated (line 51) | def unpropagated(self, challenges: ChalListType) -> ErrataListType:
method clear (line 54) | def clear(self, challenges: ChalListType) -> ErrataListType:
class HTTPProviderBase (line 58) | class HTTPProviderBase(ProviderBase):
method __init__ (line 67) | def __init__(self, **kwargs: Any) -> None:
class DNSProviderBase (line 73) | class DNSProviderBase(ProviderBase):
method __init__ (line 81) | def __init__(self, *, alias: str = "", **kwargs: Any) -> None:
method cname_domain (line 89) | def cname_domain(self, chal: Dict[str, str]) -> Union[str, None]:
method target_domain (line 94) | def target_domain(self, chal: Dict[str, str]) -> str:
FILE: sewer/catalog.py
class ProviderDescriptor (line 7) | class ProviderDescriptor:
method __init__ (line 8) | def __init__(
method __str__ (line 33) | def __str__(self) -> str:
method get_provider (line 36) | def get_provider(self) -> ProviderBase:
class ProviderCatalog (line 44) | class ProviderCatalog:
method __init__ (line 45) | def __init__(self, filepath: str = "") -> None:
method get_item_list (line 63) | def get_item_list(self) -> List[ProviderDescriptor]:
method get_descriptor (line 70) | def get_descriptor(self, name: str) -> ProviderDescriptor:
method get_provider (line 75) | def get_provider(self, name: str) -> ProviderBase:
FILE: sewer/cli.py
function setup_parser (line 12) | def setup_parser(catalog):
function get_provider (line 161) | def get_provider(provider_name, provider_kwargs, catalog, logger):
function main (line 328) | def main():
FILE: sewer/client.py
class Client (line 13) | class Client:
method __init__ (line 18) | def __init__(
method GET (line 124) | def GET(self, url: str) -> requests.Response:
method HEAD (line 135) | def HEAD(self, url: str) -> requests.Response:
method POST (line 138) | def POST(
method _request (line 143) | def _request(
method get_user_agent (line 182) | def get_user_agent():
method get_acme_endpoints (line 191) | def get_acme_endpoints(self):
method acme_register (line 208) | def acme_register(self):
method apply_for_cert_issuance (line 248) | def apply_for_cert_issuance(self):
method get_identifier_authorization (line 298) | def get_identifier_authorization(self, auth_url: str) -> Dict[str, str]:
method get_keyauthorization (line 349) | def get_keyauthorization(self, token):
method check_authorization_status (line 357) | def check_authorization_status(self, authorization_url, desired_status...
method respond_to_challenge (line 399) | def respond_to_challenge(self, acme_keyauthorization, challenge_url):
method send_csr (line 429) | def send_csr(self, finalize_url):
method download_certificate (line 466) | def download_certificate(self, certificate_url: str) -> str:
method get_nonce (line 485) | def get_nonce(self):
method get_acme_header (line 496) | def get_acme_header(self, url, needs_jwk=False):
method make_signed_acme_request (line 516) | def make_signed_acme_request(self, url, payload, needs_jwk=False):
method get_certificate (line 533) | def get_certificate(self):
method sleep_iter (line 600) | def sleep_iter(self):
method propagation_delay (line 608) | def propagation_delay(self, challenges: ChalListType) -> Tuple[str, Er...
method cert (line 655) | def cert(self):
method renew (line 659) | def renew(self):
FILE: sewer/crypto.py
class AcmeAbstractError (line 20) | class AcmeAbstractError(AcmeError):
class AcmeKeyError (line 27) | class AcmeKeyError(AcmeError):
class AcmeKeyTypeError (line 31) | class AcmeKeyTypeError(AcmeError):
class AcmeKidError (line 35) | class AcmeKidError(AcmeKeyError):
class KeyDesc (line 50) | class KeyDesc:
method __init__ (line 54) | def __init__(
method generate (line 70) | def generate(self) -> PrivateKeyType:
method sign (line 73) | def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:
method match (line 76) | def match(self, pk: PrivateKeyType) -> bool:
class RsaKeyDesc (line 82) | class RsaKeyDesc(KeyDesc):
method __init__ (line 86) | def __init__(self, key_size: int) -> None:
method generate (line 90) | def generate(self) -> PrivateKeyType:
method sign (line 93) | def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:
class EcKeyDesc (line 99) | class EcKeyDesc(KeyDesc):
method __init__ (line 103) | def __init__(self, key_size: int, hash_type, alg: str, key_bytes: int)...
method generate (line 112) | def generate(self) -> PrivateKeyType:
method sign (line 115) | def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:
function resolve_key_desc (line 134) | def resolve_key_desc(key: Union[str, PrivateKeyType]) -> KeyDesc:
class AcmeKey (line 162) | class AcmeKey:
method __init__ (line 182) | def __init__(self, pk: PrivateKeyType, key_desc: KeyDesc) -> None:
method create (line 191) | def create(cls: Type["AcmeKey"], key_type_name: str) -> "AcmeKey":
method from_pem (line 200) | def from_pem(cls: Type["AcmeKey"], pem_data: bytes) -> "AcmeKey":
method read_pem (line 213) | def read_pem(cls: Type["AcmeKey"], filename: str) -> "AcmeKey":
method to_pem (line 221) | def to_pem(self) -> bytes:
method write_pem (line 229) | def write_pem(self, filename: str) -> None:
method sign_message (line 235) | def sign_message(self, message: bytes) -> bytes:
class AcmeAccount (line 242) | class AcmeAccount(AcmeKey):
method __init__ (line 247) | def __init__(self, pk: PrivateKeyType, key_desc: KeyDesc = None) -> None:
method get_kid (line 257) | def get_kid(self) -> str:
method set_kid (line 262) | def set_kid(self, kid: str, timestamp: float = None) -> None:
method del_kid (line 270) | def del_kid(self) -> None:
method has_kid (line 276) | def has_kid(self) -> bool:
method jwk (line 282) | def jwk(self) -> Dict[str, str]:
method write_key (line 316) | def write_key(self, filename: str) -> None:
method read_key (line 327) | def read_key(cls: Type["AcmeAccount"], filename: str) -> "AcmeAccount":
class AcmeCsr (line 349) | class AcmeCsr:
method __init__ (line 350) | def __init__(self, *, cn: str, san: List[str], key: AcmeKey) -> None:
method public_bytes (line 367) | def public_bytes(self) -> bytes:
FILE: sewer/dns_providers/acmedns.py
class AcmeDnsDns (line 11) | class AcmeDnsDns(common.BaseDns):
method __init__ (line 12) | def __init__(self, ACME_DNS_API_USER, ACME_DNS_API_KEY, ACME_DNS_API_B...
method create_dns_record (line 23) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 54) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/aliyundns.py
class _ResponseForAliyun (line 14) | class _ResponseForAliyun(object):
method __init__ (line 19) | def __init__(self, status_code=200, content=None, headers=None):
method json (line 26) | def json(self):
class AliyunDns (line 30) | class AliyunDns(common.BaseDns):
method __init__ (line 31) | def __init__(self, key, secret, endpoint="cn-beijing", debug=False, **...
method _send_reqeust (line 46) | def _send_reqeust(self, request):
method query_recored_items (line 69) | def query_recored_items(self, host, zone=None, tipe=None, page=1, psiz...
method query_recored_id (line 125) | def query_recored_id(self, root, zone, tipe="TXT"):
method extract_zone (line 142) | def extract_zone(domain_name):
method create_dns_record (line 158) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 180) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/auroradns.py
class AuroraDns (line 13) | class AuroraDns(common.BaseDns):
method __init__ (line 19) | def __init__(self, AURORA_API_KEY, AURORA_SECRET_KEY, **kwargs):
method create_dns_record (line 24) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 43) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/cloudflare.py
class CloudFlareDns (line 9) | class CloudFlareDns(common.BaseDns):
method __init__ (line 10) | def __init__(
method find_dns_zone (line 41) | def find_dns_zone(self, domain_name):
method create_dns_record (line 73) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 107) | def delete_dns_record(self, domain_name, domain_dns_value):
method _get_auth_header (line 160) | def _get_auth_header(self):
FILE: sewer/dns_providers/cloudns.py
function _split_domain_name (line 9) | def _split_domain_name(domain_name):
class ClouDNSDns (line 20) | class ClouDNSDns(common.BaseDns):
method __init__ (line 21) | def __init__(self, **kwargs):
method create_dns_record (line 24) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 38) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/common.py
class BaseDns (line 7) | class BaseDns(DNSProviderBase):
method __init__ (line 12) | def __init__(self, **kwargs: Any) -> None:
method setup (line 21) | def setup(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Erra...
method unpropagated (line 26) | def unpropagated(self, challenges: Sequence[Dict[str, str]]) -> Sequen...
method clear (line 29) | def clear(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Erra...
method create_dns_record (line 36) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 69) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/dnspod.py
class DNSPodDns (line 8) | class DNSPodDns(common.BaseDns):
method __init__ (line 9) | def __init__(
method create_dns_record (line 24) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 69) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/duckdns.py
class DuckDNSDns (line 8) | class DuckDNSDns(common.BaseDns):
method __init__ (line 9) | def __init__(self, duckdns_token, DUCKDNS_API_BASE_URL="https://www.du...
method _common_dns_record (line 20) | def _common_dns_record(self, logger_info, domain_name, payload_end_arg):
method create_dns_record (line 52) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 55) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/gandi.py
class GandiDns (line 9) | class GandiDns(common.BaseDns):
method __init__ (line 10) | def __init__(
method create_dns_record (line 37) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 66) | def delete_dns_record(self, domain_name, domain_dns_value):
method _get_subdomain_records (line 69) | def _get_subdomain_records(self, subdomain, all_records):
method delete_record (line 72) | def delete_record(self, domain_name):
method get_zone_records_href (line 87) | def get_zone_records_href(self, base_domain):
method get_all_zone_records (line 98) | def get_all_zone_records(self, zone_records_href):
method split_domain (line 107) | def split_domain(domain_name):
method subdomain_to_challenge_domain (line 126) | def subdomain_to_challenge_domain(subdomain):
FILE: sewer/dns_providers/hurricane.py
class _Response (line 11) | class _Response(object):
method __init__ (line 16) | def __init__(self, status_code=200, content=None, headers=None):
method json (line 23) | def json(self):
class HurricaneDns (line 27) | class HurricaneDns(common.BaseDns):
method __init__ (line 28) | def __init__(self, username, password, **kwargs):
method extract_zone (line 33) | def extract_zone(domain_name):
method create_dns_record (line 49) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 57) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/powerdns.py
class PowerDNSDns (line 8) | class PowerDNSDns(common.BaseDns):
method __init__ (line 23) | def __init__(self, powerdns_api_key, powerdns_api_url, *kwargs):
method validate_powerdns_zone (line 28) | def validate_powerdns_zone(self, domain_name):
method _common_dns_record (line 54) | def _common_dns_record(self, domain_name, domain_dns_value, changetype):
method create_dns_record (line 93) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 96) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/rackspace.py
class RackspaceDns (line 9) | class RackspaceDns(common.BaseDns):
method __init__ (line 10) | def __init__(self, RACKSPACE_USERNAME, RACKSPACE_API_KEY, **kwargs):
method get_rackspace_credentials (line 22) | def get_rackspace_credentials(self):
method get_dns_zone (line 59) | def get_dns_zone(self, domain_name):
method find_dns_zone_id (line 64) | def find_dns_zone_id(self, domain_name):
method find_dns_record_id (line 96) | def find_dns_record_id(self, domain_name, domain_dns_value):
method poll_callback_url (line 131) | def poll_callback_url(self, callback_url):
method create_dns_record (line 159) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 195) | def delete_dns_record(self, domain_name, domain_dns_value):
FILE: sewer/dns_providers/route53.py
class Route53Dns (line 10) | class Route53Dns(common.BaseDns):
method __init__ (line 15) | def __init__(self, access_key_id=None, secret_access_key=None, client=...
method create_dns_record (line 42) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 46) | def delete_dns_record(self, domain_name, domain_dns_value):
method _find_zone_id_for_domain (line 50) | def _find_zone_id_for_domain(self, domain):
method _change_txt_record (line 77) | def _change_txt_record(self, action, domain_name, domain_dns_value):
FILE: sewer/dns_providers/tests/test_acmedns.py
class Testacmedns (line 10) | class Testacmedns(TestCase):
method setUp (line 14) | def setUp(self):
method tearDown (line 32) | def tearDown(self):
method test_acmedns_is_called_by_create_dns_record (line 35) | def test_acmedns_is_called_by_create_dns_record(self):
method test_acmedns_is_not_called_by_delete_dns_record (line 57) | def test_acmedns_is_not_called_by_delete_dns_record(self):
FILE: sewer/dns_providers/tests/test_aliyundns.py
class TestAliyunDNS (line 9) | class TestAliyunDNS(TestCase):
method setUp (line 13) | def setUp(self):
method tearDown (line 26) | def tearDown(self):
method test_extract_zone_sub_domain (line 29) | def test_extract_zone_sub_domain(self):
method test_extract_zone_root (line 38) | def test_extract_zone_root(self):
method test_aliyun_is_called_by_create_dns_record (line 45) | def test_aliyun_is_called_by_create_dns_record(self):
method test_aliyun_is_not_called_by_delete_dns_record (line 59) | def test_aliyun_is_not_called_by_delete_dns_record(self):
FILE: sewer/dns_providers/tests/test_aurora.py
class TestAurora (line 9) | class TestAurora(TestCase):
method setUp (line 17) | def setUp(self):
method tearDown (line 32) | def tearDown(self):
method test_delete_dns_record_is_not_called_by_create_dns_record (line 35) | def test_delete_dns_record_is_not_called_by_create_dns_record(self):
method test_aurora_is_called_by_delete_dns_record (line 55) | def test_aurora_is_called_by_delete_dns_record(self):
FILE: sewer/dns_providers/tests/test_cloudflare.py
class TestCloudflare (line 10) | class TestCloudflare(TestCase):
method setUp (line 14) | def setUp(self):
method tearDown (line 43) | def tearDown(self):
method test_delete_dns_record_is_not_called_by_create_dns_record (line 46) | def test_delete_dns_record_is_not_called_by_create_dns_record(self):
method test_cloudflare_is_called_by_create_dns_record (line 68) | def test_cloudflare_is_called_by_create_dns_record(self):
method test_cloudflare_is_called_by_delete_dns_record (line 113) | def test_cloudflare_is_called_by_delete_dns_record(self):
class TestCloudflareTokens (line 157) | class TestCloudflareTokens(TestCase):
method setUp (line 162) | def setUp(self):
method test_init_auth_validation (line 167) | def test_init_auth_validation(self):
FILE: sewer/dns_providers/tests/test_cloudns.py
class TestClouDNS (line 9) | class TestClouDNS(TestCase):
method setUp (line 14) | def setUp(self):
method test_cloudns_is_called_by_create_dns_record (line 25) | def test_cloudns_is_called_by_create_dns_record(self):
method test_cloudns_is_called_by_delete_dns_record (line 52) | def test_cloudns_is_called_by_delete_dns_record(self):
FILE: sewer/dns_providers/tests/test_common.py
class TestCommon (line 6) | class TestCommon(TestCase):
method setUp (line 10) | def setUp(self):
method tearDown (line 16) | def tearDown(self):
method test_create_dns_record (line 19) | def test_create_dns_record(self):
method test_delete_dns_record (line 27) | def test_delete_dns_record(self):
method test_setup_empty (line 35) | def test_setup_empty(self):
method test_clear_empty (line 38) | def test_clear_empty(self):
method test_setup_mocked (line 41) | def test_setup_mocked(self):
method test_clear_mocked (line 46) | def test_clear_mocked(self):
FILE: sewer/dns_providers/tests/test_dnspod.py
class TestDNSPod (line 9) | class TestDNSPod(TestCase):
method setUp (line 13) | def setUp(self):
method tearDown (line 51) | def tearDown(self):
method test_delete_dns_record_is_not_called_by_create_dns_record (line 54) | def test_delete_dns_record_is_not_called_by_create_dns_record(
method test_dnspod_is_called_by_create_dns_record (line 72) | def test_dnspod_is_called_by_create_dns_record(self):
method test_dnspod_is_called_by_delete_dns_record (line 101) | def test_dnspod_is_called_by_delete_dns_record(self):
method test_exception_is_raised_if_unsuccessful (line 134) | def test_exception_is_raised_if_unsuccessful(self):
FILE: sewer/dns_providers/tests/test_duckdns.py
class TestDuckDNS (line 8) | class TestDuckDNS(TestCase):
method setUp (line 9) | def setUp(self):
method tearDown (line 22) | def tearDown(self):
method test_duckdns_is_called_by_create_dns_record (line 25) | def test_duckdns_is_called_by_create_dns_record(self):
method test_duckdns_is_called_by_delete_dns_record (line 45) | def test_duckdns_is_called_by_delete_dns_record(self):
FILE: sewer/dns_providers/tests/test_gandi.py
class MockResponseObject (line 9) | class MockResponseObject:
method __init__ (line 10) | def __init__(self, status_code=200, body=None):
method json (line 14) | def json(self):
function add_side_effect_to_request_get (line 18) | def add_side_effect_to_request_get(mock_requests_get):
function add_side_effect_to_request_post (line 41) | def add_side_effect_to_request_post(mock_requests_post):
function add_side_effect_to_request_delete (line 51) | def add_side_effect_to_request_delete(mock_requests_delete):
function mock_requests (line 64) | def mock_requests(func, mock_requests_get, mock_requests_post, mock_requ...
class TestGandiDns (line 86) | class TestGandiDns(TestCase):
method check_correct_headers_passed (line 92) | def check_correct_headers_passed(self, calls, expectedHeaders):
method test_delete_existing_record (line 97) | def test_delete_existing_record(self, mock_requests_lib):
method test_delete_non_existing_record (line 114) | def test_delete_non_existing_record(self, mock_requests_lib):
method test_create_record (line 119) | def test_create_record(self, mock_requests_lib):
method test_create_non_existing_record (line 149) | def test_create_non_existing_record(self, mock_requests_lib):
FILE: sewer/dns_providers/tests/test_hedns.py
class TestHEDNS (line 9) | class TestHEDNS(TestCase):
method setUp (line 13) | def setUp(self):
method tearDown (line 26) | def tearDown(self):
method test_extract_zone_sub_domain (line 29) | def test_extract_zone_sub_domain(self):
method test_extract_zone_root (line 38) | def test_extract_zone_root(self):
method test_hedns_is_called_by_create_dns_record (line 45) | def test_hedns_is_called_by_create_dns_record(self):
method test_hedns_is_not_called_by_delete_dns_record (line 65) | def test_hedns_is_not_called_by_delete_dns_record(self):
FILE: sewer/dns_providers/tests/test_powerdns.py
class TestPowerDNS (line 9) | class TestPowerDNS(TestCase):
method setUp (line 14) | def setUp(self):
method tearDown (line 28) | def tearDown(self):
method test_validate_powerdns_zone (line 31) | def test_validate_powerdns_zone(self):
method test_could_not_determine_apex_domain (line 46) | def test_could_not_determine_apex_domain(self):
method test_powerdns_has_correct_changetype (line 54) | def test_powerdns_has_correct_changetype(self):
method test_powerdns_returns_correct_status_code (line 63) | def test_powerdns_returns_correct_status_code(self):
method test_powerdns_is_called_by_create_dns_record (line 78) | def test_powerdns_is_called_by_create_dns_record(self):
method test_powerdns_is_called_by_delete_dns_record (line 92) | def test_powerdns_is_called_by_delete_dns_record(self):
FILE: sewer/dns_providers/tests/test_rackspace.py
class TestRackspace (line 9) | class TestRackspace(TestCase):
method setUp (line 13) | def setUp(self):
method tearDown (line 35) | def tearDown(self):
method test_find_dns_zone_id (line 38) | def test_find_dns_zone_id(self):
method test_find_dns_record_id (line 60) | def test_find_dns_record_id(self):
method test_delete_dns_record_is_not_called_by_create_dns_record (line 90) | def test_delete_dns_record_is_not_called_by_create_dns_record(self):
method test_rackspace_is_called_by_create_dns_record (line 114) | def test_rackspace_is_called_by_create_dns_record(self):
method test_rackspace_is_called_by_delete_dns_record (line 144) | def test_rackspace_is_called_by_delete_dns_record(self):
FILE: sewer/dns_providers/tests/test_route53.py
class TestRoute53 (line 7) | class TestRoute53(TestCase):
method setUp (line 11) | def setUp(self):
method tearDown (line 18) | def tearDown(self):
method mocked_route53_set_record_response (line 22) | def mocked_route53_set_record_response():
method make_change_batch (line 25) | def make_change_batch(self, action, domain_name, domain_value):
method mocked_find_zone_response (line 41) | def mocked_find_zone_response(self):
method test_user_given_credential (line 71) | def test_user_given_credential(self, mock_client):
method test_user_given_client (line 81) | def test_user_given_client(self, mock_client):
method test_user_given_creds_and_client (line 88) | def test_user_given_creds_and_client(self, mock_client):
method test_user_not_given_credential (line 93) | def test_user_not_given_credential(self, mock_client):
method test_route53_create_record (line 98) | def test_route53_create_record(self, mock_client):
method test_route53_delete_record (line 117) | def test_route53_delete_record(self, mock_client):
FILE: sewer/dns_providers/tests/test_unbound_ssh.py
class response (line 9) | class response:
method __init__ (line 12) | def __init__(self, *, content_val="", json_val=None):
method json (line 16) | def json(self):
class MockObj (line 22) | class MockObj:
method __init__ (line 23) | def __init__(self, **kwargs) -> None:
function patch_subprocess_run (line 27) | def patch_subprocess_run(returncode, **kwargs):
class TestLib (line 34) | class TestLib(TestCase):
method test01_init_requires_ssh_des (line 38) | def test01_init_requires_ssh_des(self):
method test02_init_okay (line 42) | def test02_init_okay(self):
method test03_init_with_alias_okay (line 45) | def test03_init_with_alias_okay(self):
method test13_unbound_command_bad_cmd_fails (line 50) | def test13_unbound_command_bad_cmd_fails(self):
method test21_create_delete_dns_record_okay (line 56) | def test21_create_delete_dns_record_okay(self):
method test22_create_dns_record_fail (line 65) | def test22_create_dns_record_fail(self):
FILE: sewer/dns_providers/tests/test_utils.py
class MockResponse (line 4) | class MockResponse(object):
method __init__ (line 9) | def __init__(self, status_code=200, content=None):
method json (line 31) | def json(self):
class mockLibcloudDriverZone (line 35) | class mockLibcloudDriverZone(object):
method create_record (line 42) | def create_record(self, name, type, data):
class mockLibcloudDriver (line 46) | class mockLibcloudDriver(object):
method __init__ (line 51) | def __init__(self, key, secret):
method get_zone (line 54) | def get_zone(self, domainSuffix):
method list_records (line 58) | def list_records(self, zone):
method get_record (line 66) | def get_record(self, zone_id, record_id):
method delete_record (line 69) | def delete_record(self, record):
function mockLibcloudGetDriver (line 73) | def mockLibcloudGetDriver(provider):
class MockDnsResolver (line 80) | class MockDnsResolver(object):
FILE: sewer/dns_providers/unbound_ssh.py
class UnboundSsh (line 6) | class UnboundSsh(BaseDns):
method __init__ (line 24) | def __init__(self, *, ssh_des, **kwargs):
method create_dns_record (line 35) | def create_dns_record(self, host_fqdn, acme_challenge):
method delete_dns_record (line 38) | def delete_dns_record(self, host_fqdn, acme_challenge):
method manage_dns_record (line 41) | def manage_dns_record(self, host_fqdn, acme_challenge, cmd):
function unbound_command (line 63) | def unbound_command(cmd, fqdn, acme_challenge):
FILE: sewer/lib.py
class SewerError (line 8) | class SewerError(Exception):
class AcmeError (line 13) | class AcmeError(SewerError):
class AcmeRegistrationError (line 18) | class AcmeRegistrationError(AcmeError):
function log_response (line 25) | def log_response(response: Any) -> str:
function create_logger (line 36) | def create_logger(name: str, log_level: Union[str, int]) -> LoggerType:
function safe_base64 (line 51) | def safe_base64(un_encoded_data: Union[str, bytes]) -> str:
function dns_challenge (line 60) | def dns_challenge(key_auth: str) -> str:
function sewer_meta (line 69) | def sewer_meta(name: str) -> str:
FILE: sewer/providers/demo.py
class ManualProvider (line 11) | class ManualProvider(ProviderBase):
method __init__ (line 12) | def __init__(self, *, chal_type: str = "http-01", **kwargs: Any) -> None:
method setup (line 23) | def setup(self, challenges: ChalListType) -> ErrataListType:
method unpropagated (line 26) | def unpropagated(self, challenges: ChalListType) -> ErrataListType:
method clear (line 30) | def clear(self, challenges: ChalListType) -> ErrataListType:
method _prompt (line 33) | def _prompt(self, mode: str, challenges: ChalListType) -> ErrataListType:
FILE: sewer/providers/tests/test_demo.py
class TestDemo (line 6) | class TestDemo(unittest.TestCase):
method test_create_dns (line 9) | def test_create_dns(self):
method test_create_http (line 13) | def test_create_http(self):
method test_create_invalid (line 17) | def test_create_invalid(self):
method test_run_dns (line 23) | def test_run_dns(self):
method test_run_http (line 35) | def test_run_http(self):
method test_accept_empty_chal_list (line 45) | def test_accept_empty_chal_list(self):
method test_fails_dns_bad_chal (line 49) | def test_fails_dns_bad_chal(self):
method test_fails_http_bad_chal (line 54) | def test_fails_http_bad_chal(self):
FILE: sewer/tests/test_Client.py
function keys_for_ACME (line 32) | def keys_for_ACME(no_kid=False):
function usual_ACME (line 40) | def usual_ACME(no_kid=False):
class TestClient (line 51) | class TestClient(TestCase):
method setUp (line 65) | def setUp(self):
method tearDown (line 76) | def tearDown(self):
method test_get_get_acme_endpoints_failure_results_in_exception (line 79) | def test_get_get_acme_endpoints_failure_results_in_exception(self):
method test_user_agent_is_generated (line 98) | def test_user_agent_is_generated(self):
method test_acme_registration_is_done (line 106) | def test_acme_registration_is_done(self):
method test_acme_registration_failure_doesnt_result_in_certificate (line 114) | def test_acme_registration_failure_doesnt_result_in_certificate(self):
method test_get_identifier_authorization_is_called (line 125) | def test_get_identifier_authorization_is_called(self):
method test_get_identifier_authorization_is_not_called (line 142) | def test_get_identifier_authorization_is_not_called(self):
method test_respond_to_challenge_called (line 162) | def test_respond_to_challenge_called(self):
method test_check_authorization_status_is_called (line 186) | def test_check_authorization_status_is_called(self):
method test_get_certificate_is_called (line 194) | def test_get_certificate_is_called(self):
method test_certificate_is_issued (line 205) | def test_certificate_is_issued(self):
method test_certificate_is_not_issued (line 214) | def test_certificate_is_not_issued(self):
method test_certificate_is_issued_for_renewal (line 242) | def test_certificate_is_issued_for_renewal(self):
method test_right_args_to_client (line 251) | def test_right_args_to_client(self):
class TestClientForSAN (line 265) | class TestClientForSAN(TestClient):
method setUp (line 272) | def setUp(self):
class TestClientForWildcard (line 294) | class TestClientForWildcard(TestClient):
method setUp (line 301) | def setUp(self):
class TestClientDnsApiCompatibility (line 324) | class TestClientDnsApiCompatibility(TestCase):
method setUp (line 331) | def setUp(self):
method test_get_get_acme_endpoints_failure_results_in_exception_with (line 344) | def test_get_get_acme_endpoints_failure_results_in_exception_with(self):
method test_create_dns_record_is_called (line 363) | def test_create_dns_record_is_called(self):
method test_delete_dns_record_is_called (line 374) | def test_delete_dns_record_is_called(self):
method test_right_args_to_client (line 385) | def test_right_args_to_client(self):
class TestClientUnits (line 399) | class TestClientUnits(TestCase):
method __init__ (line 403) | def __init__(self, *args, **kwargs):
method mock_sewer (line 409) | def mock_sewer(self, provider):
method test01_sleep_iter_sticky (line 414) | def test01_sleep_iter_sticky(self):
method test02_prop_timeout_okay (line 421) | def test02_prop_timeout_okay(self):
method test03_prop_timeout_timeout (line 425) | def test03_prop_timeout_timeout(self):
method test04_prop_timeout_delayed_okay (line 431) | def test04_prop_timeout_delayed_okay(self):
FILE: sewer/tests/test_auth.py
function pbj (line 6) | def pbj(**kwargs):
class TestAuth01 (line 10) | class TestAuth01(unittest.TestCase):
method test01_requires_chal_types (line 15) | def test01_requires_chal_types(self):
method test02_accepts_valid_chal_types (line 19) | def test02_accepts_valid_chal_types(self):
method _rejects_invalid_value_chal_types (line 25) | def _rejects_invalid_value_chal_types(self, chal_types):
method test03_rejects_str_chal_types (line 29) | def test03_rejects_str_chal_types(self):
method test04_rejects_iter_chal_types (line 32) | def test04_rejects_iter_chal_types(self):
method test05_rejects_unknown_parameters (line 37) | def test05_rejects_unknown_parameters(self):
method test06_accepts_logger (line 43) | def test06_accepts_logger(self):
method test07_accepts_log_level (line 46) | def test07_accepts_log_level(self):
method test08_rejects_logger_and_log_level (line 51) | def test08_rejects_logger_and_log_level(self):
method test09_prop_timeout_and_times_default (line 57) | def test09_prop_timeout_and_times_default(self):
method test10_prop_timeout_accepted (line 62) | def test10_prop_timeout_accepted(self):
method test11_prop_sleep_times_int_accepted (line 65) | def test11_prop_sleep_times_int_accepted(self):
method test12_prop_sleep_times_list_accepted (line 68) | def test12_prop_sleep_times_list_accepted(self):
method test13_prop_sleep_times_tuple_accepted (line 71) | def test13_prop_sleep_times_tuple_accepted(self):
method test14_prop_sleep_times_rejects (line 74) | def test14_prop_sleep_times_rejects(self):
class TestAuth02 (line 79) | class TestAuth02(unittest.TestCase):
method test01_notimplemented_setup (line 82) | def test01_notimplemented_setup(self):
method test02_notimplemented_unpropagated (line 86) | def test02_notimplemented_unpropagated(self):
method test03_notimplemented_clear (line 90) | def test03_notimplemented_clear(self):
class TestAuthHTTP (line 95) | class TestAuthHTTP(unittest.TestCase):
method test01_requires_nothing (line 98) | def test01_requires_nothing(self):
method test02_accepts_chal_types (line 101) | def test02_accepts_chal_types(self):
class TestAuthDNS (line 105) | class TestAuthDNS(unittest.TestCase):
method test01_requires_nothing (line 108) | def test01_requires_nothing(self):
method test02_accepts_chal_types (line 111) | def test02_accepts_chal_types(self):
method test03_accepts_alias (line 114) | def test03_accepts_alias(self):
method test04_without_alias (line 117) | def test04_without_alias(self):
method test05_with_alias (line 124) | def test05_with_alias(self):
FILE: sewer/tests/test_catalog.py
class TestCatalog (line 6) | class TestCatalog(unittest.TestCase):
method test01_ProviderCatalog_create (line 7) | def test01_ProviderCatalog_create(self):
method test02_catalog_get_item_list_okay (line 11) | def test02_catalog_get_item_list_okay(self):
method test03_catalog_get_descriptor_okay (line 15) | def test03_catalog_get_descriptor_okay(self):
method test04_catalog_get_provider_okay (line 19) | def test04_catalog_get_provider_okay(self):
method test_05_catalog__str__okay (line 24) | def test_05_catalog__str__okay(self):
FILE: sewer/tests/test_lib.py
class response (line 7) | class response:
method __init__ (line 8) | def __init__(self, *, content_val="", json_val=None):
method json (line 12) | def json(self):
class TestLib (line 18) | class TestLib(unittest.TestCase):
method test01_log_response_json_okay (line 19) | def test01_log_response_json_okay(self):
method test02_log_response_content_okay (line 22) | def test02_log_response_content_okay(self):
method test11_create_logger_okay (line 25) | def test11_create_logger_okay(self):
method test21_safe_base64_str_or_bytes_okay (line 31) | def test21_safe_base64_str_or_bytes_okay(self):
method test31_dns_challenge_okay (line 35) | def test31_dns_challenge_okay(self):
method test41_sewer_meta_okay (line 39) | def test41_sewer_meta_okay(self):
FILE: sewer/tests/test_utils.py
class ExmpleDnsProvider (line 7) | class ExmpleDnsProvider(BaseDns):
method __init__ (line 8) | def __init__(self, **kwargs):
method create_dns_record (line 12) | def create_dns_record(self, domain_name, domain_dns_value):
method delete_dns_record (line 15) | def delete_dns_record(self, domain_name, domain_dns_value):
class ExmpleDNS (line 19) | class ExmpleDNS(ExmpleDnsProvider):
method __init__ (line 22) | def __init__(self, fail_prop_count, **kwargs):
method unpropagated (line 26) | def unpropagated(self, challenges):
class ExmpleHttpProvider (line 33) | class ExmpleHttpProvider(ProviderBase):
method __init__ (line 34) | def __init__(self):
method setup (line 37) | def setup(self, challenges):
method unpropagated (line 40) | def unpropagated(self, challenges):
method clear (line 43) | def clear(self, challenges):
class MockResponse (line 47) | class MockResponse(object):
method __init__ (line 52) | def __init__(self, status_code=201, content=None):
method json (line 93) | def json(self):
FILE: tests/crypto_test.py
function fromfile_privbytes_frombytes_sign_key (line 14) | def fromfile_privbytes_frombytes_sign_key(key_type: KeyType) -> None:
function test11_rsa_kitchen_sink (line 42) | def test11_rsa_kitchen_sink():
function test12_secp_kitchen_sink (line 47) | def test12_secp_kitchen_sink():
function generate_test (line 52) | def generate_test(key_types: Sequence[KeyType]) -> None:
function test21_generate_rsa_keys (line 58) | def test21_generate_rsa_keys():
function test22_generate_ec_keys (line 62) | def test22_generate_ec_keys():
function test31_read_key_write_acct_read_acct (line 66) | def test31_read_key_write_acct_read_acct():
Condensed preview — 90 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (357K chars).
[
{
"path": ".circleci/codecov.yml",
"chars": 322,
"preview": "codecov:\n notify:\n require_ci_to_pass: yes\n\ncoverage:\n range: 70..100\n round: down\n precision: 2\n status:\n pr"
},
{
"path": ".circleci/config.yml",
"chars": 1678,
"preview": "# Python CircleCI 2.0 configuration file\n#\n# Check https://circleci.com/docs/2.0/language-python/ for more details\n#\nver"
},
{
"path": ".github/CONTRIBUTING.md",
"chars": 4118,
"preview": "# Contributing to sewer\n\nThank you for thinking of contributing to sewer. Every contribution to\nsewer is important to u"
},
{
"path": ".github/ISSUE_TEMPLATE.md",
"chars": 856,
"preview": "## Which version of python are you using?\n\n## What operating system and version of operating system are you using?\n\n## W"
},
{
"path": ".github/PULL_REQUEST_TEMPLATE.md",
"chars": 798,
"preview": "Thank you for contributing to sewer. \nEvery contribution to sewer is important to us. "
},
{
"path": ".github/workflows/build.yml",
"chars": 1040,
"preview": "name: Build\non: push\n \njobs:\n linux:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - "
},
{
"path": ".gitignore",
"chars": 546,
"preview": "# Compiled Python modules\n*.pyo\n*.pyc\n\n# SQLite\n*.db\n*.s3db\n*.sqlite3\n\n# Vagrant private files\n.vagrant/*\n.vagrant*\n\n#lo"
},
{
"path": "CONTRIBUTORS.md",
"chars": 1355,
"preview": "Thank you for contributing to sewer.\nEvery contribution to sewer is important to us.\n\n> Contributor offers to license ce"
},
{
"path": "LICENSE.txt",
"chars": 1078,
"preview": "The MIT License (MIT)\n\nCopyright (c) 2017 Komu Wairagu\n\nPermission is hereby granted, free of charge, to any person obta"
},
{
"path": "Makefile",
"chars": 2987,
"preview": "# we may need something other than the system default here\n# use $ PYTHON=python3 make ... for example\npython = python"
},
{
"path": "README.md",
"chars": 3704,
"preview": "## Sewer\n\n[](https://github.com/komuw/"
},
{
"path": "docs/ACME.md",
"chars": 5260,
"preview": "# ACME, RFCs, and confusion, oh my!\n\nACME grew out of early, ad-hoc procedures designed to let CAs issue large\nnumbers o"
},
{
"path": "docs/Aliasing.md",
"chars": 4738,
"preview": "# Aliasing for ACME Validation\n\nThe idea is presented (for dns-01 authorizations) in [an article at\nletsencrypt.org](htt"
},
{
"path": "docs/CHANGELOG.md",
"chars": 6552,
"preview": "# `sewer` changelog:\n\n## **pre-release** 0.8.5\n\n- driver for Windows DNS server (local only) [IN PROGRESS]\n\n- cleanup th"
},
{
"path": "docs/DNS-Propagation.md",
"chars": 5337,
"preview": "# Waiting for Mr. DNS or Someone Like Him\n\nQ: How long does it take after you've setup the challenge response TXT record"
},
{
"path": "docs/LegacyDNS.md",
"chars": 2244,
"preview": "## Legacy DNS challenge providers\n\n### `BaseDns` shim class\n\nA child of `DNSProviderBase` that acts as an adapter betwee"
},
{
"path": "docs/UnifiedProvider.md",
"chars": 10507,
"preview": "# DNS and HTTP challenges unified\n\n_There's still a draft when the wind is blowing, but it's getting less._\n\n## Dedicati"
},
{
"path": "docs/catalog.md",
"chars": 6229,
"preview": "# The Catalog of Drivers\n\nThe driver catalog, `sewer/catalog.json`, replaces scattered facilities that\nwere used to stit"
},
{
"path": "docs/crypto.md",
"chars": 4259,
"preview": "# A crypto module for ACME\n\nThere were several motivations behind the creation of `crypto.py`:\n\n- a desire to convert th"
},
{
"path": "docs/dns-01.md",
"chars": 5081,
"preview": "# DNS service drivers\n\nACME's dns-01 authorization was sewer's original target. There are a number\nof DNS services supp"
},
{
"path": "docs/drivers/route53.md",
"chars": 710,
"preview": "## route53 - driver for AWS DNS service\n\n### Command line use\n\nroute53 has never been wired into `sewer-cli`, and that h"
},
{
"path": "docs/drivers/unbound_ssh.md",
"chars": 1788,
"preview": "## unbound_ssh legacy DNS driver\n\nA working, if somewhat quirky, driver to setup challenges in local data of\nthe [unboun"
},
{
"path": "docs/http-01.md",
"chars": 252,
"preview": "# HTTP challenge providers\n\nThere are no http-01 drivers in sewer yet.\n\n## Bring your own HTTP provider\n\n**To be rewritt"
},
{
"path": "docs/index.md",
"chars": 2691,
"preview": "## sewer, the ACME library and command-line client\n\nThis is a quick & dirty directory of the docs directory, which is st"
},
{
"path": "docs/notes/0.8.2-notes.md",
"chars": 1852,
"preview": "## 0.8.2 release\n\n0.8.2 contains a lot more work - and changes - than recent releases,\nhence this verbose guide to what'"
},
{
"path": "docs/notes/0.8.3-notes.md",
"chars": 4035,
"preview": "## Sewer 0.8.3 Release Notes\n\nThis will attempt to list all the changes that affect users of the\n`sewer-cli` program, in"
},
{
"path": "docs/notes/0.8.4-notes.md",
"chars": 2755,
"preview": "# Sewer 0.8.4 Release Notes\n\n## What's New\n\n- ECDSA keys supported (account and/or certificate keys)\n\n- optional `client"
},
{
"path": "docs/preview/cloaca.md",
"chars": 2512,
"preview": "# Cloaca, aka sewer-cli-next?\n\n_This is so preliminary that it hasn't been written yet. A design essay in\na rambling st"
},
{
"path": "docs/preview/cloaca_config.md",
"chars": 5319,
"preview": "# Configuration file for cloaca\n\n_This is a pre-coding, let alone release, preview of a more config-driven\nuser command "
},
{
"path": "docs/sewer-as-a-library.md",
"chars": 5044,
"preview": "# Sewer as a Python Library\n\n>`sewer-the-library` is in a period of heavy change (summer 2020 - ?). I'll\ntry to keep th"
},
{
"path": "docs/sewer-cli.md",
"chars": 9078,
"preview": "## Sewer's user command (so many --options!)\n\nSewer's command line interface, historically named just \"sewer\" or\n\"sewer-"
},
{
"path": "docs/unpropagated.md",
"chars": 3241,
"preview": "# Waiting for the Challenge to Propagate\n\nWhen you use a service provider's API to setup a challenge response, how\nlong "
},
{
"path": "docs/wildcards.md",
"chars": 2432,
"preview": "# Wildcard Certificates\n\nSince 0.8.2, sewer should be able to request and receive simple wildcard\ncertificates using any"
},
{
"path": "mypy.ini",
"chars": 503,
"preview": "# ignoring missing annotations is still a way of life. In general, add\n# mypy-package here for general-purpose librarie"
},
{
"path": "pyproject.toml",
"chars": 622,
"preview": "[tool.coverage.run]\ncommand_line = \"-m pytest\"\nsource = [\"sewer\"]\nomit = [\n \"*test*\",\n \"*__init__.py\"\n]\ndata_file "
},
{
"path": "setup.py",
"chars": 2968,
"preview": "import codecs, json, os\nfrom setuptools import setup, find_packages\n\n# long description comes from README.md\nwith codecs"
},
{
"path": "sewer/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "sewer/__main__.py",
"chars": 30,
"preview": "from . import cli\n\ncli.main()\n"
},
{
"path": "sewer/auth.py",
"chars": 3658,
"preview": "from typing import Any, Dict, Optional, Sequence, Tuple, Union, cast\n\nfrom .lib import create_logger, LoggerType\n\nChalIt"
},
{
"path": "sewer/catalog.json",
"chars": 5847,
"preview": "[\n {\n \"name\": \"acmedns\",\n \"desc\": \"AcmeDns DNS provider\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n "
},
{
"path": "sewer/catalog.py",
"chars": 2472,
"preview": "import codecs, importlib, json, os\nfrom typing import Dict, List, Sequence\n\nfrom .auth import ProviderBase\n\n\nclass Provi"
},
{
"path": "sewer/cli.py",
"chars": 14141,
"preview": "import argparse, os\n\nfrom . import client, config, lib\n\nfrom .catalog import ProviderCatalog\nfrom .crypto import AcmeKey"
},
{
"path": "sewer/client.py",
"chars": 28550,
"preview": "import json, time, platform\nfrom hashlib import sha256\nfrom typing import Dict, Sequence, Tuple, Union\n\nimport requests\n"
},
{
"path": "sewer/config.py",
"chars": 167,
"preview": "ACME_DIRECTORY_URL_STAGING = \"https://acme-staging-v02.api.letsencrypt.org/directory\"\nACME_DIRECTORY_URL_PRODUCTION = \"h"
},
{
"path": "sewer/crypto.py",
"chars": 12160,
"preview": "import time\n\nfrom cryptography import x509\nfrom cryptography.x509.oid import NameOID\nfrom cryptography.hazmat.primitives"
},
{
"path": "sewer/dns_providers/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "sewer/dns_providers/acmedns.py",
"chars": 2359,
"preview": "import urllib.parse\n\nimport requests\n\nfrom dns.resolver import Resolver\n\nfrom . import common\nfrom ..lib import log_resp"
},
{
"path": "sewer/dns_providers/aliyundns.py",
"chars": 7689,
"preview": "import json\n\nfrom aliyunsdkcore import client # type: ignore\nimport aliyunsdkalidns.request.v20150109 # type:ignore\nfr"
},
{
"path": "sewer/dns_providers/auroradns.py",
"chars": 3118,
"preview": "# DNS Provider for AuroRa DNS from the dutch hosting provider pcextreme\n# https://www.pcextreme.nl/aurora/dns\n# Aurora u"
},
{
"path": "sewer/dns_providers/cloudflare.py",
"chars": 6679,
"preview": "import urllib.parse\n\nimport requests\n\nfrom . import common\nfrom ..lib import log_response\n\n\nclass CloudFlareDns(common.B"
},
{
"path": "sewer/dns_providers/cloudns.py",
"chars": 2183,
"preview": "from cloudns_api import record # type: ignore\n\nfrom . import common\n\n\n### FIX ME ### this assumes there are only two le"
},
{
"path": "sewer/dns_providers/common.py",
"chars": 3542,
"preview": "from typing import Any, Dict, Sequence\n\nfrom ..auth import ErrataItemType, DNSProviderBase\nfrom ..lib import dns_challen"
},
{
"path": "sewer/dns_providers/dnspod.py",
"chars": 4837,
"preview": "import urllib.parse\n\nimport requests\n\nfrom . import common\n\n\nclass DNSPodDns(common.BaseDns):\n def __init__(\n "
},
{
"path": "sewer/dns_providers/duckdns.py",
"chars": 2299,
"preview": "import urllib.parse\n\nimport requests\n\nfrom . import common\n\n\nclass DuckDNSDns(common.BaseDns):\n def __init__(self, du"
},
{
"path": "sewer/dns_providers/gandi.py",
"chars": 4584,
"preview": "import os\nfrom itertools import chain\n\nimport requests\n\nfrom . import common\n\n\nclass GandiDns(common.BaseDns):\n def _"
},
{
"path": "sewer/dns_providers/hurricane.py",
"chars": 2214,
"preview": "\"\"\"\nHurricane Electric DNS Support\n\"\"\"\nimport json\n\nimport HurricaneDNS as _hurricanedns # type: ignore\n\nfrom . import "
},
{
"path": "sewer/dns_providers/powerdns.py",
"chars": 3841,
"preview": "import json\n\nimport requests\n\nfrom . import common\n\n\nclass PowerDNSDns(common.BaseDns):\n \"\"\"\n For PowerDNS, all su"
},
{
"path": "sewer/dns_providers/rackspace.py",
"chars": 10730,
"preview": "import time, urllib.parse\n\nimport requests, tldextract\n\nfrom . import common\nfrom ..lib import log_response\n\n\nclass Rack"
},
{
"path": "sewer/dns_providers/route53.py",
"chars": 4443,
"preview": "import collections\n\nimport boto3 # type: ignore\nfrom botocore.client import Config # type: ignore\n\nfrom . import commo"
},
{
"path": "sewer/dns_providers/tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "sewer/dns_providers/tests/test_acmedns.py",
"chars": 2558,
"preview": "from unittest import mock\nimport json\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.acmedns import AcmeDnsDns\n"
},
{
"path": "sewer/dns_providers/tests/test_aliyundns.py",
"chars": 2533,
"preview": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.aliyundns import AliyunDns\n\nfrom . imp"
},
{
"path": "sewer/dns_providers/tests/test_aurora.py",
"chars": 2900,
"preview": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.auroradns import AuroraDns\n\nfrom . imp"
},
{
"path": "sewer/dns_providers/tests/test_cloudflare.py",
"chars": 7926,
"preview": "from unittest import mock\nimport json\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.cloudflare import CloudFla"
},
{
"path": "sewer/dns_providers/tests/test_cloudns.py",
"chars": 2825,
"preview": "import os, sys\nfrom unittest import mock, skipIf, TestCase\n\nfrom sewer.dns_providers.cloudns import ClouDNSDns\n\nfrom . i"
},
{
"path": "sewer/dns_providers/tests/test_common.py",
"chars": 1596,
"preview": "from unittest import mock, TestCase\n\nimport sewer.dns_providers.common\n\n\nclass TestCommon(TestCase):\n \"\"\"\n \"\"\"\n\n "
},
{
"path": "sewer/dns_providers/tests/test_dnspod.py",
"chars": 6692,
"preview": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.dnspod import DNSPodDns\n\nfrom . import"
},
{
"path": "sewer/dns_providers/tests/test_duckdns.py",
"chars": 1949,
"preview": "import json\nfrom unittest import mock, TestCase\n\nfrom . import test_utils\nfrom ..duckdns import DuckDNSDns\n\n\nclass TestD"
},
{
"path": "sewer/dns_providers/tests/test_gandi.py",
"chars": 7028,
"preview": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.gandi import GandiDns\n\nMOCK_GANDI_API_"
},
{
"path": "sewer/dns_providers/tests/test_hedns.py",
"chars": 2920,
"preview": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.hurricane import HurricaneDns\n\nfrom . "
},
{
"path": "sewer/dns_providers/tests/test_powerdns.py",
"chars": 3867,
"preview": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.powerdns import PowerDNSDns\n\nfrom . im"
},
{
"path": "sewer/dns_providers/tests/test_rackspace.py",
"chars": 8481,
"preview": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.rackspace import RackspaceDns\n\nfrom . "
},
{
"path": "sewer/dns_providers/tests/test_route53.py",
"chars": 5134,
"preview": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.route53 import Route53Dns\n\n\nclass Test"
},
{
"path": "sewer/dns_providers/tests/test_unbound_ssh.py",
"chars": 2403,
"preview": "from unittest import mock, TestCase\n\nfrom .. import unbound_ssh\n\n\n####### Mocks and other helpers #######\n\n\nclass respon"
},
{
"path": "sewer/dns_providers/tests/test_utils.py",
"chars": 1877,
"preview": "import json\n\n\nclass MockResponse(object):\n \"\"\"\n mock python-requests Response object\n \"\"\"\n\n def __init__(sel"
},
{
"path": "sewer/dns_providers/unbound_ssh.py",
"chars": 2531,
"preview": "import subprocess\n\nfrom .common import BaseDns\n\n\nclass UnboundSsh(BaseDns):\n \"\"\"\n Working demo of using aliasing w"
},
{
"path": "sewer/lib.py",
"chars": 2051,
"preview": "import base64, codecs, json, logging, os\nfrom hashlib import sha256\nfrom typing import Any, Union\n\nLoggerType = logging."
},
{
"path": "sewer/meta.json",
"chars": 320,
"preview": "{\n \"name\": \"sewer\",\n \"description\": \"Sewer is a programmatic Lets Encrypt(ACME) client\",\n \"url\": \"https://github.com/"
},
{
"path": "sewer/providers/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "sewer/providers/demo.py",
"chars": 1924,
"preview": "\"demo.py - examples of implementing non-or-minimally-functional challenge providers\"\n\n# still minimally functional - too"
},
{
"path": "sewer/providers/tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "sewer/providers/tests/test_demo.py",
"chars": 2192,
"preview": "import unittest\n\nfrom .. import demo\n\n\nclass TestDemo(unittest.TestCase):\n \"this actually tests nothing non-trivial, "
},
{
"path": "sewer/tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "sewer/tests/test_Client.py",
"chars": 17321,
"preview": "# the test dir is a sub dir of sewer/sewer so as\n# not to pollute the global namespace.\n# see: https://python-packaging."
},
{
"path": "sewer/tests/test_auth.py",
"chars": 4432,
"preview": "import unittest\n\nfrom .. import auth, lib\n\n\ndef pbj(**kwargs):\n return auth.ProviderBase(chal_types=[\"dns-01\"], **kwa"
},
{
"path": "sewer/tests/test_catalog.py",
"chars": 885,
"preview": "import unittest\n\nfrom .. import auth, catalog\n\n\nclass TestCatalog(unittest.TestCase):\n def test01_ProviderCatalog_cre"
},
{
"path": "sewer/tests/test_lib.py",
"chars": 1357,
"preview": "import logging\nimport unittest\n\nfrom .. import lib\n\n\nclass response:\n def __init__(self, *, content_val=\"\", json_val="
},
{
"path": "sewer/tests/test_utils.py",
"chars": 3037,
"preview": "import json\n\nfrom ..auth import ProviderBase\nfrom ..dns_providers.common import BaseDns\n\n\nclass ExmpleDnsProvider(BaseDn"
},
{
"path": "tests/crypto_test.py",
"chars": 2072,
"preview": "import unittest\n\nfrom typing import Sequence, Tuple\n\nfrom sewer.crypto import AcmeAccount, AcmeCsr, AcmeKey\n\n\nKeyType = "
},
{
"path": "tests/data/README",
"chars": 39,
"preview": "Directory for test data and artifacts.\n"
}
]
About this extraction
This page contains the full source code of the komuw/sewer GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 90 files (330.9 KB), approximately 80.7k tokens, and a symbol index with 428 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.