[ { "path": ".circleci/codecov.yml", "content": "codecov:\n notify:\n require_ci_to_pass: yes\n\ncoverage:\n range: 70..100\n round: down\n precision: 2\n status:\n project:\n default:\n # basic\n target: 85\n\ncomment:\n layout: \"reach, diff, flags, files\"\n behavior: default\n require_changes: yes # if true: only post the comment if coverage changes\n" }, { "path": ".circleci/config.yml", "content": "# Python CircleCI 2.0 configuration file\n#\n# Check https://circleci.com/docs/2.0/language-python/ for more details\n#\nversion: 2\njobs:\n build:\n docker:\n # specify the version you desire here\n # use `-browsers` prefix for selenium tests, e.g. `3.6.1-browsers`\n - image: circleci/python:3.6\n # Specify service dependencies here if necessary\n # CircleCI maintains a library of pre-built images\n # documented at https://circleci.com/docs/2.0/circleci-images/\n # - image: circleci/postgres:9.4\n\n working_directory: ~/repo\n\n steps:\n - checkout\n\n - run:\n name: install dependencies\n command: |\n sudo pip3 install -e .[dev,test,alldns]\n\n # run tests! Most config is in pyprojects.toml rather than --options since 0.8.4\n - run:\n name: run tests\n command: |\n make testdata\n find . -type f -name *.pyc -delete | echo\n coverage erase\n coverage run && ln -s tests/coverage/.coverage .coverage && bash <(curl -s https://codecov.io/bash)\n\n - run:\n name: run tests reports\n command: | \n coverage report --fail-under=85\n\n - run:\n name: run static analyzers\n command: |\n black --check . || { printf \"\\\\n\\\\t please use black to format your code.\"; exit 77; }\n pylint --enable=E --disable=W,R,C --unsafe-load-any-extension=y sewer/\n\n - run:\n name: run sewer cli\n command: |\n sewer --version && sewer --help\n\n # run make upload\n\n - store_artifacts:\n path: test-reports\n destination: test-reports\n" }, { "path": ".github/CONTRIBUTING.md", "content": "# Contributing to sewer\n\nThank you for thinking of contributing to sewer. Every contribution to\nsewer is important to us. You may not know it, but you are about to\ncontribute towards making the world a safer and more secure place.\n\nContributor offers to license certain software (a “Contribution” or multiple\n“Contributions”) to sewer, and sewer agrees to accept said Contributions,\nunder the terms of the MIT License. Contributor understands and agrees that\nsewer shall have the irrevocable and perpetual right to make and distribute\ncopies of any Contribution, as well as to create and distribute collective\nworks and derivative works of any Contribution, under the MIT License.\n\n## To contribute:\n\n- fork this repo.\n\n- cd sewer\n\n- open an issue on this repo. In your issue, outline what it is you want to add and why.\n\n- install pre-requiste software:\n```shell\npip3 install -e .[dev,test]\n```\n\n- python cryptography generally only requires openssl's libraries. To run\n the full set of tests (make test), you **also need the openssl program**\n\n- make the changes you want on your fork.\n\n- your changes should have backward compatibility in mind unless it is impossible to do so.\n\n- add your name and contact(optional) to CONTRIBUTORS.md\n\n- add tests\n\n- format your code using [black](https://github.com/ambv/black) *NB:*\nrequires black 19.3.b0 or newer (19.10b0 is used by the CI):\n```shell\nblack -l 100 -t py35 .\n```\n\n- run [pylint](https://pypi.python.org/pypi/pylint) on the code and fix any issues:\n```shell\npylint --enable=E --disable=W,R,C sewer/\n```\n\n- run tests and make sure everything is passing:\n```shell\nmake test\n```\n- open a pull request on this repo.\n\nNB: I make no commitment of accepting your pull requests.\n\n##Styles\n\nMartin (@mmaney) has a few things to say about what he's looking for aside\nfrom code that works:\n\n- Python is not Java. There is rarely an excuse for @staticmethod, we can\n use first-class non-member _functions_ when _self_ would just be baggage.\n\n- When fixing things, I approve of trying to identify a minimal change that\n repairs the bug (or adds a feature, etc.). But be prepared to get\n feedback asking (sometimes sketching) a more intrusive refactoring that\n perhaps incidentally fixes the bug. It's not that I don't appreciate a\n small, focused patch, but there's a lot of houscleaning going on these days!\n\n- And sometimes your PR (or less often a bug report itself) will get me\n thinking about a piece of work I hadn't focused on yet (or get me thinking\n about it in a more productive way), and all of a sudden I've stolen your\n patch and wrapped it in a larger refactoring. Don't doubt that your\n contribution was appreciated, you just yodeled and triggered an avalanche\n that you weren't expecting!\n\n- black is the current fad, but its indifference to actual readability in\n favor of slavish consistency to simplistic rules sometimes makes me ill. \n But for now it's a thing.\n\n\n## Creating a new release:\nTo create a new release on [https://pypi.org/project/sewer](https://pypi.org/project/sewer);\n\n- Create a new branch\n\n- Update `sewer/meta.json` with the new version number.\n The version numbers should follow semver.\n\n- Update `docs/CHANGELOG.md` with information about what is changing in the new release.\n\n- Open a pull request and have it go through the normal review process.\n\n- Upon approval of the pull request, squash and merge it. \n Remember that the squash & merge commit message should ideally be the message that was in the pull request template. \n\n- Once succesfully merged, run; \n```bash\ngit checkout master\ngit pull --tags\n# if plain \"python\" runs Py2, prefix \"make ...\" with \"PYTHON=python3\"\n# or have it set in your shell environment.\nmake uploadprod\n```\n That should upload the new release on pypi. You do need to have\n permissions to upload to pypi. Currently only\n [@komuw](https://github.com/komuw) and\n [@mmaney](https://github.com/mmaney) have pypi permissions, so if you\n need to create a new release, do talk to him to do that. In the future,\n more contributors may be availed permissions to pypi.\n" }, { "path": ".github/ISSUE_TEMPLATE.md", "content": "## Which version of python are you using?\n\n## What operating system and version of operating system are you using?\n\n## What version of sewer are you using?\n\n## What did you do? (be as detailed as you can)\n\n## What did you expect to see/happen/not happen?\n\n## What did you actually see/happen? \n\n## Paste here the log output generated by `sewer`, if any. Please remember to remove any sensitive items from the log before pasting here.\n## If you can, run sewer with loglevel set to debug; eg `sewer --loglevel DEBUG` \n\n\nAlternatively if you want to conribute to this repo, answer this questions instead in your issue: \n\n## What is it that you would like to propose to add/remove/change?\n\n## Why do you want to add/remove/change that?\n\n## How do you want to go about adding/removing/changing that?\n" }, { "path": ".github/PULL_REQUEST_TEMPLATE.md", "content": "Thank you for contributing to sewer. \nEvery contribution to sewer is important to us. \nYou may not know it, but you have just contributed to making the world a more safer and secure place. \n\nContributor offers to license certain software (a “Contribution” or multiple\n“Contributions”) to sewer, and sewer agrees to accept said Contributions,\nunder the terms of the MIT License.\nContributor understands and agrees that sewer shall have the irrevocable and perpetual right to make\nand distribute copies of any Contribution, as well as to create and distribute collective works and\nderivative works of any Contribution, under the MIT License.\n\n\nNow, \n\n## What(What have you changed?)\n\n\n## Why(Why did you change it?)\n\n" }, { "path": ".github/workflows/build.yml", "content": "name: Build\non: push\n \njobs:\n linux:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-python@v5\n with:\n python-version: '3.10.x'\n cache: 'pip'\n\n - name: install dependencies\n shell: bash\n run: |\n sudo apt install build-essential\n python -m pip install -e .[dev,test,alldns]\n - name: run tests\n shell: bash\n run: |\n make testdata\n find . -type f -name *.pyc -delete | echo\n coverage erase\n coverage run\n - name: generate test reports\n shell: bash\n run: |\n coverage report --fail-under=85\n - name: run static analyzers\n shell: bash\n run: |\n black --check . || { printf \"\\\\n\\\\t please use black to format your code.\"; exit 77; }\n pylint --enable=E --disable=W,R,C --unsafe-load-any-extension=y sewer/\n - name: run sewer cli\n shell: bash\n run: |\n sewer --version && sewer --help\n" }, { "path": ".gitignore", "content": "# Compiled Python modules\n*.pyo\n*.pyc\n\n# SQLite\n*.db\n*.s3db\n*.sqlite3\n\n# Vagrant private files\n.vagrant/*\n.vagrant*\n\n#logs\n*.log\n*.log.*\n\n#pids\n*.pid\n\n#ppa public keys\n*.asc*\n\n# markdown preview\nREADME.html\n\n# vscode editor\n.vscode/\n.DS_Store\n\n# testing artifacts, temp files, etc.\n/tests/coverage/\n/tests/tmp/\n\n# Setuptools distribution folder.\n/dist/\n/build/\n\n# Python egg metadata, regenerated from source files by setuptools.\n/*.egg-info\n\n# virtualenv\n.venv\n\n# certificates and keys\n*.csr\n*.crt\n*.key\n*.pem\n.idea\n.vscode\n\n.mypy_cache/\nlocal/\n" }, { "path": "CONTRIBUTORS.md", "content": "Thank you for contributing to sewer.\nEvery contribution to sewer is important to us.\n\n> Contributor offers to license certain software (a _Contribution_ or\nmultiple _Contributions_) to sewer, and sewer agrees to accept said\nContributions, under the terms of the MIT License. Contributor understands\nand agrees that sewer shall have the irrevocable and perpetual right to make\nand distribute copies of any Contribution, as well as to create and\ndistribute collective works and derivative works of any Contribution, under\nthe MIT License.\n\nContributors\n------------\n\n- Author: [komu W](https://www.komu.engineer)\n- Current maintainer: [@mmaney](https://github.com/mmaney)\n- [Wilfried Jonker](wjonker.nl)\n- [András Veres-Szentkirályi, dnet](https://techblog.vsza.hu/)\n- [menduo](https://menduo.net)\n- [m4ldonado](https://github.com/m4ldonado)\n- [luisbarrueco](https://github.com/luisbarrueco)\n- [Tungsteno74](https://github.com/Tungsteno74)\n- [Harold Bradley III](https://haroldbradleyiii.com/)\n- [@etienne-napoleone](https://github.com/etienne-napoleone)\n- [@soloradish](https://github.com/soloradish)\n- [Moritz Ulmer](https://www.protohaus.org)\n- [rozgonik](https://github.com/rozgonik)\n- [alec T](https://github.com/AlecTroemel)\n- [Don S](https://github.com/donspaulding)\n- [Julien Demoor](https://github.com/jdkx)\n- [@tkalus](https://github.com/tkalus)\n" }, { "path": "LICENSE.txt", "content": "The MIT License (MIT)\n\nCopyright (c) 2017 Komu Wairagu\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE." }, { "path": "Makefile", "content": "# we may need something other than the system default here\n# use $ PYTHON=python3 make ... for example\npython = python\nifdef PYTHON\n\tpython = ${PYTHON}\nendif\n\n# invoke these using ${python} -m to avoid yet more xxxx3 naming issues\ntwine = ${python} -m twine\npip = ${python} -m pip\ncoverage = ${python} -m coverage\nblack = ${python} -m black\npylint = ${python} -m pylint\nmypy = ${python} -m mypy\n\n\nVERSION_STRING=$$(sed -n -e '/\"version\"/ s/.*version\": *\"\\([^\"]*\\)\".*/\\1/p' * Generate a PKCS#10 [RFC2986] Certificate Signing Request (CSR).\n> * Cut and paste the CSR into a CA's web page.\n> * Prove ownership of the domain(s) in the CSR by one of the following methods:\n> + Put a CA-provided challenge at a specific place on the web server\n> + Put a CA-provided challenge in a DNS record corresponding to the target domain\n> + Receive a CA-provided challenge at (hopefully) an administrator-controlled email \n> address corresponding to the domain, and then respond to it on the CA's web page\n> * Download the issued certificate and install it on the user's web server\n>\n> With the exception of the CSR itself and the certificates that are\n> issued, these are all completely ad hoc procedures and are\n> accomplished by getting the human user to follow interactive natural\n> language instructions from the CA rather than by machine-implemented\n> published protocols.\n\nHTTP validation was the first mechanism, matching the first method of\nproving ownership in the above. The rest of what\n[Let's Encrypt](https://letsencrypt.org)\nadded was automating the process (and rearranging it a bit, having the proof\nof control happen before the CSR, etc.). Years later, the IETF standardized\nthe ACME protocol, and there are other variants that have been (or will be)\nstandardized.\n\n## RFC8555\n\nThe [IETF](https://www.ietf.org/) has published\n[RFC8555](https://tools.ietf.org/html/rfc8555) defining the ACME protocol\nfor http-01 and dns-01 validations of dns-name authorizations. These are\nthe sort of ACME authorizations that we usually think of, and which sewer\nworks with. The RFC was published in the spring of 2019, but it wasn't\nuntil near the end of that year that Let's Encrypt adopted the full v.2 on\nonly their *staging* server. There's some elaborate and, from what I can\nmake out, often-shifting schedule for various partial transitions, but I'm\nnot going to try to make sense of them. As of the beginning of 2020, the\nonly immediate effect on sewer was that one could no longer run it against\nthe *staging* server. The next big change is when that same restriction is\nrolled out on LE's *production* server later in the year. Since sewer\nv0.8.2, which implemented the final RFC8555 protocol at least well enough to\nwork with LE's server implementation, our tl;dr is just this:\n\n> If you get a failure running an older version of sewer, get v0.8.2 or\n later. This is a known problem: v0.8.2 or later is the fix.\n\n### RSA Keys\n\nLE probably accepts RSA keys of a wide range of sizes. Traditionally sewer\nhas used a 2048 bit RSA key by default, with a _bits_ option available in\nthe Client() interface, with no support from the command line. As part of\nthe crypto overhaul, 0.8.4 has introduced support for several sizes of RSA\nkey by name (matching the elliptical curve keys it added). 2048, 3072 and\n4096 bit RSA keys can be called for, and the default (for keys not\nexplicitly provided and so generated by the cli program) is changed to a\n3072 bit RSA key for both the account key and the certificate key. Either\nor both can use an externally generated key (of one of these sizes only) or\nyou can (from the cli) change the generated key type & size for each.\n\nSHA256 is the only hash algorithm for signing with an RSA key that LE\ncurrently accepts, and so there is no option to change it.\n\n### Elliptic Curve Keys and Certificates\n\nRFC8555 specifies _An ACME server MUST implement the \"ES256\" signature algorithm\n[RFC7518] and SHOULD implement the \"EdDSA\" signature algorithm using\nthe \"Ed25519\" variant (indicated by \"crv\") [RFC8037]. As of 0.8.4, sewer\nsupports all the EC curves which LE currently accepts. These are\n_secp256r1_ (P-256 curve using ES256 for signatures) and _secp384r1_ (P-384\nand ES384). So once again there's no option for changing the signature\nhashing algorithm.\n\nLE does not currently accept an Ed25519 elliptic key not for any technical\nreason but because the CAB (?) hasn't yet added it to their list of approved\nkeys/algorithms. Apparently they see little benefit, and much potential\nconfusion, in accepting such as an account key when they are constrained not\nto accept one in a csr.\n\n## Other RFCs\n\nYou think RFC8555 is a lot to take in? There is, as they say, much more:\nthis collection of RFCs that describe details referred to but not expounded\nin good ol' 8555.\n\n[RFC7515](https://tools.ietf.org/html/rfc7515) specifies the Json Web\nSignature and JOSE Header which is used by the ACME protocol.\n\n[RFC7517](https://tools.ietf.org/html/rfc7517) describes the Json Web Key\n\n[RFC7518](https://www.rfc-editor.org/rfc/rfc7518) describes Json Web\nAlgorithms, a few of which are used in the ACME protocol\n\n[RFC7807](https://tools.ietf.org/html/rfc7807) describes the JSON structure\nof an error document such as those used by ACME\n\n[RFC8037](https://tools.ietf.org/html/rfc8037) covers the ED25519 details\n(not yet supported in either LE nor sewer).\n\nOh yes, there's another RFC for the new TLS authorization method which sewer\nhas (so far) no interest in.\n" }, { "path": "docs/Aliasing.md", "content": "# Aliasing for ACME Validation\n\nThe idea is presented (for dns-01 authorizations) in [an article at\nletsencrypt.org](https://letsencrypt.org/2019/10/09/onboarding-your-customers-with-lets-encrypt-and-acme.html)\nwhich shows an example of DNS aliasing and describes what was likely the\noriginal motivation - a hosting provider running the certificate process on\nbehalf of his customers. Like all DNS aliasing, it uses a CNAME at the\ncanonical name `_acme-challenge.domain.tld` to redirect the ACME server to a\ndifferent fqdn that is more convenient for provisioning of the validation\nresponses. I'm not going to try to convince you that you should use\naliasing, because if you need it you probably already know that, or at least\nknow that the process isn't working smoothly as-is.\n\nThe `alias` option in sewer is available to drivers that derive from\n`DNSProviderBase`.\n\n>Added in 0.8.3: `--p_opts alias=...`, but legacy drivers don't take\nadvantage of the aliasing support in their parent classes yet.\n\n## Isn't aliasing just for DNS?\n\nNo. HTTP has had, as a side effect of common web server and client behavior,\na kind of aliasing since the very beginning of ACME. Usually it's\nconvenient enough to provision the validation at the canonical\n`/.well-known/acme-challenge/` location. But if it isn't, either\n`acme-challenge` or `.well-known` can be mapped (by server configuration or\nexternally by symlink, usually) to some other location. If it's desired to\nserve the validation file from some other domain or server altogether, an\nHTTP redirect can often be used (and that's also a third way to place the\nfile elsewhere within the web server's accesible filesystem).\n\nThe RFC says that (for http-01 challenges) the ACME server \"SHOULD follow\nredirects\", which would allow for an analogous aliasing. Lets' Encrypt's\nservers [do follow redirects and\nCNAMES](https://letsencrypt.org/docs/challenge-types/).\n\nSo aliasing can be used with HTTP validations, though it's probably less\noften needed since the privilege needed to directly configure the canonical\nresponse file is likely to be the same (or even less) than that needed to\nsetup the new certificate. And it's possible that you've already used it\nwithout thinking of it as _aliasing_ because it uses such basic HTTP\nbehavior (and so needs no support from sewer).\n\n## Preparing for DNS aliasing\n\nThe first thing which you must have is a way to manage DNS TXT records. In\nfact, you need to be able to control both the real domain's records (in\norder to setup the CNAME entries, but that's something that needs be done\nonly once) as well as managing the alias domain records through the\nservice-specific driver. Generally, expect to need a new-model driver\nrather than an existing legacy driver, on the assumption that it's not much\nmore work to migrate the legacy driver to the new interface while adding the\nalias support. I have my fingers crossed, at any rate...\n\nWith alias-capable driver in hand, you then setup CNAME records for every\nDNS name that you wish to use with the alias domain. In traditional zone\nfile form that might look something like this [excerpt]:\n\n ; existing record for your web or other server\n name.example.com. IN A 111.222.333.444\n\n ; then add the CNAME at the ACME-prefixed name\n _acme-challenge.name.example.com. IN CNAME name.example.com.alias.org\n\nIn online domain editors, the names are usually given without the full\ndomain suffix that's shown here (example.com). The A record (or it could be\na CNAME) for `name.example.com` that directs to your server is shown as an\nexample here.\nThe added CNAME record is the redirect from the conventional ACME challenge\nDNS name, pointing to the TXT record in the alias domain. When it sees that\nCNAME, the ACME server will proceed to look for the challenge's TXT record\nat `name.example.com.alias.org`. Since the alias-aware driver will have\nsetup that TXT record, the server will retrieve it and validate your right\nto issue for `name.example.com`.\n\nNote that the alias domain can be ANY valid domain that you can manage. In\nparticular, it can be in a different tld (as shown here) or a different\ndomain in the same tld, or even in a sub-domain (eg. \n`validation.example.com`) of the target's domain that has been delgated to\nthat more convenient DNS server. And you can setup aliased TXT challenge\nrecords for names from any number of _real_ domains as long as the CNAME\nredirects can be provisioned.\n\n## Using DNS aliases in sewer\n\nThis is really pretty short & sweet.\nAll that's needed, once the setup is done, is to pass `alias=alias.org` to\nthe alias-supporting driver when it's created.\nFrom the command line, that's `--p_opts alias=alias.org`.\n" }, { "path": "docs/CHANGELOG.md", "content": "# `sewer` changelog:\n\n## **pre-release** 0.8.5\n\n- driver for Windows DNS server (local only) [IN PROGRESS]\n\n- cleanup that was deferred from 0.8.4 (affects developers, not cli users)\n\n - crypto.py refactored\n\n - mypy added to tests\n\n - dns_providers have had non-base imports cleaned up: use local `# type:\n ignore` annotations\n\n - a few non-service-specific libs marked globally to be ignored\n\n - REMOVED obsolescent dns_provider_name class variables (use the JSON\n catalog, added in 0.8.3)\n\n - REMOVED obsolescent guards around service-specific imports and the\n corresponding delayed exceptions (the unnecessary imports that used to\n require the guards were removed in 0.8.3)\n\n - crypto.py's tests migrated to pytest format as tests/crypto_test.py\n\n- Fixed the alias support code and unbound_ssh, its only in-tree client, to\n use correct names for alias option parameters\n\n- Aliasing document updated to current client options\n\n- in-tree tests began migrating to pytest format (and moving to ./tests)\n\n## **version:** 0.8.4\n\n- add support for ECDSA keys\n\nCLI changes:\n\n- `--acct_key` & `--cert_key` should be used to designate the file that\n holds the keys to be used (rather than having new ones generated). \n `--account_key` & `--certificate_key` are still accepted as synonyms.\n\n- add `--acct_key_type` & `--cert_key_type` to allow choice of RSA or EC\n keys and key sizes when sewer is generating them for you.\n\n- changed default for generated keys to 3072 bit RSA (had been 2048 bit)\n\n- add `--is_new_key` to allow for first-time registration of your own\n account key (using `--acct_key`) generated outside of sewer.\n\nInternal changes for library clients:\n\n- Client methods cert() and renew() are deprecated; just call\n get_certificate() directly instead.\n\n- Client **no longer generates keys**. (see below)\n\n- crytographic refactoring\n\n - AcmeKey, AcmeAccount & AcmeCsr in crypto.py; uses only cryptography library\n\n- Client interface changes due to crypto refactoring\n\n - dropped `account_key` and `certificate_key` optional arguments to Client\n\n - added `acct_key` and `cert_key` REQUIRED arguments to Client taking\n AcmeAccount and AcmeKey objects, respectively.\n\n - add `is_new_acct` argument to force registration of the supplied account\n key\n\n - dropped `bits` argument because Client no longer generates keys!\n\n - dropped `digest` argument since there are currently no alternate digest\n methods for the different key types. (was this ever used?)\n\n## **version:** 0.8.3\n\nFeatures and Improvements:\n- added `--acme-timeout ` option to adjust timeout on queries to\n the ACME server\n- `--action {run,renew}` has been doing nothing useful and is now deprecated.\n- added `--p_opt =` for passing kwargs to drivers\n- Added optional parameters accepted by base class for DNS drivers:\n - `alias=` specifies a separate domain for DNS challenges\n (requires driver support, see [Aliasing](Aliasing))\n - `prop_delay=` gives a fixed delay (sleep) after challenge setup\n- gandi (legacy DNS driver) fixed internal bugs that broke common wildcard\n use cases (eg., `*.domain.tld`) as well as the \"wildcard plus\" pattern\n- added unbound_ssh legacy-style DNS provider as a working demo of adding\n new features to legacy drivers. It does work in the right environment, and\n could be useful to someone other than myself (mm).\n\nInternals:\n- added [catalog.py](catalog) to manage provider catalogs; includes\n get_provider(name) method to replace `import ......{name.}ClassName`\n- replace __version__.py with meta.json; setup.py converted; add sewer_meta()\n in lib.py; cli.py converted; client.py converted\n- added catalog.json defining known drivers and their interfaces; also\n information about dependencies for setup.py\n- added `**kwargs` to all legacy providers to allow new options that are\n handled in a parent class to pass through (for `alias`, `prop_delay`, etc.)\n- removed imports that were in `sewer/__init__` and\n `sewer/dns_providers/__init__`; fixed all uses in cli.py and tests.\n- began cleanup/refactor of cli.py (there will be more to come and/or a new,\n more config driven, alternative command (0.9?))\n- added `__main__.py` to support `python -m sewer` invocation of `sewer-cli`\n- fixed imports in client.py that didn't actually import the parts of\n OpenSSL and cryptography that we use (worked because we import requests?)\n\nSee also [release notes](notes/0.8.3-notes).\n\n## **version:** 0.8.2\nFeature additions:\n\n- support current RFC8555 protocol (LE staging current, production requires in Nov)\n- added DNS providers powerdns and gandi\n\nInternals (features and/or annoying changes for sewer-as-a-library users)\n\n- unified dns-01 and http-01 providers; support challenge propagation check\n- added support for non-dns (http-01 challenge) provider\n- collect shared (internal) functions into lib.py\n- use unitest.mock rather than external module\n- client no longer prepends`*.` to wildcards; remove spotty code in providers to strip it\n- begin addition of annotations, mostly opportunistically\n\nSee also [release notes](notes/0.8.2-notes).\n\n## **version:** 0.8.1\n- Fix bug where `sewer` was unable to delete wildcard names from clouflare: https://github.com/komuw/sewer/pull/139 \n- Fix a StopIteration bug: https://github.com/komuw/sewer/pull/148 \n- Add guide on how to create a new pypi release\n\n## **version:** 0.8.0\n- Fix bug where `sewer` would log twice: https://github.com/komuw/sewer/pull/137 \n Thanks to [@mmaney](https://github.com/mmaney) for this\n\n## **version:** 0.7.9\n- Fix bug where Aliyun response is in bytes: https://github.com/komuw/sewer/pull/133 \n Thanks to [@ButterflyTech](https://github.com/ButterflyTech) for this \n\n## **version:** 0.7.8\n- Add support for Cloudflare token auth: https://github.com/komuw/sewer/pull/130 \n Thanks to [@moritz89](https://github.com/moritz89) for this \n\n## **version:** 0.7.7\n- Add support for Support AWS Route53: https://github.com/komuw/sewer/pull/126 \n Thanks to [@soloradish](https://github.com/soloradish) for this\n\n## **version:** 0.7.6\n- Fix logging, sewer was redefining root logger: https://github.com/komuw/sewer/pull/125 \n Thanks to [@etienne-napoleone](https://github.com/etienne-napoleone) for this\n\n## **version:** 0.7.5\n- Fix pypi upload script\n\n## **version:** 0.7.4\n- Adds support for [ClouDNS](https://www.cloudns.net/): https://github.com/komuw/sewer/pull/122 \n Thanks to [@hbradleyiii](https://github.com/hbradleyiii) for this \n" }, { "path": "docs/DNS-Propagation.md", "content": "# Waiting for Mr. DNS or Someone Like Him\n\nQ: How long does it take after you've setup the challenge response TXT records\nuntil they're actually accessible to the ACME server?\n\nA: Good Question!\n\nAccording to [Let's Encrypt](https://letsencrypt.org/docs/challenge-types/#dns-01-challenge)\nit can take up to an hour. Depends on the DNS service. Some provide a way\nto check that your changes are fully propagated to all their servers. With\nmany, however, you just have to wait. But be sure you wait long enough,\nbecause Let's Encrypt DOES NOT implement automatic or triggered retry of a\nfailed authorization - you have to restart the [same] order or else start\nall over again.\n\nSewer provides a flexible _delay until actually published_ mechanism through\nthree optional driver parameters, `prop_delay`, `prop_timeout`,\n`prop_sleep_times`, and the [`unpropagated` method](unpropagated).\nLet's see how they're used in various circumstances.\n\n## No API support, no reliable way to check: just delay\n\nIf you can't check that the TXT records are fully published, then all you\ncan do is delay for a while. Perhaps the DNS service will suggest a safe\ntime. If not, you'll have to start with a guess and adjust from there based\non your experience. Choosing the right number - long enough but not\nexcessively long - can be hard, but applying it is easy. Just add\n`prop_delay=SIMPLE_DELAY_TIME` to the driver's initialization parameters,\nand sewer's engine will add that many seconds of delay after the challenge\nsetup returns before it signals the ACME server to validate those\nchallenges.\n\n**CLI option --p_opt prop_delay=... is available for all drivers since 0.8.3**\n\n## API support or can check: use a timeout\n\nIf the DNS service gives you a way to check that the propagation is\ncomplete, or if there are not too many authoritative servers (viz., not an\nanycast system), you can use that actual check (implemented in the driver's\n`unpropagated` method) and the engine will run that check until it succeeds\nor until a timeout you specify is exceeded. However the check is being\ndone, you setup the timeout by adding `prop_timeout=MAX_WAIT_TIME` to the\ndriver parameters. If you know that it takes at least some minimum time to\npropagate, you may also pass `prop_delay` to make the engine delay that long\nbefore it starts checking. And there's a delay between checks that has a\nhopefully sensible default, but which you can adjust if necessary through\nthe `prop_sleep_times` parameter.\n\n**no drivers implement `unpropagated` as of 0.8.3**\n\n## You probably don't need to change `prop_sleep_times`\n\nUnless you do, but if it's not obvious, just leave it.\n\nThis parameter defines the lengths of sleeps the engine will add following a\ncall to `unpropagated` that reports not ready. As an optional parameter\npassed to the driver, `prop_sleep_times` can be an integer number of seconds\nor a list or tuple of such delays which will be used in order. The final\nvalue in the sequence will be reused indefinitely.\n\nExample: the default value is (1, 2, 4, 8) which provides an exponential\nbackoff up to an 8 second delay, then sticks there. _[the values could\nchange - it's just what seemed reasonable to me]_ So if there's no delay,\nand the check call takes no measurable time (and reports not ready each\ntime), it will look something like this with `prop_timeout=20`:\n\n| time | action |\n| ---: | --- |\n| 0 | call unpropagated, sleep(1) |\n| 1 | call unproagagted, sleep(2) |\n| 2 | call unpropagated, sleep(4) |\n| 6 | call unpropagated, sleep(8) |\n| 14 | call unpropagated, sleep(8) |\n| 22 | call unpropagated, timeout! |\n\nThis shows both the last value repeating and the way the timeout and sleeps\ninteract. The check for timeout is done only AFTER a call to unpropagated\nAND the chance to exit with success if it finally reports the challenges are\nready. So the timeout isn't a hard maximum time, but it's bounded to be no\nmore than one sleep interval (plus actual time to run `unpropagated`, of\ncourse) over `prop_timeout`.\n\n## Other Notes and Advanced Use\n\nThese values are setup through the Provider on the reasonable assumption\nthat they will vary most directly with the choice of service provider, so\nthe individual drivers are best suited to provide sensible defaults where\nappropriate (and possible!). Sewer's engine implements the delay and check\nloop (with timeout) because the mechanism is the same for all providers (and\nmay be useful for other than the DNS-based challenges for which it has been\nimplemented).\n\nIf you are using sewer as a library and find that you can make a better\nestimate of the propagation after the driver is setup (perhaps using a\nservice-specific method to access part of the service's API or run some\ntests), you could adjust those parameters through the same-named attributes\non the Provider instance. This is solidly in the categories of don't do it\nunless you're sure you need to, and be prepared to own both the pieces if\nyou break it!\n\n## Could this be used with non-DNS drivers?\n\nYes! I have no experience with http-01 in any setting where such a delay\nmight be needed, but the mechanism is implemented in sewer's engine, and all\nthat needs be done is to setup the parameters (and implement unpropagated in\nthe driver if using more than just `prop_delay`) as described above and\nthere you go!\n" }, { "path": "docs/LegacyDNS.md", "content": "## Legacy DNS challenge providers\n\n### `BaseDns` shim class\n\nA child of `DNSProviderBase` that acts as an adapter between the new\nProvider interface and the legacy DNS provider interface.\n\n#### `__init__(self, **kwargs: Any) -> None`\n\nAccepts no arguments itself; doesn't expect any to be passed by Legacy code.\nInjects `chal_types=[\"dns-01\"]`.\n\n#### `setup(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`\n\nIterates over the challenges, extracting the values needed for the legacy\nDNS interface from each challenge in the list, and passing them to\n`create_dns_record`. Always returns an empty list since there is no error\nreturn from `create_dns_record` other than raising an exception.\n\n#### `unpropagated(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`\n\nAlways returns an empty list, signalling \"all ready as far as I know\".\nA legacy DNS driver wishing to do something useful here MAY implement\n`unpropagated` without updating the rest of its interface.\n\n#### `clear(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Dict[str, str]]`\n\nSame as setup except it calls the legacy `delete_dns_record`, of course.\n\n### Legacy DNS class\n\n#### `__init__(self, ..., **kwargs)`\n\nArgs handled by the driver should be explicitly named, with defaults where\nthat makes sense. Starting in 0.8.3, the `**kwargs` bucket has been added\nto provide pass-through to the base class.\n\n#### `def create_dns_record(self, domain_name, domain_dns_value)`\n\nMinimum is to add `_acme-challenge` prefix to domain_name and post the\nchallenge response (domain_dns_value) as that name's TXT value.\nAll very provider-dependent.\n\n#### `def delete_dns_record(self, domain_name, domain_dns_value)`\n\nIn theory it should undo the effects of setup.\nIn practice, at least one of the services is unable to do that\n(according to the author's comment).\n\n### Legacy DNS vs Aliasing\n\nLegacy DNS drivers MAY change to use the [aliasing](ALiasing) methods\ninherited from `DNSProviderBase`, though this will require a potentially\nfragile faking of the new-model challenge dict in the driver. See the\n`unbound_ssh` example driver, and bear in mind that a change to the data\ntype of the challenge items IS anticipated, perhaps in 0.9.\n" }, { "path": "docs/UnifiedProvider.md", "content": "# DNS and HTTP challenges unified\n\n_There's still a draft when the wind is blowing, but it's getting less._\n\n## Dedication\n\nIt is indisputable that this is, in the first instance, Alec Troemel's fault,\nsince he added support for http-01 challenges.\nAlso indisputable is that the many changes to both code and overall design\nmade in the process of unifying the two types of challenges,\nwhile influenced by Alec's code and our discussions, are entirely my fault.\nAlec cannot be blamed for my choices!\n\n## A few words about words\n\nBecause the word \"provider\" is so overloaded, I'm going to refer to the\nservice-specific implementations as \"drivers\", except when I forget, or\nmissed changing an old use. \"Provider\" is still used in the class names. And\nthen there are the \"service providers\", viz., DNS services or web hosts,\netc.\n\n## Overview (tl;dr)\n\n`ProviderBase` described here defines the interface the ACME engine uses\nwith new-model drivers (all http-01 drivers, as there are no old ones). New\ndrivers normally should inherit from the `DNSProviderBase` or\n`HTTPProviderBase` classes in auth.py.\n\n`DNSProviderBase` has support for [aliasing](Aliasing), though the\nindividual drivers need to be created (or modified) to support it at this\ntime. _unbound_ssh is a quirky but working example that supports aliasing.`\n\n## ProviderBase interface for ACME engine\n\nThe interface between the ACME protocol code and any driver implementation\nconsists of three methods, `setup`, `unpropagated` and `clear`. The first\nand last are not unlike methods used by the legacy drivers, but they accept\na list of one or more challenges rather than one challenge at a time.\n\n--- most of this is in [unpropagated], or should be. ToDo: reconcile\n\nThe `unpropagated` method was added with DNS propagation delays in mind. It\nshould be possible for legacy drivers to implement this without a full\nconversion to the new-model as a temporary adaptation for those who need\nthis feature. Just override the null version provided by `BaseDns`. \n\n`unpropagated` checks all the challenges in the list it is passed, and\nreturns a list containing the ones which are *not* yet ready to be\nvalidated. This should be more reliable than adding an ad-hoc delay before\n_responding_ to the ACME server as well as avoiding wasting time.\n\nThe errata list returned by by all three methods has tuples for elements,\nwhere each tuple holds three values: the status string, the msg string, and\nthe original, unmodified challenge item (dictionary). This is defined as\ntypes `ErrataItemType` and `ErrataListType` in auth.py\n\n> LE's ACME server, for one, implements neither automatic nor triggered\nretries, so it's important not to _respond_ to a challenge before the\nvalidation response is actually accessible. And yes, the RFC's language\ndoes encourage confusing the respond-to-challenge API request with the\nchallenge response (TXT) that the server has to find when it probes for it.\n\n> My thinking on this is that, while the ACME engine's code can know what\nnames to check, in the really interesting case of widely distributed\n(anycast?) DNS service, figuring out which DNS server to query must be left\nto the service-specific driver. In some cases the service may provide an\nAPI for checking some internal status that might be faster and/or more\nreliable than polling DNS servers. For cases where all that can (or needs)\nto be done is some DNS lookups, well, that can be packaged as a function.\n\n--- end reconcile block\n\nThis is the pattern which all three methods use: accept a list of challenges\n(each a dictionary) to process, and return an errata list containing the\nsubset which have problems or are not ready. So in all cases an empty list\nreturned means that all went well.\n\n## `ProviderBase`\n\nAbstract base class for driver implementations ultimately inherit from.\n\n### ProviderBase __init__\n\n __init__(self,\n *,\n chal_types: Sequence[str],\n logger: Optional[LoggerType] = None,\n LOG_LEVEL: Optional[str] = \"INFO\",\n prop_delay: int = 0\n prop_timeout: int = 0,\n prop_sleep_times: Union[Sequence[int], int] = (1, 2, 4, 8)\n ) -> None:\n\n\nThe drivers' `__init__` methods accept only keyword arguments. We can see\nthat ProviderBase has become the final recipient of quite a few, mostly\noptional, arguments. They ended up here because they are not specific to a\nsubclass; a counterexample is the `alias` parameter, which is handled in\n`DNSProviderBase`. The conventions for ProviderBase and its subclasses are:\n- keyword only arguments (other than self, of course)\n- Required arguments never have a default value\n- Optional arguments must have a default, of course\n- Everything not explicitly handled is left to kwargs\n\nConveniently, ProviderBase's __init__ demonstrates all of these aside from\nthe use of kwargs (because it is the final base class, so any unrecognized\narguments would be in error):\n- chal_types is required, so it has no default value and will be diagnosed by\n Python's calling mechanism if omitted.\n- logger/LOG_LEVEL are... weird. Without the legacy DNS providers I would\n be inclined to just require logger, pushing the job of setting up logging\n firmly back up the stack. As it is, logger cannot be required (yet), so\n both have defaults that work with the __init__ logic to setup logging as\n sanely as possible. Eventually LOG_LEVEL should get deprecated and then\n dropped, and logger is just required...\n- the prop_* arguments are all optional, and receive default values that the\n engine code is designed to deal with - by disabling the optional behavior\n they control. These are all parameters that were introduced for a\n lower-level driver or driverBase class, but which have migrated up to\n ProviderBase because they may apply to any sort of Provider.\n\nIn all subclasses, kwargs is expected to catch parameters that may need to\npass up the Provider classes, and so it must be passed to super()__init__. \nIt is allowed to add, change, or even remove items from kwargs if necessary\n\n--- see the intermediate *ProviderBase classes.\n\n--- (see where for args documentation? DNS-Alias and DNS-Propagation & ???)\n\n### `setup(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Tuple[str, str, Dict[str, str]]]`\n\nThe `setup` method is called to publish one or more challenges. Each item\nin the list describes one challenge.\n\n(_the description of the challenges list is common to all three methods_)\n\nThe items are dictionaries with keys and values that come from the ACME\nauthz query, or are derived from it and the account key [see note].\nFor dns-01 and http-01 challenges, the required keys are:\n\n* ident_value - the value of the identifier to be validated (1)\n* token - the validation nonce\n* key_auth - validation value (hash of nonce + secret key's thumbprint)\n\n> The current per-challenge dict holds a subset of the authz values, and\nsome of the names (and structure) are different. *This is likely to change\nin the future!* (0.9?)\n\nThe plan is to include other values from the ACME _authorization object_\nresponse, as well as non-authz values, so the driver implementation MUST\naccept additional keys in the dictionary. Likewise, the list SHOULD include\nonly outstanding challenges, and the call(s) to the driver SHOULD be omitted\nif there are none. But the latter, especially, is just the plan, so\nthrowing an exception if the challenges list is empty is JUST NOT ON.\n\n> Allowing an empty challenges list is also convenient for unit tests.\n\nEach of the three methods return an errata list of the challenge items which\nencountered an issue - couldn't create, isn't published, removal failed. So\nin all cases, an empty list means all is well.\n\nThe *errata list* is a list-like containing a tuple for each failed or\nunready challenge. The tuples have three elements: a status (str), a msg\n(str) intended to enlighten a human observer, and the original challenge\nitem (the dictionary from the argument list). The status MUST be one of the\ndefined values:\n\n| status | applies to | meaning |\n| --- | --- | --- |\n| \"failed\" | all | challenge for which a failure occurred |\n| \"skipped\" | setup | may skip challenges after one has a hard failure |\n| \"unready\" | unpropagated | soft fail: record not deployed to authoritative server(s). If a non-recoverable error is detected, then use _failed_. |\n\n### `unpropagated(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Tuple[str, str, Dict[str, str]]]`\n\nThis method is expected to be needed mostly for DNS challenges, but it\nshould be used whenever a service provider has a relatively slow or\nunpredictable lag between the challenge being posted by `setup` and that\nchallenge data being visible to the world. When there's no expectation of\nsuch lag, or no way to reliably check that the challenge has propagated,\nthis may as well just return an empty list, and we'll all hope for the best.\n\n### `clear(self, challenges: Sequence[Dict[str, str]]) -> Sequence[Tuple[str, str, Dict[str, str]]]`\n\n`clear`, unlike `setup`, SHOULD NOT stop processing challenges after hitting\nan error. It's possible that any reported errors will be treated as\npotential soft errors and the operation retried (with only the unready\nchallenges).\n\n_? should have a status word for \"this one's hard failed, forget about it\"?_\n\n## `DNSProviderBase`\n\nThe driver *interface* is the same for everything except legacy DNS drivers,\nbut there are some differences which it makes no sense to push into\n`ProviderBase`. `DNSProviderBase` provides a nice example of this:\n\n`__init__(self, *, alias: str = \"\", **kwargs: Any) -> None`\n\ndef cname_domain(self, chal: Dict[str, str]) -> Union[str, None]\n\ndef target_domain(self, chal: Dict[str, str]) -> str\n\nThe class's `__init__` handles the `alias` argument, and provides chal_types\nsuitable for a DNS driver if they weren't already present. Its value, if\nany, is stored locally for use by the helper methods. `target_domain` is to\nbe used in the driver to get the actual DNS name for the challenge TXT,\nhandling both the aliasing and non-aliasing case. `cname_domain` forms the\nDNS name for the CNAME that should exist in the aliasing case and returns it\nfor the use of a hypothetical sanity check, or None when not aliasing.\n\n## `HTTPProviderBase`\n\nThis intermediate base class stands ready to handle any HTTP-specific\noptions or helper methods. No additions are expected until sewer has had\nsome actual drivers added. It also provides chal_types if needed.\n" }, { "path": "docs/catalog.md", "content": "# The Catalog of Drivers\n\nThe driver catalog, `sewer/catalog.json`, replaces scattered facilities that\nwere used to stitch things together. The import farms in `sewer.__init__`\nand `sewer.dns_providers.__init__` have already been removed; with the\ncatalog in place, redundant lists in cli.py and setup.py are also removed,\nreplaced by use of the catalog's data and a few lines of code. The\n`dns_provider_name` is deprecated in favor of the catalog as well.\n\n`catalog.py` wraps the catalog in a class, adding some convenience methods\nfor listing the known drivers, looking up a driver's data by name, and\nloading a driver's implementation class by name. But using the catalog\nwithout `catalog.py` is as easy as loading it using the standard lib's json\nfacilities - it's all lists & dicts (see eg. setup.py which loads the\ncatalog this way to avoid potential issues with trying to call into the\npackage's code before it's installed).\n\n## Catalog structure\n\nThe catalog resides in a JSON file that loads as an array of dictionaries,\none element for each registered driver. The per-driver record contains the\nfollowing items (some optional):\n\n- **name** The name used to identify this driver, eg., `--provider `\n to `sewer-cli`. These names need to be unique, but are not required to\n match the module or implementing class names. (legacy DNS drivers usually\n matched the module name, but not always)\n- **desc** A brief description of the driver, intended for display to humans\n to help them understand what each driver is, eg. in --known_providers output\n- **chals** list of strings for the type of challenge(s) this driver\n handles. (if more than one type, in order of preference?)\n- **args** A list of the driver-specific [parameters](#args-parameter-descriptors)\n- **path** The path to use to import the driver's Python module.\n _Default_ is `sewer.providers.{name}`\n- **cls** Name of module attribute which is called with parameters to get a\n working instance of the driver. Usually a class, but a factory function\n may be used. _Default_ is `Provider`.\n- **features** - a list of strings that name the optional features that this\n driver supports. _Default_ is an empty list.\n- **memo** Additional text/comments about the driver, the descriptor, etc.\n- **deps** list of additional projects this driver requires (for setup)\n\n## args - parameter desciptors\n\nThis is a bit of a mess due to legacy drivers that ignored the established\nconventions. To be fair to them, those conventions weren't clearly\ndocumented (then - see below). This adds some complications to preserve\ncompatibility, as usual. Let's begin with a minimal descriptor for a driver\nthat conforms to the new convention (hint: it's imaginary at this time):\n\n {\n \"name\": \"well_behaved\",\n \"desc\": \"made-up example driver that's mostly defaults\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n { \"name\": \"api_id\", \"req\": 1 },\n { \"name\": \"api_key\", \"req\": 1},\n ],\n \"features\": [ \"alias\" ],\n }\n\nThis describes a dns-01 challenge driver that is found in the module\n`sewer.providers.well_behaved`, constructed from a class named `Provider`.\nThe constructor takes two required arguments, `api_id` and `api_key`, which\nthe program should accept from environment variables `WELL_BEHAVED_API_ID\"\nand \"WELL_BEHAVED_API_KEY\". Since it is a `dns-01` challenge provider and\nup to date, it adds the claim that it supports the `alias` feature. It\ndoesn't support the \"unpropagated\" feature - perhaps the DNS service has no\nAPI to check the propagation of changes.\n\nIf this had been a difficult old legacy driver, the descriptor might have\nlooked more like this:\n\n {\n \"name\": \"difficult\",\n \"desc\": \"made-up example driver that's as non-default as can be\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n { \"name\": \"api_id\",\n \"req\": 1,\n \"param\": \"difficult_api_id\",\n \"envvar\": \"DIFFICULT_DNS_API_ID\",\n },\n { \"name\": \"api_key\",\n \"req\": 1,\n \"param\": \"DIFFICULT_API_KEY\",\n \"envvar\": \"DIFFICULT_DNS_API_KEY\"},\n },\n\t{ \"name\": \"api_base_url\",\n\t \"param\": \"API_BASE_URL\",\n\t \"envvar\": \"\",\n\t},\n ],\n \"path\": \"sewer.dns_providers.difficultdns\",\n \"cls\": \"DifficultDNSDns\",\n \"features\": [],\n \"memo\", \"difficult, indeed...\"\n }\n\nThis driver has both parameter names and envvar names that defy convention,\nso both the parameter and envvar name must be given explicitly. There is\nalso an optional parameter that has never had an associated envvar that the\nimplementation used.\n\n## driver parameter and environment variable names\n\nThe convention is that the envvar name (if any) SHOULD be formed from the\ndriver name and the individual args' names (see the first envvar rule\nbelow). This gives envvar names similar to, sometimes identical to, the\nones already used with legacy DNS drivers. One thing that is changing is\nthat the parameter names, which in the old convention were\nTHE_SAME_AS_ENVVAR_NAMES, are changing to be lower case and losing\ndriver-name prefixes, etc. Where appropriate, the new names will use just a\nfew shared names, viz., `id`, `key`, `token`.\n\nObviously the drivers and envvar names are not so consistent among the\nlegacy DNS drivers. Therefore the descriptor has both `param` and `envvar`\nvalues, along with a set of rules for resolving the names to be used.\n\n### parameter name rules\n\n1. `descriptor.args[n].name` is the \"modern\" name for the nth parameter\n2. if `param` is given, it overrides the \"modern\" name\n\n### environment name rules\n\n1. f\"{descriptor.name}_{descriptor.args[n].name}\".upper() is the default\n2. if `envvar` is given, it overrides the default\n\nTwo guidelines for the use of envvars:\n\n1. If `envvar` is given, is not the empty string, and the so-named envvar is\n not found, the invoking code MAY also look for the default-named envvar\n before reporting a missing envvar.\n\n2. If `envvar` is set to the empty string, then catalog using code will not\n look for a matching envvar at all.\n\n## catalog representation in Python\n\nFor now, see the brief implementation in sewer/catalog.py for the way the\nJSON structure is mapped into a ProviderDescriptor instance.\n" }, { "path": "docs/crypto.md", "content": "# A crypto module for ACME\n\nThere were several motivations behind the creation of `crypto.py`:\n\n- a desire to convert the OpenSSL (Python code) usage to the preferred\n cryptography package. Note that this is only a change in which Python\n wrapper around the openssl binary is used!\n\n- breaking the Client global mess up into more cohesive components\n\n- Oh, and @jfb's PR adding ECDSA certificate keys was the spark that\n triggered the whole thing into [code] motion.\n\n## Preliminaries\n\nI use the term `private key` quite a bit, as it is the term widely used for\nsome representation of both the public and private parts of an asymmetric\nkey.\n\n## AcmeKey\n\n`AcmeKey` is a parameterized holder for the primitive key classes of the\nunderlying implementation (the cryptography package).\n\nSo now Client and the cli code can stop schlepping around a string that\nholds the key in PEM format. That's part of the reason this work\nintentionally broke the names in Client (eg., acct_key in place of account\nkey) when it switched to an AcmeKey object.\n\n#### Factory (class)methods\n\nTwo essential factories:\n\n- `create(key_type: str) -> AcmeKeyType` generates a new private key\n of the named kind.\n\n- `from_pem(pem_data: bytes) -> AcmeKeyType` returns an AcmeKey object\n containing the key serialized in the PEM bytes, assuming it is one of the\n types of keys that are known (viz., implemented in a subclass of AcmeKey).\n\nAnd an inessential convenience:\n\n- `read_pem(filename: str) -> AcmeKeyType` loads the key from a PEM format\n file.\n\n### AcmeKey attributes\n\n- pk, the private key in the form of cryptography's key class\n\nSewer's code never needs to touch the pk directly.\n\n### AcmeKey methods\n\n- `to_pem(self) -> bytes` Returns all the key's info in PEM format\n\n- `write_pem(self, filename:str) -> None` Writes private key to file as PEM\n\nprivate_bytes is the only method that all ACME clients must use, and that\nwill often be done indirectly through to_file.\n\n- `sign_message(self, message: bytes) -> bytes` Calculate the signature for\n the given message. (uses SHA256 only because that's what ACME specifies)\n\n> You will have noticed the symmetry of the method names: to/from_pem for\n in-memory byte strings, read/write_pem for keys in files.\n\n## AcmeAccount\n\nAccounts are keys which are registered with the ACME service and thereafter\nused to identify and authenticate the origina of most of the messages sent\nto the service. They are a subclass of AcmeKey that extends the interface\nto include things only an account key has to do.\n\n### AcmeAccount attributes\n\n- _kid: str — the _key identifier_ that comes from registering pk with ACME.\n\n- _timestamp: float — when the account was [most recently] registered\n\n- _jwk: Dict[str, str] — cached JWK or None\n\n_kid is needed only for constructing the signed ACME requests, and is used\nmostly as an in-memory value. There is experimental support for saving the\nKey ID and timestamp along with the account key.\n\n### AcmeAccount methods\n\n- `jwk(self) -> Dict[str, str]` Returns the JSON Web Key as a dictionary\n (with binary values base64 encoded). (value is cached)\n\n- `set_kid(self, kid: str, timestamp: float = None) -> None` Hook for ACME\n register_account or its caller to use to attach the registered key's kid\n to the AcmeKey object. If timestamp is not given, the current time will\n be used.\n\nset_kid() is pure implementation detail, a stash for the account's registered\nURL on a specific ACME server. timestamp is what time.time() returns.\n\n> not yet implemented: write_extended_pem, read_extended_pem (or will read\n just be an override that loads the extended values if present?)\n\n\n## AcmeCsr\n\nThis is currently a minimal replacement for the OpenSSL-based create_csr\nmethod. Which might be all an ACME client requires, so perhaps the current\ndesign will be more or less how it comes out? There will be an additional\nflag for setting the \"must be stapled\" certificate extension, but there's\nreally not much else to add, based on a review of the de-facto standard of\ncert-bot's options.\n\nOne thing to note: yes, the choice of DER format is intentional and\nnecessary, as the ACME protocol requires that format, base64-url encoded,\n_without_ the starting and ending text lines that PEM adds.\n" }, { "path": "docs/dns-01.md", "content": "# DNS service drivers\n\nACME's dns-01 authorization was sewer's original target. There are a number\nof DNS services supported in-tree, and implementations for other services\nare difficult to write only if the service's API is difficult.\n\n## DNS services supported\n\nCurrently, these are all _legacy_ drivers, built on the original DNS-only\ninterface. That's okay, there's no plan to drop them (just a hope that\ninterested users will step up to get them updated), but that does mean that\nsupport for some features varies.\n\n| service | driver name | wc+ | alias | prop | notes |\n| --- | --- | :-: | :-: | :-: | --- |\n| [acme-dns](https://github.com/joohoi/acme-dns) | acmedns | ? | no | no | |\n| [Aliyun](https://help.aliyun.com/document_detail/29739.html) | aliyun | ? | no | no | |\n| [Aurora](https://www.pcextreme.com/aurora/dns) | aurora | ? | no | no | |\n| [Cloudflare](https://www.cloudflare.com/dns) | cloudflare | ? | no | no | patch in #123, needs confirmation? |\n| [ClouDNS](https://www.cloudns.net) | cloudns | ? | no | no | test coverage 75% |\n| [DNSPod](https://www.dnspod.cn/) | dnspod | ? | no | no | |\n| [DuckDNS](https://www.duckdns.org/) | duckdns | ? | no | no | |\n| [Gandi](https://doc.livedns.gandi.net/) | gandi | OK | no | no | wildcard & other fixes in 0.8.3 |\n| [Hurricane Electric](https://dns.he.net/) | hurricane | ? | no | no | test coverage 70% |\n| [PowerDNS](https://doc.powerdns.com/authoritative/http-api/index.html) | powerdns | NO | no | no | apparently not in 0.8.2; bug #195 |\n| [Rackspace](https://www.rackspace.com/cloud/dns) | rackspace | ? | no | no | test coverage 69% | \n| [Route 53 (AWS)](https://aws.amazon.com/route53/) | route53 (1) | OK | no | no | wc+ in 0.8.2; not in CLI |\n| Unbound | unbound_ssh | OK | yes | no | Working demonstrator model for local unbound server |\n\n- _wc+_ (wilcard plus) is specifically about a single certificate that has\n at least two registered names: `domain.tld` and `*.domain.tld`. This\n specific combination has issues with some service providers/s.p.'s\n API/drivers. So far it's been possible to make it work by changing the\n drivers, but it has to be done one by one.\n\n- _alias_ publishing challenges in a different [sub]domain than the\n identities being authorized. See [Aliasing](Aliasing).\n\n- _prop_ support for the [unpropagated](unpropagated) interface. Can be\n added to any driver but may only be worthwhile with service API support?\n\n## Add a driver for your DNS service\n\nMost (?) of the DNS drivers came about because someone wanted to use sewer\nwith their DNS service provider, but there wasn't a driver to use with the\nSP yet. This involvement of sewer and DNS-service users is a practical\nnecessity, as there is no substitute for being able to test the driver\nagainst the DNS-service, and many such services are for-pay or bundled with\nother for-pay services.\n\nsewer's [legacy DNS driver interface](LegacyDNS) (BaseDns in dns_providers/common.py)\nis deprecated, although there is no plan for its removal other than\n_after they have all migrated to the new interface_.\nNew DNS drivers should use DNSProviderBase from the start, of course,\nand will be placed in sewer/providers/ if added to the project.\n\n # sketch of simple dns-01 provider, including alias support\n\n from .. import auth\n from .. import lib\n\n class Provider(auth.DNSProviderBase):\n def __init__(self, *, my_api_url, my_api_id, my_api_key, **kwargs):\n super().__init__(self, **kwargs)\n self.api_url = my_api_url\n self.api_id = my_api_id\n self.api_key = my_api_key\n\n def setup(self, challenges):\n for challenge in challenges:\n fqdn = self.target_domain(challenge)\n txt_value = lib.dns_challenge(challenge[\"key_auth\"])\n self.my_api_add_txt(fqdn, txt_value)\n\n def unpropagated(self, challenges):\n return [] # if service has a propagation check, use it here\n\n def clear(self, challenges):\n # like setup, but calling my_api_del_txt; may not need txt_value\n\n def my_api_add_txt(self, fqdn, txt_value):\n # this is where you talk to the DNS service to add a TXT\n\n def my_api_del_txt(self, fqdn):\n # talk to DNS service to remove TXT\n\nMost of your work is in implementing the two methods (or one method, or\ninline code, but inline makes testing without access to the service more\ndifficult) which actually communicate with the DNS service. This can be\neasy or very difficult, depending on the service provider's API (or lack of\ndesigned API if you have to use a mix of web scraping and HTTP request\ngeneration to operate a mechanism that was designed for interactive use).\n\nThe above is bare-bones, not taking advantage of the batching of challenges\nwhich the new-model interface provides - that can be a big win for large-SAN\ncertificates if you have to grovel the service's API (or web pages) to guide\nthe construction of your commands to them. It does show the use of\ntarget_domain to support [DNS aliasing](Aliasing).\n" }, { "path": "docs/drivers/route53.md", "content": "## route53 - driver for AWS DNS service\n\n### Command line use\n\nroute53 has never been wired into `sewer-cli`, and that hasn't really\nchanged in 0.8.3. It does appear in the list of \"known providers\", but it\nisn't usable, and raises an exception if named by `--provider`.\n\nAdding that integration is on the list, but seeing as no one has complained\nabout this lack up to now it's nowhere near the top. :-(\n\n### Programmatic use\n\nApparently everyone using sewer's route53 has been rolling their own\nwrapper, since it has only been available for such use to date. There is a\npatch to extend that Route53Dns.__init__ to allow additional AWS-specific\nmethods of authentication which I expect will ship in 0.8.3.\n" }, { "path": "docs/drivers/unbound_ssh.md", "content": "## unbound_ssh legacy DNS driver\n\nA working, if somewhat quirky, driver to setup challenges in local data of\nthe [unbound](https://nlnetlabs.nl/projects/unbound/about/) caching\nresolver. As the name suggests, it relies upon ssh to provide an\nauthenticated connection the server; inside that connection the\n`unbound-control` program is used to add and remove the records. The driver\ndoes NOT handle the login authorization, assuming that it is running\ninteractively and ssh will prompt for your input, or that a key agent (eg.,\nssh-agent) is active to supply the cryptographic credentials. That's the\n_somewhat quirky_ part!\n\n### `__init__(self, *, ssh_des, **kwargs)`\n\nThere is one REQUIRED parameter, `ssh_des`, which is the login target, such\nas acme_user@ns1.example.com. This is simply passed to the ssh command,\nalong with the `unbound-control` commands to be executed on the destination\nmachine.\n\n### Driver features\n\nunbound_ssh supports the `alias` parameter.\n\nOnly `prop_delay` is supported; there is no custom `unpropagated` method.\n\n### Usage\n\nFrom the command line:\n\n python3 -m sewer ... --provider=unbound_ssh --p_opt ssh_des=acme@ns.example.com ...\n\nFrom custom code:\n\n from sewer.dns_providers.unbound_ssh import UnboundSSH\n\n provider = UnboundSSH(ssh_des=\"acme@ns.example.com\", alias=\"validation.example.com\")\n ...\n\n### Bugs\n\nSadly, This was written using the old paradigm where both the module name\nand the class name were more-or-less the same name aside from\ncapitalization... and often less predictable changes. Should have been\nunbound_ssh.Provider ...\n\nThe `unbound-control` commands generated could be run locally with not very\nmuch change to the driver. Perhaps that will become part of a demonstration\nof some different features in the future.\n" }, { "path": "docs/http-01.md", "content": "# HTTP challenge providers\n\nThere are no http-01 drivers in sewer yet.\n\n## Bring your own HTTP provider\n\n**To be rewritten.** For now, see `providers/demo.py` for some hints, and\n[UnifiedProviders](UnifiedProviders) for doumentation of the interface.\n" }, { "path": "docs/index.md", "content": "## sewer, the ACME library and command-line client\n\nThis is a quick & dirty directory of the docs directory, which is still a\nwork in progress - in particular, some of the files are not yet properly\nlinked from the README or other docs. There's some overlap and redundancy,\nand doubtless some out of date bits. The _internals_ documents include some\nthat were written more as technical essays when I was sorting out some\nissue, and those may not have been updated since before the features were\nimplemented.\n\n### General and \"user\" docs.\n\n- [README](https://github.com/komuw/sewer) and project page.\n- [CHANGELOG](CHANGELOG) with links to per-release notes when they exist\n- [sewer-cli](sewer-cli) Documentation for the command line tool\n\n### Internals (docs for direct users of Client, etc.)\n\n- [sewer-as-a-library](sewer-as-a-library) Rewritten _Usage_ section from\n README using new interfaces, etc. WIP.\n\n- [Aliasing.md](Aliasing), just renamed from DNS-Aliasing, so the contents must\n still be in flux, too. Probably a lumpy blending of implementation and\n user notes, still.\n- [catalog](catalog) the driver catalog and support in catalog.py\n- [dns-01](dns-01) sewer's existing (legacy) DNS providers as well as some\n skeleton code for implementing a new-model driver\n- [DNS-Propagation.md](DNS-Propagation) has become (is becoming?) the document\n about _what_ propagation means to the ACME process and _how_ we might\n manage it. From the title you can tell it began when I was still thinking\n of propagation as DNS thing.\n- [http-01](http-01) guide for writing HTTP driver **TO DO**\n- [LegacyDNS.md](LegacyDNS) Documents the new-model shim (still named\n BaseDNS) that allows unmigrated Legacy DNS drivers to continue working. \n Authors of any DNS module, Legacy or new, should review this... and,\n hopefully, migrate to or start anew as new-model Providers.\n- [UnifiedProvider.md](UnifiedProvider) was the first intentional thought\n doc. It has been written and revised repeatedly since Alec's original\n http-01 changes got me thinking about how best to accomodate both, as well\n as other validation methods that were already RFCs or on their way.\n- [unpropagated.md](unpropagated) was the first separate piece. Begun as I was\n figuring out what to do, and changes upon changes here as well. Heading\n towards being documentation of the implementation.\n- [wildcards.md](wildcards) Notes about wildcard certificates - they should\n work for all drivers now! - and a special case where there are probably\n still issues.\n\n- [ACME](ACME) A bit of history, a start on a technical essay, or ??? \n Notes on various things related to ACME and Let's Encrypt's servers.\n" }, { "path": "docs/notes/0.8.2-notes.md", "content": "## 0.8.2 release\n\n0.8.2 contains a lot more work - and changes - than recent releases,\nhence this verbose guide to what's been going on in sewer this spring.\n\nTo my mind, the big change has been landing the revised RFC protocol changes.\nThis allows sewer to operate against LE's staging server again,\nand to continue to work with their production server when they drop compatibility\nwith the earlier version of the protocol in November.\n\nOther changes that may be equally important to some users have been the addition\nof drivers for the powerdns and gandi DNS services,\nand changes to accomodate http-01 challenge providers.\nThe interface for dns-01 and http-01 challenge providers has been unified\nfrom its initial form, and hopefully that interface is general enough\nto accomodate not only dns-01 and http-01, but other future challenge types.\n\n### bugs, fixed or known\n\nThere are two related issues with wildcard certificates that have turned up\nin some providers.\nThe first of these was fixed in 0.8.1, when we stopped Client from prefixing\nwildcard names with \"*.\" when passing them to the providers.\nThat issue has been known for a long time, and some providers already had a\nworkaround - but sometimes the workaround wasn't complete (PR #139, eg.).\n\nThe second issue arises only when requesting a wildcard certificate (for\n*.domain.tld, say) that is to also cover the naked domain (domain.tld).\nThis arises when the DNS service has issues with setting up two TXT records\nfor the two separate challenges ACME needs, because they both are on\ndomain.tld.\nThere doesn't seem to be any easy global fix for this, as there was for the\nfirst problem, so it's being fixed provider by provider as it arises (and\nthere's a user of that service to help with the fix).\n\n### other changes\n\nThe *cli* program has, I believe, no user-visible incompatibilties.\n" }, { "path": "docs/notes/0.8.3-notes.md", "content": "## Sewer 0.8.3 Release Notes\n\nThis will attempt to list all the changes that affect users of the\n`sewer-cli` program, including even cosmetic changes. If you use sewer as a\nlibrary you may find internal changes not called out here.\n\n**New `sewer-cli` features are usually just mentioned, see\n[sewer-cli.md](sewer-cli) for more complete documentation.**\n\n### What's New\n\n- added many words in the /docs directory. A lot of it is internals\n documentation; a lot of it was written to help me understand exactly how\n some things worked and decide how they should be improved - design essays,\n as it were. Much of it is sure to still be in some intermediate state.\n\n- new documentation of the [`sewer-cli` user command](sewer-cli). Tries to\n be _as-is in 0.8.3_ while warning about things that are sure to change\n later.\n\n- `--acme_timeout` has been added (revised from menduo's PR #154)\n\n- `--p_opt name=value ...` allows passing multiple driver options through\n the command line. This is preferred over single-purpose cli options that\n were briefly present in pre-release work since they eliminate the need to\n manually add new driver options to the cli parser.\n\n- `alias` parameter (in DNSProviderBase) added; available through `--p_opt\n alias=...` for `sewer-cli`. **NB: legacy drivers do NOT implement\n aliasing yet - code changes required** except for the demonstration DNS\n driver, unbound_ssh.\n\n- `prop_delay`, `prop_timeout` and `prop_sleep_times` (in DNSProviderBase)\n Available through the `--p_opt` option in `sewer-cli`, though no legacy\n provider has the driver support necessary to make `prop_timeout` and\n `prop_sleep_times` work.\n\n### What's Changed\n\nMostly I've tried to avoid changes that were likely to break things. More\nso for `sewer-cli` than those who use the inner workings of Client, of\ncourse.\n\n- the default log level for providers changed from INFO to WARNING,\n so by default they don't natter so much. Makes `sewer-cli` a little quieter\n when there are no problems. (there's more to be done - it looks like some\n messages that are clearly informational are written at a higher priority.)\n\n- all the legacy DNS providers have been minimally revised to accept some\n new options (alias and prop_delay) and pass them up to their parent classes.\n\n- `sewer-cli` interface to providers has been augmented to pass some new options\n (--acme_timeout, driver parameters from --p_opts)\n\n- `sewer-cli` has had long option abbreviations disabled in argparse. This\n was never documented in sewer, and is an attractive nuisance since the\n addition of another option can break an abbreviation. Everyone wants more\n options, right? \n\n- `unbound_ssh` driver, a working demonstration of using aliasing support in\n a legacy DNS driver. Needs a rather specific environment to work, but I\n just renewed a handful of certificates using it the other day. :-)\n\n- JSON configuration has arrived. meta.json replaces __version__.py.\n catalog.json adds a central description of known drivers, replacing the\n mess of imports (removed, see below), and some non-DRY lists in setup.py\n and cli.py.\n\n- `sewer.catalog` provides loading of the JSON catalog as well as methods to\n lookup the driver's descriptor by name and load the module; replaces the\n mess of imports\n\n### Breakage\n\n- removed all the imports in __init__.py (both sewer & dns_providers). This\n *will* affect you if you've just done `import sewer` and access especially\n the provider classes as eg. `sewer.ThatDNSDns`. ~~Using proper imports,\n eg., `import sewer.dns_providers.thatdns.ThatDNSDns` is the current\n workaround, sorry.~~ Recommended use is now something like\n ```\n from sewer import catalog\n\n pro_cls = catalog.ProviderCatalog().get_provider(\"route53\")\n provider = pro_cls(...arguments as needed...)\n ```\n **This does NOT affect `sewer-cli` users.**\n\n### Deprecated\n\n- `--action {run,renew}` option has never actually had any effect and is no\n longer required (since 0.8.2?). LOGS A WARNING IN 0.8.3.\n" }, { "path": "docs/notes/0.8.4-notes.md", "content": "# Sewer 0.8.4 Release Notes\n\n## What's New\n\n- ECDSA keys supported (account and/or certificate keys)\n\n- optional `client` argument to route53 driver for special use cases\n\n## What's Changed in the sewer Command\n\nBasically nothing for many users, other than the change in default for\nsewer-generated RSA key size to follow current security guidance. You will\nhave to use the new `*_key_type` options if you have a hard requirement for\n2048 bit RSA keys (most likely for the certificate key only).\n\n- `--acct_key` & `--cert_key` should be used to designate the file that\n holds the keys to be used (rather than having new ones generated). \n `--account_key` & `--certificate_key` are still accepted as synonyms but\n will be phased out later.\n\n- add `--acct_key_type` & `--cert_key_type` to allow choice of RSA or EC\n keys and key sizes when sewer is generating them for you.\n\n- changed default for generated keys to 3072 bit RSA (had been 2048 bit)\n\n- add `--is_new_key` to allow for first-time registration of your own\n account key (using `--acct_key`) generated outside of sewer.\n\n`--is-new-key` allows an externally generated account key (specified with\n`--acct_key`) to be registered with ACME when first used. Since there's\nbeen basically no way to do this from the CLI, I doubt the addition will\naffect anyone who isn't interested in the new capability.\n\n## Internal changes for library clients:\n\n- Client class methods cert() and renew() are deprecated; just call\n get_certificate() directly instead.\n\n- Client class **no longer generates keys**. (see below)\n\n- crytographic refactoring\n\n - AcmeKey, AcmeAccount & AcmeCsr in crypto.py; uses only cryptography library\n\n - This is what enables ECDSA keys and provides selection by name\n\n- Client.__init__ interface changes due to crypto refactoring\n\n - dropped `account_key` and `certificate_key` optional arguments to Client\n\n - added `acct_key` and `cert_key` REQUIRED arguments to Client taking\n AcmeAccount and AcmeKey objects, respectively.\n\n - add `is_new_acct` argument to force registration of the supplied account\n key\n\n - dropped `bits` argument because Client no longer generates keys!\n\n - dropped `digest` argument since there are currently no alternate digest\n methods for the different key types. (was this ever used?)\n\n## Breakage\n\nNone that I know of (aside from hard changes in Client() interface listed\nabove), so let me know if you find any!\n\n## Deprecated\n\n- `--action {run,renew}` option has never actually had any effect and is no\n longer required (since 0.8.2?). LOGS A WARNING IN 0.8.3. **Will be\n removed in 0.8.5.**\n\n- As mentioned above, CLI `*_key` argument names are changing. Expect they\n will issue a warning in 0.8.5 and go away in 0.8.6.\n" }, { "path": "docs/preview/cloaca.md", "content": "# Cloaca, aka sewer-cli-next?\n\n_This is so preliminary that it hasn't been written yet. A design essay in\na rambling style is all it is._\n\nThe design of the sewer-cli program is focused on getting one certificate\nwith no state (aside from the optionally reused account key) on disk. I can\nsay from much personal experience that this is wonderful for doing ad-hoc\ntests while changing the implementation, and it's workable even for getting\na handful of certificates (ah, shell script, how I both love and hate\nthee...), but I've long thought about a better way - better, at least, for\nhow I'd like to handle certificate renewal.\n\n## Shortcomings of sewer-cli\n\nOne thing that I kept tripping on was the apparent lack of a simple way to\nsetup a new account key for later use. And strictly speaking, sewer-cli\njust doesn't do that. Enlightenment comes when you give proper\nconsideration to sewer-cli's scope - it gets only one certificate per\ninvocation, so you can harvest the one it created and reuse it later. I\nmissed that because I was already looking to get several (three, IIRC)\ncertificates when I first started using sewer.\n\nThe other \"issues\" that come to mind are just limitations of the intended\nscope of sewer-cli. As mentioned above, I ended up doing some shell\nscripting to work around _all those --options, some the same and some that\nchange for each certificate_. And the other part I wanted to automate (and\nfelt less happy doing in a shell script for $REASONS) was getting the new\ncertificates installed after they were created.\n\nSo cloaca addresses these.\n\n## The cloaca command\n\nFirst change - the cloaca command takes, as its first, required, non-option\nargument, the sub-command which selects the operation to carry out. The\nshort list so far goes like this:\n\n- account - create key, register, deregister, maybe transfer, others\n- renew - with no options, renew & install (if configured) all certs in cloaca.ini\n\nAnd more to come. Plenty of operations are defined in RFC8555 which never\ngot into sewer-cli. Goal will be to support all the operations in the RFC\nthat Let's Encrypt has implemented.\n\nSecond change - cloaca is more [config-driven](cloaca_config) than\n--option-driven. Which doesn't mean options won't be available, but maybe\nnot to the point of emulating sewer-cli's ability to renew one certificate\nwithout any configuration.\n\n_Okay, that's it for now. Need to get some proof of concept code to see if\nany of the untested ideas are more problematic than I believe._\n" }, { "path": "docs/preview/cloaca_config.md", "content": "# Configuration file for cloaca\n\n_This is a pre-coding, let alone release, preview of a more config-driven\nuser command that's just starting to bubble in the crock pot here. It could\nend up being cli-next if it's practical to combine the two different\napproaches in one. Either way, it needs a less awkward name than cli-next. :-)_\n\nCloaca's configuration has been driven by a couple use cases. If you have\nonly a single certificate, for one or a few identities (SANs), the\ntraditional option-driven CLI program is perhaps easier. Cloaca is designed\nfor the case where several certificates are being managed together. It also\nadds installation support to get those certificates onto the servers. \nSomewhat to my surprise, it has ended up using the simple \"ini\" file format.\n\n## Introductory examples\n\nA minimal configuration for one certificate for test.example.com that just\nleaves test.example.com.key and test.example.com.key sitting in the current\nworking directory:\n\n [cert_test.example.com]\n account_email = webmaster@example.com\n provider = demo_dns\n\nThis demonstrates an important convention: certificate sections are named by\nprepending \"cert_\" to the domain identity. The only other data it requires\nto produce a new key and certificate, is the name of the Provider class that\nwill handle publishing the challenge responses. As always, the driver's\nauhtentication parameters are here assumed to be passed in through\nenvironment variables (but they could be additional items in that section if\nyou like).\n\nNow, let's add the promised installation to this example:\n\n [cert_test.example.com]\n account_email = webmaster@example.com\n provider = demo_dns\n install_transport = ssh\n install_ssh_to = root@test.example.com\n install_dir = /etc/apache2/ssl\n install_post_cmd = system restart apache2\n\nGetting a little longer, and it glosses over how it gets the ssh key it will\nneed for the installation. So what happens when we have multiple\ncertificates?\n\n [cert_default]\n account_key = account.key\n account_email = webmaster@example.com\n provider = demo_dns\n install_transport = ssh\n install_dir = /etc/apache2/ssl\n install_post_cmd = systemctl restart apache2\n\n [cert_test.example.com]\n install_ssh_to = root@test.example.com\n\n [cert_example.com]\n SAN = www.example.com\n install_ssh_to = hostmaster@www.example.com\n\n [cert_webmail.example.com]\n install_ssh_to = postmaster@webmail.example.com\n install_dir = /etc/certificates\n install_post_cmd = systemctl restart dovecot\n\nThere's a reason this avoids the native [DEFAULT] section, which we'll come\nback to later.\n\n## Configuration Reference\n\nAll of the actual configuration options can appear in a certificate section. \nA certificate section is any section whose name is formed by prepending\n\"cert_\" to the principle domain name (CN in certificate speak). The\navailable keys:\n\n- account_key = filepath to account key to use for this certificate\n- account_email = what it says, obviously\n- account_file = filepath to file with account key, email, and other info TBD\n (this might replace _key and _email, those coming as args to account creation?)\n- provider = name of Provider class\n- provider_KW = value, passed to class constructor as KW=value\n- SAN = san1, san2, ... comma-separated list of additional identifiers to add to certificate\n- install_transport = name of method to use, eg., ssh, cp, ...\n- install_ssh_to = user@fqdn for any ssh-based transport (also for post_cmd, etc)\n- install_dir = path (should be absolute) to directory where new key & crt are installed\n- install_post_cmd = command to be run after key is installed (to make it take effect)\n\nAs seen above, a [cert_default] section will be folded in to every [cert_CN]\nsection. This is convenient for truly universal settings, but the `_method`\nmechanism is preferred for settings that apply less universally. In any\nevent, an explicit setting in [cert_CN] will always take precedence over\nthose injected from other sources.\n\n### The `_method` Method\n\nThis allows a group of settings that are shared among some of the\ncertificates to be grouped and defined once, then included by name. More or\nless:\n\n [cert_default]\n account_key = account.key\n provider = demo_dns\n\n [install_apache2]\n transport = ssh\n dir = /etc/apache2/ssl\n post_cmd = systemctl restart apache2\n\n [cert_test.example.com]\n install_method = apache2\n install_ssh_to = root@test.example.com\n\n [cert_example.com]\n SAN = www.example.com\n install_method = apache2\n install_ssh_to = hostmaster@www.example.com\n\nThe mechanism finds keys of the form PREFIX_method = NAME and looks for\na section [PREFIX_NAME]. It then replaces the PREFIX_method item with\nthe items in that section after prepending PREFIX_ to each key. So in the\nabove, each occurence of\n\n install_method = apache2\n\nis replaced by items from [install_apache2] which become\n\n install_transport = ssh\n install_dir = /etc/apache2/ssl\n install_post_cmd = systemctl restart apache2\n\nWhich is much like what a [cert_default] section would do except it wouldn't\ntry to add the apache config items into certs that are for an nginx hosted\ndomain, say. The `_method` method only injects the settings it's explicitly\nasked to insert.\n" }, { "path": "docs/sewer-as-a-library.md", "content": "# Sewer as a Python Library\n\n>`sewer-the-library` is in a period of heavy change (summer 2020 - ?). I'll\ntry to keep the examples (below) and other docs up to date, but I'm sure\nthings will lag sometimes.\n\nThis document is neither a \"cookbook\" nor in any way a substitute for the\ndocumentation of sewer's parts and internals, such as they are. Let's try\nto list the existing docs:\n\n- [Cryptographic library](crypto) this is in decent shape because it's been\n created and kept up to date in sync with crypto.py - both too new to have\n bit rot yet.\n\n- [ACME protocol](ACME) was a piece I started writing while learning the\n quirks of the ACMEprotocol. Quite incomplete, the main thing it brings to\n the table is the link to RFC8555, which is the protocol's definition. Of\n course there are other foundational RFCs to be read...\n\n- The [driver catalog](catalog) is another new part that isn't yet being\n used to its fullest. It glues the drivers and the CLI program together,\n and stands ready to help your bespoke front end likewise unless your\n target is so specific you can just manually import the only driver you\n need.\n\n- Drivers! So much of this is about those intermediaries between sewer and\n the diverse services that actually publish our challenge responses.\n\n + [Unified provider](UnifiedProvider) began as a technical essay when I\n was starting to sort through the problems and possibilities Alec's\n original http-01 driver support introduced. It's an uneven blend of\n design philosophy and code documentation, with plenty of ToDo in it.\n\n + [Wildcard certificates](wildcards) are one of the things the dns-01\n challenge type brought to the table. Some notes on how they work and\n what issues remain.\n\n + [Aliasing](Aliasing) can be a handy technique to manage dns-01\n challenges without needing to deal with a primary DNS provider whose\n support for fast-propagating, short-lived TXT records leaves something\n to be desired.\n + Speaking of DNS Propagation, we find [DNS propagation](DNS-Propagation),\n which talks about what it is and why you might need it, and offers what\n documentation there is on the parameters to pass to the drivers to\n control it. And for driver writers, mostly,\n [unpropagated](unpropagated) discusses what's needed to add a probe &\n wait timeout loop.\n\n_more to do _\n\n## Usage examples\n\nKeep in mind that these are untested code intended to demonstrate how the\nmajor features are used. Supporting details may not be repeated for each\nsimilar example, or may not be present in any of them.\n\n```python\nimport sewer.client\nfrom sewer.crypto import AcmeKey\n\n# [[ change this to load using the catalog! ]]\nimport sewer.dns_providers.cloudflare\n\ndns_class = sewer.dns_providers.cloudflare.CloudFlareDns(\n CLOUDFLARE_EMAIL='example@example.com',\n CLOUDFLARE_API_KEY='nsa-grade-api-key'\n)\n\n# 1. to create a new certificate (new account and certificate keys)\n\nclient = sewer.client.Client(\n domain_name='example.com',\n dns_class=dns_class,\n acct_key=AcmeKey.create(\"rsa2048\"),\n cert_key=AcmeKey.create(\"rsa2048\")\n)\ncertificate = client.cert()\n\n# NB: new crypto keeps keys & certs in python objects. They intentionally\n# do not convert to printable form automatically (__str__, etc.)\n\nprint(\"your certificate is:\", certificate.private_bytes())\nprint(\"your certificate's key is:\", cert_key.private_bytes())\nprint(\"your letsencrypt.org account key is:\", acct_key.private_bytes())\n\n# NB: your certificate_key and account_key should be SECRET.\n# You can write these out to individual files, eg::\n\nwith open('certificate.crt', 'wb') as f:\n f.write(certificate.private_bytes())\nwith open('certificate.key', 'wb') as f:\n f.write(certkey.private_bytes())\nwith open('account.key', 'w') f:\n f.write(acctkey.private_bytes())\n\n# the acct_key also contains ACME's \"kid\" identifier if you're interested\n\n# 2. to renew a certificate:\n\ndns_class = sewer.dns_providers.cloudflare.CloudFlareDns(\n CLOUDFLARE_EMAIL='example@example.com',\n CLOUDFLARE_API_KEY='nsa-grade-api-key'\n)\n\n# load saved keys or create new as you prefer\nacct_key = AcmeKey.from_file(\"account.key\")\ncert_key = AcmeKey.from_file(\"certificate_key\")\n\nclient = sewer.client.Client(\n domain_name='example.com',\n dns_class=dns_class,\n acct_key=acct_key,\n cert_key=cert_key,\n)\ncertificate = client.renew()\ncertificate_key = client.certificate_key\n\nwith open('certificate.crt', 'w') as certificate_file:\n certificate_file.write(certificate)\nwith open('certificate.key', 'w') as certificate_key_file:\n certificate_key_file.write(certificate_key)\n\n# 3. You can also request/renew wildcard certificates:\n\ndns_class = sewer.dns_providers.cloudflare.CloudFlareDns(\n CLOUDFLARE_EMAIL='example@example.com',\n CLOUDFLARE_API_KEY='nsa-grade-api-key'\n)\nclient = sewer.client.Client(\n domain_name='*.example.com',\n dns_class=dns_class,\n # load or create keys\n)\ncertificate = client.cert()\ncert_key = client.cert_key\nacct_key = client.acct_key\n```\n" }, { "path": "docs/sewer-cli.md", "content": "## Sewer's user command (so many --options!)\n\nSewer's command line interface, historically named just \"sewer\" or\n\"sewer-cli\", and implemented in `sewer/cli.py`, is now also available using\nthe python command line option (eg. `python3 -m sewer`). In these docs\nwe'll call it `sewer-cli` in order to avoid already overloaded or generic\nnames.\n\nThe command line tool, however invoked, is still a good vehicle for creating\nor renewing a single certificate. Simple cases may need only a few\n--options, but as time goes by the possibilities keep increasing. The\nofficial doumentation of the options that `sewer-cli` supports remains the\noutput from running `sewer-cli --help`, but that can be rather terse. Here\nwe will discuss what the options are and why they are needed, especially\nsome recently [or soon-to-be!] added options.\n\n### _key_type_ values\n\nThis is new with the crypto overhaul (pre-0.8.4). You can now choose the\ntype and size of both the account and certificate keys to be generated by\nsewer (if you don't pass it existing keys for one or both).\n| _key_type_ | key & size | notes |\n| --- | :-: | --- |\n| rsa2048 | RSA 2048 bits | old sewer default |\n| rsa3072 | RSA 3072 bits | NEW sewer default |\n| rsa4096 | RSA 4096 bits | |\n| secp256r1 | ECDSA 256 bits | |\n| secp384r1 | ECDSA 384 bits | |\n| secp521r1 | ECDSA 521 bits | **not accepted for certificate key** |\n\n> NOTE that the default generated key has changed from 2048 bit RSA to 3072\nbit RSA. This is in keeping with current NIST reccomendations. Unless you\nhave a need to continue to use RSA account keys (existing scripts assume\nRSA, perhaps), one of the ECDSA types is suggested.\n\nThe choice of key_type would be easy if not for external factors: ECDSA is\nwidely preferred on most grounds, but RSA may be required for backwards\ncompatibility with old software or appliances. Some new applications and\ndevices, OTOH, are dropping RSA due to its resource demands (CPU time and\nmemory).\n\nThe 521 bit EC key is still valid in the specs, but currently most (?) browsers\ndon't support it, as LE has chosen to reject certificates using that key\ntype and size.\n\n### sewer-cli General Options\n\n`--version`\n`--known_providers`\n> These are both immediate action options. They print their information and\nexit, ignoring all other arguments (to include argparse errors).\n\n`--log_level`\n\n`--action` \"run\"|\"renew\"\n> **OBSOLESCENT**. No longer required in 0.8.3! Default is \"renew\".\nHas no effect other than changing one word used in one message text.\nWhether to create a server key and certificate de novo or reuse the existing\nserver key (the only thing that CAN be reused) depends only on whether\n`--certificate_key` is given.\n\n`--acme_timeout` _seconds_ {7}\n> Used to adjust the timeout applied to all requests to the ACME server.\nIf you need to increase this timeout you'll know it .\n_added by #188 in pre-0.8.3; reworked from #154 from @menduo_\n\n### ACME options\n\n`--endpoint` \"production\"|\"staging\"\n> Default is \"production\", viz., issue a legitimate certificate. Use\n\"staging\" for testing!\n_protocol changes enforced since late 2019 for staging are fixed in 0.8.2_\n\n### Account options\n\nTo an ACME server, an account is a key pair which has been registered with\nthat server. Oh, there's other information that MAY be attached when it is\nregistered - if you pass you email address to LE you can get timely reminders\nabout certificates that need to be renewed soon. But basically, it's that\nkey pair.\n\nBy default, `sewer-cli` will create a new, unique key pair each time it's\nrun. And this is okay, because it will also save the key alongside the\ncertificate and the key that's attested to by the certificate. But if you\ndon't want every cert to be issued to a new identity, you'll need to use\n`--acct_key` to provide the already-registered one. **New in 0.8.4:** you\ncan use a new, unregistered account key if you also use the --is_new_acct\noption (and email, of course).\n\nAfter the certificate has been created and downloaded, the account key\n`sewer-cli` used will be saved alongside the certificate and certificate\nkey. After this,the new account key is registered with the ACME endpoint\nand may be used for future certificate requests using `--acct_key`.\n\n`--acct_key` _filepath_\n> Filepath to existing, already registered ACME account key. Default is to\ncreate a new key and register it. Preferred over `--account_key` now.\n\n`acct_key_type` _key_type_\n> Type of key to generate if `--acct_key` not given. Default is rsa3087.\n\n`--is_new_acct`\n> Used with `--acct_key`, allows a key you created outside of sewer to be\nregistered as an account key the first time it's used.\n\n`--email` _email_address_\n\n### Challenge publisher options\n\n`--provider`|`--dns` **name**\n> Name of the [DNS] provider to use.\n`--dns` is OBSOLESCENT, prefer `--provider` which will be required in 0.9.\nAs of 0.8.3 this still only supports the legacy DNS providers.\n\n> _ALTERNATE FOR 0.9: make `provider` a required positional parameter,\nin accordance with argparse's good advice that\n\"users expect options to be optional\"._\n\n#### Driver parameters\n\nDuring the pre-0.8.3 work, several long options were added to `sewer-cli`\nfor individual new driver parameters. These single-use options will be\nretired with the release of 0.8.3, as they are all redundant since the\nintroduction of `--p_opts`.\n\n`--p_opts` name=value ...\n>Added late in 0.8.3 development, this will be the only way to pass\nparameters into the drivers when 0.8.3 releases. Like `--alt_domains`,\nthere can be any number of named parameters following `--p_opts`.\n\n##### Propagation management parameters\n\nThere are two sorts of things for which we have to wait: the ACME server\n(see `--acme_timeout`) and the service provider (especially DNS propagation\nacross a global anycast service). Although these are USED in the core\nengine code, they are SET through the driver. The reasoning is that the\ndriver is the only part of sewer that might sensibly \"know\" what sensible\ndefaults might be for its service provider, and what features are available. \nSo eg., although `--prop_timeout` is available to all drivers, legacy DNS\ndrivers might want to issue a warning if it is specified since (unless\nthey're updated) they do not support the `prop_timeout` mechanism.\n\n`--p_opts prop_delay=`\n> Adds a fixed delay after the challenge response have all been setup to\nallow the challenge to propagate before any other processing; the default is\nno delay. This is the simplest of the propagation waiting methods, and the\nonly one available to ~~unmodified~~ minimally modified legacy DNS drivers\nsuch as those in 0.8.3.\n_This was `--prop_delay ` during 0.8.3. development_\n\n`--p_opts prop_timeout=`\n> Activates the active propagation checks and sets the timeout for that\nprocess; default is to not do these checks. This requires an implementation\nof the driver `unpropagated` method to be useful. Legacy DNS drivers\ninherit a null implementation which always reports _all are ready_ which\nshort-circuits this process. _to be added when there's driver support_\n\n`--p_opts prop_sleep_times=`\n> Comma-separated list of integer number of seconds to sleep after the\nfirst, second, ... _not all ready_ response from the driver's\n`unpropagated` method. The last value is re-used after the list has been\nused up. Default is \"1,2,4,8\". _to be added when there's driver support_\n\n##### DNS driver parameters\n\n`--p_opts alias_domain=`\n> Configure an alternate DNS domain in which the challenge responses will be\nplaced. See [Aliasing](Aliasing) for details. **Legacy DNS\nproviders accept this, but require further modification to actually apply\nthe aliasing that's supported by their parent classes.**\n_This was `--alias_domain ` during 0.8.3 development.`\n\n### Certificate info\n\n`--domain` **CN-name**\n> The primary identity for the certificate. REQUIRED, no default. CN-name\nis also used to form the default names for a number of files.\n\n`--alt_domains` _SAN-name ..._\n> List of alternate identities to be included in the certificate. Not quite\nwhat pedants would call \"SAN\", since this should NOT include the CN-name. \nMultiple identities may be given, and sewer-cli will take all parameters\n(aka words) as SAN-names until it encounters another option (double-dash). \nDefault is an empty list.\n\n`--out_dir` _dirpath_\n> Set directory where the certificate and key files will be stored. Default\nis to use the current working directory where sewer was run.\n\n`--cert_key` _filepath_\n> File path to your existing certificate key. Preferred over\n`--certificate_key`. As with `acct_key`, if this is not specified, a new\nkey will be created. Has a similar effect to certbot's `--reuse-key` (sp?)\nif it points to the key file from the previous run.\n\n`--cert_key_type` key_type\n> Type of key to generate if `--cert_key` not given. Default is rsa3087.\n\n--bundle_name _basename_\n> Base name to use for output file, eg., out_dir/basename.{account.key,crt,key}\nDefault is to use the CN-name\n" }, { "path": "docs/unpropagated.md", "content": "# Waiting for the Challenge to Propagate\n\nWhen you use a service provider's API to setup a challenge response, how\nlong does it take before the ACME server can reliably get that answer? \nEspecially with global anycast DNS services, it can take a while! This\ndelay between _posted to API_ and _actually online everywhere that matters_\nis the propagation delay we're talking about here.\n\n## How shall we wait, let me count the ways\n\nsewer provides two kinds of delay that can be used to deal with propagation\nwithin the service provider's systems. Although this was designed mostly\nfor DNS validation, it may be needed for other types depending on the\nservice provider. The two kinds of delay are (1) an unconditional sleep and\n(2) an iterative probe/sleep delay loop that has a timeout to keep it from\nwaiting _too long_.\n\n### To sleep, perchance t'will be enough\n\nThe unconditional sleep is implemented in the sewer core logic and is\navailable to any driver which can pass `prop_delay=seconds_to_sleep` along\nto its parent, and so upwards to `ProviderBase`. If set to a positive\nvalue, it simply sleeps for that number of seconds after the challenges have\nall been setup.\n\n--- Available in drivers in 0.8.3.\n\n### Check twice, respond once\n\nThe iterative probe is a more active sort of delay: it repeatedly calls the\ndriver's `unpropagated` method to test whether the challenges are all in\nplace. Sadly, the service providers who most need this kind of check are\nprobably the ones it is most difficult to meaningfully test: by design, you\ncannot know which anycast DNS or CDN machine(s) the ACME server will query,\nnow which ones you'll get in a simple probe by DNS name. Some service\nproviders may give you a way to query through their API. Do what you can...\n\nThere are two parameters that manage this: `prop_timeout` and the optional\n`prop_sleep_times`. The first has to be present for the probing to happen\nat all; by default this checking is skipped. the sleep times is an array of\nintegers, with a default value [1, 2, 4, 8] which causes the loop to sleep 1\nsecond after the first probe if the challenges are not all ready, then 2, 4,\n8, and ever after 8 seconds, continuing until it's been at least\nprop_timeout seconds since the first probe.\n\nThis does also require an implementaion of the `unpropagated` method in the\ndriver. The only sane default, used in the legacy drivers' shim class, is\nto always return success without any actual checking.\n\n## Advice to driver authors and users\n\nAuthors: If the service gives you a way to do a meaningful check and it's\nneeded, please implement `unpropagated`, and mention that in the driver's\ndocumentation. Otherwise, just make sure it inherits or implements a null\ncheck. Feel free to set default values for delay and/or timeout if its\npredictable enough, but be sure not to overide the values if the user passes\nhis own into the driver. And document, document, document!\n\nUsers: Check those driver docs to see what's supported. Most of the time, a\ngoodly `prop_delay` will get you past the propagation most of the time, and\nis more likely to be available.\n\n_Driver docs are a WIP. Currently not much to see other than the features\ntable in [dns-01](dns-01), such as it is._\n" }, { "path": "docs/wildcards.md", "content": "# Wildcard Certificates\n\nSince 0.8.2, sewer should be able to request and receive simple wildcard\ncertificates using any of the DNS drivers. In earlier versions there was an\neccentric re-naming of wildcard targets in the core logic which the drivers\nwould, sometimes unreliably, remove. _tl;dr: before 0.8.2 it depended on the\ndriver._\n\n## One issue remains in 0.8.3\n\nCertificates with a wildcard CN name, eg., `domain=*.example.com`, are valid\nfor all and only the immediate sub domains of example.com. They do NOT\nvalidate for example.com itself, which may come as a surprise if you have\nused some other (commercial?) providers, as they may silently add the\nnaked domain as described below.\n\nTo create such \"wildcard-plus\" certificates in sewer, you would still use\n`domain=*.example.com`, then add `alt_domains=example.com`. Sewer itself,\nboth through sewer-cli as well as the library interface (Client), is fully\ncapable of handling this. The issue arises when publishing the challenge\nresponse. To a DNS ([1](#footnote1)) driver, this will appear as two\ndifferent TXT values for the same name (in this case \"example.com\"). \nTraditional DNS systems (inevitable eg.: bind) have no problem with having\nmultiple TXT records like this, but many DNS service providers are using\nvery different software. To be honest, the problem some of sewer's drivers\nhave with this may be in the service provider's core system or just in their\nAPI layer. But we have had a problem using those APIs when setting up such\nwildcard-plus-naked-domain certificate's validations, and from here on the\noutside we can only deal with them one by one.\n\n(1) HTTP challenges don't have this issue\nbecause each challenge uses a unique file name/URL component.\n\n> There is a general fix that OUGHT to be possible: have sewer's core logic\nrecognize cases where such \"duplicate\" challenges exist, and if the driver\ndoesn't announce itself capable of handling it, use a multi-step process to\npublish a non-overlapping subset of the challenges, wait for propagation,\nrespond to the ACME server, and wait for the challenges to be validated,\nthen remove the subset; then repeat the whole process for the \"duplicate\"\nvalidation. For now, my (mmaney) \"plan\" is to continue helping driver\nauthors fix the drivers they're familiar with as a bug is reported (and try\nto talk them into migrating to the new-model interface, of course!).\n" }, { "path": "mypy.ini", "content": "# ignoring missing annotations is still a way of life. In general, add\n# mypy-package here for general-purpose libraries (cryptography.x509.oid and\n# tldextract are good examples); use local \"# type: ignore\" markup for the\n# service-specific libraries used in drivers (eg. boto3 in route53.py)\n#\n# sadly, Guido resists using pyproject.toml, so this file must add its clutter.\n\n[mypy]\n\n\n[mypy-cryptography.x509.oid]\n ignore_missing_imports = True\n\n[mypy-tldextract]\n ignore_missing_imports = True\n" }, { "path": "pyproject.toml", "content": "[tool.coverage.run]\ncommand_line = \"-m pytest\"\nsource = [\"sewer\"]\nomit = [\n \"*test*\",\n \"*__init__.py\"\n]\ndata_file = \"tests/coverage/.coverage\"\n\n[tool.coverage.report]\nfail_under = 85\nshow_missing = true\nomit = [\n \"*__main__.py\",\n \"*acme.py\",\n \"*cli.py\"\n]\n\n[tool.coverage.html]\ndirectory = \"tests/data/coverage/html\"\n\n\n[tool.pytest.ini_options]\nconsole_output_style = \"classic\"\naddopts = \"--color=no\"\n\n\n# nigri delenda est - getting to be more trouble than help (no, NOT kwargs, )\n[tool.black]\nline-length = 100\ntarget-version = [\n \"py37\",\n \"py38\",\n \"py39\",\n \"py310\",\n \"py311\",\n \"py312\",\n]\n" }, { "path": "setup.py", "content": "import codecs, json, os\nfrom setuptools import setup, find_packages\n\n# long description comes from README.md\nwith codecs.open(\"README.md\", \"r\", encoding=\"utf8\") as f:\n long_description = f.read()\nldct = \"text/markdown\"\n\n# version and other fields in about, with envvar override\nwith codecs.open(os.path.join(\"sewer\", \"meta.json\"), \"r\", encoding=\"utf8\") as f:\n meta = json.load(f)\n\nfor k in meta:\n if \"SETUP_\" + k in os.environ:\n meta[k] = os.environ[\"SETUP_\" + k]\n\n# provider catalog, used to construct the list of extras and their deps, and all their deps\nwith codecs.open(os.path.join(\"sewer\", \"catalog.json\"), \"r\", encoding=\"utf8\") as f:\n catalog = json.load(f)\n\nprovider_deps_map = dict((i[\"name\"], i[\"deps\"]) for i in catalog)\n\nall_deps_of_all_providers = list(set(sum((i[\"deps\"] for i in catalog), [])))\n\n\nsetup(\n long_description=long_description,\n long_description_content_type=ldct,\n classifiers=[\n \"Development Status :: 4 - Beta\",\n \"Environment :: Console\",\n \"Intended Audience :: Developers\",\n \"Topic :: Software Development :: Build Tools\",\n \"Topic :: Internet :: WWW/HTTP\",\n \"Topic :: Security\",\n \"Topic :: System :: Installation/Setup\",\n \"Topic :: System :: Networking\",\n \"Topic :: System :: Systems Administration\",\n \"Topic :: Utilities\",\n \"License :: OSI Approved :: MIT License\",\n # Specify the Python versions you support here. In particular, ensure\n # that you indicate whether you support Python 2, Python 3 or both.\n \"Programming Language :: Python :: 3\",\n \"Programming Language :: Python :: 3.7\",\n \"Programming Language :: Python :: 3.8\",\n \"Programming Language :: Python :: 3.9\",\n \"Programming Language :: Python :: 3.10\",\n \"Programming Language :: Python :: 3.11\",\n \"Programming Language :: Python :: 3.12\",\n ],\n packages=find_packages(exclude=[\"docs\", \"*tests*\"]),\n install_requires=[\"requests\", \"cryptography\"],\n extras_require=dict(\n provider_deps_map,\n dev=[\"twine\", \"wheel\"],\n test=[\"mypy>=0.780\", \"coverage>=5.0\", \"pytest>=6.0\", \"pylint>=2.6.0\", \"black==19.10b0\"],\n alldns=all_deps_of_all_providers,\n ),\n # data files to be placed in project directory, not zip safe but zips suck anyway\n package_data={\"sewer\": [\"*.json\"]},\n zip_safe=False,\n # To provide executable scripts, use entry points in preference to the\n # \"scripts\" keyword. Entry points provide cross-platform support and allow\n # pip to create the appropriate form of executable for the target platform.\n # entry_points={\n # 'console_scripts': [\n # 'sample=sample:main',\n # ],\n # },\n entry_points={\"console_scripts\": [\"sewer=sewer.cli:main\", \"sewer-cli=sewer.cli:main\"]},\n ### CANNOT FIX ### black sometimes ignores explicit version and adds the invalid comma anyway\n # fmt: off\n **meta\n # fmt: on\n)\n" }, { "path": "sewer/__init__.py", "content": "" }, { "path": "sewer/__main__.py", "content": "from . import cli\n\ncli.main()\n" }, { "path": "sewer/auth.py", "content": "from typing import Any, Dict, Optional, Sequence, Tuple, Union, cast\n\nfrom .lib import create_logger, LoggerType\n\nChalItemType = Dict[str, str]\nChalListType = Sequence[ChalItemType]\nErrataItemType = Tuple[str, str, ChalItemType]\nErrataListType = Sequence[ErrataItemType]\n\n\nclass ProviderBase:\n \"\"\"\n New-model driver documentation is in docs/UnifiedProvider.md\n \"\"\"\n\n def __init__(\n self,\n *,\n chal_types: Sequence[str],\n logger: Optional[LoggerType] = None,\n LOG_LEVEL: str = \"INFO\",\n prop_delay: int = 0,\n prop_timeout: int = 0,\n prop_sleep_times: Union[Sequence[int], int] = (1, 2, 4, 8),\n ) -> None:\n\n # TypeError if missing, still check that it's a sequencey value; non-str vals, meh\n if not isinstance(chal_types, (list, tuple)):\n raise ValueError(\"chal_types must be a list or tuple of strings, not: %s\" % chal_types)\n self.chal_types = chal_types\n\n # setup logging. let it pass if both are given; logger supersedes old LOG_LEVEL\n if logger:\n self.logger = logger\n else:\n self.logger = create_logger(__name__, LOG_LEVEL)\n\n # prop_* control delay before and timeout of checking loop as well as internal sleeps\n self.prop_delay = int(prop_delay)\n self.prop_timeout = int(prop_timeout)\n\n ### eratta ### accepts str value(s) that pass int(); low importance\n if isinstance(prop_sleep_times, (list, tuple)):\n self.prop_sleep_times = tuple(int(v) for v in prop_sleep_times)\n else:\n self.prop_sleep_times = (int(cast(int, prop_sleep_times)),)\n\n def setup(self, challenges: ChalListType) -> ErrataListType:\n raise NotImplementedError(\"setup method not implemented by %s\" % self.__class__)\n\n def unpropagated(self, challenges: ChalListType) -> ErrataListType:\n raise NotImplementedError(\"unpropagated method not implemented by %s\" % self.__class__)\n\n def clear(self, challenges: ChalListType) -> ErrataListType:\n raise NotImplementedError(\"clear method not implemented by %s\" % self.__class__)\n\n\nclass HTTPProviderBase(ProviderBase):\n \"\"\"\n Base class for new-model HTTP drivers\n\n Currently this is a null adapter, holding a place in line for any future shared\n implementation. It may never become non-null aside from the default chal_types\n it provides, but it's a small price to pay to avoid having to stuff it in later.\n \"\"\"\n\n def __init__(self, **kwargs: Any) -> None:\n if \"chal_types\" not in kwargs:\n kwargs[\"chal_types\"] = [\"http-01\"]\n super().__init__(**kwargs)\n\n\nclass DNSProviderBase(ProviderBase):\n \"\"\"\n Base class for new-model DNS drivers - legacy drivers use the one in common.py\n\n Accepts the alias optional argument and adds cname_domain and target_domain\n to support the implementation of aliasing in drivers that inherit from it.\n \"\"\"\n\n def __init__(self, *, alias: str = \"\", **kwargs: Any) -> None:\n if \"chal_types\" not in kwargs:\n kwargs[\"chal_types\"] = [\"dns-01\"]\n super().__init__(**kwargs)\n self.alias = alias\n\n ### support for using a DNS alias\n\n def cname_domain(self, chal: Dict[str, str]) -> Union[str, None]:\n \"returns fqdn where CNAME should be if aliasing, else None\"\n\n return \"_acme-challenge.\" + chal[\"ident_value\"] if self.alias else None\n\n def target_domain(self, chal: Dict[str, str]) -> str:\n \"returns fqdn where challenge TXT should be placed\"\n\n d = chal[\"ident_value\"]\n return \"_acme-challenge.\" + d if not self.alias else d + \".\" + self.alias\n" }, { "path": "sewer/catalog.json", "content": "[\n {\n \"name\": \"acmedns\",\n \"desc\": \"AcmeDns DNS provider\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"api_user\",\n \"req\": 1,\n \"old_param\": \"ACME_DNS_API_USER\"\n },\n {\n \"name\": \"api_key\",\n \"req\": 1,\n \"old_param\": \"ACME_DNS_API_KEY\"\n },\n {\n \"name\": \"api_base_url\",\n \"req\": 1,\n \"old_param\": \"ACME_DNS_API_BASE_URL\"\n }\n ],\n \"path\": \"sewer.dns_providers.acmedns\",\n \"cls\": \"AcmeDnsDns\",\n \"deps\": [\"dnspython\"]\n },\n {\n \"name\": \"aliyun\",\n \"desc\": \"Alibaba Cloud DNS service\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"ak\",\n \"req\": 1,\n \"old_param\": \"aliyun_ak\",\n \"old_envvar\": \"ALIYUN_AK_ID\"\n },\n {\n \"name\": \"secret\",\n \"req\": 1,\n \"old_param\": \"aliyun_secret\",\n \"old_envvar\": \"ALIYUN_AK_SECRET\"\n },\n {\n \"name\": \"endpoint\",\n \"old_param\": \"aliyun_endpoint\",\n \"old_envvar\": \"ALIYUN_ENDPOINT\"\n }\n ],\n \"path\": \"sewer.dns_providers.aliyundns\",\n \"cls\": \"AliyunDns\",\n \"deps\": [\"aliyun-python-sdk-core-v3\", \"aliyun-python-sdk-alidns\"],\n \"memo\": \"default value of endpoint in ad-hoc cli.py code - add default to args?\"\n },\n {\n \"name\": \"aurora\",\n \"desc\": \"Aurora DNS service from pcextreme hosting\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"api_key\",\n \"req\": 1\n },\n {\n \"name\": \"secret_key\",\n \"req\": 1\n }\n ],\n \"path\": \"sewer.dns_providers.auroradns\",\n \"cls\": \"AuroraDns\",\n \"deps\": [\"tldextract\", \"apache-libcloud\"]\n },\n {\n \"name\": \"cloudflare\",\n \"desc\": \"Cloudflare DNS using either email & key or just a token\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"email\"\n },\n {\n \"name\": \"api_key\"\n },\n {\n \"name\": \"api_base_url\"\n },\n {\n \"name\": \"token\"\n }\n ],\n \"path\": \"sewer.dns_providers.cloudflare\",\n \"cls\": \"CloudFlareDns\",\n \"deps\": [],\n \"memo\": \"accepts EITHER token OR both email & key; driver MUST sanity check\"\n },\n {\n \"name\": \"cloudns\",\n \"desc\": \"ClouDNS service\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n ],\n \"path\": \"sewer.dns_providers.cloudns\",\n \"cls\": \"ClouDNSDns\",\n \"deps\": [\"cloudns-api\"],\n \"memo\": \"API library grovels the environment for its access parameters directly\"\n },\n {\n \"name\": \"dnspod\",\n \"desc\": \"DNSPod DNS provider\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"id\",\n \"req\": 1\n },\n {\n \"name\": \"api_key\",\n \"req\": 1\n },\n {\n \"name\": \"api_base_url\"\n }\n ],\n \"path\": \"sewer.dns_providers.dnspod\",\n \"cls\": \"DNSPodDns\",\n \"deps\": [],\n \"memo\": \"api_base_url not usually used? [VERIFY]\"\n },\n {\n \"name\": \"duckdns\",\n \"desc\": \"DuckDNS DNS provider\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"token\",\n \"req\": 1,\n \"old_param\": \"duckdns_token\",\n \"old_envvar\": \"DUCKDNS_TOKEN\"\n },\n {\n \"name\": \"api_base_url\",\n \"old_param\": \"DUCKDNS_API_BASE_URL\",\n \"old_envvar\": \"\"\n }\n ],\n \"path\": \"sewer.dns_providers.duckdns\",\n \"cls\": \"DuckDNSDns\",\n \"deps\": [],\n \"memo\": \"as-is code does not look for envvar for base_url; maybe for testing only?\"\n },\n {\n \"name\": \"gandi\",\n \"desc\": \"Gandi DNS service\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"api_key\",\n \"req\": 1,\n \"old_param\": \"GANDI_API_KEY\"\n }\n ],\n \"path\": \"sewer.dns_providers.gandi\",\n \"cls\": \"GandiDns\",\n \"deps\": []\n },\n {\n \"name\": \"hurricane\",\n \"desc\": \"Hurricane Electric DNS service\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"username\",\n \"req\": 1,\n \"old_param\": \"he_username\"\n },\n {\n \"name\": \"password\",\n \"req\": 1,\n \"old_param\": \"he_password\"\n }\n ],\n \"path\": \"sewer.dns_providers.hurricane\",\n \"cls\": \"HurricaneDns\",\n \"deps\": [\"hurricanedns\"]\n },\n {\n \"name\": \"powerdns\",\n \"desc\": \"PowerDNS DNS provider\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"api_key\",\n \"req\": 1,\n \"old_param\": \"powerdns_api_key\",\n \"old_envvar\": \"POWERDNS_API_KEY\"\n },\n {\n \"name\": \"api_url\",\n \"req\": 1,\n \"old_param\": \"powerdns_api_url\",\n \"old_envvar\": \"POWERDNS_API_URL\"\n }\n ],\n \"path\": \"sewer.dns_providers.powerdns\",\n \"cls\": \"PowerDNSDns\",\n \"deps\": [],\n \"memo\": \"could drop old_envvar if prediction ignored old_param?\"\n },\n {\n \"name\": \"rackspace\",\n \"desc\": \"Rackspace DNS service\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"username\",\n \"req\": 1\n },\n {\n \"name\": \"api_key\",\n \"req\": 1\n }\n ],\n \"path\": \"sewer.dns_providers.rackspace\",\n \"cls\": \"RackspaceDns\",\n \"deps\": [\"tldextract\"]\n },\n {\n \"name\": \"route53\",\n \"desc\": \"Amazon cloud DNS service\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"id\",\n \"req\": 1,\n \"old_param\": \"access_key_id\"\n },\n {\n \"name\": \"key\",\n \"req\": 1,\n \"old_param\": \"secret_access_key\"\n }\n ],\n \"path\": \"sewer.dns_providers.route53\",\n \"cls\": \"Route53Dns\",\n \"deps\": [\"boto3\"],\n \"memo\": \"DUMMY LISTING: route53 has never been integrated into cli.py, so this DOESN'T WORK yet\"\n },\n {\n \"name\": \"unbound_ssh\",\n \"desc\": \"Working demonstrater of legacy DNS adopting new features\",\n \"chals\": [\"dns-01\"],\n \"args\": [\n {\n \"name\": \"ssh_des\",\n \"req\": 1,\n \"envvar\": \"\"\n }\n ],\n \"path\": \"sewer.dns_providers.unbound_ssh\",\n \"cls\": \"UnboundSsh\",\n \"features\": [\"alias\"],\n \"deps\": []\n } \n]\n" }, { "path": "sewer/catalog.py", "content": "import codecs, importlib, json, os\nfrom typing import Dict, List, Sequence\n\nfrom .auth import ProviderBase\n\n\nclass ProviderDescriptor:\n def __init__(\n self,\n *,\n name: str,\n desc: str,\n chals: Sequence[str],\n args: Sequence[Dict[str, str]],\n deps: Sequence[str],\n path: str = None,\n cls: str = None,\n features: Sequence[str] = None,\n memo: str = None,\n ) -> None:\n \"initialize a driver descriptor from one item in the catalog\"\n\n self.name = name\n self.desc = desc\n self.chals = chals\n self.args = args\n self.deps = deps\n self.path = path\n self.cls = cls\n self.features = [] if features is None else features\n self.memo = memo\n\n def __str__(self) -> str:\n return \"Descriptor %s\" % self.name\n\n def get_provider(self) -> ProviderBase:\n \"return the class that implements this driver\"\n\n module_name = self.path if self.path else (\"sewer.providers.\" + self.name)\n module = importlib.import_module(module_name)\n return getattr(module, self.cls if self.cls else \"Provider\")\n\n\nclass ProviderCatalog:\n def __init__(self, filepath: str = \"\") -> None:\n \"intialize a catalog from either the default catalog.json or one named by filepath\"\n\n if not filepath:\n here = os.path.abspath(os.path.dirname(__file__))\n filepath = os.path.join(here, \"catalog.json\")\n with codecs.open(filepath, \"r\", encoding=\"utf8\") as f:\n raw_catalog = json.load(f)\n\n items = {} # type: Dict[str, ProviderDescriptor]\n for item in raw_catalog:\n k = item[\"name\"]\n if k in items:\n print(\"WARNING: duplicate name %s skipped in catalog %s\" % (k, filepath))\n else:\n items[k] = ProviderDescriptor(**item)\n self.items = items\n\n def get_item_list(self) -> List[ProviderDescriptor]:\n \"return the list of items in the catalog, sorted by name\"\n\n res = [i for i in self.items.values()]\n res.sort(key=lambda i: i.name)\n return res\n\n def get_descriptor(self, name: str) -> ProviderDescriptor:\n \"return the ProviderDescriptor that matches name\"\n\n return self.items[name]\n\n def get_provider(self, name: str) -> ProviderBase:\n \"return the class that implements the named driver\"\n\n return self.get_descriptor(name).get_provider()\n" }, { "path": "sewer/cli.py", "content": "import argparse, os\n\nfrom . import client, config, lib\n\nfrom .catalog import ProviderCatalog\nfrom .crypto import AcmeKey, AcmeAccount, key_type_choices\n\n\nDEFAULT_KEY_TYPE = \"rsa3072\"\n\n\ndef setup_parser(catalog):\n \"\"\"\n return configured ArgumentParser - catalog-driven list of providers\n \"\"\"\n\n parser = argparse.ArgumentParser(\n prog=\"sewer\",\n description=\"Sewer is an ACME client for getting certificates from Let's Encrypt\",\n allow_abbrev=False,\n formatter_class=argparse.RawTextHelpFormatter,\n )\n\n ### immediate action \"options\"\n\n parser.add_argument(\n \"--version\",\n action=\"version\",\n version=\"%(prog)s {version}\".format(version=lib.sewer_meta(\"version\")),\n help=\"The currently installed sewer version.\",\n )\n parser.add_argument(\n \"--known_providers\",\n action=\"version\",\n version=\"Known Providers:\\n \"\n + \"\\n \".join(\"%s %s\" % (i.name, i.desc) for i in catalog.get_item_list()),\n help=\"Show a list of the known providers and exit.\",\n )\n\n ### ACME account options\n\n parser.add_argument(\n \"--acct_key\",\n \"--account_key\",\n dest=\"acct_key_file\",\n type=argparse.FileType(\"rb\"),\n help=\"File to load registered ACME account key from. Default is to create one.\",\n )\n\n parser.add_argument(\n \"--acct_key_type\",\n choices=key_type_choices,\n default=DEFAULT_KEY_TYPE,\n help=(\n \"Type of acct key to generate if not loaded by --acct_key. Default %s.\"\n % DEFAULT_KEY_TYPE\n ),\n ),\n\n parser.add_argument(\"--email\", help=\"Email to be used for registration of an ACME account.\")\n\n parser.add_argument(\n \"--is_new_acct\",\n action=\"store_true\",\n help=\"Register the key (from --acct_key) rather than assuming it's already registered.\",\n ),\n\n ### certificate options\n\n parser.add_argument(\n \"--cert_key\",\n \"--certificate_key\",\n dest=\"cert_key_file\",\n type=argparse.FileType(\"rb\"),\n help=\"File to load existing certificate key from. Default is to create key.\",\n )\n\n parser.add_argument(\n \"--cert_key_type\",\n choices=[kt for kt in key_type_choices if kt != \"secp521r1\"],\n default=DEFAULT_KEY_TYPE,\n help=(\n \"Type of cert key to generate if not loaded by --cert_key. Default %s.\"\n % DEFAULT_KEY_TYPE\n ),\n ),\n\n parser.add_argument(\n \"--domain\",\n required=True,\n help=\"The DNS identity which will be the certificate's Common Name. May be a wildcard.\",\n )\n\n parser.add_argument(\n \"--alt_domains\",\n default=[],\n nargs=\"*\",\n help=\"Optional alternate (SAN) identities to be added to the CN on this certificate.\",\n )\n\n parser.add_argument(\n \"--bundle_name\",\n help=\"The basename for the output files. Default is the CN given by --domain.\",\n )\n\n parser.add_argument(\n \"--out_dir\",\n default=os.getcwd(),\n help=\"Directory that stores certificate and keys files; current dir is default.\",\n )\n\n ### challenge provider options\n\n parser.add_argument(\n \"--provider\",\n \"--dns\",\n metavar=\"\",\n dest=\"provider\",\n required=True,\n choices=[i.name for i in catalog.get_item_list()],\n help=\"Name of the challenge provider to use. (--dns is OBSOLESCENT; prefer --provider)\",\n )\n parser.add_argument(\n \"--p_opts\", default=[], nargs=\"*\", help=\"Option(s) to pass to provider, each is key=value\"\n )\n\n ### protocol options\n\n parser.add_argument(\n \"--endpoint\",\n default=\"production\",\n choices=[\"production\", \"staging\"],\n help=\"Select between Let's Encrypt's endpoints. Default is production.\",\n )\n parser.add_argument(\n \"--acme_timeout\",\n type=int,\n default=7,\n help=\"Timeout (maximum wait) for all requests to the ACME service. Default is 7\",\n )\n\n ### sewer command options\n\n parser.add_argument(\n \"--loglevel\",\n default=\"INFO\",\n choices=[\"DEBUG\", \"INFO\", \"WARNING\", \"ERROR\", \"CRITICAL\"],\n help=\"The log level to output log messages at. \\\n eg: --loglevel DEBUG\",\n )\n parser.add_argument(\n \"--action\",\n choices=[\"run\", \"renew\"],\n default=\"none\",\n help=\"[DEPRECATED] The action that you want to perform (has never done anything).\",\n )\n\n return parser\n\n\ndef get_provider(provider_name, provider_kwargs, catalog, logger):\n \"\"\"\n return class (or callable) that will return the Provider instance to use\n \"\"\"\n\n ### TODO ### part of catalog's motivation is to replace all this ad hoc copypasta.\n\n if provider_name == \"cloudflare\":\n from .dns_providers.cloudflare import CloudFlareDns\n\n CLOUDFLARE_EMAIL = os.environ.get(\"CLOUDFLARE_EMAIL\", None)\n CLOUDFLARE_API_KEY = os.environ.get(\"CLOUDFLARE_API_KEY\", None)\n CLOUDFLARE_TOKEN = os.environ.get(\"CLOUDFLARE_TOKEN\", None)\n\n if CLOUDFLARE_EMAIL and CLOUDFLARE_API_KEY and not CLOUDFLARE_TOKEN:\n dns_class = CloudFlareDns(\n CLOUDFLARE_EMAIL=CLOUDFLARE_EMAIL,\n CLOUDFLARE_API_KEY=CLOUDFLARE_API_KEY,\n **provider_kwargs,\n )\n elif CLOUDFLARE_TOKEN and not CLOUDFLARE_EMAIL and not CLOUDFLARE_API_KEY:\n dns_class = CloudFlareDns(CLOUDFLARE_TOKEN=CLOUDFLARE_TOKEN, **provider_kwargs)\n else:\n err = (\n \"ERROR:: Please supply either CLOUDFLARE_EMAIL and CLOUDFLARE_API_KEY\"\n \"or CLOUDFLARE_TOKEN as environment variables.\"\n )\n logger.error(err)\n raise KeyError(err)\n\n elif provider_name == \"aurora\":\n from .dns_providers.auroradns import AuroraDns\n\n try:\n AURORA_API_KEY = os.environ[\"AURORA_API_KEY\"]\n AURORA_SECRET_KEY = os.environ[\"AURORA_SECRET_KEY\"]\n\n dns_class = AuroraDns(\n AURORA_API_KEY=AURORA_API_KEY,\n AURORA_SECRET_KEY=AURORA_SECRET_KEY,\n **provider_kwargs,\n )\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"acmedns\":\n from .dns_providers.acmedns import AcmeDnsDns\n\n try:\n ACME_DNS_API_USER = os.environ[\"ACME_DNS_API_USER\"]\n ACME_DNS_API_KEY = os.environ[\"ACME_DNS_API_KEY\"]\n ACME_DNS_API_BASE_URL = os.environ[\"ACME_DNS_API_BASE_URL\"]\n\n dns_class = AcmeDnsDns(\n ACME_DNS_API_USER=ACME_DNS_API_USER,\n ACME_DNS_API_KEY=ACME_DNS_API_KEY,\n ACME_DNS_API_BASE_URL=ACME_DNS_API_BASE_URL,\n **provider_kwargs,\n )\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"aliyun\":\n from .dns_providers.aliyundns import AliyunDns\n\n try:\n aliyun_ak = os.environ[\"ALIYUN_AK_ID\"]\n aliyun_secret = os.environ[\"ALIYUN_AK_SECRET\"]\n aliyun_endpoint = os.environ.get(\"ALIYUN_ENDPOINT\", \"cn-beijing\")\n dns_class = AliyunDns(aliyun_ak, aliyun_secret, aliyun_endpoint, **provider_kwargs)\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"hurricane\":\n from .dns_providers.hurricane import HurricaneDns\n\n try:\n he_username = os.environ[\"HURRICANE_USERNAME\"]\n he_password = os.environ[\"HURRICANE_PASSWORD\"]\n dns_class = HurricaneDns(he_username, he_password, **provider_kwargs)\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"rackspace\":\n from .dns_providers.rackspace import RackspaceDns\n\n try:\n RACKSPACE_USERNAME = os.environ[\"RACKSPACE_USERNAME\"]\n RACKSPACE_API_KEY = os.environ[\"RACKSPACE_API_KEY\"]\n dns_class = RackspaceDns(RACKSPACE_USERNAME, RACKSPACE_API_KEY, **provider_kwargs)\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"dnspod\":\n from .dns_providers.dnspod import DNSPodDns\n\n try:\n DNSPOD_ID = os.environ[\"DNSPOD_ID\"]\n DNSPOD_API_KEY = os.environ[\"DNSPOD_API_KEY\"]\n dns_class = DNSPodDns(DNSPOD_ID, DNSPOD_API_KEY, **provider_kwargs)\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"duckdns\":\n from .dns_providers.duckdns import DuckDNSDns\n\n try:\n duckdns_token = os.environ[\"DUCKDNS_TOKEN\"]\n dns_class = DuckDNSDns(duckdns_token=duckdns_token, **provider_kwargs)\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"cloudns\":\n from .dns_providers.cloudns import ClouDNSDns\n\n try:\n dns_class = ClouDNSDns(**provider_kwargs)\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"powerdns\":\n from .dns_providers.powerdns import PowerDNSDns\n\n try:\n powerdns_api_key = os.environ[\"POWERDNS_API_KEY\"]\n powerdns_api_url = os.environ[\"POWERDNS_API_URL\"]\n dns_class = PowerDNSDns(powerdns_api_key, powerdns_api_url, **provider_kwargs)\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"gandi\":\n from .dns_providers.gandi import GandiDns\n\n try:\n gandi_api_key = os.environ[\"GANDI_API_KEY\"]\n dns_class = GandiDns(GANDI_API_KEY=gandi_api_key, **provider_kwargs)\n except KeyError as e:\n logger.error(\"ERROR:: Please supply {0} as an environment variable.\".format(str(e)))\n raise\n\n elif provider_name == \"unbound_ssh\":\n from .dns_providers.unbound_ssh import UnboundSsh\n\n # check & report, let calling protocol crash it.\n if \"ssh_des\" not in provider_kwargs:\n logger.error(\"ERROR: unbound_ssh REQUIRES ssh_des option.\")\n dns_class = UnboundSsh(**provider_kwargs) # pylint: disable=E1125\n\n elif provider_name == \"route53\":\n raise ValueError(\"route53 driver can only be used programmatically at this time, sorry\")\n\n else:\n raise ValueError(\"The dns provider {0} is not recognised.\".format(provider_name))\n\n logger.info(\"Using %s as registered provider.\", provider_name)\n return dns_class\n\n\ndef main():\n \"See docs/sewer-cli.md for docs & examples\"\n\n catalog = ProviderCatalog()\n\n parser = setup_parser(catalog)\n args = parser.parse_args()\n\n loglevel = args.loglevel\n logger = lib.create_logger(None, loglevel)\n\n provider_name = args.provider\n domain = args.domain\n alt_domains = args.alt_domains\n if args.action != \"none\":\n logger.warning(\"DEPRECATION WARNING: --action option is obsolete and will be dropped soon\")\n bundle_name = args.bundle_name\n endpoint = args.endpoint\n email = args.email\n out_dir = args.out_dir\n\n ### FIX ME ### to keep special options --domain_alias & --prop-*, or use -p_opts instead?\n\n provider_kwargs = {}\n\n for p in args.p_opts:\n parts = p.split(\"=\")\n if len(parts) == 2:\n provider_kwargs[parts[0]] = parts[1]\n\n # Make sure the output dir user specified is writable\n if not os.access(out_dir, os.W_OK):\n raise OSError(\"The dir '{0}' is not writable\".format(out_dir))\n\n if args.acct_key_file:\n account = AcmeAccount.from_pem(args.acct_key_file.read())\n is_new_acct = args.is_new_acct\n else:\n account = AcmeAccount.create(args.acct_key_type)\n is_new_acct = True\n\n if args.cert_key_file:\n cert_key = AcmeKey.from_pem(args.cert_key.read())\n else:\n cert_key = AcmeKey.create(args.cert_key_type)\n\n if bundle_name:\n file_name = bundle_name\n else:\n file_name = \"{0}\".format(domain)\n\n if endpoint == \"staging\":\n ACME_DIRECTORY_URL = config.ACME_DIRECTORY_URL_STAGING\n else:\n ACME_DIRECTORY_URL = config.ACME_DIRECTORY_URL_PRODUCTION\n\n dns_class = get_provider(provider_name, provider_kwargs, catalog, logger)\n\n acme_client = client.Client(\n provider=dns_class,\n domain_name=domain,\n domain_alt_names=alt_domains,\n contact_email=email,\n account=account,\n is_new_acct=is_new_acct,\n cert_key=cert_key,\n ACME_DIRECTORY_URL=ACME_DIRECTORY_URL,\n LOG_LEVEL=loglevel,\n ACME_REQUEST_TIMEOUT=args.acme_timeout,\n )\n\n # prepare file path\n account_key_file_path = os.path.join(out_dir, \"{0}.account.key\".format(file_name))\n crt_file_path = os.path.join(out_dir, \"{0}.crt\".format(file_name))\n crt_key_file_path = os.path.join(out_dir, \"{0}.key\".format(file_name))\n\n # write out account_key in out_dir directory\n account.write_pem(account_key_file_path)\n logger.info(\"account key succesfully written to {0}.\".format(account_key_file_path))\n\n certificate = acme_client.get_certificate()\n\n # write out certificate and certificate key in out_dir directory\n with open(crt_file_path, \"w\") as certificate_file:\n certificate_file.write(certificate)\n logger.info(\"certificate succesfully written to {0}.\".format(crt_file_path))\n\n cert_key.write_pem(crt_key_file_path)\n logger.info(\"certificate key succesfully written to {0}.\".format(crt_key_file_path))\n" }, { "path": "sewer/client.py", "content": "import json, time, platform\nfrom hashlib import sha256\nfrom typing import Dict, Sequence, Tuple, Union\n\nimport requests\n\nfrom .auth import ChalListType, ErrataListType, ProviderBase\nfrom .config import ACME_DIRECTORY_URL_PRODUCTION\nfrom .crypto import AcmeCsr, AcmeKey, AcmeAccount\nfrom .lib import create_logger, log_response, safe_base64, sewer_meta, AcmeRegistrationError\n\n\nclass Client:\n \"\"\"\n refer to docs/sewer-as-a-library for usage, etc.\n \"\"\"\n\n def __init__(\n self,\n *,\n domain_name: str,\n account: AcmeAccount,\n cert_key: AcmeKey,\n is_new_acct=False,\n dns_class: ProviderBase = None,\n domain_alt_names: Sequence[str] = None,\n contact_email: str = None,\n provider: ProviderBase = None,\n ACME_REQUEST_TIMEOUT: int = 7,\n ACME_AUTH_STATUS_WAIT_PERIOD: int = 8,\n ACME_AUTH_STATUS_MAX_CHECKS: int = 3,\n ACME_DIRECTORY_URL: str = ACME_DIRECTORY_URL_PRODUCTION,\n ACME_VERIFY: bool = True,\n LOG_LEVEL: str = \"INFO\",\n ):\n\n ### do some type checking of some parameters\n\n ### FIX ME ### spotty and not always complete; some should raise TypeError, not ValueError\n\n if not isinstance(domain_alt_names, (type(None), list)):\n raise ValueError(\n \"domain_alt_names should be None or a list of strings, not %s\" % domain_alt_names\n )\n\n if not isinstance(contact_email, (type(None), str)):\n raise ValueError(\"contact_email should be None or a string, not %s\" % contact_email)\n\n if LOG_LEVEL.upper() not in [\"DEBUG\", \"INFO\", \"WARNING\", \"ERROR\", \"CRITICAL\"]:\n raise ValueError(\n \"LOG_LEVEL must be one of 'DEBUG', 'INFO', 'WARNING', 'ERROR' or 'CRITICAL'\"\n )\n\n if dns_class is not None and provider is not None:\n raise ValueError(\n \"Client was passed both the DEPRECATED dns_class argument and provider.\"\n )\n\n if not isinstance(account, AcmeAccount):\n raise TypeError(\"The account argument must be an AcmeAccount.\")\n\n if not isinstance(cert_key, AcmeKey):\n raise TypeError(\"The argument cert_key must be an AcmeKey.\")\n\n ### setup Client's global variables\n\n self.domain_name = domain_name\n\n # long winded is both stricter check as well as giving mypy a clear enough hint\n if isinstance(provider, ProviderBase):\n self.provider = provider\n elif isinstance(dns_class, ProviderBase):\n self.provider = dns_class\n\n if not domain_alt_names:\n domain_alt_names = []\n self.domain_alt_names = list(set(domain_alt_names))\n self.contact_email = contact_email\n self.ACME_REQUEST_TIMEOUT = ACME_REQUEST_TIMEOUT\n self.ACME_AUTH_STATUS_WAIT_PERIOD = ACME_AUTH_STATUS_WAIT_PERIOD\n self.ACME_AUTH_STATUS_MAX_CHECKS = ACME_AUTH_STATUS_MAX_CHECKS\n self.ACME_DIRECTORY_URL = ACME_DIRECTORY_URL\n self.ACME_VERIFY = ACME_VERIFY\n self.LOG_LEVEL = LOG_LEVEL.upper()\n\n self.account = account\n self.cert_key = cert_key\n self.is_new_acct = is_new_acct\n\n self.logger = create_logger(__name__, LOG_LEVEL)\n\n try:\n self.all_domain_names = [self.domain_name] + self.domain_alt_names\n self.User_Agent = self.get_user_agent()\n acme_endpoints = self.get_acme_endpoints().json()\n self.ACME_GET_NONCE_URL = acme_endpoints[\"newNonce\"]\n self.ACME_TOS_URL = acme_endpoints[\"meta\"][\"termsOfService\"]\n self.ACME_KEY_CHANGE_URL = acme_endpoints[\"keyChange\"]\n self.ACME_NEW_ACCOUNT_URL = acme_endpoints[\"newAccount\"]\n self.ACME_NEW_ORDER_URL = acme_endpoints[\"newOrder\"]\n self.ACME_REVOKE_CERT_URL = acme_endpoints[\"revokeCert\"]\n\n self.acme_csr = AcmeCsr(cn=domain_name, san=domain_alt_names, key=self.cert_key)\n\n if dns_class is not None:\n self.logger.warning(\n \"DEPRECATED parameter 'dns_class' will be removed in 0.9; use 'provider' instead\"\n )\n\n self.logger.info(\n \"intialise_success, sewer_version={0}, domain_names={1}, acme_server={2}\".format(\n sewer_meta(\"version\"),\n self.all_domain_names,\n self.ACME_DIRECTORY_URL[:20] + \"...\",\n )\n )\n\n ### FIX ME ### [:100] is bandaid to reduce spew during tests\n\n except Exception as e:\n self.logger.error(\"Unable to intialise Client. error={0}\".format(str(e)[:100]))\n raise e\n\n def GET(self, url: str) -> requests.Response:\n \"\"\"\n wrap requests.get (and post and head, below) to allow:\n * injection of e.g. UserAgent header in one place rather than all over\n * hides requests itself to allow for change (unlikely) or use of Session\n * paves the way to inject the verify option, required to use pebble\n \"\"\"\n\n return self._request(\"GET\", url)\n\n # HEAD is still waiting for the test rewrite to let it be used... very low priority :-(\n def HEAD(self, url: str) -> requests.Response:\n return self._request(\"HEAD\", url)\n\n def POST(\n self, url: str, *, data: bytes = None, headers: Dict[str, str] = None\n ) -> requests.Response:\n return self._request(\"POST\", url, data=data, headers=headers)\n\n def _request(\n self, method: str, url: str, *, data: bytes = None, headers: Dict[str, str] = None\n ) -> requests.Response:\n \"\"\"\n shared implementation for GET, POST and HEAD\n * injects standard request options unless they are already given in headers\n * header:UserAgent, timeout\n * verify - this is a hack to make sewer accept pebble's intentionally bogus cert\n \"\"\"\n\n if headers is None:\n headers = {}\n\n if \"UserAgent\" not in headers:\n headers[\"UserAgent\"] = self.User_Agent\n\n kwargs = {\"timeout\": self.ACME_REQUEST_TIMEOUT} # type: Dict[str, Union[str, int]]\n\n ### FIX ME ### can get current bogus cert from pebble, figure out how to use it here?\n\n # if ACME_VERIFY is false, disable certificate check in request\n if not self.ACME_VERIFY:\n kwargs[\"verify\"] = False\n\n # this is what we'd do if damn near every test didn't mock requests.{get,post}\n # response = requests.request(method, url, headers=headers, **kwargs)\n\n # awkward implementation to maintain compatibility with current mocked tests\n if method == \"GET\":\n # mypy seems to be confused if params isn't explicitly passed, wtf?\n response = requests.get(url, params=None, headers=headers, **kwargs)\n elif method == \"HEAD\":\n response = requests.head(url, headers=headers, **kwargs)\n elif method == \"POST\":\n response = requests.post(url, data, headers=headers, **kwargs)\n\n return response\n\n @staticmethod\n def get_user_agent():\n return \"python-requests/{requests_version} ({system}: {machine}) sewer {sewer_version} ({sewer_url})\".format(\n requests_version=requests.__version__,\n system=platform.system(),\n machine=platform.machine(),\n sewer_version=sewer_meta(\"version\"),\n sewer_url=sewer_meta(\"url\"),\n )\n\n def get_acme_endpoints(self):\n self.logger.debug(\"get_acme_endpoints\")\n get_acme_endpoints = self.GET(self.ACME_DIRECTORY_URL)\n self.logger.debug(\n \"get_acme_endpoints_response. status_code={0}\".format(get_acme_endpoints.status_code)\n )\n if get_acme_endpoints.status_code not in [200, 201]:\n raise ValueError(\n \"Error while getting Acme endpoints: status_code={status_code} response={response}\".format(\n status_code=get_acme_endpoints.status_code,\n response=log_response(get_acme_endpoints),\n )\n )\n return get_acme_endpoints\n\n ### FIX ME ### this is a kludge to fix Alec's needs until there's time to do the Acme* refactor\n\n def acme_register(self):\n\n self.logger.info(\"acme_register%s\" % \" (is new account)\" if self.is_new_acct else \"\")\n\n if self.account.has_kid():\n self.logger.info(\"acme_register: key was already registered\")\n return None\n\n if not self.is_new_acct:\n payload = {\"onlyReturnExisting\": True}\n elif self.contact_email:\n payload = {\n \"termsOfServiceAgreed\": True,\n \"contact\": [\"mailto:{0}\".format(self.contact_email)],\n }\n else:\n payload = {\"termsOfServiceAgreed\": True}\n\n url = self.ACME_NEW_ACCOUNT_URL\n response = self.make_signed_acme_request(\n url=url, payload=json.dumps(payload), needs_jwk=True\n )\n self.logger.debug(\n \"response. status_code={0}. response={1}\".format(\n response.status_code, log_response(response)\n )\n )\n\n if response.status_code not in [201, 200, 409]:\n raise AcmeRegistrationError(\n \"Error while registering: status_code={status_code} response={response}\".format(\n status_code=response.status_code, response=log_response(response),\n )\n )\n\n self.account.set_kid(response.headers[\"Location\"])\n\n self.logger.info(\"acme_register_success\")\n return response\n\n def apply_for_cert_issuance(self):\n \"\"\"\n https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.4\n The order object returned by the server represents a promise that if\n the client fulfills the server's requirements before the \"expires\"\n time, then the server will be willing to finalize the order upon\n request and issue the requested certificate. In the order object,\n any authorization referenced in the \"authorizations\" array whose\n status is \"pending\" represents an authorization transaction that the\n client must complete before the server will issue the certificate.\n\n Once the client believes it has fulfilled the server's requirements,\n it should send a POST request to the order resource's finalize URL.\n The POST body MUST include a CSR:\n\n The date values seem to be ignored by LetsEncrypt although they are\n in the ACME draft spec; https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.4\n \"\"\"\n self.logger.info(\"apply_for_cert_issuance (newOrder)\")\n identifiers = []\n for domain_name in self.all_domain_names:\n identifiers.append({\"type\": \"dns\", \"value\": domain_name})\n\n payload = {\"identifiers\": identifiers}\n url = self.ACME_NEW_ORDER_URL\n apply_for_cert_issuance_response = self.make_signed_acme_request(\n url=url, payload=json.dumps(payload)\n )\n self.logger.debug(\n \"apply_for_cert_issuance_response. status_code={0}. response={1}\".format(\n apply_for_cert_issuance_response.status_code,\n log_response(apply_for_cert_issuance_response),\n )\n )\n\n if apply_for_cert_issuance_response.status_code != 201:\n raise ValueError(\n \"Error applying for certificate issuance: status_code={status_code} response={response}\".format(\n status_code=apply_for_cert_issuance_response.status_code,\n response=log_response(apply_for_cert_issuance_response),\n )\n )\n\n apply_for_cert_issuance_response_json = apply_for_cert_issuance_response.json()\n finalize_url = apply_for_cert_issuance_response_json[\"finalize\"]\n authorizations = apply_for_cert_issuance_response_json[\"authorizations\"]\n\n self.logger.info(\"apply_for_cert_issuance_success\")\n return authorizations, finalize_url\n\n def get_identifier_authorization(self, auth_url: str) -> Dict[str, str]:\n \"\"\"\n https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.5\n When a client receives an order from the server it downloads the\n authorization resources by sending GET requests to the indicated\n URLs. If the client initiates authorization using a request to the\n new authorization resource, it will have already received the pending\n authorization object in the response to that request.\n\n This is also where we get the challenges/tokens.\n \"\"\"\n self.logger.info(\"get_identifier_authorization for %s\" % auth_url)\n response = self.make_signed_acme_request(auth_url, payload=\"\")\n\n self.logger.debug(\n \"get_identifier_authorization_response. status_code={0}. response={1}\".format(\n response.status_code, log_response(response)\n )\n )\n if response.status_code not in [200, 201]:\n raise ValueError(\n \"Error getting identifier authorization: status_code={status_code} response={response}\".format(\n status_code=response.status_code, response=log_response(response)\n )\n )\n response_json = response.json()\n domain = response_json[\"identifier\"][\"value\"]\n wildcard = response_json.get(\"wildcard\")\n\n for i in response_json[\"challenges\"]:\n if i[\"type\"] in self.provider.chal_types:\n challenge = i\n challenge_token = challenge[\"token\"]\n challenge_url = challenge[\"url\"]\n\n identifier_auth = {\n \"domain\": domain,\n \"url\": auth_url,\n \"wildcard\": wildcard,\n \"token\": challenge_token,\n \"challenge_url\": challenge_url,\n }\n\n self.logger.debug(\n \"get_identifier_authorization_success. identifier_auth={0}\".format(identifier_auth)\n )\n self.logger.info(\n \"get_identifier_authorization got %s, token=%s\" % (challenge_url, challenge_token)\n )\n return identifier_auth\n\n def get_keyauthorization(self, token):\n self.logger.debug(\"get_keyauthorization\")\n acme_header_jwk_json = json.dumps(self.account.jwk(), sort_keys=True, separators=(\",\", \":\"))\n acme_thumbprint = safe_base64(sha256(acme_header_jwk_json.encode(\"utf8\")).digest())\n acme_keyauthorization = \"{0}.{1}\".format(token, acme_thumbprint)\n\n return acme_keyauthorization\n\n def check_authorization_status(self, authorization_url, desired_status=None):\n \"\"\"\n https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.5.1\n To check on the status of an authorization, the client sends a GET(polling)\n request to the authorization URL, and the server responds with the\n current authorization object.\n\n https://tools.ietf.org/html/draft-ietf-acme-acme#section-8.2\n Clients SHOULD NOT respond to challenges until they believe that the\n server's queries will succeed. If a server's initial validation\n query fails, the server SHOULD retry[intended to address things like propagation delays in\n HTTP/DNS provisioning] the query after some time.\n The server MUST provide information about its retry state to the\n client via the \"errors\" field in the challenge and the Retry-After\n \"\"\"\n self.logger.debug(\"check_authorization_status\")\n desired_status = desired_status or [\"pending\", \"valid\"]\n number_of_checks = 0\n while True:\n time.sleep(self.ACME_AUTH_STATUS_WAIT_PERIOD)\n response = self.make_signed_acme_request(authorization_url, payload=\"\")\n authorization_status = response.json()[\"status\"]\n number_of_checks = number_of_checks + 1\n self.logger.debug(\n \"response. status_code={0}. response={1}\".format(\n response.status_code, log_response(response),\n )\n )\n if authorization_status in desired_status:\n break\n if number_of_checks == self.ACME_AUTH_STATUS_MAX_CHECKS:\n raise StopIteration(\n \"Checks done={0}. Max checks allowed={1}. Interval between checks={2}seconds.\".format(\n number_of_checks,\n self.ACME_AUTH_STATUS_MAX_CHECKS,\n self.ACME_AUTH_STATUS_WAIT_PERIOD,\n )\n )\n\n self.logger.debug(\"check_authorization_status_success\")\n return response\n\n def respond_to_challenge(self, acme_keyauthorization, challenge_url):\n \"\"\"\n https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.5.1\n To prove control of the identifier and receive authorization, the\n client needs to respond with information to complete the challenges.\n The server is said to \"finalize\" the authorization when it has\n completed one of the validations, by assigning the authorization a\n status of \"valid\" or \"invalid\".\n\n Usually, the validation process will take some time, so the client\n will need to poll the authorization resource to see when it is finalized.\n To check on the status of an authorization, the client sends a GET(polling)\n request to the authorization URL, and the server responds with the\n current authorization object.\n \"\"\"\n self.logger.info(\n \"respond_to_challenge for %s at %s\" % (acme_keyauthorization, challenge_url)\n )\n payload = json.dumps({\"keyAuthorization\": \"{0}\".format(acme_keyauthorization)})\n respond_to_challenge_response = self.make_signed_acme_request(challenge_url, payload)\n self.logger.debug(\n \"respond_to_challenge_response. status_code={0}. response={1}\".format(\n respond_to_challenge_response.status_code,\n log_response(respond_to_challenge_response),\n )\n )\n\n self.logger.info(\"respond_to_challenge_success\")\n return respond_to_challenge_response\n\n def send_csr(self, finalize_url):\n \"\"\"\n https://tools.ietf.org/html/draft-ietf-acme-acme#section-7.4\n Once the client believes it has fulfilled the server's requirements,\n it should send a POST request(include a CSR) to the order resource's finalize URL.\n A request to finalize an order will result in error if the order indicated does not have status \"pending\",\n if the CSR and order identifiers differ, or if the account is not authorized for the identifiers indicated in the CSR.\n The CSR is sent in the base64url-encoded version of the DER format(OpenSSL.crypto.FILETYPE_ASN1)\n\n A valid request to finalize an order will return the order to be finalized.\n The client should begin polling the order by sending a\n GET request to the order resource to obtain its current state.\n \"\"\"\n self.logger.info(\"send_csr\")\n payload = {\"csr\": safe_base64(self.acme_csr.public_bytes())}\n send_csr_response = self.make_signed_acme_request(\n url=finalize_url, payload=json.dumps(payload)\n )\n self.logger.debug(\n \"send_csr_response. status_code={0}. response={1}\".format(\n send_csr_response.status_code, log_response(send_csr_response)\n )\n )\n\n if send_csr_response.status_code not in [200, 201]:\n raise ValueError(\n \"Error sending csr: status_code={status_code} response={response}\".format(\n status_code=send_csr_response.status_code,\n response=log_response(send_csr_response),\n )\n )\n send_csr_response_json = send_csr_response.json()\n certificate_url = send_csr_response_json[\"certificate\"]\n\n self.logger.info(\"send_csr_success\")\n return certificate_url\n\n def download_certificate(self, certificate_url: str) -> str:\n self.logger.info(\"download_certificate\")\n\n response = self.make_signed_acme_request(certificate_url, payload=\"\")\n self.logger.debug(\n \"download_certificate_response. status_code={0}. response={1}\".format(\n response.status_code, log_response(response)\n )\n )\n if response.status_code not in [200, 201]:\n raise ValueError(\n \"Error fetching signed certificate: status_code={status_code} response={response}\".format(\n status_code=response.status_code, response=log_response(response)\n )\n )\n pem_certificate = response.content.decode(\"utf-8\")\n self.logger.info(\"download_certificate_success\")\n return pem_certificate\n\n def get_nonce(self):\n \"\"\"\n https://tools.ietf.org/html/draft-ietf-acme-acme#section-6.4\n Each request to an ACME server must include a fresh unused nonce\n in order to protect against replay attacks.\n \"\"\"\n self.logger.debug(\"get_nonce\")\n response = self.GET(self.ACME_GET_NONCE_URL)\n nonce = response.headers[\"Replay-Nonce\"]\n return nonce\n\n def get_acme_header(self, url, needs_jwk=False):\n \"\"\"\n https://tools.ietf.org/html/draft-ietf-acme-acme#section-6.2\n The JWS Protected Header MUST include the following fields:\n - \"alg\" (Algorithm)\n - \"jwk\" (JSON Web Key, only for requests to new-account and revoke-cert resources)\n - \"kid\" (Key ID, for all other requests). gotten from self.ACME_NEW_ACCOUNT_URL\n - \"nonce\". gotten from self.ACME_GET_NONCE_URL\n - \"url\"\n \"\"\"\n self.logger.debug(\"get_acme_header\")\n header = {\"alg\": self.account.key_desc.alg, \"nonce\": self.get_nonce(), \"url\": url}\n\n if needs_jwk:\n header[\"jwk\"] = self.account.jwk()\n else:\n header[\"kid\"] = self.account.kid\n\n return header\n\n def make_signed_acme_request(self, url, payload, needs_jwk=False):\n self.logger.debug(\"make_signed_acme_request\")\n headers = {}\n payload64 = safe_base64(payload)\n protected = self.get_acme_header(url, needs_jwk)\n protected64 = safe_base64(json.dumps(protected))\n message = (\"%s.%s\" % (protected64, payload64)).encode(\"utf-8\")\n # signature = self.sign_message(message=\"{0}.{1}\".format(protected64, payload64)) # bytes\n # signature64 = safe_base64(signature) # str\n signature64 = safe_base64(self.account.sign_message(message))\n data = json.dumps(\n {\"protected\": protected64, \"payload\": payload64, \"signature\": signature64}\n )\n headers.update({\"Content-Type\": \"application/jose+json\"})\n response = self.POST(url, data=data.encode(\"utf8\"), headers=headers)\n return response\n\n def get_certificate(self):\n self.logger.debug(\"get_certificate\")\n challenges = []\n\n try:\n self.acme_register()\n authorizations, finalize_url = self.apply_for_cert_issuance()\n\n for auth_url in authorizations:\n identifier_auth = self.get_identifier_authorization(auth_url)\n token = identifier_auth[\"token\"]\n challenge = {\n \"ident_value\": identifier_auth[\"domain\"],\n \"token\": token,\n \"key_auth\": self.get_keyauthorization(token), # responder acme_keyauth..\n \"wildcard\": identifier_auth[\"wildcard\"],\n \"auth_url\": auth_url, # responder auth.._url\n \"chal_url\": identifier_auth[\"challenge_url\"], # responder challenge_url\n }\n challenges.append(challenge)\n\n # any errors in setup are fatal (here - they are all necessary for same cert)\n failures = self.provider.setup(challenges)\n if failures:\n raise RuntimeError(\"get_certificate: challenge setup failed for %s\" % failures)\n\n ### FIX ME ### should abort cert and try to clear on error\n\n error, errata_list = self.propagation_delay(challenges)\n\n # for a case where you want certificates for *.example.com and example.com\n # you have to create both auth records AND then respond to the challenge.\n # see issues/83\n for chal in challenges:\n # Make sure the authorization is in a status where we can submit a challenge\n # response. The authorization can be in the \"valid\" state before submitting\n # a challenge response if there was a previous authorization for these hosts\n # that was successfully validated, still cached by the server.\n auth_status_response = self.check_authorization_status(chal[\"auth_url\"])\n if auth_status_response.json()[\"status\"] == \"pending\":\n self.respond_to_challenge(chal[\"key_auth\"], chal[\"chal_url\"])\n\n ### TO DO ### this is the obfuscated timeout loop. Clean this mess up!\n ### # # # ### it also keeps trying even when the auth is failed :-(\n\n ### FIX? ### shouldn't this be checking the ORDER's status for completion?\n # that is at least the most frugal of queries approach...\n\n for chal in challenges:\n # Before sending a CSR, we need to make sure the server has completed the\n # validation for all the authorizations\n self.check_authorization_status(chal[\"auth_url\"], [\"valid\"])\n\n certificate_url = self.send_csr(finalize_url)\n certificate = self.download_certificate(certificate_url)\n\n ### FIX ME ### [:100] is a bandaid to reduce spew during tests\n\n except Exception as e:\n self.logger.error(\"Error: Unable to issue certificate. error={0}\".format(str(e)[:100]))\n raise e\n finally:\n # best-effort attempt to clear challenges\n failures = self.provider.clear(challenges)\n\n return certificate\n\n def sleep_iter(self):\n \"returns values from list, then repeats last value forever\"\n\n for cur_time in self.provider.prop_sleep_times:\n yield cur_time\n while True:\n yield cur_time\n\n def propagation_delay(self, challenges: ChalListType) -> Tuple[str, ErrataListType]:\n \"\"\"\n Wait for the challenges to propagate through the service.\n\n Returns (error: str, errata_list)\n * (\"\", []) is complete success\n * (\"timeout\", [...]) list contains challenges that weren't ready\n * (\"failure\", [...]) list contains both failed and not-yet-ready challenges\n\n See docs/unpropagated.md for the details.\n \"\"\"\n\n if self.provider.prop_delay:\n time.sleep(self.provider.prop_delay)\n\n if self.provider.prop_timeout:\n unready = challenges\n end_time = time.time() + self.provider.prop_timeout\n sleep_time = self.sleep_iter()\n num_checks = 0\n\n while unready:\n errata = self.provider.unpropagated(unready)\n num_checks += 1\n\n # right idea, but details aren't yet nailed down?\n # failed = [e for e in errata if e['status'].startswith(\"FAIL\")]\n\n if errata:\n poll_time = time.time()\n # intentional: do an \"extra\" check rather than running short\n if end_time < poll_time:\n break\n # wait a while to let more propagation happen\n time.sleep(next(sleep_time))\n\n unready = [err[2] for err in errata]\n\n if unready:\n ### FIX ME ### might be good for mock tests, but really should try to clear, eh?\n # return (\"timeout\", unready)\n raise RuntimeError(\n \"propagation_delay: time out after %s probes: %s\" % (num_checks, unready)\n )\n\n return (\"\", [])\n\n def cert(self):\n self.logger.warning(\"DEPRECATED: Client.cert is deprecated as of 0.8.4\")\n return self.get_certificate()\n\n def renew(self):\n self.logger.warning(\"DEPRECATED: Client.renew is deprecated as of 0.8.4\")\n return self.cert()\n" }, { "path": "sewer/config.py", "content": "ACME_DIRECTORY_URL_STAGING = \"https://acme-staging-v02.api.letsencrypt.org/directory\"\nACME_DIRECTORY_URL_PRODUCTION = \"https://acme-v02.api.letsencrypt.org/directory\"\n" }, { "path": "sewer/crypto.py", "content": "import time\n\nfrom cryptography import x509\nfrom cryptography.x509.oid import NameOID\nfrom cryptography.hazmat.primitives import hashes\nfrom cryptography.hazmat.primitives.asymmetric import ec, padding, rsa, utils\nfrom cryptography.hazmat.primitives.serialization import (\n load_pem_private_key,\n Encoding,\n NoEncryption,\n PrivateFormat,\n)\nfrom cryptography.hazmat.backends import default_backend, openssl\n\nfrom typing import Any, Callable, cast, Dict, List, Optional, Tuple, Type, Union\n\nfrom .lib import AcmeError, safe_base64\n\n\nclass AcmeAbstractError(AcmeError):\n pass\n\n\n### Exceptions specific to ACME crypto operations\n\n\nclass AcmeKeyError(AcmeError):\n pass\n\n\nclass AcmeKeyTypeError(AcmeError):\n pass\n\n\nclass AcmeKidError(AcmeKeyError):\n pass\n\n\n### types for things defined here\n\n### FIX ME ### is there any way to eliminate the repetition here? cryptography classes...\n\nprivate_key_types = (openssl.rsa._RSAPrivateKey, openssl.ec._EllipticCurvePrivateKey)\nPrivateKeyType = Union[openssl.rsa._RSAPrivateKey, openssl.ec._EllipticCurvePrivateKey]\n\n\n### low level key type table\n\n\nclass KeyDesc:\n\n pk_type: PrivateKeyType\n\n def __init__(\n self,\n key_size: int,\n type_name: str,\n jwk_const: Dict[str, str],\n jwk_attr: Dict[str, str],\n alg: str,\n key_bytes: int,\n ) -> None:\n self.key_size = key_size\n self.type_name = type_name\n self.jwk_const = jwk_const\n self.jwk_attr = jwk_attr\n self.alg = alg\n self.key_bytes = key_bytes\n\n def generate(self) -> PrivateKeyType:\n raise AcmeAbstractError(\"KeyDesc.generate\")\n\n def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:\n raise AcmeAbstractError(\"KeyDesc.sign\")\n\n def match(self, pk: PrivateKeyType) -> bool:\n if isinstance(pk, self.pk_type) and pk.key_size == self.key_size:\n return True\n return False\n\n\nclass RsaKeyDesc(KeyDesc):\n\n pk_type = rsa.RSAPrivateKey\n\n def __init__(self, key_size: int) -> None:\n type_name = \"rsa%s\" % key_size\n super().__init__(key_size, type_name, {\"kty\": \"RSA\"}, {\"e\": \"e\", \"n\": \"n\"}, \"RS256\", 0)\n\n def generate(self) -> PrivateKeyType:\n return rsa.generate_private_key(65537, self.key_size, default_backend())\n\n def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:\n \"Yes, SHA256 is hardwired. As of Sep 2020, LE rejects other hashes for RSA\"\n\n return pk.sign(message, padding.PKCS1v15(), hashes.SHA256())\n\n\nclass EcKeyDesc(KeyDesc):\n\n pk_type = ec.EllipticCurvePrivateKey\n\n def __init__(self, key_size: int, hash_type, alg: str, key_bytes: int) -> None:\n name = \"secp%sr1\" % key_size\n curve = \"P-%s\" % key_size\n super().__init__(\n key_size, name, {\"kty\": \"EC\", \"crv\": curve}, {\"x\": \"x\", \"y\": \"y\"}, alg, key_bytes\n )\n self.curve = getattr(ec, name.upper())\n self.hash_type = hash_type\n\n def generate(self) -> PrivateKeyType:\n return ec.generate_private_key(self.curve, default_backend())\n\n def sign(self, pk: PrivateKeyType, message: bytes) -> bytes:\n # EC sign method returns ASN.1 encoded values for some inane reason\n r, s = utils.decode_dss_signature(pk.sign(message, ec.ECDSA(self.hash_type())))\n return r.to_bytes(self.key_bytes, \"big\") + s.to_bytes(self.key_bytes, \"big\")\n\n\nkey_table = [\n RsaKeyDesc(2048),\n RsaKeyDesc(3072),\n RsaKeyDesc(4096),\n EcKeyDesc(256, hashes.SHA256, \"ES256\", 32),\n EcKeyDesc(384, hashes.SHA384, \"ES384\", 48),\n # EcKeyDesc(521, hashes.SHA512, 64, \"ES512\", 66), this is where the key size != hash size?\n]\n\n# extract just the names for option choice lists, etc.\nkey_type_choices = [kd.type_name for kd in key_table]\n\n\ndef resolve_key_desc(key: Union[str, PrivateKeyType]) -> KeyDesc:\n \"\"\"\n Given a private key or a registered key type name, find the unique matching\n descriptor and return it.\n\n Raises exceptions if no match is found or if more than one matches (internal\n table error!).\n \"\"\"\n\n if isinstance(key, private_key_types):\n kdl = [kd for kd in key_table if kd.match(key)]\n kt = str(type(key))\n else:\n kdl = [kd for kd in key_table if kd.type_name == key]\n kt = key\n if not kdl:\n raise AcmeKeyTypeError(\"Unknown key type: %s\", kt)\n if len(kdl) != 1:\n raise AcmeKeyError(\"Internal error: key type %s matches %s entries!\" % (kt, len(kdl)))\n return kdl[0]\n\n\n### AcmeKey, finally!\n\n\nAcmeKeyType = Union[\"AcmeKey\", \"AcmeAccount\"]\n\n\nclass AcmeKey:\n \"\"\"\n AcmeKey is a parameterized wrapper around the private key type that are\n useful with ACME services. Key creation, loading and storing, message\n signing, and generating the key's JWK are all provided. Only key creation\n needs to be told the kind or size of key, and other differences in these\n operations are hidden away.\n\n See the key_table (or key_type_choices if you just want a list of the\n valid type names for the create method ( eg., sewer's cli program).\n\n These are based on what Let's Encrypt's servers accept as of Sep 2020:\n RSA with SHA256, P-256 with SHA256, and P-384 with SHA384. LE doubtless\n accepts many other key sizes than our simple list-based setup provides,\n but these are the ones sewer has actually tested (which is how we found\n out that RSA was SHA256 only, and P-521 wasn't available at all). Of\n course this can change, which is much of the reason for the table-driven\n approach used here.\n \"\"\"\n\n def __init__(self, pk: PrivateKeyType, key_desc: KeyDesc) -> None:\n self.pk = pk\n self.key_desc = key_desc\n #\n self._jwk: Optional[Dict[str, str]] = None\n\n ### Key Constructors\n\n @classmethod\n def create(cls: Type[\"AcmeKey\"], key_type_name: str) -> \"AcmeKey\":\n \"\"\"\n Factory method to create a new key of key_type, returned as an AcmeKey.\n \"\"\"\n\n kd = resolve_key_desc(key_type_name)\n return cls(kd.generate(), kd)\n\n @classmethod\n def from_pem(cls: Type[\"AcmeKey\"], pem_data: bytes) -> \"AcmeKey\":\n \"\"\"\n load a key from the PEM-format bytes, return an AcmeKey\n\n NB: since it's not stored in the PEM, the kid is empty (None)\n \"\"\"\n\n pk = load_pem_private_key(pem_data, None, default_backend())\n kd = resolve_key_desc(pk)\n\n return cls(pk, kd)\n\n @classmethod\n def read_pem(cls: Type[\"AcmeKey\"], filename: str) -> \"AcmeKey\":\n \"convenience method to load a PEM-format key; returns the AcmeKey\"\n\n with open(filename, \"rb\") as f:\n return cls.from_pem(f.read())\n\n ### shared methods\n\n def to_pem(self) -> bytes:\n \"return private key's serialized (PEM) form\"\n\n pem_data = self.pk.private_bytes(\n encoding=Encoding.PEM, format=PrivateFormat.PKCS8, encryption_algorithm=NoEncryption()\n )\n return pem_data\n\n def write_pem(self, filename: str) -> None:\n \"convenience method to write out the key in PEM form\"\n\n with open(filename, \"wb\") as f:\n f.write(self.to_pem())\n\n def sign_message(self, message: bytes) -> bytes:\n return self.key_desc.sign(self.pk, message)\n\n\n### An ACME account is identified by a key. When registered there is a Key ID as well.\n\n\nclass AcmeAccount(AcmeKey):\n \"\"\"\n Only an account key needs (or has) a Key ID associated with it.\n \"\"\"\n\n def __init__(self, pk: PrivateKeyType, key_desc: KeyDesc = None) -> None:\n if key_desc is None:\n key_desc = resolve_key_desc(pk)\n super().__init__(pk, key_desc)\n self.__kid: Optional[str] = None\n self._timestamp: Optional[float] = None\n self.__jwk: Optional[Dict[str, str]] = None\n\n ### kid's descriptor methods\n\n def get_kid(self) -> str:\n if self.__kid is None:\n raise AcmeKidError(\"Attempt to access a Key ID that hasn't been set. Register key?\")\n return self.__kid\n\n def set_kid(self, kid: str, timestamp: float = None) -> None:\n \"The kid can be set only once, but we overlook exact duplicate set calls\"\n\n if self.__kid and self.__kid != kid:\n raise AcmeKidError(\"Cannot alter a key's kid\")\n self.__kid = kid\n self._timestamp = timestamp if timestamp is not None else time.time()\n\n def del_kid(self) -> None:\n \"Doesn't actually del the hidden attribute, just resets the value to None (empty)\"\n self.__kid = None\n\n kid = property(get_kid, set_kid, del_kid)\n\n def has_kid(self) -> bool:\n \"need a non-exploding test for the presence of a Key ID\"\n return not self.__kid is None\n\n ### extend AcmeKey with new methods\n\n def jwk(self) -> Dict[str, str]:\n \"\"\"\n Returns the key's JWK as a dictionary\n\n CACHES result in _jwk\n \"\"\"\n\n if not self.__jwk:\n jwk = {}\n pubnums = self.pk.public_key().public_numbers()\n jwk.update(self.key_desc.jwk_const)\n for name, attr_name in self.key_desc.jwk_attr.items():\n val = getattr(pubnums, attr_name)\n numbytes = self.key_desc.key_bytes\n if numbytes == 0:\n numbytes = (val.bit_length() + 7) // 8\n jwk[name] = safe_base64(val.to_bytes(numbytes, \"big\"))\n self.__jwk = jwk\n return self.__jwk\n\n ### TODO ### store & load file format with kid, timestamp and pk.\n #\n # RFC7568 says that at least most implementations accept text outside the\n # BEGIN...END lines, especially in PKIX certificates. So I plan to do\n # something like this:\n #\n # KID: https://acme-v02.api.letsencrypt.org/acme/account/1a2b3c4d5e6f7g8h9i0j\n # Timestamp: 1600452956.446775\n # -----BEGIN PRIVATE KEY-----\n #\n # Both openssl pkey and the cryptography library can load a PEM decorated\n # like that. Only possible question is whether the '\\n' line ending needs\n # to be adjusted for non-Unix systems.\n\n def write_key(self, filename: str) -> None:\n \"Like write_pem but prepends the KID and timestamp if those are present\"\n\n with open(filename, \"wb\") as f:\n if self.__kid:\n f.write((\"KID: %s\\n\" % self.__kid).encode())\n if self._timestamp:\n f.write((\"Timestamp: %s\\n\" % self._timestamp).encode())\n f.write(self.to_pem())\n\n @classmethod\n def read_key(cls: Type[\"AcmeAccount\"], filename: str) -> \"AcmeAccount\":\n with open(filename, \"rb\") as f:\n data = f.read()\n prefix = b\"\"\n n = data.find(b\"-----BEGIN\")\n if 0 < n:\n prefix = data[:n]\n data = data[n:]\n acct = cast(\"AcmeAccount\", cls.from_pem(data))\n if prefix:\n parts = prefix.split(b\"\\n\")\n for p in parts:\n if p.startswith(b\"KID: \"):\n acct.__kid = p[5:].decode()\n elif p.startswith(b\"Timestamp: \"):\n acct._timestamp = float(p[11:])\n return acct\n\n\n### We also need to generate Certificate Signing Requests\n\n\nclass AcmeCsr:\n def __init__(self, *, cn: str, san: List[str], key: AcmeKey) -> None:\n \"\"\"\n temporary \"just like Client.create_csr\", more or less\n\n TODO: \"must staple\" extension; NOT elaborating subject name, since LE\n suggests that even CN may be replaced by a meaningless number in some\n vague future version of the server. I guess they're right that\n browsers ignore the CN already (aside from displaying it if asked).\n \"\"\"\n\n csrb = x509.CertificateSigningRequestBuilder()\n csrb = csrb.subject_name(x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, cn)]))\n all_names = list(set([cn] + san))\n SAN: List[x509.GeneralName] = [x509.DNSName(name) for name in all_names]\n csrb = csrb.add_extension(x509.SubjectAlternativeName(SAN), critical=False)\n self.csr = csrb.sign(key.pk, hashes.SHA256(), default_backend())\n\n def public_bytes(self) -> bytes:\n return self.csr.public_bytes(Encoding.DER)\n" }, { "path": "sewer/dns_providers/__init__.py", "content": "" }, { "path": "sewer/dns_providers/acmedns.py", "content": "import urllib.parse\n\nimport requests\n\nfrom dns.resolver import Resolver\n\nfrom . import common\nfrom ..lib import log_response\n\n\nclass AcmeDnsDns(common.BaseDns):\n def __init__(self, ACME_DNS_API_USER, ACME_DNS_API_KEY, ACME_DNS_API_BASE_URL, **kwargs):\n self.ACME_DNS_API_USER = ACME_DNS_API_USER\n self.ACME_DNS_API_KEY = ACME_DNS_API_KEY\n self.HTTP_TIMEOUT = 65 # seconds\n\n if ACME_DNS_API_BASE_URL[-1] != \"/\":\n self.ACME_DNS_API_BASE_URL = ACME_DNS_API_BASE_URL + \"/\"\n else:\n self.ACME_DNS_API_BASE_URL = ACME_DNS_API_BASE_URL\n super().__init__(**kwargs)\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"create_dns_record\")\n\n resolver = Resolver(configure=False)\n resolver.nameservers = [\"8.8.8.8\"]\n answer = resolver.query(\"_acme-challenge.{0}.\".format(domain_name), \"TXT\")\n subdomain, _ = str(answer.canonical_name).split(\".\", 1)\n\n url = urllib.parse.urljoin(self.ACME_DNS_API_BASE_URL, \"update\")\n headers = {\"X-Api-User\": self.ACME_DNS_API_USER, \"X-Api-Key\": self.ACME_DNS_API_KEY}\n body = {\"subdomain\": subdomain, \"txt\": domain_dns_value}\n update_acmedns_dns_record_response = requests.post(\n url, headers=headers, json=body, timeout=self.HTTP_TIMEOUT\n )\n self.logger.debug(\n \"update_acmedns_dns_record_response. status_code={0}. response={1}\".format(\n update_acmedns_dns_record_response.status_code,\n log_response(update_acmedns_dns_record_response),\n )\n )\n if update_acmedns_dns_record_response.status_code != 200:\n # raise error so that we do not continue to make calls to ACME\n # server\n raise ValueError(\n \"Error creating acme-dns dns record: status_code={status_code} response={response}\".format(\n status_code=update_acmedns_dns_record_response.status_code,\n response=log_response(update_acmedns_dns_record_response),\n )\n )\n self.logger.info(\"create_dns_record_end\")\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"delete_dns_record\")\n # acme-dns doesn't support this\n self.logger.info(\"delete_dns_record_success\")\n" }, { "path": "sewer/dns_providers/aliyundns.py", "content": "import json\n\nfrom aliyunsdkcore import client # type: ignore\nimport aliyunsdkalidns.request.v20150109 # type:ignore\nfrom aliyunsdkalidns.request.v20150109 import (\n DescribeDomainRecordsRequest,\n AddDomainRecordRequest,\n DeleteDomainRecordRequest,\n)\n\nfrom . import common\n\n\nclass _ResponseForAliyun(object):\n \"\"\"\n wrapper aliyun resp to the format sewer wanted.\n \"\"\"\n\n def __init__(self, status_code=200, content=None, headers=None):\n self.status_code = status_code\n self.headers = headers or {}\n self.content = content or {}\n self.content = json.dumps(content)\n super(_ResponseForAliyun, self).__init__()\n\n def json(self):\n return json.loads(self.content)\n\n\nclass AliyunDns(common.BaseDns):\n def __init__(self, key, secret, endpoint=\"cn-beijing\", debug=False, **kwargs):\n \"\"\"\n aliyun dns client\n :param str key: access key\n :param str secret: access sceret\n :param str endpoint: endpoint\n :param bool debug: if debug?\n \"\"\"\n super().__init__(**kwargs)\n self._key = key\n self._secret = secret\n self._endpoint = endpoint\n self._debug = debug\n self.clt = client.AcsClient(self._key, self._secret, self._endpoint, debug=self._debug)\n\n def _send_reqeust(self, request):\n \"\"\"\n send request to aliyun\n \"\"\"\n request.set_accept_format(\"json\")\n try:\n status, headers, result = self.clt.implementation_of_do_action(request)\n if isinstance(result, bytes):\n result = result.decode()\n result = json.loads(result)\n if \"Message\" in result or \"Code\" in result:\n result[\"Success\"] = False\n self.logger.warning(\"aliyundns resp error: %s\", result)\n except Exception as exc:\n self.logger.warning(\"aliyundns failed to send request: %s, %s\", str(exc), request)\n status, headers, result = 502, {}, b'{\"Success\": false}'\n result = json.loads(result)\n\n if self._debug:\n self.logger.info(\"aliyundns request name: %s\", request.__class__.__name__)\n self.logger.info(\"aliyundns request query: %s\", request.get_query_params())\n return _ResponseForAliyun(status, result, headers)\n\n def query_recored_items(self, host, zone=None, tipe=None, page=1, psize=200):\n \"\"\"\n query recored items.\n :param str host: like example.com\n :param str zone: like menduo.example.com\n :param str tipe: TXT, CNAME, IP or other\n :param int page:\n :param int psize:\n :return dict: res = {\n 'DomainRecords':\n {'Record': [\n {\n 'DomainName': 'menduo.net',\n 'Line': 'default',\n 'Locked': False,\n 'RR': 'zb',\n 'RecordId': '3989515483698964',\n 'Status': 'ENABLE',\n 'TTL': 600,\n 'Type': 'A',\n 'Value': '127.0.0.1',\n 'Weight': 1\n },\n {\n 'DomainName': 'menduo.net',\n 'Line': 'default',\n 'Locked': False,\n 'RR': 'a.sub',\n 'RecordId': '3989515480778964',\n 'Status': 'ENABLE',\n 'TTL': 600,\n 'Type': 'CNAME',\n 'Value': 'h.p.menduo.net',\n 'Weight': 1\n }\n ]\n },\n 'PageNumber': 1,\n 'PageSize': 20,\n 'RequestId': 'FC4D02CD-EDCC-4EE8-942F-1497CCC3B10E',\n 'TotalCount': 95\n }\n \"\"\"\n request = DescribeDomainRecordsRequest.DescribeDomainRecordsRequest()\n request.get_action_name()\n request.set_DomainName(host)\n request.set_PageNumber(page)\n request.set_PageSize(psize)\n if zone:\n request.set_RRKeyWord(zone)\n if tipe:\n request.set_TypeKeyWord(tipe)\n resp = self._send_reqeust(request)\n body = resp.json()\n return body\n\n def query_recored_id(self, root, zone, tipe=\"TXT\"):\n \"\"\"\n find recored\n :param str root: root host, like example.com\n :param str zone: sub zone, like menduo.example.com\n :param str tipe: record tipe, TXT, CNAME, IP. we use TXT\n :return str:\n \"\"\"\n record_id = None\n recoreds = self.query_recored_items(root, zone, tipe=tipe)\n recored_list = recoreds.get(\"DomainRecords\", {}).get(\"Record\", [])\n recored_item_list = [i for i in recored_list if i[\"RR\"] == zone]\n if len(recored_item_list):\n record_id = recored_item_list[0][\"RecordId\"]\n return record_id\n\n @staticmethod\n def extract_zone(domain_name):\n \"\"\"\n extract domain to root, sub, acme_txt\n :param str domain_name: the value sewer client passed in, like *.menduo.example.com\n :return tuple: root, zone, acme_txt\n \"\"\"\n if domain_name.count(\".\") > 1:\n zone, middle, last = str(domain_name).rsplit(\".\", 2)\n root = \".\".join([middle, last])\n acme_txt = \"_acme-challenge.%s\" % zone\n else:\n zone = \"\"\n root = domain_name\n acme_txt = \"_acme-challenge\"\n return root, zone, acme_txt\n\n def create_dns_record(self, domain_name, domain_dns_value):\n \"\"\"\n create a dns record\n :param str domain_name: the value sewer client passed in, like *.menduo.example.com\n :param str domain_dns_value: the value sewer client passed in.\n :return _ResponseForAliyun:\n \"\"\"\n self.logger.info(\"create_dns_record start: %s\", (domain_name, domain_dns_value))\n root, _, acme_txt = self.extract_zone(domain_name)\n\n request = AddDomainRecordRequest.AddDomainRecordRequest()\n request.set_DomainName(root)\n request.set_TTL(600)\n request.set_RR(acme_txt)\n request.set_Type(\"TXT\")\n request.set_Value(domain_dns_value)\n resp = self._send_reqeust(request)\n\n self.logger.info(\"create_dns_record end: %s\", (domain_name, domain_dns_value, resp.json()))\n\n return resp\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n \"\"\"\n delete a txt record we created just now.\n :param str domain_name: the value sewer client passed in, like *.menduo.example.com\n :param str domain_dns_value: the value sewer client passed in. we do not use this.\n :return _ResponseForAliyun:\n :return:\n \"\"\"\n self.logger.info(\"delete_dns_record start: %s\", (domain_name, domain_dns_value))\n\n root, _, acme_txt = self.extract_zone(domain_name)\n\n record_id = self.query_recored_id(root, acme_txt)\n if not record_id:\n msg = \"failed to find record_id of domain: %s, value: %s\", domain_name, domain_dns_value\n self.logger.warning(msg)\n return\n\n self.logger.info(\"start to delete dns record, id: %s\", record_id)\n\n request = DeleteDomainRecordRequest.DeleteDomainRecordRequest()\n request.set_RecordId(record_id)\n resp = self._send_reqeust(request)\n\n self.logger.info(\"delete_dns_record end: %s\", (domain_name, domain_dns_value, resp.json()))\n return resp\n" }, { "path": "sewer/dns_providers/auroradns.py", "content": "# DNS Provider for AuroRa DNS from the dutch hosting provider pcextreme\n# https://www.pcextreme.nl/aurora/dns\n# Aurora uses libcloud from apache\n# https://libcloud.apache.org/\nimport tldextract\n\nfrom libcloud.dns.providers import get_driver # type: ignore\nfrom libcloud.dns.types import Provider, RecordType # type: ignore\n\nfrom . import common\n\n\nclass AuroraDns(common.BaseDns):\n \"\"\"\n Todo: re-organize this class so that we make it easier to mock things out to\n facilitate better tests.\n \"\"\"\n\n def __init__(self, AURORA_API_KEY, AURORA_SECRET_KEY, **kwargs):\n self.AURORA_API_KEY = AURORA_API_KEY\n self.AURORA_SECRET_KEY = AURORA_SECRET_KEY\n super().__init__(**kwargs)\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"create_dns_record\")\n\n extractedDomain = tldextract.extract(domain_name)\n domainSuffix = extractedDomain.domain + \".\" + extractedDomain.suffix\n\n if extractedDomain.subdomain == \"\":\n subDomain = \"_acme-challenge\"\n else:\n subDomain = \"_acme-challenge.\" + extractedDomain.subdomain\n\n cls = get_driver(Provider.AURORADNS)\n driver = cls(key=self.AURORA_API_KEY, secret=self.AURORA_SECRET_KEY)\n zone = driver.get_zone(domainSuffix)\n zone.create_record(name=subDomain, type=RecordType.TXT, data=domain_dns_value)\n\n self.logger.info(\"create_dns_record_success\")\n return\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"delete_dns_record\")\n\n extractedDomain = tldextract.extract(domain_name)\n domainSuffix = extractedDomain.domain + \".\" + extractedDomain.suffix\n\n if extractedDomain.subdomain == \"\":\n subDomain = \"_acme-challenge\"\n else:\n subDomain = \"_acme-challenge.\" + extractedDomain.subdomain\n\n cls = get_driver(Provider.AURORADNS)\n driver = cls(key=self.AURORA_API_KEY, secret=self.AURORA_SECRET_KEY)\n zone = driver.get_zone(domainSuffix)\n\n records = driver.list_records(zone)\n for x in records:\n if x.name == subDomain and x.type == \"TXT\":\n record_id = x.id\n self.logger.info(\n \"Found record \"\n + subDomain\n + \".\"\n + domainSuffix\n + \" with id : \"\n + record_id\n + \".\"\n )\n record = driver.get_record(zone_id=zone.id, record_id=record_id)\n driver.delete_record(record)\n self.logger.info(\n \"Deleted record \"\n + subDomain\n + \".\"\n + domainSuffix\n + \" with id : \"\n + record_id\n + \".\"\n )\n else:\n self.logger.info(\n \"Record \" + subDomain + \".\" + domainSuffix + \" not found. No record to delete.\"\n )\n\n self.logger.info(\"delete_dns_record_success\")\n return\n" }, { "path": "sewer/dns_providers/cloudflare.py", "content": "import urllib.parse\n\nimport requests\n\nfrom . import common\nfrom ..lib import log_response\n\n\nclass CloudFlareDns(common.BaseDns):\n def __init__(\n self,\n CLOUDFLARE_EMAIL=None,\n CLOUDFLARE_API_KEY=None,\n CLOUDFLARE_API_BASE_URL=\"https://api.cloudflare.com/client/v4/\",\n CLOUDFLARE_TOKEN=None,\n **kwargs,\n ):\n self.CLOUDFLARE_DNS_ZONE_ID = None\n self.CLOUDFLARE_EMAIL = CLOUDFLARE_EMAIL\n self.CLOUDFLARE_API_KEY = CLOUDFLARE_API_KEY\n self.CLOUDFLARE_API_BASE_URL = CLOUDFLARE_API_BASE_URL\n self.CLOUDFLARE_TOKEN = CLOUDFLARE_TOKEN\n self.HTTP_TIMEOUT = 65 # seconds\n\n if CLOUDFLARE_API_BASE_URL[-1] != \"/\":\n self.CLOUDFLARE_API_BASE_URL = CLOUDFLARE_API_BASE_URL + \"/\"\n else:\n self.CLOUDFLARE_API_BASE_URL = CLOUDFLARE_API_BASE_URL\n\n # Either only pass a token or only the email and API key\n if not (\n (CLOUDFLARE_TOKEN and not CLOUDFLARE_EMAIL and not CLOUDFLARE_API_KEY)\n or (not CLOUDFLARE_TOKEN and CLOUDFLARE_EMAIL and CLOUDFLARE_API_KEY)\n ):\n raise ValueError(\n \"Error initializing Cloudflare DNS adapter. Pass either email and API key or a token.\"\n )\n\n super().__init__(**kwargs)\n\n def find_dns_zone(self, domain_name):\n self.logger.debug(\"find_dns_zone\")\n url = urllib.parse.urljoin(self.CLOUDFLARE_API_BASE_URL, \"zones?status=active\")\n headers = self._get_auth_header()\n find_dns_zone_response = requests.get(url, headers=headers, timeout=self.HTTP_TIMEOUT)\n self.logger.debug(\n \"find_dns_zone_response. status_code={0}\".format(find_dns_zone_response.status_code)\n )\n if find_dns_zone_response.status_code != 200:\n raise ValueError(\n \"Error creating cloudflare dns record: status_code={status_code} response={response}\".format(\n status_code=find_dns_zone_response.status_code,\n response=log_response(find_dns_zone_response),\n )\n )\n\n result = find_dns_zone_response.json()[\"result\"]\n for i in result:\n if i[\"name\"] in domain_name:\n setattr(self, \"CLOUDFLARE_DNS_ZONE_ID\", i[\"id\"])\n if isinstance(self.CLOUDFLARE_DNS_ZONE_ID, type(None)):\n raise ValueError(\n \"No DNS zone for %s: status = %s, response=%s\"\n % (\n domain_name,\n find_dns_zone_response.status_code,\n log_response(find_dns_zone_response),\n )\n )\n\n self.logger.debug(\"find_dns_zone_success\")\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"create_dns_record\")\n self.find_dns_zone(domain_name)\n\n url = urllib.parse.urljoin(\n self.CLOUDFLARE_API_BASE_URL,\n \"zones/{0}/dns_records\".format(self.CLOUDFLARE_DNS_ZONE_ID),\n )\n headers = self._get_auth_header()\n body = {\n \"type\": \"TXT\",\n \"name\": \"_acme-challenge\" + \".\" + domain_name + \".\",\n \"content\": \"{0}\".format(domain_dns_value),\n }\n create_cloudflare_dns_record_response = requests.post(\n url, headers=headers, json=body, timeout=self.HTTP_TIMEOUT\n )\n self.logger.debug(\n \"create_cloudflare_dns_record_response. status_code={0}. response={1}\".format(\n create_cloudflare_dns_record_response.status_code,\n log_response(create_cloudflare_dns_record_response),\n )\n )\n if create_cloudflare_dns_record_response.status_code != 200:\n # raise error so that we do not continue to make calls to ACME\n # server\n raise ValueError(\n \"Error creating cloudflare dns record: status_code={status_code} response={response}\".format(\n status_code=create_cloudflare_dns_record_response.status_code,\n response=log_response(create_cloudflare_dns_record_response),\n )\n )\n self.logger.info(\"create_dns_record_end\")\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"delete_dns_record\")\n\n class MockResponse(object):\n def __init__(self, status_code=200, content=\"mock-response\"):\n self.status_code = status_code\n self.content = content\n super(MockResponse, self).__init__()\n\n def json(self):\n return {}\n\n delete_dns_record_response = MockResponse()\n headers = self._get_auth_header()\n\n dns_name = \"_acme-challenge\" + \".\" + domain_name\n list_dns_payload = {\"type\": \"TXT\", \"name\": dns_name}\n list_dns_url = urllib.parse.urljoin(\n self.CLOUDFLARE_API_BASE_URL,\n \"zones/{0}/dns_records\".format(self.CLOUDFLARE_DNS_ZONE_ID),\n )\n\n list_dns_response = requests.get(\n list_dns_url, params=list_dns_payload, headers=headers, timeout=self.HTTP_TIMEOUT\n )\n\n for i in range(0, len(list_dns_response.json()[\"result\"])):\n dns_record_id = list_dns_response.json()[\"result\"][i][\"id\"]\n url = urllib.parse.urljoin(\n self.CLOUDFLARE_API_BASE_URL,\n \"zones/{0}/dns_records/{1}\".format(self.CLOUDFLARE_DNS_ZONE_ID, dns_record_id),\n )\n headers = self._get_auth_header()\n delete_dns_record_response = requests.delete(\n url, headers=headers, timeout=self.HTTP_TIMEOUT\n )\n self.logger.debug(\n \"delete_dns_record_response. status_code={0}. response={1}\".format(\n delete_dns_record_response.status_code, log_response(delete_dns_record_response)\n )\n )\n if delete_dns_record_response.status_code != 200:\n # extended logging for debugging\n # we do not need to raise exception\n self.logger.error(\n \"delete_dns_record_response. status_code={0}. response={1}\".format(\n delete_dns_record_response.status_code,\n log_response(delete_dns_record_response),\n )\n )\n\n self.logger.info(\"delete_dns_record_success\")\n\n def _get_auth_header(self):\n if self.CLOUDFLARE_TOKEN:\n return {\"Authorization\": \"Bearer \" + self.CLOUDFLARE_TOKEN}\n else:\n return {\"X-Auth-Email\": self.CLOUDFLARE_EMAIL, \"X-Auth-Key\": self.CLOUDFLARE_API_KEY}\n" }, { "path": "sewer/dns_providers/cloudns.py", "content": "from cloudns_api import record # type: ignore\n\nfrom . import common\n\n\n### FIX ME ### this assumes there are only two levels above the host (no bbc.co.uk eg.)\n\n\ndef _split_domain_name(domain_name):\n \"\"\"ClouDNS requires the domain name and host to be split.\"\"\"\n full_domain_name = \"_acme-challenge.{}\".format(domain_name)\n domain_parts = full_domain_name.split(\".\")\n\n domain_name = \".\".join(domain_parts[-2:])\n host = \".\".join(domain_parts[:-2])\n\n return domain_name, host\n\n\nclass ClouDNSDns(common.BaseDns):\n def __init__(self, **kwargs):\n super().__init__(**kwargs)\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"create_dns_record\")\n domain_name, host = _split_domain_name(domain_name)\n response = record.create(\n domain_name=domain_name, host=host, record_type=\"TXT\", record=domain_dns_value, ttl=60\n )\n\n if not response.success:\n self.logger.info(\"ClouDNS could not create DNS record.\")\n raise Exception(\"ClouDNS responded with an error.\")\n\n self.logger.info(\"create_dns_record_success\")\n return\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"delete_dns_record\")\n domain_name, host = _split_domain_name(domain_name)\n response = record.list(domain_name=domain_name, host=host, record_type=\"TXT\")\n\n if not response.success:\n self.logger.info(\"ClouDNS could not find DNS record to delete.\")\n raise Exception(\"ClouDNS responded with an error.\")\n\n for record_id, item in response.payload.items():\n if item[\"record\"] == domain_dns_value:\n response = record.delete(domain_name=domain_name, record_id=record_id)\n if not response.success:\n self.logger.info(\"ClouDNS could not delete DNS record.\")\n raise Exception(\"ClouDNS responded with an error.\")\n self.logger.info(\"delete_dns_record_success\")\n return\n\n self.logger.info(\"ClouDNS could not find DNS record to delete.\")\n raise Exception(\"ClouDNS responded with an error.\")\n" }, { "path": "sewer/dns_providers/common.py", "content": "from typing import Any, Dict, Sequence\n\nfrom ..auth import ErrataItemType, DNSProviderBase\nfrom ..lib import dns_challenge\n\n\nclass BaseDns(DNSProviderBase):\n \"\"\"\n Shim for legacy DNS provider interface.\n \"\"\"\n\n def __init__(self, **kwargs: Any) -> None:\n if \"chal_types\" not in kwargs:\n kwargs[\"chal_types\"] = [\"dns-01\"]\n if \"LOG_LEVEL\" not in kwargs:\n kwargs[\"LOG_LEVEL\"] = \"WARNING\"\n super().__init__(**kwargs)\n\n ### shim methods\n\n def setup(self, challenges: Sequence[Dict[str, str]]) -> Sequence[ErrataItemType]:\n for chal in challenges:\n self.create_dns_record(chal[\"ident_value\"], dns_challenge(chal[\"key_auth\"]))\n return []\n\n def unpropagated(self, challenges: Sequence[Dict[str, str]]) -> Sequence[ErrataItemType]:\n return []\n\n def clear(self, challenges: Sequence[Dict[str, str]]) -> Sequence[ErrataItemType]:\n for chal in challenges:\n self.delete_dns_record(chal[\"ident_value\"], dns_challenge(chal[\"key_auth\"]))\n return []\n\n ### legacy DNS methods\n\n def create_dns_record(self, domain_name, domain_dns_value):\n \"\"\"\n Method that creates/adds a dns TXT record for a domain/subdomain name on\n a chosen DNS provider.\n\n :param domain_name: :string: The domain/subdomain name whose dns record ought to be\n created/added on a chosen DNS provider.\n :param domain_dns_value: :string: The value/content of the TXT record that will be\n created/added for the given domain/subdomain\n\n This method should return None\n\n Basic Usage:\n If the value of the `domain_name` variable is example.com and the value of\n `domain_dns_value` is HAJA_4MkowIFByHhFaP8u035skaM91lTKplKld\n Then, your implementation of this method ought to create a DNS TXT record\n whose name is '_acme-challenge' + '.' + domain_name + '.' (ie: _acme-challenge.example.com. )\n and whose value/content is HAJA_4MkowIFByHhFaP8u035skaM91lTKplKld\n\n Using a dns client like dig(https://linux.die.net/man/1/dig) to do a dns lookup should result\n in something like:\n dig TXT _acme-challenge.example.com\n ...\n ;; ANSWER SECTION:\n _acme-challenge.example.com. 120 IN TXT \"HAJA_4MkowIFByHhFaP8u035skaM91lTKplKld\"\n _acme-challenge.singularity.brandur.org. 120 IN TXT \"9C0DqKC_4MkowIFByHhFaP8u0Zv4z7Wz2IHM91lTKec\"\n Optionally, you may also use an online dns client like: https://toolbox.googleapps.com/apps/dig/#TXT/\n\n Please consult your dns provider on how/format of their DNS TXT records.\n You may also want to consult the cloudflare DNS implementation that is found in this repository.\n \"\"\"\n raise NotImplementedError(\"create_dns_record method must be implemented.\")\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n \"\"\"\n Method that deletes/removes a dns TXT record for a domain/subdomain name on\n a chosen DNS provider.\n\n :param domain_name: :string: The domain/subdomain name whose dns record ought to be\n deleted/removed on a chosen DNS provider.\n :param domain_dns_value: :string: The value/content of the TXT record that will be\n deleted/removed for the given domain/subdomain\n\n This method should return None\n \"\"\"\n raise NotImplementedError(\"delete_dns_record method must be implemented.\")\n" }, { "path": "sewer/dns_providers/dnspod.py", "content": "import urllib.parse\n\nimport requests\n\nfrom . import common\n\n\nclass DNSPodDns(common.BaseDns):\n def __init__(\n self, DNSPOD_ID, DNSPOD_API_KEY, DNSPOD_API_BASE_URL=\"https://dnsapi.cn/\", **kwargs\n ):\n self.DNSPOD_ID = DNSPOD_ID\n self.DNSPOD_API_KEY = DNSPOD_API_KEY\n self.DNSPOD_API_BASE_URL = DNSPOD_API_BASE_URL\n self.HTTP_TIMEOUT = 65 # seconds\n self.DNSPOD_LOGIN = \"{0},{1}\".format(self.DNSPOD_ID, self.DNSPOD_API_KEY)\n\n if DNSPOD_API_BASE_URL[-1] != \"/\":\n self.DNSPOD_API_BASE_URL = DNSPOD_API_BASE_URL + \"/\"\n else:\n self.DNSPOD_API_BASE_URL = DNSPOD_API_BASE_URL\n super().__init__(**kwargs)\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"create_dns_record\")\n\n ### FIX ME ### domain is exactly last two parts (service has list of zones API?)\n ### YUCK ### Odd, unpythonic implementation\n ### REFACTOR ### duplicated code (here and delete_dns_record)\n\n subd = \"\"\n if domain_name.count(\".\") != 1: # not top level domain\n pos = domain_name.rfind(\".\", 0, domain_name.rfind(\".\"))\n subd = domain_name[:pos]\n domain_name = domain_name[pos + 1 :]\n if subd != \"\":\n subd = \".\" + subd\n\n url = urllib.parse.urljoin(self.DNSPOD_API_BASE_URL, \"Record.Create\")\n body = {\n \"record_type\": \"TXT\",\n \"domain\": domain_name,\n \"sub_domain\": \"_acme-challenge\" + subd,\n \"value\": domain_dns_value,\n \"record_line_id\": \"0\",\n \"format\": \"json\",\n \"login_token\": self.DNSPOD_LOGIN,\n }\n create_dnspod_dns_record_response = requests.post(\n url, data=body, timeout=self.HTTP_TIMEOUT\n ).json()\n self.logger.debug(\n \"create_dnspod_dns_record_response. status_code={0}. response={1}\".format(\n create_dnspod_dns_record_response[\"status\"][\"code\"],\n create_dnspod_dns_record_response[\"status\"][\"message\"],\n )\n )\n if create_dnspod_dns_record_response[\"status\"][\"code\"] != \"1\":\n # raise error so that we do not continue to make calls to ACME\n # server\n raise ValueError(\n \"Error creating dnspod dns record: status_code={status_code} response={response}\".format(\n status_code=create_dnspod_dns_record_response[\"status\"][\"code\"],\n response=create_dnspod_dns_record_response[\"status\"][\"message\"],\n )\n )\n self.logger.info(\"create_dns_record_end\")\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"delete_dns_record\")\n\n subd = \"\"\n if domain_name.count(\".\") != 1: # not top level domain\n pos = domain_name.rfind(\".\", 0, domain_name.rfind(\".\"))\n subd = domain_name[:pos]\n domain_name = domain_name[pos + 1 :]\n if subd != \"\":\n subd = \".\" + subd\n\n url = urllib.parse.urljoin(self.DNSPOD_API_BASE_URL, \"Record.List\")\n # pos = domain_name.rfind(\".\",0, domain_name.rfind(\".\"))\n subdomain = \"_acme-challenge.\" + subd\n rootdomain = domain_name\n body = {\n \"login_token\": self.DNSPOD_LOGIN,\n \"format\": \"json\",\n \"domain\": rootdomain,\n \"subdomain\": subdomain,\n \"record_type\": \"TXT\",\n }\n list_dns_response = requests.post(url, data=body, timeout=self.HTTP_TIMEOUT).json()\n if list_dns_response[\"status\"][\"code\"] != \"1\":\n self.logger.error(\n \"list_dns_record_response. status_code={0}. message={1}\".format(\n list_dns_response[\"status\"][\"code\"], list_dns_response[\"status\"][\"message\"]\n )\n )\n for i in range(0, len(list_dns_response[\"records\"])):\n rid = list_dns_response[\"records\"][i][\"id\"]\n urlr = urllib.parse.urljoin(self.DNSPOD_API_BASE_URL, \"Record.Remove\")\n bodyr = {\n \"login_token\": self.DNSPOD_LOGIN,\n \"format\": \"json\",\n \"domain\": rootdomain,\n \"record_id\": rid,\n }\n delete_dns_record_response = requests.post(\n urlr, data=bodyr, timeout=self.HTTP_TIMEOUT\n ).json()\n if delete_dns_record_response[\"status\"][\"code\"] != \"1\":\n self.logger.error(\n \"delete_dns_record_response. status_code={0}. message={1}\".format(\n delete_dns_record_response[\"status\"][\"code\"],\n delete_dns_record_response[\"status\"][\"message\"],\n )\n )\n\n self.logger.info(\"delete_dns_record_success\")\n" }, { "path": "sewer/dns_providers/duckdns.py", "content": "import urllib.parse\n\nimport requests\n\nfrom . import common\n\n\nclass DuckDNSDns(common.BaseDns):\n def __init__(self, duckdns_token, DUCKDNS_API_BASE_URL=\"https://www.duckdns.org\", **kwargs):\n\n self.duckdns_token = duckdns_token\n self.HTTP_TIMEOUT = 65 # seconds\n\n if DUCKDNS_API_BASE_URL[-1] != \"/\":\n self.DUCKDNS_API_BASE_URL = DUCKDNS_API_BASE_URL + \"/\"\n else:\n self.DUCKDNS_API_BASE_URL = DUCKDNS_API_BASE_URL\n super().__init__(**kwargs)\n\n def _common_dns_record(self, logger_info, domain_name, payload_end_arg):\n self.logger.info(\"{0}\".format(logger_info))\n # add provider domain to the domain name if not present\n provider_domain = \".duckdns.org\"\n if domain_name.rfind(provider_domain) == -1:\n \"\".join((domain_name, provider_domain))\n\n url = urllib.parse.urljoin(self.DUCKDNS_API_BASE_URL, \"update\")\n\n payload = dict([(\"domains\", domain_name), (\"token\", self.duckdns_token), payload_end_arg])\n update_duckdns_dns_record_response = requests.get(\n url, params=payload, timeout=self.HTTP_TIMEOUT\n )\n\n normalized_response = update_duckdns_dns_record_response.text\n self.logger.debug(\n \"update_duckdns_dns_record_response. status_code={0}. response={1}\".format(\n update_duckdns_dns_record_response.status_code, normalized_response\n )\n )\n\n if update_duckdns_dns_record_response.status_code != 200 or normalized_response != \"OK\":\n # raise error so that we do not continue to make calls to DuckDNS\n # server\n raise ValueError(\n \"Error creating DuckDNS dns record: status_code={status_code} response={response}\".format(\n status_code=update_duckdns_dns_record_response.status_code,\n response=normalized_response,\n )\n )\n self.logger.info(\"{0}_success\".format(logger_info))\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self._common_dns_record(\"create_dns_record\", domain_name, (\"txt\", domain_dns_value))\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self._common_dns_record(\"delete_dns_record\", domain_name, (\"clear\", \"true\"))\n" }, { "path": "sewer/dns_providers/gandi.py", "content": "import os\nfrom itertools import chain\n\nimport requests\n\nfrom . import common\n\n\nclass GandiDns(common.BaseDns):\n def __init__(\n self,\n GANDI_API_KEY=None,\n GANDI_API_BASE_URL=\"https://dns.api.gandi.net/api/v5/\",\n DEFAULT_TTL=10800,\n requests_lib=requests,\n **kwargs,\n ):\n self.GANDI_API_KEY = GANDI_API_KEY\n self.HTTP_TIMEOUT = 65 # seconds\n self.RECORD_TTL = DEFAULT_TTL\n self.requests = requests_lib\n\n if GANDI_API_BASE_URL[-1] != \"/\":\n self.GANDI_API_BASE_URL = GANDI_API_BASE_URL + \"/\"\n else:\n self.GANDI_API_BASE_URL = GANDI_API_BASE_URL\n\n # pass an API key\n if not GANDI_API_KEY:\n raise ValueError(\"Error initializing Gandi DNS adapter. Pass an API key.\")\n\n self.GET_HEADERS = {\"X-Api-Key\": self.GANDI_API_KEY}\n self.POST_HEADERS = {\"X-Api-Key\": self.GANDI_API_KEY, \"Content-Type\": \"application/json\"}\n\n super().__init__(**kwargs)\n\n def create_dns_record(self, domain_name, domain_dns_value):\n [subdomain, base_domain] = GandiDns.split_domain(domain_name)\n zone_records_href = self.get_zone_records_href(base_domain)\n all_records = self.get_all_zone_records(zone_records_href)\n subdomain_records = self._get_subdomain_records(\n GandiDns.subdomain_to_challenge_domain(subdomain), all_records\n )\n\n request_body = {\n \"rrset_type\": \"TXT\",\n \"rrset_ttl\": self.RECORD_TTL,\n \"rrset_name\": GandiDns.subdomain_to_challenge_domain(subdomain),\n \"rrset_values\": list(\n chain.from_iterable(\n [subdomain_record[\"rrset_values\"] for subdomain_record in subdomain_records]\n )\n )\n + [domain_dns_value],\n }\n\n if subdomain_records:\n self.delete_record(domain_name)\n\n create_record_resp = self.requests.post(\n zone_records_href, headers=self.POST_HEADERS, json=request_body\n )\n if not create_record_resp.status_code < 300:\n raise RuntimeError(\"createRecord failed\")\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self.delete_record(domain_name)\n\n def _get_subdomain_records(self, subdomain, all_records):\n return list(filter(lambda rec: rec[\"rrset_name\"] == subdomain, all_records))\n\n def delete_record(self, domain_name):\n [subdomain, base_domain] = GandiDns.split_domain(domain_name)\n zone_records_href = self.get_zone_records_href(base_domain)\n all_records = self.get_all_zone_records(zone_records_href)\n\n subdomain_records = self._get_subdomain_records(\n GandiDns.subdomain_to_challenge_domain(subdomain), all_records\n )\n for subdomain_record in subdomain_records:\n del_record_resp = self.requests.delete(\n subdomain_record[\"rrset_href\"], headers=self.GET_HEADERS\n )\n if not del_record_resp.status_code < 300:\n raise RuntimeError(\"deleteRecord failed\")\n\n def get_zone_records_href(self, base_domain):\n domain_resp = self.requests.get(\n os.path.join(self.GANDI_API_BASE_URL, \"domains\", base_domain), headers=self.GET_HEADERS\n )\n\n if not domain_resp.status_code < 300:\n raise RuntimeError(\"getZoneHref failed\")\n\n domain_info = domain_resp.json()\n return domain_info[\"zone_records_href\"]\n\n def get_all_zone_records(self, zone_records_href):\n all_records_resp = self.requests.get(zone_records_href, headers=self.GET_HEADERS)\n if not all_records_resp.status_code < 300:\n raise RuntimeError(\"getAllZoneRecords failed\")\n\n all_records = all_records_resp.json()\n return all_records\n\n @staticmethod\n def split_domain(domain_name):\n def has_second_level_domain(split_domains):\n return len(split_domains) >= 2\n\n split_domains = domain_name.split(\".\")\n if not has_second_level_domain(split_domains):\n raise RuntimeError(\n \"domain: \" + str(domain_name) + \"does not have a second level domain\"\n )\n\n separator = \".\"\n base_domain = separator.join(split_domains[-2:])\n subdomain = separator.join(split_domains[0:-2])\n if not subdomain:\n subdomain = \"@\"\n\n return [subdomain, base_domain]\n\n @staticmethod\n def subdomain_to_challenge_domain(subdomain):\n return \"_acme-challenge\" + ((\".\" + subdomain) if subdomain != \"@\" else \"\")\n" }, { "path": "sewer/dns_providers/hurricane.py", "content": "\"\"\"\nHurricane Electric DNS Support\n\"\"\"\nimport json\n\nimport HurricaneDNS as _hurricanedns # type: ignore\n\nfrom . import common\n\n\nclass _Response(object):\n \"\"\"\n wrapper aliyun resp to the format sewer wanted.\n \"\"\"\n\n def __init__(self, status_code=200, content=None, headers=None):\n self.status_code = status_code\n self.headers = headers or {}\n self.content = content or {}\n self.content = json.dumps(content)\n super(_Response, self).__init__()\n\n def json(self):\n return json.loads(self.content)\n\n\nclass HurricaneDns(common.BaseDns):\n def __init__(self, username, password, **kwargs):\n super().__init__(**kwargs)\n self.clt = _hurricanedns.HurricaneDNS(username, password)\n\n @staticmethod\n def extract_zone(domain_name):\n \"\"\"\n extract domain to root, sub, acme_txt\n :param str domain_name: the value sewer client passed in, like *.menduo.example.com\n :return tuple: root, zone, acme_txt\n \"\"\"\n if domain_name.count(\".\") > 1:\n zone, middle, last = str(domain_name).rsplit(\".\", 2)\n root = \".\".join([middle, last])\n acme_txt = \"_acme-challenge.%s\" % zone\n else:\n zone = \"\"\n root = domain_name\n acme_txt = \"_acme-challenge\"\n return root, zone, acme_txt\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"create_dns_record start: %s\", (domain_name, domain_dns_value))\n\n root, _, acme_txt = self.extract_zone(domain_name)\n self.clt.add_record(root, acme_txt, \"TXT\", domain_dns_value, ttl=300)\n\n self.logger.info(\"create_dns_record end: %s\", (domain_name, domain_dns_value))\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"delete_dns_record start: %s\", (domain_name, domain_dns_value))\n\n root, _, acme_txt = self.extract_zone(domain_name)\n host = \"%s.%s\" % (acme_txt, root)\n\n recored_list = self.clt.get_records(root, host, \"TXT\")\n\n for i in recored_list:\n self.clt.del_record(root, i[\"id\"])\n\n self.logger.info(\"delete_dns_record end: %s\", (domain_name, domain_dns_value))\n" }, { "path": "sewer/dns_providers/powerdns.py", "content": "import json\n\nimport requests\n\nfrom . import common\n\n\nclass PowerDNSDns(common.BaseDns):\n \"\"\"\n For PowerDNS, all subdomains for a given domain live under the apex zone `zone_id`.\n For example, if you want a cert for domain.tld and www.domain.tld, you need\n to create two DNS records:\n 1) `acme-challenge.domain.tld. IN TXT`\n 2) `acme-challenge.www.domain.tld. IN TXT`\n\n However, both of these records must be created under `/servers/{server_id}/zones/{zone_id}`,\n where `zone_id` is the apex domain (`domain.tld`)\n\n So, we must be smart about stripping out subdomains as part of the URL passed\n to `requests.patch`, but must maintain the FQDN in the `name` field of the `payload`.\n \"\"\"\n\n def __init__(self, powerdns_api_key, powerdns_api_url, *kwargs):\n self.powerdns_api_key = powerdns_api_key\n self.powerdns_api_url = powerdns_api_url\n super().__init__(*kwargs)\n\n def validate_powerdns_zone(self, domain_name):\n \"\"\"\n Walk `domain_name` backwards, trying to find the apex domain.\n E.g.: For `fu.bar.baz.domain.com`, `response.status_code` will only be\n `200` for `domain.com`\n \"\"\"\n d = \".\"\n count = domain_name.count(d)\n\n while True:\n url = self.powerdns_api_url + \"/\" + domain_name\n response = requests.get(url, headers={\"X-API-Key\": self.powerdns_api_key})\n\n if response.status_code == 200:\n return domain_name\n elif count <= 0:\n raise ValueError(\n \"Could not determine apex domain: (count: %s, domain_name: %s)\"\n % (count, domain_name)\n )\n else:\n split = domain_name.split(d)\n split.pop(0)\n domain_name = d.join(split)\n count -= 1\n\n def _common_dns_record(self, domain_name, domain_dns_value, changetype):\n if changetype not in (\"REPLACE\", \"DELETE\"):\n raise ValueError(\"changetype is not valid.\")\n\n payload = {\n \"rrsets\": [\n {\n \"name\": \"_acme-challenge\" + \".\" + domain_name + \".\",\n \"type\": \"TXT\",\n \"ttl\": 60,\n \"changetype\": changetype,\n \"records\": [{\"content\": f'\"{domain_dns_value}\"', \"disabled\": False}],\n }\n ]\n }\n self.logger.debug(\"PowerDNS domain name: %s\", domain_name)\n self.logger.debug(\"PowerDNS payload: %s\", payload)\n\n apex_domain = self.validate_powerdns_zone(domain_name)\n url = self.powerdns_api_url + \"/\" + apex_domain\n self.logger.debug(\"apex_domain: %s\", apex_domain)\n self.logger.debug(\"url: %s\", url)\n\n try:\n response = requests.patch(\n url, data=json.dumps(payload), headers={\"X-API-Key\": self.powerdns_api_key}\n )\n self.logger.debug(\"PowerDNS response: %s, %s\", response.status_code, response.text)\n except requests.exceptions.RequestException as e:\n self.logger.error(\"Unable to communicate with PowerDNS API: %s\", e)\n raise\n\n # Per https://doc.powerdns.com/authoritative/http-api/zone.html:\n # PATCH /servers/{server_id}/zones/{zone_id}\n # Creates/modifies/deletes RRsets present in the payload and their comments.\n # Returns 204 No Content on success.\n if response.status_code != 204:\n raise ValueError(\"Error creating or deleting PowerDNS record: %s\" % response.text)\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self._common_dns_record(domain_name, domain_dns_value, \"REPLACE\")\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self._common_dns_record(domain_name, domain_dns_value, \"DELETE\")\n" }, { "path": "sewer/dns_providers/rackspace.py", "content": "import time, urllib.parse\n\nimport requests, tldextract\n\nfrom . import common\nfrom ..lib import log_response\n\n\nclass RackspaceDns(common.BaseDns):\n def __init__(self, RACKSPACE_USERNAME, RACKSPACE_API_KEY, **kwargs):\n self.RACKSPACE_DNS_ZONE_ID = None\n self.RACKSPACE_USERNAME = RACKSPACE_USERNAME\n self.RACKSPACE_API_KEY = RACKSPACE_API_KEY\n self.HTTP_TIMEOUT = 65 # seconds\n super().__init__(**kwargs)\n self.RACKSPACE_API_TOKEN, self.RACKSPACE_API_BASE_URL = self.get_rackspace_credentials()\n self.RACKSPACE_HEADERS = {\n \"X-Auth-Token\": self.RACKSPACE_API_TOKEN,\n \"Content-Type\": \"application/json\",\n }\n\n def get_rackspace_credentials(self):\n self.logger.debug(\"get_rackspace_credentials\")\n RACKSPACE_IDENTITY_URL = \"https://identity.api.rackspacecloud.com/v2.0/tokens\"\n payload = {\n \"auth\": {\n \"RAX-KSKEY:apiKeyCredentials\": {\n \"username\": self.RACKSPACE_USERNAME,\n \"apiKey\": self.RACKSPACE_API_KEY,\n }\n }\n }\n find_rackspace_api_details_response = requests.post(RACKSPACE_IDENTITY_URL, json=payload)\n self.logger.debug(\n \"find_rackspace_api_details_response. status_code={0}\".format(\n find_rackspace_api_details_response.status_code\n )\n )\n if find_rackspace_api_details_response.status_code != 200:\n raise ValueError(\n \"Error getting token and URL details from rackspace identity server: status_code={status_code} response={response}\".format(\n status_code=find_rackspace_api_details_response.status_code,\n response=log_response(find_rackspace_api_details_response),\n )\n )\n data = find_rackspace_api_details_response.json()\n api_token = data[\"access\"][\"token\"][\"id\"]\n url_data = next(\n (item for item in data[\"access\"][\"serviceCatalog\"] if item[\"type\"] == \"rax:dns\"), None\n )\n if url_data is None:\n raise ValueError(\n \"Error finding url data for the rackspace dns api in the response from the identity server\"\n )\n else:\n api_base_url = url_data[\"endpoints\"][0][\"publicURL\"] + \"/\"\n return (api_token, api_base_url)\n\n def get_dns_zone(self, domain_name):\n self.logger.debug(\"get_dns_zone\")\n extracted_domain = tldextract.extract(domain_name)\n self.RACKSPACE_DNS_ZONE = \".\".join([extracted_domain.domain, extracted_domain.suffix])\n\n def find_dns_zone_id(self, domain_name):\n self.logger.debug(\"find_dns_zone_id\")\n self.get_dns_zone(domain_name)\n url = self.RACKSPACE_API_BASE_URL + \"domains\"\n find_dns_zone_id_response = requests.get(url, headers=self.RACKSPACE_HEADERS)\n self.logger.debug(\n \"find_dns_zone_id_response. status_code={0}\".format(\n find_dns_zone_id_response.status_code\n )\n )\n if find_dns_zone_id_response.status_code != 200:\n raise ValueError(\n \"Error getting rackspace dns domain info: status_code={status_code} response={response}\".format(\n status_code=find_dns_zone_id_response.status_code,\n response=log_response(find_dns_zone_id_response),\n )\n )\n result = find_dns_zone_id_response.json()\n domain_data = next(\n (item for item in result[\"domains\"] if item[\"name\"] == self.RACKSPACE_DNS_ZONE), None\n )\n if domain_data is None:\n raise ValueError(\n \"Error finding information for {dns_zone} in dns response data:\\n{response_data})\".format(\n dns_zone=self.RACKSPACE_DNS_ZONE,\n response_data=log_response(find_dns_zone_id_response),\n )\n )\n dns_zone_id = domain_data[\"id\"]\n self.logger.debug(\"find_dns_zone_id_success\")\n return dns_zone_id\n\n def find_dns_record_id(self, domain_name, domain_dns_value):\n self.logger.debug(\"find_dns_record_id\")\n self.RACKSPACE_DNS_ZONE_ID = self.find_dns_zone_id(domain_name)\n url = self.RACKSPACE_API_BASE_URL + \"domains/{0}/records\".format(self.RACKSPACE_DNS_ZONE_ID)\n find_dns_record_id_response = requests.get(url, headers=self.RACKSPACE_HEADERS)\n self.logger.debug(\n \"find_dns_record_id_response. status_code={0}\".format(\n find_dns_record_id_response.status_code\n )\n )\n self.logger.debug(url)\n if find_dns_record_id_response.status_code != 200:\n raise ValueError(\n \"Error finding dns records for {dns_zone}: status_code={status_code} response={response}\".format(\n dns_zone=self.RACKSPACE_DNS_ZONE,\n status_code=find_dns_record_id_response.status_code,\n response=log_response(find_dns_record_id_response),\n )\n )\n records = find_dns_record_id_response.json()[\"records\"]\n RACKSPACE_RECORD_DATA = next(\n (item for item in records if item[\"data\"] == domain_dns_value), None\n )\n if RACKSPACE_RECORD_DATA is None:\n raise ValueError(\n \"Couldn't find record with name {domain_name}\\ncontaining data: {domain_dns_value}\\nin the response data:{response_data}\".format(\n domain_name=domain_name,\n domain_dns_value=domain_dns_value,\n response_data=log_response(find_dns_record_id_response),\n )\n )\n record_id = RACKSPACE_RECORD_DATA[\"id\"]\n self.logger.debug(\"find_dns_record_id success\")\n return record_id\n\n def poll_callback_url(self, callback_url):\n start_time = time.time()\n while True:\n callback_url_response = requests.get(callback_url, headers=self.RACKSPACE_HEADERS)\n if time.time() > start_time + self.HTTP_TIMEOUT:\n raise ValueError(\n \"Timed out polling callbackurl for dns record status. Last status_code={status_code} last response={response}\".format(\n status_code=callback_url_response.status_code,\n response=log_response(callback_url_response),\n )\n )\n if callback_url_response.status_code != 200:\n raise Exception(\n \"Could not get dns record status from callback url. Status code ={status_code}. response={response}\".format(\n status_code=callback_url_response.status_code,\n response=log_response(callback_url_response),\n )\n )\n if callback_url_response.json()[\"status\"] == \"ERROR\":\n raise Exception(\n \"Error in creating/deleting dns record: status_Code={status_code}. response={response}\".format(\n status_code=callback_url_response.status_code,\n response=log_response(callback_url_response),\n )\n )\n if callback_url_response.json()[\"status\"] == \"COMPLETED\":\n break\n\n def create_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"create_dns_record\")\n self.RACKSPACE_DNS_ZONE_ID = self.find_dns_zone_id(domain_name)\n record_name = \"_acme-challenge.\" + domain_name\n url = urllib.parse.urljoin(\n self.RACKSPACE_API_BASE_URL, \"domains/{0}/records\".format(self.RACKSPACE_DNS_ZONE_ID)\n )\n body = {\n \"records\": [{\"name\": record_name, \"type\": \"TXT\", \"data\": domain_dns_value, \"ttl\": 3600}]\n }\n create_rackspace_dns_record_response = requests.post(\n url, headers=self.RACKSPACE_HEADERS, json=body, timeout=self.HTTP_TIMEOUT\n )\n self.logger.debug(\n \"create_rackspace_dns_record_response. status_code={status_code}\".format(\n status_code=create_rackspace_dns_record_response.status_code\n )\n )\n if create_rackspace_dns_record_response.status_code != 202:\n raise ValueError(\n \"Error creating rackspace dns record: status_code={status_code} response={response}\".format(\n status_code=create_rackspace_dns_record_response.status_code,\n response=create_rackspace_dns_record_response.text,\n )\n )\n # response=log_response(create_rackspace_dns_record_response)))\n # After posting the dns record we want created, the response gives us a url to check that will\n # update when the job is done\n callback_url = create_rackspace_dns_record_response.json()[\"callbackUrl\"]\n self.poll_callback_url(callback_url)\n self.logger.info(\n \"create_dns_record_success. Name: {record_name} Data: {data}\".format(\n record_name=record_name, data=domain_dns_value\n )\n )\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n self.logger.info(\"delete_dns_record\")\n record_name = \"_acme-challenge.\" + domain_name\n self.RACKSPACE_DNS_ZONE_ID = self.find_dns_zone_id(domain_name)\n self.RACKSPACE_RECORD_ID = self.find_dns_record_id(domain_name, domain_dns_value)\n url = self.RACKSPACE_API_BASE_URL + \"domains/{domain_id}/records/?id={record_id}\".format(\n domain_id=self.RACKSPACE_DNS_ZONE_ID, record_id=self.RACKSPACE_RECORD_ID\n )\n delete_dns_record_response = requests.delete(url, headers=self.RACKSPACE_HEADERS)\n # After sending a delete request, if all goes well, we get a 202 from the server and a URL that we can poll\n # to see when the job is done\n self.logger.debug(\n \"delete_dns_record_response={0}\".format(delete_dns_record_response.status_code)\n )\n if delete_dns_record_response.status_code != 202:\n raise ValueError(\n \"Error deleting rackspace dns record: status_code={status_code} response={response}\".format(\n status_code=delete_dns_record_response.status_code,\n response=log_response(delete_dns_record_response),\n )\n )\n callback_url = delete_dns_record_response.json()[\"callbackUrl\"]\n self.poll_callback_url(callback_url)\n self.logger.info(\n \"delete_dns_record_success. Name: {record_name} Data: {data}\".format(\n record_name=record_name, data=domain_dns_value\n )\n )\n" }, { "path": "sewer/dns_providers/route53.py", "content": "import collections\n\nimport boto3 # type: ignore\nfrom botocore.client import Config # type: ignore\n\nfrom . import common\n\n\n# most code of this class is copy from certbot's route53 dns plugin.\nclass Route53Dns(common.BaseDns):\n ttl = 10\n connect_timeout = 30\n read_timeout = 30\n\n def __init__(self, access_key_id=None, secret_access_key=None, client=None, **kwargs):\n if (access_key_id or secret_access_key) and client:\n raise RuntimeError(\"Pass keys OR preconfigured client, not both\")\n\n self.aws_config = Config(\n connect_timeout=self.connect_timeout, read_timeout=self.read_timeout\n )\n if access_key_id and secret_access_key:\n # use user given credential\n self.r53 = boto3.client(\n \"route53\",\n aws_access_key_id=access_key_id,\n aws_secret_access_key=secret_access_key,\n config=self.aws_config,\n )\n elif client:\n # Use the client passed in from the caller.\n self.r53 = client\n else:\n # let boto3 find credential\n # https://boto3.readthedocs.io/en/latest/guide/configuration.html#best-practices-for-configuring-credentials\n self.r53 = boto3.client(\"route53\", config=self.aws_config)\n\n self._resource_records = collections.defaultdict(list)\n\n super().__init__(**kwargs)\n\n def create_dns_record(self, domain_name, domain_dns_value):\n challenge_domain = \"_acme-challenge\" + \".\" + domain_name + \".\"\n return self._change_txt_record(\"UPSERT\", challenge_domain, domain_dns_value)\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n challenge_domain = \"_acme-challenge\" + \".\" + domain_name + \".\"\n return self._change_txt_record(\"DELETE\", challenge_domain, domain_dns_value)\n\n def _find_zone_id_for_domain(self, domain):\n \"\"\"Find the zone id responsible a given FQDN.\n That is, the id for the zone whose name is the longest parent of the\n domain.\n \"\"\"\n paginator = self.r53.get_paginator(\"list_hosted_zones\")\n zones = []\n target_labels = domain.rstrip(\".\").split(\".\")\n for page in paginator.paginate():\n for zone in page[\"HostedZones\"]:\n if zone[\"Config\"][\"PrivateZone\"]:\n continue\n\n candidate_labels = zone[\"Name\"].rstrip(\".\").split(\".\")\n if candidate_labels == target_labels[-len(candidate_labels) :]:\n zones.append((zone[\"Name\"], zone[\"Id\"]))\n\n if not zones:\n raise RuntimeError(\"Unable to find a Route53 hosted zone for {0}\".format(domain))\n\n # Order the zones that are suffixes for our desired to domain by\n # length, this puts them in an order like:\n # [\"foo.bar.baz.com\", \"bar.baz.com\", \"baz.com\", \"com\"]\n # And then we choose the first one, which will be the most specific.\n zones.sort(key=lambda z: len(z[0]), reverse=True)\n return zones[0][1]\n\n def _change_txt_record(self, action, domain_name, domain_dns_value):\n zone_id = self._find_zone_id_for_domain(domain_name)\n\n rrecords = self._resource_records[domain_name]\n challenge = {\"Value\": '\"{0}\"'.format(domain_dns_value)}\n if action == \"DELETE\":\n # Remove the record being deleted from the list of tracked records\n rrecords.remove(challenge)\n if rrecords:\n # Need to update instead, as we're not deleting the rrset\n action = \"UPSERT\"\n else:\n # Create a new list containing the record to use with DELETE\n rrecords = [challenge]\n else:\n rrecords.append(challenge)\n\n response = self.r53.change_resource_record_sets(\n HostedZoneId=zone_id,\n ChangeBatch={\n \"Comment\": \"certbot-dns-route53 certificate validation \" + action,\n \"Changes\": [\n {\n \"Action\": action,\n \"ResourceRecordSet\": {\n \"Name\": domain_name,\n \"Type\": \"TXT\",\n \"TTL\": self.ttl,\n \"ResourceRecords\": rrecords,\n },\n }\n ],\n },\n )\n return response[\"ChangeInfo\"][\"Id\"]\n" }, { "path": "sewer/dns_providers/tests/__init__.py", "content": "" }, { "path": "sewer/dns_providers/tests/test_acmedns.py", "content": "from unittest import mock\nimport json\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.acmedns import AcmeDnsDns\n\nfrom . import test_utils\n\n\nclass Testacmedns(TestCase):\n \"\"\"\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.acmedns_API_USER = \"mock-email@example.com\"\n self.acmedns_API_KEY = \"mock-api-key\"\n self.acmedns_API_BASE_URL = \"https://some-mock-url.com\"\n\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.dns_class = AcmeDnsDns(\n ACME_DNS_API_USER=self.acmedns_API_USER,\n ACME_DNS_API_KEY=self.acmedns_API_KEY,\n ACME_DNS_API_BASE_URL=self.acmedns_API_BASE_URL,\n )\n\n def tearDown(self):\n pass\n\n def test_acmedns_is_called_by_create_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"sewer.dns_providers.acmedns.AcmeDnsDns.delete_dns_record\"\n ) as mock_delete_dns_record, mock.patch(\"dns.resolver.Resolver.query\") as mock_dns_resolver:\n mock_requests_post.return_value = (\n mock_delete_dns_record.return_value\n ) = test_utils.MockResponse()\n mock_dns_resolver.return_value = test_utils.MockDnsResolver()\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n\n expected = {\n \"headers\": {\"X-Api-User\": self.acmedns_API_USER, \"X-Api-Key\": self.acmedns_API_KEY},\n \"data\": '{\"subdomain\": \"canonical\", \"txt\": \"mock-domain_dns_value\"}',\n }\n\n self.assertDictEqual(expected[\"headers\"], mock_requests_post.call_args[1][\"headers\"])\n self.assertDictEqual(\n json.loads(expected[\"data\"]), mock_requests_post.call_args[1][\"json\"]\n )\n\n def test_acmedns_is_not_called_by_delete_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post:\n mock_requests_post.return_value = test_utils.MockResponse()\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertFalse(mock_requests_post.called)\n" }, { "path": "sewer/dns_providers/tests/test_aliyundns.py", "content": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.aliyundns import AliyunDns\n\nfrom . import test_utils\n\n\nclass TestAliyunDNS(TestCase):\n \"\"\"\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.API_KEY = \"mock-api-key\"\n self.API_SECRET = \"mock-api-secret\"\n\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.dns_class = AliyunDns(key=self.API_KEY, secret=self.API_SECRET)\n\n def tearDown(self):\n pass\n\n def test_extract_zone_sub_domain(self):\n _zone = \"sub-domain\"\n domain = \"%s.%s\" % (_zone, self.domain_name)\n root, zone, acme_txt = self.dns_class.extract_zone(domain)\n\n self.assertEqual(root, self.domain_name)\n self.assertEqual(zone, _zone)\n self.assertEqual(acme_txt, \"_acme-challenge.%s\" % zone)\n\n def test_extract_zone_root(self):\n domain = self.domain_name\n root, zone, acme_txt = self.dns_class.extract_zone(domain)\n self.assertEqual(root, self.domain_name)\n self.assertEqual(zone, \"\")\n self.assertEqual(acme_txt, \"_acme-challenge\")\n\n def test_aliyun_is_called_by_create_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"sewer.dns_providers.aliyundns.AliyunDns.delete_dns_record\"\n ) as mock_delete_dns_record, mock.patch(\"dns.resolver.Resolver.query\") as mock_dns_resolver:\n mock_requests_post.return_value = (\n mock_delete_dns_record.return_value\n ) = test_utils.MockResponse()\n mock_dns_resolver.return_value = test_utils.MockDnsResolver()\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n\n self.assertFalse(mock_requests_post.called)\n\n def test_aliyun_is_not_called_by_delete_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post:\n mock_requests_post.return_value = test_utils.MockResponse()\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertFalse(mock_requests_post.called)\n" }, { "path": "sewer/dns_providers/tests/test_aurora.py", "content": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.auroradns import AuroraDns\n\nfrom . import test_utils\n\n\nclass TestAurora(TestCase):\n \"\"\"\n Test the functionality of Aurora DNS.\n Todo: add proper tests that test that AuroraDns servers are actually called,\n and called with the right parameters. Currently the way the AuroraDns class is,\n makes it hard to mock stuff out.\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.AURORA_API_KEY = \"mock-aurora-api-key\"\n self.AURORA_SECRET_KEY = \"mock-aurora-secret-key\"\n\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.dns_class = AuroraDns(\n AURORA_API_KEY=self.AURORA_API_KEY, AURORA_SECRET_KEY=self.AURORA_SECRET_KEY\n )\n\n def tearDown(self):\n pass\n\n def test_delete_dns_record_is_not_called_by_create_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.auroradns.get_driver\"\n ) as mock_get_driver, mock.patch(\n \"sewer.dns_providers.auroradns.AuroraDns.delete_dns_record\"\n ) as mock_delete_dns_record:\n mock_requests_post.return_value = (\n mock_requests_get.return_value\n ) = (\n mock_requests_delete.return_value\n ) = mock_delete_dns_record.return_value = test_utils.MockResponse()\n mock_get_driver.return_value = test_utils.mockLibcloudGetDriver(\"mock-provider\")\n\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertFalse(mock_delete_dns_record.called)\n\n def test_aurora_is_called_by_delete_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.auroradns.get_driver\"\n ) as mock_get_driver:\n mock_requests_post.return_value = (\n mock_requests_get.return_value\n ) = mock_requests_delete.return_value = test_utils.MockResponse()\n mock_get_driver.return_value = test_utils.mockLibcloudGetDriver(\"mock-provider\")\n\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n" }, { "path": "sewer/dns_providers/tests/test_cloudflare.py", "content": "from unittest import mock\nimport json\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.cloudflare import CloudFlareDns\n\nfrom . import test_utils\n\n\nclass TestCloudflare(TestCase):\n \"\"\"\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.CLOUDFLARE_EMAIL = \"mock-email@example.com\"\n self.CLOUDFLARE_API_KEY = \"mock-api-key\"\n self.CLOUDFLARE_API_BASE_URL = \"https://some-mock-url.com\"\n self.CLOUDFLARE_TOKEN = \"mock-token\"\n\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.dns_class_api_key = CloudFlareDns(\n CLOUDFLARE_EMAIL=self.CLOUDFLARE_EMAIL,\n CLOUDFLARE_API_KEY=self.CLOUDFLARE_API_KEY,\n CLOUDFLARE_API_BASE_URL=self.CLOUDFLARE_API_BASE_URL,\n )\n\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.dns_class_token = CloudFlareDns(\n CLOUDFLARE_TOKEN=self.CLOUDFLARE_TOKEN,\n CLOUDFLARE_API_BASE_URL=self.CLOUDFLARE_API_BASE_URL,\n )\n\n def tearDown(self):\n pass\n\n def test_delete_dns_record_is_not_called_by_create_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.cloudflare.CloudFlareDns.delete_dns_record\"\n ) as mock_delete_dns_record:\n mock_requests_post.return_value = (\n mock_requests_get.return_value\n ) = (\n mock_requests_delete.return_value\n ) = mock_delete_dns_record.return_value = test_utils.MockResponse()\n\n self.dns_class_api_key.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertFalse(mock_delete_dns_record.called)\n\n self.dns_class_token.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertFalse(mock_delete_dns_record.called)\n\n def test_cloudflare_is_called_by_create_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.cloudflare.CloudFlareDns.delete_dns_record\"\n ) as mock_delete_dns_record:\n mock_requests_post.return_value = (\n mock_requests_get.return_value\n ) = (\n mock_requests_delete.return_value\n ) = mock_delete_dns_record.return_value = test_utils.MockResponse()\n\n # Test with API key authentication\n self.dns_class_api_key.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n expected = {\n \"headers\": {\n \"X-Auth-Email\": self.CLOUDFLARE_EMAIL,\n \"X-Auth-Key\": self.CLOUDFLARE_API_KEY,\n },\n \"data\": '{\"content\": \"mock-domain_dns_value\", \"type\": \"TXT\", \"name\": \"_acme-challenge.example.com.\"}',\n \"timeout\": 65,\n }\n\n self.assertDictEqual(expected[\"headers\"], mock_requests_post.call_args[1][\"headers\"])\n self.assertDictEqual(\n json.loads(expected[\"data\"]), mock_requests_post.call_args[1][\"json\"]\n )\n\n # Test with token authentication\n self.dns_class_token.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n expected = {\n \"headers\": {\"Authorization\": \"Bearer \" + self.CLOUDFLARE_TOKEN},\n \"data\": '{\"content\": \"mock-domain_dns_value\", \"type\": \"TXT\", \"name\": \"_acme-challenge.example.com.\"}',\n \"timeout\": 65,\n }\n\n self.assertDictEqual(expected[\"headers\"], mock_requests_post.call_args[1][\"headers\"])\n self.assertDictEqual(\n json.loads(expected[\"data\"]), mock_requests_post.call_args[1][\"json\"]\n )\n\n def test_cloudflare_is_called_by_delete_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete:\n mock_requests_post.return_value = (\n mock_requests_delete.return_value\n ) = test_utils.MockResponse()\n\n mock_requests_get.return_value = test_utils.MockResponse()\n\n # Test with API key authentication\n self.dns_class_api_key.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertTrue(mock_requests_get.called)\n expected = {\n \"headers\": {\n \"X-Auth-Email\": self.CLOUDFLARE_EMAIL,\n \"X-Auth-Key\": self.CLOUDFLARE_API_KEY,\n },\n \"timeout\": 65,\n }\n self.assertDictEqual(expected, mock_requests_delete.call_args[1])\n self.assertIn(\n \"https://some-mock-url.com/zones/None/dns_records/some-mock-dns-zone-id\",\n str(mock_requests_delete.call_args),\n )\n\n # Test with token authentication\n self.dns_class_token.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertTrue(mock_requests_get.called)\n expected = {\n \"headers\": {\"Authorization\": \"Bearer \" + self.CLOUDFLARE_TOKEN},\n \"timeout\": 65,\n }\n self.assertDictEqual(expected, mock_requests_delete.call_args[1])\n self.assertIn(\n \"https://some-mock-url.com/zones/None/dns_records/some-mock-dns-zone-id\",\n str(mock_requests_delete.call_args),\n )\n\n\nclass TestCloudflareTokens(TestCase):\n \"\"\"\n Test that the authorization parameters are validated (either email + API key or token)\n \"\"\"\n\n def setUp(self):\n self.CLOUDFLARE_EMAIL = \"mock-email@example.com\"\n self.CLOUDFLARE_API_KEY = \"mock-api-key\"\n self.CLOUDFLARE_TOKEN = \"some-token\"\n\n def test_init_auth_validation(self):\n # Invalid inputs\n with self.assertRaises(ValueError):\n CloudFlareDns(\n CLOUDFLARE_TOKEN=self.CLOUDFLARE_TOKEN,\n CLOUDFLARE_EMAIL=self.CLOUDFLARE_EMAIL,\n CLOUDFLARE_API_KEY=self.CLOUDFLARE_API_KEY,\n )\n\n with self.assertRaises(ValueError):\n CloudFlareDns(\n CLOUDFLARE_TOKEN=self.CLOUDFLARE_TOKEN, CLOUDFLARE_API_KEY=self.CLOUDFLARE_API_KEY\n )\n\n with self.assertRaises(ValueError):\n CloudFlareDns()\n\n with self.assertRaises(ValueError):\n CloudFlareDns(\n CLOUDFLARE_TOKEN=self.CLOUDFLARE_TOKEN, CLOUDFLARE_EMAIL=self.CLOUDFLARE_EMAIL\n )\n\n # Valid inputs\n CloudFlareDns(\n CLOUDFLARE_EMAIL=self.CLOUDFLARE_EMAIL, CLOUDFLARE_API_KEY=self.CLOUDFLARE_API_KEY\n )\n CloudFlareDns(CLOUDFLARE_TOKEN=self.CLOUDFLARE_TOKEN)\n" }, { "path": "sewer/dns_providers/tests/test_cloudns.py", "content": "import os, sys\nfrom unittest import mock, skipIf, TestCase\n\nfrom sewer.dns_providers.cloudns import ClouDNSDns\n\nfrom . import test_utils\n\n\nclass TestClouDNS(TestCase):\n \"\"\"\n Tests the ClouDNS DNS provider class.\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.cloudns_auth_id = \"mock-api-id\"\n self.cloudns_auth_password = \"mock-api-password\"\n\n self.dns_class = ClouDNSDns()\n\n os.environ[\"CLOUDNS_API_AUTH_ID\"] = self.cloudns_auth_id\n os.environ[\"CLOUDNS_API_AUTH_PASSWORD\"] = self.cloudns_auth_password\n\n def test_cloudns_is_called_by_create_dns_record(self):\n with mock.patch(\n \"cloudns_api.api.CLOUDNS_API_AUTH_ID\", new=self.cloudns_auth_id\n ), mock.patch(\n \"cloudns_api.api.CLOUDNS_API_AUTH_PASSWORD\", new=self.cloudns_auth_password\n ), mock.patch(\n \"requests.post\"\n ) as mock_requests_post:\n mock_requests_post.return_value = test_utils.MockResponse()\n\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n\n expected = {\n \"auth-id\": \"mock-api-id\",\n \"auth-password\": \"mock-api-password\",\n \"domain-name\": \"example.com\",\n \"host\": \"_acme-challenge\",\n \"record\": \"mock-domain_dns_value\",\n \"record-type\": \"TXT\",\n \"ttl\": 60,\n }\n\n self.assertDictEqual(expected, mock_requests_post.call_args[1][\"params\"])\n\n @skipIf(sys.version_info[:2] == (3, 5), \"mysterious failure with Py3.5 only\")\n def test_cloudns_is_called_by_delete_dns_record(self):\n with mock.patch(\n \"cloudns_api.api.CLOUDNS_API_AUTH_ID\", new=self.cloudns_auth_id\n ), mock.patch(\n \"cloudns_api.api.CLOUDNS_API_AUTH_PASSWORD\", new=self.cloudns_auth_password\n ), mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\n \"requests.post\"\n ) as mock_requests_post:\n\n mock_requests_get.return_value = test_utils.MockResponse(\n content={\"1234567\": {\"record\": \"mock-domain_dns_value\"}}\n )\n mock_requests_post.return_value = test_utils.MockResponse()\n\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n\n expected = {\n \"auth-id\": \"mock-api-id\",\n \"auth-password\": \"mock-api-password\",\n \"domain-name\": \"example.com\",\n \"record-id\": \"1234567\",\n }\n\n self.assertDictEqual(expected, mock_requests_post.call_args[1][\"params\"])\n" }, { "path": "sewer/dns_providers/tests/test_common.py", "content": "from unittest import mock, TestCase\n\nimport sewer.dns_providers.common\n\n\nclass TestCommon(TestCase):\n \"\"\"\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"wwfw2402if\"\n self.challenges = [{\"ident_value\": \"example.com\", \"key_auth\": \"abcdefgh12345678\"}]\n self.dns_class = sewer.dns_providers.common.BaseDns()\n\n def tearDown(self):\n pass\n\n def test_create_dns_record(self):\n def mock_create_dns_record():\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n\n self.assertRaises(NotImplementedError, mock_create_dns_record)\n\n def test_delete_dns_record(self):\n def mock_delete_dns_record():\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n\n self.assertRaises(NotImplementedError, mock_delete_dns_record)\n\n def test_setup_empty(self):\n self.assertFalse(self.dns_class.setup([]))\n\n def test_clear_empty(self):\n self.assertFalse(self.dns_class.clear([]))\n\n def test_setup_mocked(self):\n with mock.patch(\"sewer.dns_providers.common.BaseDns.create_dns_record\") as cdr:\n self.dns_class.setup(self.challenges)\n self.assertTrue(cdr.called)\n\n def test_clear_mocked(self):\n with mock.patch(\"sewer.dns_providers.common.BaseDns.delete_dns_record\") as ddr:\n self.dns_class.clear(self.challenges)\n self.assertTrue(ddr.called)\n" }, { "path": "sewer/dns_providers/tests/test_dnspod.py", "content": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.dnspod import DNSPodDns\n\nfrom . import test_utils\n\n\nclass TestDNSPod(TestCase):\n \"\"\"\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.DNSPOD_ID = \"0123456\"\n self.DNSPOD_API_KEY = \"mock-api-key\"\n self.DNSPOD_API_BASE_URL = \"https://some-mock-url.com\"\n self.test_datas = [\n {\n \"domain_name\": \"example.com\",\n \"domain_dns_value\": \"mock-domain_dns_value\",\n \"expected_domain_name\": \"example.com\",\n \"expected_sub_domain_name\": \"_acme-challenge\",\n },\n {\n \"domain_name\": \"sub1.example.com\",\n \"domain_dns_value\": \"mock-domain_dns_value\",\n \"expected_domain_name\": \"example.com\",\n \"expected_sub_domain_name\": \"_acme-challenge.sub1\",\n },\n {\n \"domain_name\": \"sub1.sub2.example.com\",\n \"domain_dns_value\": \"mock-domain_dns_value\",\n \"expected_domain_name\": \"example.com\",\n \"expected_sub_domain_name\": \"_acme-challenge.sub1.sub2\",\n },\n ]\n\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.dns_class = DNSPodDns(\n DNSPOD_ID=self.DNSPOD_ID,\n DNSPOD_API_KEY=self.DNSPOD_API_KEY,\n DNSPOD_API_BASE_URL=self.DNSPOD_API_BASE_URL,\n )\n\n def tearDown(self):\n pass\n\n def test_delete_dns_record_is_not_called_by_create_dns_record(\n self,\n ): # actually I don't know the purpose of this.\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\n \"sewer.dns_providers.dnspod.DNSPodDns.delete_dns_record\"\n ) as mock_delete_dns_record:\n mock_resp = {\"status\": {\"code\": \"1\", \"message\": \"Action completed successful\"}}\n mock_requests_post.return_value = (\n mock_requests_get.return_value\n ) = mock_delete_dns_record.return_value = test_utils.MockResponse(content=mock_resp)\n\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertFalse(mock_delete_dns_record.called)\n\n def test_dnspod_is_called_by_create_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.dnspod.DNSPodDns.delete_dns_record\"\n ) as mock_delete_dns_record:\n for test_data in self.test_datas:\n mock_resp = {\"status\": {\"code\": \"1\", \"message\": \"Action completed successful\"}}\n mock_requests_post.return_value = (\n mock_requests_get.return_value\n ) = (\n mock_requests_delete.return_value\n ) = mock_delete_dns_record.return_value = test_utils.MockResponse(content=mock_resp)\n\n self.dns_class.create_dns_record(\n domain_name=test_data[\"domain_name\"],\n domain_dns_value=test_data[\"domain_dns_value\"],\n )\n expected = {\n \"record_type\": \"TXT\",\n \"domain\": test_data[\"expected_domain_name\"],\n \"sub_domain\": test_data[\"expected_sub_domain_name\"],\n \"value\": test_data[\"domain_dns_value\"],\n \"record_line_id\": \"0\",\n \"format\": \"json\",\n \"login_token\": \"0123456,mock-api-key\",\n }\n self.assertDictEqual(expected, mock_requests_post.call_args[1][\"data\"])\n\n def test_dnspod_is_called_by_delete_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete:\n for test_data in self.test_datas:\n mock_resp = {\n \"status\": {\"code\": \"1\", \"message\": \"Action completed successful\"},\n \"records\": [\n {\n \"id\": \"123456789\",\n \"value\": \"32156546311615\",\n \"name\": test_data[\"expected_sub_domain_name\"],\n \"type\": \"TXT\",\n }\n ],\n }\n\n mock_requests_post.return_value = (\n mock_requests_delete.return_value\n ) = test_utils.MockResponse(content=mock_resp)\n\n mock_requests_get.return_value = test_utils.MockResponse()\n\n self.dns_class.delete_dns_record(\n domain_name=test_data[\"domain_name\"],\n domain_dns_value=test_data[\"domain_dns_value\"],\n )\n self.assertTrue(mock_requests_post.called)\n self.assertIn(\"123456789\", str(mock_requests_post.call_args))\n self.assertIn(\n \"https://some-mock-url.com/Record.Remove\", str(mock_requests_post.call_args)\n )\n\n def test_exception_is_raised_if_unsuccessful(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.dnspod.DNSPodDns.delete_dns_record\"\n ) as mock_delete_dns_record:\n for test_data in self.test_datas:\n mock_resp = {\"status\": {\"code\": \"-15\", \"message\": \"Domain name has been banned.\"}}\n mock_requests_post.return_value = (\n mock_requests_get.return_value\n ) = (\n mock_requests_delete.return_value\n ) = mock_delete_dns_record.return_value = test_utils.MockResponse(content=mock_resp)\n\n self.assertRaises(\n ValueError,\n self.dns_class.create_dns_record,\n test_data[\"domain_name\"],\n domain_dns_value=self.domain_dns_value,\n )\n" }, { "path": "sewer/dns_providers/tests/test_duckdns.py", "content": "import json\nfrom unittest import mock, TestCase\n\nfrom . import test_utils\nfrom ..duckdns import DuckDNSDns\n\n\nclass TestDuckDNS(TestCase):\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.duckdns_API_KEY = \"mock-api-key\"\n self.duckdns_API_BASE_URL = \"https://some-mock-url.com\"\n\n self.duck_mresponse = test_utils.MockResponse(content=\"OK\")\n with mock.patch(\"requests.get\") as mock_requests_get:\n mock_requests_get.return_value = self.duck_mresponse\n self.dns_class = DuckDNSDns(\n duckdns_token=self.duckdns_API_KEY, DUCKDNS_API_BASE_URL=self.duckdns_API_BASE_URL\n )\n\n def tearDown(self):\n pass\n\n def test_duckdns_is_called_by_create_dns_record(self):\n with mock.patch(\"requests.get\") as mock_requests_get, mock.patch(\n \"sewer.dns_providers.duckdns.DuckDNSDns.delete_dns_record\"\n ) as mock_delete_dns_record:\n\n mock_requests_get.return_value = (\n mock_delete_dns_record.return_value\n ) = self.duck_mresponse\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n\n expected = {\n \"data\": '{\"domains\": \"example.com\", \"token\": \"mock-api-key\", \"txt\": \"mock-domain_dns_value\"}'\n }\n\n self.assertDictEqual(\n json.loads(expected[\"data\"]), mock_requests_get.call_args[1][\"params\"]\n )\n\n def test_duckdns_is_called_by_delete_dns_record(self):\n with mock.patch(\"requests.get\") as mock_requests_get:\n mock_requests_get.return_value = self.duck_mresponse\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertTrue(mock_requests_get.called)\n" }, { "path": "sewer/dns_providers/tests/test_gandi.py", "content": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.gandi import GandiDns\n\nMOCK_GANDI_API_KEY = \"gandi-Api-Key\"\n\n\nclass MockResponseObject:\n def __init__(self, status_code=200, body=None):\n self.status_code = status_code\n self.body = body\n\n def json(self):\n return self.body\n\n\ndef add_side_effect_to_request_get(mock_requests_get):\n def mock_implementation(*args, **kwargs):\n if args[0] == \"https://dns.api.gandi.net/api/v5/domains/second-level-domain.tld\":\n return MockResponseObject(body={\"zone_records_href\": \"mock_zone_records_href\"})\n\n if args[0] == \"mock_zone_records_href\":\n return MockResponseObject(\n body=[\n {\n \"rrset_href\": \"mock_record_href\",\n \"rrset_type\": \"TXT\",\n \"rrset_ttl\": 10800,\n \"rrset_name\": \"_acme-challenge.subsubdomain.subdomain\",\n \"rrset_values\": [\"challenge_text\"],\n }\n ]\n )\n\n return MockResponseObject(status_code=404)\n\n mock_requests_get.side_effect = mock_implementation\n\n\ndef add_side_effect_to_request_post(mock_requests_post):\n def mock_implementation(*args, **kwargs):\n if args[0] == \"mock_zone_records_href\":\n return MockResponseObject()\n\n return MockResponseObject(status_code=404)\n\n mock_requests_post.side_effect = mock_implementation\n\n\ndef add_side_effect_to_request_delete(mock_requests_delete):\n def mock_implementation(*args, **kwargs):\n if args[0] == \"mock_record_href\":\n return MockResponseObject()\n\n return MockResponseObject(status_code=404)\n\n mock_requests_delete.side_effect = mock_implementation\n\n\n@mock.patch(\"requests.delete\")\n@mock.patch(\"requests.post\")\n@mock.patch(\"requests.get\")\ndef mock_requests(func, mock_requests_get, mock_requests_post, mock_requests_delete):\n class MockRequests:\n def __init__(self, mock_get, mock_post, mock_delete):\n self.get = mock_get\n self.post = mock_post\n self.delete = mock_delete\n\n def add_mock_implementations():\n add_side_effect_to_request_get(mock_requests_get)\n add_side_effect_to_request_post(mock_requests_post)\n add_side_effect_to_request_delete(mock_requests_delete)\n\n def inner(*args, **kwargs):\n mock_requests_lib = MockRequests(\n mock_requests_get, mock_requests_post, mock_requests_delete\n )\n return func(*[*args, mock_requests_lib], **kwargs)\n\n add_mock_implementations()\n return inner\n\n\nclass TestGandiDns(TestCase):\n\n EXPECTED_GET_HEADERS = {\"X-Api-Key\": MOCK_GANDI_API_KEY}\n\n EXPECTED_POST_HEADERS = {\"X-Api-Key\": MOCK_GANDI_API_KEY, \"Content-Type\": \"application/json\"}\n\n def check_correct_headers_passed(self, calls, expectedHeaders):\n for call in calls:\n self.assertEqual(call[1][\"headers\"], expectedHeaders)\n\n @mock_requests\n def test_delete_existing_record(self, mock_requests_lib):\n gandi_dns = GandiDns(GANDI_API_KEY=MOCK_GANDI_API_KEY, requests_lib=mock_requests_lib)\n gandi_dns.delete_dns_record(\"subsubdomain.subdomain.second-level-domain.tld\", \"val\")\n\n get_calls = mock_requests_lib.get.call_args_list\n delete_calls = mock_requests_lib.delete.call_args_list\n\n self.check_correct_headers_passed(get_calls, TestGandiDns.EXPECTED_GET_HEADERS)\n self.check_correct_headers_passed(delete_calls, TestGandiDns.EXPECTED_GET_HEADERS)\n\n self.assertEqual(\n get_calls[0][0][0], \"https://dns.api.gandi.net/api/v5/domains/second-level-domain.tld\"\n )\n self.assertEqual(get_calls[1][0][0], \"mock_zone_records_href\")\n self.assertEqual(delete_calls[0][0][0], \"mock_record_href\")\n\n @mock_requests\n def test_delete_non_existing_record(self, mock_requests_lib):\n gandi_dns = GandiDns(GANDI_API_KEY=MOCK_GANDI_API_KEY, requests_lib=mock_requests_lib)\n gandi_dns.delete_dns_record(\"no-exist.second-level-domain.tld\", \"val\")\n\n @mock_requests\n def test_create_record(self, mock_requests_lib):\n gandi_dns = GandiDns(GANDI_API_KEY=MOCK_GANDI_API_KEY, requests_lib=mock_requests_lib)\n gandi_dns.create_dns_record(\"subsubdomain.subdomain.second-level-domain.tld\", \"val\")\n\n get_calls = mock_requests_lib.get.call_args_list\n post_calls = mock_requests_lib.post.call_args_list\n delete_calls = mock_requests_lib.delete.call_args_list\n\n self.check_correct_headers_passed(get_calls, TestGandiDns.EXPECTED_GET_HEADERS)\n self.check_correct_headers_passed(post_calls, TestGandiDns.EXPECTED_POST_HEADERS)\n self.check_correct_headers_passed(delete_calls, TestGandiDns.EXPECTED_GET_HEADERS)\n\n self.assertEqual(\n get_calls[0][0][0], \"https://dns.api.gandi.net/api/v5/domains/second-level-domain.tld\"\n )\n self.assertEqual(get_calls[1][0][0], \"mock_zone_records_href\")\n self.assertEqual(delete_calls[0][0][0], \"mock_record_href\")\n\n self.assertEqual(\n get_calls[2][0][0], \"https://dns.api.gandi.net/api/v5/domains/second-level-domain.tld\"\n )\n self.assertEqual(post_calls[0][0][0], \"mock_zone_records_href\")\n self.assertEqual(post_calls[0][1][\"json\"][\"rrset_type\"], \"TXT\")\n self.assertEqual(post_calls[0][1][\"json\"][\"rrset_ttl\"], 10800)\n self.assertEqual(\n post_calls[0][1][\"json\"][\"rrset_name\"], \"_acme-challenge.subsubdomain.subdomain\"\n )\n self.assertEqual(post_calls[0][1][\"json\"][\"rrset_values\"], [\"challenge_text\", \"val\"])\n\n @mock_requests\n def test_create_non_existing_record(self, mock_requests_lib):\n gandi_dns = GandiDns(GANDI_API_KEY=MOCK_GANDI_API_KEY, requests_lib=mock_requests_lib)\n gandi_dns.create_dns_record(\"subdomain.second-level-domain.tld\", \"val\")\n\n get_calls = mock_requests_lib.get.call_args_list\n post_calls = mock_requests_lib.post.call_args_list\n delete_calls = mock_requests_lib.delete.call_args_list\n\n self.check_correct_headers_passed(get_calls, TestGandiDns.EXPECTED_GET_HEADERS)\n self.check_correct_headers_passed(post_calls, TestGandiDns.EXPECTED_POST_HEADERS)\n self.check_correct_headers_passed(delete_calls, TestGandiDns.EXPECTED_GET_HEADERS)\n\n self.assertEqual(\n get_calls[0][0][0], \"https://dns.api.gandi.net/api/v5/domains/second-level-domain.tld\"\n )\n self.assertEqual(get_calls[1][0][0], \"mock_zone_records_href\")\n self.assertEqual(len(delete_calls), 0)\n\n self.assertEqual(post_calls[0][0][0], \"mock_zone_records_href\")\n self.assertEqual(post_calls[0][1][\"json\"][\"rrset_type\"], \"TXT\")\n self.assertEqual(post_calls[0][1][\"json\"][\"rrset_ttl\"], 10800)\n self.assertEqual(post_calls[0][1][\"json\"][\"rrset_name\"], \"_acme-challenge.subdomain\")\n self.assertEqual(post_calls[0][1][\"json\"][\"rrset_values\"], [\"val\"])\n" }, { "path": "sewer/dns_providers/tests/test_hedns.py", "content": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.hurricane import HurricaneDns\n\nfrom . import test_utils\n\n\nclass TestHEDNS(TestCase):\n \"\"\"\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.he_uesrname = \"mock-username\"\n self.he_password = \"mock-password\"\n\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.dns_class = HurricaneDns(username=self.he_uesrname, password=self.he_password)\n\n def tearDown(self):\n pass\n\n def test_extract_zone_sub_domain(self):\n _zone = \"sub-domain\"\n domain = \"%s.%s\" % (_zone, self.domain_name)\n root, zone, acme_txt = self.dns_class.extract_zone(domain)\n\n self.assertEqual(root, self.domain_name)\n self.assertEqual(zone, _zone)\n self.assertEqual(acme_txt, \"_acme-challenge.%s\" % zone)\n\n def test_extract_zone_root(self):\n domain = self.domain_name\n root, zone, acme_txt = self.dns_class.extract_zone(domain)\n self.assertEqual(root, self.domain_name)\n self.assertEqual(zone, \"\")\n self.assertEqual(acme_txt, \"_acme-challenge\")\n\n def test_hedns_is_called_by_create_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"sewer.dns_providers.hurricane.HurricaneDns.delete_dns_record\"\n ) as mock_delete_dns_record, mock.patch(\"dns.resolver.Resolver.query\") as mock_dns_resolver:\n mock_requests_post.return_value = (\n mock_delete_dns_record.return_value\n ) = test_utils.MockResponse()\n mock_dns_resolver.return_value = test_utils.MockDnsResolver()\n\n # cause we use mock username & passworkd, the client will raise a\n # Auth Error\n try:\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n except Exception:\n pass\n\n self.assertFalse(mock_requests_post.called)\n\n def test_hedns_is_not_called_by_delete_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post:\n mock_requests_post.return_value = test_utils.MockResponse()\n\n # cause we use mock username & passworkd, the client will raise a\n # Auth Error\n try:\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n except Exception:\n pass\n self.assertFalse(mock_requests_post.called)\n" }, { "path": "sewer/dns_providers/tests/test_powerdns.py", "content": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.powerdns import PowerDNSDns\n\nfrom . import test_utils\n\n\nclass TestPowerDNS(TestCase):\n \"\"\"\n Tests for PowerDNS DNS provider class.\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.powerdns_api_key = \"mock-api-key\"\n self.powerdns_api_url = \"https://some-mock-url.com\"\n\n self.common_response = test_utils.MockResponse(status_code=204)\n self.apex_response = test_utils.MockResponse(status_code=200)\n with mock.patch(\"requests.patch\") as mock_requests_get:\n mock_requests_get.return_value = self.common_response\n self.dns_class = PowerDNSDns(\n powerdns_api_key=self.powerdns_api_key, powerdns_api_url=self.powerdns_api_url\n )\n\n def tearDown(self):\n pass\n\n def test_validate_powerdns_zone(self):\n fqdn = f\"fu.bar.baz.{self.domain_name}\"\n\n with mock.patch(\"requests.get\") as mock_requests_get, mock.patch(\n \"sewer.dns_providers.powerdns.PowerDNSDns.validate_powerdns_zone\"\n ) as mock_validate_powerdns_zone:\n\n mock_requests_get.return_value.status_code = self.apex_response\n mock_validate_powerdns_zone.return_value = self.domain_name\n\n response = self.dns_class.validate_powerdns_zone(fqdn)\n\n self.assertEqual(response, self.domain_name)\n mock_validate_powerdns_zone.assert_called_with(fqdn)\n\n def test_could_not_determine_apex_domain(self):\n with mock.patch(\"requests.get\") as mock_requests_get:\n mock_requests_get.return_value.status_code = 666\n\n self.assertRaises(\n ValueError, self.dns_class.validate_powerdns_zone, domain_name=self.domain_name\n )\n\n def test_powerdns_has_correct_changetype(self):\n self.assertRaises(\n ValueError,\n self.dns_class._common_dns_record,\n domain_name=self.domain_name,\n domain_dns_value=self.domain_dns_value,\n changetype=\"fubar\",\n )\n\n def test_powerdns_returns_correct_status_code(self):\n with mock.patch(\"requests.patch\") as mock_requests_patch, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n\n mock_requests_get.return_value = self.apex_response\n mock_requests_patch.return_value.status_code = 666\n\n self.assertRaises(\n ValueError,\n self.dns_class.create_dns_record,\n domain_name=self.domain_name,\n domain_dns_value=self.domain_dns_value,\n )\n\n def test_powerdns_is_called_by_create_dns_record(self):\n with mock.patch(\"requests.patch\") as mock_requests_patch, mock.patch(\n \"sewer.dns_providers.powerdns.PowerDNSDns.delete_dns_record\"\n ) as mock_delete_dns_record, mock.patch(\"requests.get\") as mock_requests_get:\n\n mock_requests_get.return_value = self.apex_response\n mock_requests_patch.return_value = (\n mock_delete_dns_record.return_value\n ) = self.common_response\n\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n\n def test_powerdns_is_called_by_delete_dns_record(self):\n with mock.patch(\"requests.patch\") as mock_requests_patch, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n\n mock_requests_get.return_value = self.apex_response\n mock_requests_patch.return_value = self.common_response\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertTrue(mock_requests_patch.called)\n" }, { "path": "sewer/dns_providers/tests/test_rackspace.py", "content": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.rackspace import RackspaceDns\n\nfrom . import test_utils\n\n\nclass TestRackspace(TestCase):\n \"\"\"\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.RACKSPACE_USERNAME = \"mock_username\"\n self.RACKSPACE_API_KEY = \"mock-api-key\"\n self.RACKSPACE_API_TOKEN = \"mock-api-token\"\n\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.get_rackspace_credentials\"\n ) as mock_get_credentials, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.find_dns_zone_id\", autospec=True\n ) as mock_find_dns_zone_id:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n mock_get_credentials.return_value = \"mock-api-token\", \"http://example.com/\"\n mock_find_dns_zone_id.return_value = \"mock_zone_id\"\n self.dns_class = RackspaceDns(\n RACKSPACE_USERNAME=self.RACKSPACE_USERNAME, RACKSPACE_API_KEY=self.RACKSPACE_API_KEY\n )\n\n def tearDown(self):\n pass\n\n def test_find_dns_zone_id(self):\n with mock.patch(\"requests.get\") as mock_requests_get:\n # see: https://developer.rackspace.com/docs/cloud-dns/v1/api-reference/domains/\n mock_dns_zone_id = 1_239_932\n mock_requests_content = {\n \"domains\": [\n {\n \"name\": self.domain_name,\n \"id\": mock_dns_zone_id,\n \"comment\": \"Optional domain comment...\",\n \"updated\": \"2011-06-24T01:23:15.000+0000\",\n \"accountId\": 1234,\n \"emailAddress\": \"sample@rackspace.com\",\n \"created\": \"2011-06-24T01:12:51.000+0000\",\n }\n ]\n }\n mock_requests_get.return_value = test_utils.MockResponse(200, mock_requests_content)\n dns_zone_id = self.dns_class.find_dns_zone_id(self.domain_name)\n self.assertEqual(dns_zone_id, mock_dns_zone_id)\n self.assertTrue(mock_requests_get.called)\n\n def test_find_dns_record_id(self):\n with mock.patch(\"requests.get\") as mock_requests_get, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.find_dns_zone_id\"\n ) as mock_find_dns_zone_id:\n # see: https://developer.rackspace.com/docs/cloud-dns/v1/api-reference/records/\n mock_dns_record_id = \"A-1234\"\n mock_requests_content = {\n \"totalEntries\": 1,\n \"records\": [\n {\n \"name\": self.domain_name,\n \"id\": mock_dns_record_id,\n \"type\": \"A\",\n \"data\": self.domain_dns_value,\n \"updated\": \"2011-05-19T13:07:08.000+0000\",\n \"ttl\": 5771,\n \"created\": \"2011-05-18T19:53:09.000+0000\",\n }\n ],\n }\n mock_requests_get.return_value = test_utils.MockResponse(200, mock_requests_content)\n mock_find_dns_zone_id.return_value = 1_239_932\n\n dns_record_id = self.dns_class.find_dns_record_id(\n self.domain_name, self.domain_dns_value\n )\n self.assertEqual(dns_record_id, mock_dns_record_id)\n self.assertTrue(mock_requests_get.called)\n self.assertTrue(mock_find_dns_zone_id.called)\n\n def test_delete_dns_record_is_not_called_by_create_dns_record(self):\n with mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.find_dns_zone_id\"\n ) as mock_find_dns_zone_id, mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\n \"requests.delete\"\n ) as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.delete_dns_record\"\n ) as mock_delete_dns_record, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.poll_callback_url\"\n ) as mock_poll_callback_url:\n mock_find_dns_zone_id.return_value = \"mock_zone_id\"\n mock_requests_get.return_value = (\n mock_requests_delete.return_value\n ) = mock_delete_dns_record.return_value = test_utils.MockResponse()\n mock_requests_content = {\"callbackUrl\": \"http://example.com/callbackUrl\"}\n mock_requests_post.return_value = test_utils.MockResponse(202, mock_requests_content)\n mock_poll_callback_url.return_value = 1\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n self.assertFalse(mock_delete_dns_record.called)\n\n def test_rackspace_is_called_by_create_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.delete_dns_record\"\n ) as mock_delete_dns_record, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.find_dns_zone_id\"\n ) as mock_find_dns_zone_id, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.poll_callback_url\"\n ) as mock_poll_callback_url:\n mock_requests_content = {\"callbackUrl\": \"http://example.com/callbackUrl\"}\n mock_requests_post.return_value = test_utils.MockResponse(202, mock_requests_content)\n mock_requests_get.return_value = (\n mock_requests_delete.return_value\n ) = mock_delete_dns_record.return_value = test_utils.MockResponse()\n mock_find_dns_zone_id.return_value = \"mock_zone_id\"\n mock_poll_callback_url.return_value = 1\n\n self.dns_class.create_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n expected = {\n \"headers\": {\"X-Auth-Token\": \"mock-api-token\", \"Content-Type\": \"application/json\"},\n \"data\": self.domain_dns_value,\n }\n self.assertDictEqual(expected[\"headers\"], mock_requests_post.call_args[1][\"headers\"])\n self.assertEqual(\n expected[\"data\"], mock_requests_post.call_args[1][\"json\"][\"records\"][0][\"data\"]\n )\n\n def test_rackspace_is_called_by_delete_dns_record(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\"requests.delete\") as mock_requests_delete, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.find_dns_zone_id\"\n ) as mock_find_dns_zone_id, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.poll_callback_url\"\n ) as mock_poll_callback_url, mock.patch(\n \"sewer.dns_providers.rackspace.RackspaceDns.find_dns_record_id\"\n ) as mock_find_dns_record_id:\n mock_requests_content = {\"callbackUrl\": \"http://example.com/callbackUrl\"}\n mock_requests_post.return_value = (\n mock_requests_get.return_value\n ) = test_utils.MockResponse()\n mock_requests_delete.return_value = test_utils.MockResponse(202, mock_requests_content)\n mock_find_dns_zone_id.return_value = \"mock_zone_id\"\n mock_poll_callback_url.return_value = 1\n mock_find_dns_record_id.return_value = \"mock_record_id\"\n\n self.dns_class.delete_dns_record(\n domain_name=self.domain_name, domain_dns_value=self.domain_dns_value\n )\n expected = {\n \"headers\": {\"X-Auth-Token\": \"mock-api-token\", \"Content-Type\": \"application/json\"},\n \"url\": \"http://example.com/domains/mock_zone_id/records/?id=mock_record_id\",\n }\n self.assertDictEqual(expected[\"headers\"], mock_requests_delete.call_args[1][\"headers\"])\n self.assertEqual(expected[\"url\"], mock_requests_delete.call_args[0][0])\n" }, { "path": "sewer/dns_providers/tests/test_route53.py", "content": "from unittest import mock\nfrom unittest import TestCase\n\nfrom sewer.dns_providers.route53 import Route53Dns\n\n\nclass TestRoute53(TestCase):\n \"\"\"\n \"\"\"\n\n def setUp(self):\n self.domain_name = \"example.com\"\n self.domain_dns_value = \"mock-domain_dns_value\"\n self.route53_key_id = \"mock-key-id\"\n self.route53_key_secret = \"mock-key-secret\"\n self.dns_class = Route53Dns(self.route53_key_id, self.route53_key_secret)\n\n def tearDown(self):\n pass\n\n @staticmethod\n def mocked_route53_set_record_response():\n return {\"ChangeInfo\": {\"Id\": \"mocked-id\"}}\n\n def make_change_batch(self, action, domain_name, domain_value):\n return {\n \"Comment\": \"certbot-dns-route53 certificate validation \" + action,\n \"Changes\": [\n {\n \"Action\": action,\n \"ResourceRecordSet\": {\n \"Name\": domain_name,\n \"Type\": \"TXT\",\n \"TTL\": 10,\n \"ResourceRecords\": [{\"Value\": domain_value}],\n },\n }\n ],\n }\n\n def mocked_find_zone_response(self):\n return [\n {\n \"HostedZones\": [\n {\n \"ResourceRecordSetCount\": 3,\n \"CallerReference\": \"32621E71-EA83-B2E0-9C59-51126A25A3C3\",\n \"Config\": {\"PrivateZone\": False},\n \"Id\": \"/hostedzone/Z2EH0L5RFW3ACH\",\n \"Name\": \"{}\".format(self.domain_name),\n }\n ],\n \"IsTruncated\": False,\n \"ResponseMetadata\": {\n \"RetryAttempts\": 0,\n \"HTTPStatusCode\": 200,\n \"RequestId\": \"09355760-92ea-456a-b64e-bdb0b2ff2bf1\",\n \"HTTPHeaders\": {\n \"x-amzn-requestid\": \"09355760-92ea-456a-b64e-bdb0b2ff2bf1\",\n \"content-type\": \"text/xml\",\n \"content-length\": \"3714\",\n \"vary\": \"accept-encoding\",\n \"date\": \"Wed, 04 Dec 2019 02:52:56 GMT\",\n },\n },\n \"MaxItems\": \"100\",\n }\n ]\n\n @mock.patch(\"sewer.dns_providers.route53.boto3.client\")\n def test_user_given_credential(self, mock_client):\n dns_class = Route53Dns(\"mock-key\", \"mock-secret\")\n mock_client.assert_called_once_with(\n \"route53\",\n aws_access_key_id=\"mock-key\",\n aws_secret_access_key=\"mock-secret\",\n config=dns_class.aws_config,\n )\n\n @mock.patch(\"sewer.dns_providers.route53.boto3.client\")\n def test_user_given_client(self, mock_client):\n passed_client = mock.MagicMock()\n dns_class = Route53Dns(client=passed_client)\n mock_client.assert_not_called()\n self.assertEqual(passed_client, dns_class.r53)\n\n @mock.patch(\"sewer.dns_providers.route53.boto3.client\")\n def test_user_given_creds_and_client(self, mock_client):\n with self.assertRaises(RuntimeError):\n Route53Dns(access_key_id=\"mock-key\", client=mock.MagicMock())\n\n @mock.patch(\"sewer.dns_providers.route53.boto3.client\")\n def test_user_not_given_credential(self, mock_client):\n dns_class = Route53Dns()\n mock_client.assert_called_once_with(\"route53\", config=dns_class.aws_config)\n\n @mock.patch(\"sewer.dns_providers.route53.boto3.client\")\n def test_route53_create_record(self, mock_client):\n dns_class = Route53Dns()\n # mock list zones paginator response\n mock_client.return_value.get_paginator.return_value.paginate.return_value = (\n self.mocked_find_zone_response()\n )\n mock_client.return_value.change_resource_record_sets.return_value = (\n self.mocked_route53_set_record_response()\n )\n\n change_id = dns_class.create_dns_record(self.domain_name, self.domain_dns_value)\n self.assertEqual(change_id, \"mocked-id\")\n\n mock_client.mock_calls[3].assert_called_once_with(\n HostedZoneId=\"mocked-id\",\n ChangeBatch=self.make_change_batch(\"UPSERT\", self.domain_name, self.domain_dns_value),\n )\n\n @mock.patch(\"sewer.dns_providers.route53.boto3.client\")\n def test_route53_delete_record(self, mock_client):\n dns_class = Route53Dns()\n # mock list zones paginator response\n mock_client.return_value.get_paginator.return_value.paginate.return_value = (\n self.mocked_find_zone_response()\n )\n mock_client.return_value.change_resource_record_sets.return_value = (\n self.mocked_route53_set_record_response()\n )\n\n dns_class.create_dns_record(self.domain_name, self.domain_dns_value)\n dns_class.delete_dns_record(self.domain_name, self.domain_dns_value)\n\n mock_client.mock_calls[4].assert_called_once_with(\n HostedZoneId=\"mocked-id\",\n ChangeBatch=self.make_change_batch(\"DELETE\", self.domain_name, self.domain_dns_value),\n )\n" }, { "path": "sewer/dns_providers/tests/test_unbound_ssh.py", "content": "from unittest import mock, TestCase\n\nfrom .. import unbound_ssh\n\n\n####### Mocks and other helpers #######\n\n\nclass response:\n \"web request body content and or JSON nominally decoded from body\"\n\n def __init__(self, *, content_val=\"\", json_val=None):\n self.content = content_val\n self._json = json_val\n\n def json(self):\n if self._json is None:\n raise ValueError(\"No json here\")\n return self._json\n\n\nclass MockObj:\n def __init__(self, **kwargs) -> None:\n self.__dict__.update(kwargs)\n\n\ndef patch_subprocess_run(returncode, **kwargs):\n return mock.patch(\"subprocess.run\", return_value=MockObj(returncode=returncode, **kwargs))\n\n\n####### TESTS #######\n\n\nclass TestLib(TestCase):\n\n # __init__ requires & accepts args, fails on missing\n\n def test01_init_requires_ssh_des(self):\n with self.assertRaises(TypeError):\n unbound_ssh.UnboundSsh() # pylint: disable=E1125\n\n def test02_init_okay(self):\n self.assertTrue(unbound_ssh.UnboundSsh(ssh_des=\"nobody@nowhere.man\"))\n\n def test03_init_with_alias_okay(self):\n self.assertTrue(unbound_ssh.UnboundSsh(ssh_des=\"nobody@nowhere.man\", alias=\"example.com\"))\n\n # local function unbound_command rejects invalid command\n\n def test13_unbound_command_bad_cmd_fails(self):\n with self.assertRaises(ValueError):\n unbound_ssh.unbound_command(\"bad\", \"fqdn\", \"acme_challenge\")\n\n # end to end tests (up to calling out to ssh, of course)\n\n def test21_create_delete_dns_record_okay(self):\n \"test create and delete with the ssh callout mocked\"\n\n provider = unbound_ssh.UnboundSsh(ssh_des=\"nobody@nowhere.man\")\n with patch_subprocess_run(0) as sub_run_mock:\n provider.create_dns_record(\"example.com\", \"a1b2c3d4e5f6g7h8i9j0\")\n provider.delete_dns_record(\"example.com\", \"a1b2c3d4e5f6g7h8i9j0\")\n self.assertTrue(sub_run_mock.call_count == 2)\n\n def test22_create_dns_record_fail(self):\n \"only runs through create since the fail point is in the method both call\"\n\n provider = unbound_ssh.UnboundSsh(ssh_des=\"nobody@nowhere.man\")\n with patch_subprocess_run(42, args=None) as sub_run_mock:\n with self.assertRaises(RuntimeError):\n provider.create_dns_record(\"example.com\", \"a1b2c3d4e5f6g7h8i9j0\")\n self.assertTrue(sub_run_mock.call_count == 1)\n" }, { "path": "sewer/dns_providers/tests/test_utils.py", "content": "import json\n\n\nclass MockResponse(object):\n \"\"\"\n mock python-requests Response object\n \"\"\"\n\n def __init__(self, status_code=200, content=None):\n if not content:\n content = {}\n\n try:\n content.update(\n {\n \"something\": \"ok\",\n \"result\": [{\"name\": \"example.com\", \"id\": \"some-mock-dns-zone-id\"}],\n }\n )\n except AttributeError:\n json_loads = json.loads\n else:\n json_loads = lambda a: a\n\n self.content = json.dumps(content).encode()\n self.text = json_loads(self.content.decode())\n\n self.headers = {}\n self.status_code = status_code\n\n def json(self):\n return json.loads(self.content.decode())\n\n\nclass mockLibcloudDriverZone(object):\n \"\"\"\n A mock of a dns zone in a libcloud drivers dns\n \"\"\"\n\n id = \"mock-zone-id-1\"\n\n def create_record(self, name, type, data):\n pass\n\n\nclass mockLibcloudDriver(object):\n \"\"\"\n a mock of libcloud.dns.drivers.auroradns.AuroraDNSDriver class\n \"\"\"\n\n def __init__(self, key, secret):\n pass\n\n def get_zone(self, domainSuffix):\n mock_zone = mockLibcloudDriverZone()\n return mock_zone\n\n def list_records(self, zone):\n import collections\n\n DnsRecords = collections.namedtuple(\"DnsRecords\", \"id name type\")\n one_dns_record = DnsRecords(id=\"1\", name=\"_acme-challenge\", type=\"TXT\")\n records = [one_dns_record]\n return records\n\n def get_record(self, zone_id, record_id):\n return \"mock-record\"\n\n def delete_record(self, record):\n pass\n\n\ndef mockLibcloudGetDriver(provider):\n \"\"\"\n a mock of the libcloud.dns.providers.get_driver function\n \"\"\"\n return mockLibcloudDriver\n\n\nclass MockDnsResolver(object):\n canonical_name = \"canonical.name\"\n" }, { "path": "sewer/dns_providers/unbound_ssh.py", "content": "import subprocess\n\nfrom .common import BaseDns\n\n\nclass UnboundSsh(BaseDns):\n \"\"\"\n Working demo of using aliasing with legacy DNS model.\n\n This punts the authorization issue to the ssh command. For unattended\n operation you'd need to run this with the key preloaded in ssh-agent or\n some equivalent.\n\n CLI options with alias (requires latest pre-0.8.3 code):\n --provider unbound_ssh --p_opts alias=... ssh_des=user@host\n\n Client usage:\n provider = UnboundSsh(ssh_des=\"user@host\", alias=\"...\")\n acme = client.Client(domain=\"host.your.domain\", provider=provider, ...)\n certificate = acme.cert()\n ...\n \"\"\"\n\n def __init__(self, *, ssh_des, **kwargs):\n \"\"\"\n ssh_des is a REQUIRED keyword option that specifies the \"destination\"\n argument for making the SSH connection (see ssh(1)). It is ASSUMED\n that the remote login has access to unbound's key so that it can run\n unbound-control to update the alias zone.\n \"\"\"\n\n super().__init__(**kwargs)\n self.ssh_des = ssh_des\n\n def create_dns_record(self, host_fqdn, acme_challenge):\n self.manage_dns_record(host_fqdn, acme_challenge, \"add\")\n\n def delete_dns_record(self, host_fqdn, acme_challenge):\n self.manage_dns_record(host_fqdn, acme_challenge, \"del\")\n\n def manage_dns_record(self, host_fqdn, acme_challenge, cmd):\n \"\"\"\n The first line of code here is really the whole of the aliasing demo - everything\n else is just the scaffolding to make this work in my quirky environment. :-)\n\n NB: faking the challenge dict like this is potentially fragile. Much better to\n migrate the legacy DNS driver to the new model if possible!\n \"\"\"\n\n fqdn = self.target_domain({\"ident_value\": host_fqdn})\n\n update_cmd = unbound_command(cmd, fqdn, acme_challenge)\n res = subprocess.run((\"ssh\", self.ssh_des, \"unbound-control -- \", update_cmd))\n if res.returncode != 0:\n raise RuntimeError(\n \"FAILURE (%s): unbound command failed:\\n %s\" % (res.returncode, res.args)\n )\n\n\n# DO # NOT # USE # @classmethod # just to hide the function in the class. Namespaces are great!\n\n\ndef unbound_command(cmd, fqdn, acme_challenge):\n if cmd == \"add\":\n return \"local_data %s. 300 IN TXT '%s'\" % (fqdn, acme_challenge)\n if cmd == \"del\":\n return \"local_data_remove %s.\" % (fqdn,)\n\n raise ValueError(\"Unrecognized command to unbound_command: %s\" % (cmd,))\n" }, { "path": "sewer/lib.py", "content": "import base64, codecs, json, logging, os\nfrom hashlib import sha256\nfrom typing import Any, Union\n\nLoggerType = logging.Logger\n\n\nclass SewerError(Exception):\n \"base class for sewer-related exceptions\"\n pass\n\n\nclass AcmeError(SewerError):\n \"base class [and, inevitably, catch-all] for ACME related errors\"\n pass\n\n\nclass AcmeRegistrationError(AcmeError):\n pass\n\n\n### FIX ME ### can be more specific about response arg's type... somehow\n\n\ndef log_response(response: Any) -> str:\n \"\"\"\n renders a python-requests response as json or as a string\n \"\"\"\n try:\n log_body = response.json()\n except ValueError:\n log_body = response.content[:40]\n return log_body\n\n\ndef create_logger(name: str, log_level: Union[str, int]) -> LoggerType:\n \"\"\"\n return a logger configured with name and log_level\n \"\"\"\n\n logger = logging.getLogger(name)\n logger.setLevel(log_level)\n if not logger.hasHandlers():\n handler = logging.StreamHandler()\n formatter = logging.Formatter(\"%(message)s\")\n handler.setFormatter(formatter)\n logger.addHandler(handler)\n return logger\n\n\ndef safe_base64(un_encoded_data: Union[str, bytes]) -> str:\n \"return ACME-safe base64 encoding of un_encoded_data as a string\"\n\n if isinstance(un_encoded_data, str):\n un_encoded_data = un_encoded_data.encode(\"utf8\")\n r = base64.urlsafe_b64encode(un_encoded_data).rstrip(b\"=\")\n return r.decode(\"utf8\")\n\n\ndef dns_challenge(key_auth: str) -> str:\n \"return the ACME challenge response for a DNS TXT record\"\n\n return safe_base64(sha256(key_auth.encode(\"utf8\")).digest())\n\n\n_sewer_meta = None\n\n\ndef sewer_meta(name: str) -> str:\n \"\"\"\n returns the named attribute from lazily-loaded meta.json (replaces __version__.py)\n \"\"\"\n\n global _sewer_meta\n\n if _sewer_meta is None:\n here = os.path.abspath(os.path.dirname(__file__))\n with codecs.open(os.path.join(here, \"meta.json\"), \"r\", encoding=\"utf8\") as f:\n _sewer_meta = json.load(f)\n return _sewer_meta[name]\n" }, { "path": "sewer/meta.json", "content": "{\n \"name\": \"sewer\",\n \"description\": \"Sewer is a programmatic Lets Encrypt(ACME) client\",\n \"url\": \"https://github.com/komuw/sewer\",\n \"version\": \"0.8.4\",\n \"author\": \"komuW\",\n \"author_email\": \"komuw05@gmail.com\",\n \"license\": \"MIT\",\n \"maintainer\": \"mmaney\",\n \"keywords\": \"letsencrypt,ACME,dns-01,http-01,RFC8555\"\n}\n" }, { "path": "sewer/providers/__init__.py", "content": "" }, { "path": "sewer/providers/demo.py", "content": "\"demo.py - examples of implementing non-or-minimally-functional challenge providers\"\n\n# still minimally functional - too handy for testing to make it wait for input\n\nfrom typing import Any\n\nfrom ..auth import ChalListType, ErrataListType, ProviderBase\nfrom ..lib import dns_challenge\n\n\nclass ManualProvider(ProviderBase):\n def __init__(self, *, chal_type: str = \"http-01\", **kwargs: Any) -> None:\n\n ### FIX ME ### poor example ignores possible chal_types in kwargs?\n\n # this is unusual: it can accept either DNS or HTTP challenges. Some sanity checks...\n if chal_type not in [\"dns-01\", \"http-01\"]:\n raise ValueError(\"ManualProvider: invalid chal_type value: %s\", chal_type)\n kwargs[\"chal_types\"] = [chal_type]\n super().__init__(**kwargs)\n self.chal_type = chal_type\n\n def setup(self, challenges: ChalListType) -> ErrataListType:\n return self._prompt(\"add\", challenges)\n\n def unpropagated(self, challenges: ChalListType) -> ErrataListType:\n # could add confirmation here, but it's just a demo\n return []\n\n def clear(self, challenges: ChalListType) -> ErrataListType:\n return self._prompt(\"clear\", challenges)\n\n def _prompt(self, mode: str, challenges: ChalListType) -> ErrataListType:\n for chal in challenges:\n if self.chal_type == \"dns-01\":\n print(\n \"Please {0} the challenge {1} as a TXT record on _acme-validation.{ident_value}\".format(\n mode, dns_challenge(chal[\"key_auth\"]), **chal\n )\n )\n else:\n print(\n \"Please {0} the challenge file named {token} with contents {key_auth}\".format(\n mode, **chal\n )\n )\n\n ### FIX ME ### using this for some tests, so no prompt \"press return when setup\"\n\n return []\n" }, { "path": "sewer/providers/tests/__init__.py", "content": "" }, { "path": "sewer/providers/tests/test_demo.py", "content": "import unittest\n\nfrom .. import demo\n\n\nclass TestDemo(unittest.TestCase):\n \"this actually tests nothing non-trivial, just exercises various code paths.\"\n\n def test_create_dns(self):\n p = demo.ManualProvider(chal_type=\"dns-01\")\n self.assertTrue(p)\n\n def test_create_http(self):\n p = demo.ManualProvider(chal_type=\"http-01\")\n self.assertTrue(p)\n\n def test_create_invalid(self):\n \"like this: tests one out of an infinity of invalid values. but 'covers' the exception\"\n\n with self.assertRaises(ValueError):\n demo.ManualProvider(chal_type=\"invalid-01\")\n\n def test_run_dns(self):\n p = demo.ManualProvider(chal_type=\"dns-01\")\n self.assertFalse(\n p.setup([{\"ident_value\": \"www.example.com\", \"key_auth\": \"abcdefghijklmnop.0123456789\"}])\n or p.unpropagated(\n [{\"ident_value\": \"www.example.com\", \"key_auth\": \"abcdefghijklmnop.0123456789\"}]\n )\n or p.clear(\n [{\"ident_value\": \"www.example.com\", \"key_auth\": \"abcdefghijklmnop.0123456789\"}]\n )\n )\n\n def test_run_http(self):\n p = demo.ManualProvider(chal_type=\"http-01\")\n self.assertFalse(\n p.setup([{\"token\": \"www.example.com\", \"key_auth\": \"abcdefghijklmnop.0123456789\"}])\n or p.unpropagated(\n [{\"token\": \"www.example.com\", \"key_auth\": \"abcdefghijklmnop.0123456789\"}]\n )\n or p.clear([{\"token\": \"www.example.com\", \"key_auth\": \"abcdefghijklmnop.0123456789\"}])\n )\n\n def test_accept_empty_chal_list(self):\n p = demo.ManualProvider(chal_type=\"http-01\")\n self.assertFalse(p.setup([]) or p.unpropagated([]) or p.clear([]))\n\n def test_fails_dns_bad_chal(self):\n p = demo.ManualProvider(chal_type=\"dns-01\")\n with self.assertRaises(KeyError):\n p.setup([{\"token\": \"www.example.com\", \"key_auth\": \"abcdefghijklmnop.0123456789\"}])\n\n def test_fails_http_bad_chal(self):\n p = demo.ManualProvider(chal_type=\"http-01\")\n with self.assertRaises(KeyError):\n p.setup([{\"ident_value\": \"www.example.com\", \"key_auth\": \"abcdefghijklmnop.0123456789\"}])\n" }, { "path": "sewer/tests/__init__.py", "content": "" }, { "path": "sewer/tests/test_Client.py", "content": "# the test dir is a sub dir of sewer/sewer so as\n# not to pollute the global namespace.\n# see: https://python-packaging.readthedocs.io/en/latest/testing.html\n\n# Have had to sprinkle the pylint pragma \"disable=E1125\" in the TestCase\n# classes because pylint just won't shut up about missing required keywords\n# that are passed as **kwargs. If I had time to piss away on it the pragma\n# could be added to individual calls. Not!\n\n# Also, you can't write out the pragma in a comment like the above without\n# having pylint notice it - and in this case give an error for it at module\n# scope. So reminiscent of the bad side of ol' lint.\n\n\nfrom unittest import expectedFailure, mock, TestCase\n\nimport cryptography\n\nimport sewer.client\n\nfrom ..config import ACME_DIRECTORY_URL_STAGING\nfrom ..crypto import AcmeKey, AcmeAccount\nfrom ..lib import AcmeRegistrationError\nfrom . import test_utils\n\nLOG_LEVEL = \"CRITICAL\"\n\n### FIX ME ### even with making the keys new each time, some tests manage to re-register!\n# luckily it's working anyway, but it's a good thing most of this will have to be scrapped soon\n\n\ndef keys_for_ACME(no_kid=False):\n acct = AcmeAccount.create(\"secp256r1\")\n ck = AcmeKey.create(\"secp256r1\")\n if not no_kid:\n acct.kid = \"https://imagine.acct.kid/here\"\n return {\"account\": acct, \"cert_key\": ck}\n\n\ndef usual_ACME(no_kid=False):\n res = {\n \"ACME_REQUEST_TIMEOUT\": 1,\n \"ACME_AUTH_STATUS_WAIT_PERIOD\": 0,\n \"ACME_DIRECTORY_URL\": ACME_DIRECTORY_URL_STAGING,\n \"LOG_LEVEL\": LOG_LEVEL,\n }\n res.update(keys_for_ACME(no_kid))\n return res\n\n\nclass TestClient(TestCase):\n \"\"\"\n Todo:\n - mock time.sleep\n - make this tests DRY\n - add tests for the cli\n - modularize this tests\n - separate happy path tests from sad path tests.\n eg test_get_identifier_authorization_is_called and test_get_identifier_authorization_is_not_called\n should be in different testClasses\n \"\"\"\n\n # pylint: disable=E1125\n\n def setUp(self):\n self.domain_name = \"example.com\"\n with mock.patch(\"requests.post\", return_value=test_utils.MockResponse()), mock.patch(\n \"requests.get\", return_value=test_utils.MockResponse()\n ):\n\n self.provider = test_utils.ExmpleHttpProvider()\n self.client = sewer.client.Client(\n domain_name=self.domain_name, provider=self.provider, **usual_ACME()\n )\n\n def tearDown(self):\n pass\n\n def test_get_get_acme_endpoints_failure_results_in_exception(self):\n with mock.patch(\n \"requests.post\", return_value=test_utils.MockResponse(status_code=409)\n ), mock.patch(\"requests.get\", return_value=test_utils.MockResponse(status_code=409)):\n\n def mock_create_acme_client():\n sewer.client.Client(\n domain_name=\"example.com\",\n provider=test_utils.ExmpleHttpProvider(),\n ACME_DIRECTORY_URL=ACME_DIRECTORY_URL_STAGING,\n LOG_LEVEL=LOG_LEVEL,\n **keys_for_ACME(),\n )\n\n self.assertRaises(ValueError, mock_create_acme_client)\n with self.assertRaises(ValueError) as raised_exception:\n mock_create_acme_client()\n self.assertIn(\"Error while getting Acme endpoints\", str(raised_exception.exception))\n\n def test_user_agent_is_generated(self):\n with mock.patch(\"requests.post\", return_value=test_utils.MockResponse()), mock.patch(\n \"requests.get\", return_value=test_utils.MockResponse()\n ):\n\n for i in [\"python-requests\", \"sewer\", \"https://github.com/komuw/sewer\"]:\n self.assertIn(i, self.client.User_Agent)\n\n def test_acme_registration_is_done(self):\n with mock.patch(\"requests.post\", return_value=test_utils.MockResponse()), mock.patch(\n \"requests.get\", return_value=test_utils.MockResponse()\n ), mock.patch(\"sewer.client.Client.acme_register\") as mock_acme_registration:\n\n self.client.cert()\n self.assertTrue(mock_acme_registration.called)\n\n def test_acme_registration_failure_doesnt_result_in_certificate(self):\n client = sewer.client.Client(\n domain_name=self.domain_name, provider=self.provider, **usual_ACME(no_kid=True)\n )\n with mock.patch(\n \"requests.post\", return_value=test_utils.MockResponse(status_code=400)\n ), mock.patch(\"requests.get\", return_value=test_utils.MockResponse(status_code=400)):\n\n with self.assertRaises(AcmeRegistrationError):\n client.get_certificate()\n\n def test_get_identifier_authorization_is_called(self):\n gia_return_value = {\n \"domain\": \"example.com\",\n \"url\": \"http://localhost\",\n \"wildcard\": None,\n \"token\": \"token\",\n \"challenge_url\": \"challenge_url\",\n }\n with mock.patch(\"requests.post\", return_value=test_utils.MockResponse()), mock.patch(\n \"requests.get\", return_value=test_utils.MockResponse()\n ), mock.patch(\n \"sewer.client.Client.get_identifier_authorization\", return_value=gia_return_value\n ) as mock_gia:\n\n self.client.cert()\n self.assertTrue(mock_gia.called)\n\n def test_get_identifier_authorization_is_not_called(self):\n with mock.patch(\n \"requests.post\", return_value=test_utils.MockResponse(status_code=400)\n ), mock.patch(\n \"requests.get\", return_value=test_utils.MockResponse(status_code=400)\n ), mock.patch(\n \"sewer.client.Client.acme_register\",\n return_value=test_utils.MockResponse(status_code=201),\n ):\n\n def mock_get_certificate():\n self.client.cert()\n\n self.assertRaises(ValueError, mock_get_certificate)\n with self.assertRaises(ValueError) as raised_exception:\n mock_get_certificate()\n self.assertIn(\n \"Error applying for certificate issuance\", str(raised_exception.exception)\n )\n\n def test_respond_to_challenge_called(self):\n pending_status_mock = mock.Mock()\n pending_status_mock.json.return_value = {\"status\": \"pending\"}\n\n valid_status_mock = mock.Mock()\n valid_status_mock.json.return_value = {\"status\": \"valid\"}\n\n with mock.patch(\"requests.post\", return_value=test_utils.MockResponse()), mock.patch(\n \"requests.get\", return_value=test_utils.MockResponse()\n ), mock.patch(\n \"sewer.client.Client.respond_to_challenge\"\n ) as mock_respond_to_challenge, mock.patch(\n \"sewer.client.Client.check_authorization_status\"\n ) as mock_check_authorization_status:\n mock_check_authorization_status.side_effect = [\n # 1st call returns 'pending', so respond_to_challenge has to be made\n pending_status_mock,\n # 2nd call returns 'valid', so loop breaks\n valid_status_mock,\n ]\n\n self.client.cert()\n self.assertTrue(mock_respond_to_challenge.called)\n\n def test_check_authorization_status_is_called(self):\n with mock.patch(\"requests.post\", return_value=test_utils.MockResponse()), mock.patch(\n \"requests.get\", return_value=test_utils.MockResponse()\n ), mock.patch(\"sewer.client.Client.check_authorization_status\") as mock_cas:\n\n self.client.cert()\n self.assertTrue(mock_cas.called)\n\n def test_get_certificate_is_called(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\n \"sewer.client.Client.get_certificate\"\n ) as mock_get_certificate:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.client.cert()\n self.assertTrue(mock_get_certificate.called)\n\n def test_certificate_is_issued(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n for i in [\"-----BEGIN CERTIFICATE-----\", \"-----END CERTIFICATE-----\"]:\n self.assertIn(i, self.client.cert())\n\n def test_certificate_is_not_issued(self):\n gia_return_value = {\n \"domain\": \"example.com\",\n \"url\": \"http://localhost\",\n \"wildcard\": None,\n \"token\": \"token\",\n \"challenge_url\": \"challenge_url\",\n }\n with mock.patch(\n \"requests.post\", return_value=test_utils.MockResponse(status_code=400)\n ), mock.patch(\n \"requests.get\", return_value=test_utils.MockResponse(status_code=400)\n ), mock.patch(\n \"sewer.client.Client.get_identifier_authorization\", return_value=gia_return_value\n ), mock.patch(\n \"sewer.client.Client.acme_register\",\n return_value=test_utils.MockResponse(status_code=409),\n ):\n\n def mock_get_certificate():\n self.client.cert()\n\n self.assertRaises(ValueError, mock_get_certificate)\n\n with self.assertRaises(ValueError) as raised_exception:\n mock_get_certificate()\n self.assertIn(\"Error applying for certificate\", str(raised_exception.exception))\n\n def test_certificate_is_issued_for_renewal(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n for i in [\"-----BEGIN CERTIFICATE-----\", \"-----END CERTIFICATE-----\"]:\n self.assertIn(i, self.client.renew())\n\n def test_right_args_to_client(self):\n def mock_instantiate_client():\n self.client = sewer.client.Client(\n domain_name=self.domain_name,\n provider=self.provider,\n domain_alt_names=\"domain_alt_names\",\n **usual_ACME(),\n )\n\n with self.assertRaises(ValueError) as raised_exception:\n mock_instantiate_client()\n self.assertIn(\"None or a list of strings\", str(raised_exception.exception))\n\n\nclass TestClientForSAN(TestClient):\n \"\"\"\n Test Acme client for SAN certificates.\n \"\"\"\n\n # pylint: disable=E1125\n\n def setUp(self):\n self.domain_alt_names = [\n \"blog.exampleSAN.com\",\n \"staging.exampleSAN.com\",\n \"www.exampleSAN.com\",\n ]\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n\n self.dns_class = test_utils.ExmpleDnsProvider()\n self.client = sewer.client.Client(\n domain_name=\"exampleSAN.com\",\n dns_class=self.dns_class,\n domain_alt_names=self.domain_alt_names,\n **usual_ACME(),\n )\n super(TestClientForSAN, self).setUp()\n\n\nclass TestClientForWildcard(TestClient):\n \"\"\"\n Test Acme client for wildard certificates.\n \"\"\"\n\n # pylint: disable=E1125\n\n def setUp(self):\n self.domain_alt_names = [\n \"blog.exampleSAN.com\",\n \"staging.exampleSAN.com\",\n \"www.exampleSAN.com\",\n ]\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n\n self.dns_class = test_utils.ExmpleDnsProvider()\n self.client = sewer.client.Client(\n domain_name=\"*.exampleSTARcom\",\n dns_class=self.dns_class,\n domain_alt_names=self.domain_alt_names,\n ACME_AUTH_STATUS_MAX_CHECKS=1,\n **usual_ACME(),\n )\n super(TestClientForWildcard, self).setUp()\n\n\nclass TestClientDnsApiCompatibility(TestCase):\n \"\"\"\n Test Acme client support with the deprecated dns_class parameter.\n \"\"\"\n\n # pylint: disable=E1125\n\n def setUp(self):\n self.domain_name = \"example.com\"\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n\n self.dns_class = test_utils.ExmpleDnsProvider()\n self.client = sewer.client.Client(\n domain_name=self.domain_name, dns_class=self.dns_class, **usual_ACME()\n )\n\n def test_get_get_acme_endpoints_failure_results_in_exception_with(self):\n with mock.patch(\n \"requests.post\", return_value=test_utils.MockResponse(status_code=409)\n ), mock.patch(\"requests.get\", return_value=test_utils.MockResponse(status_code=409)):\n\n def mock_create_acme_client():\n sewer.client.Client(\n domain_name=\"example.com\",\n dns_class=test_utils.ExmpleDnsProvider(), # NOTE: dns_class used here\n ACME_DIRECTORY_URL=ACME_DIRECTORY_URL_STAGING,\n LOG_LEVEL=LOG_LEVEL,\n **keys_for_ACME(),\n )\n\n self.assertRaises(ValueError, mock_create_acme_client)\n with self.assertRaises(ValueError) as raised_exception:\n mock_create_acme_client()\n self.assertIn(\"Error while getting Acme endpoints\", str(raised_exception.exception))\n\n def test_create_dns_record_is_called(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\n \"sewer.tests.test_utils.ExmpleDnsProvider.create_dns_record\"\n ) as mock_create_dns_record:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.client.cert()\n self.assertTrue(mock_create_dns_record.called)\n\n def test_delete_dns_record_is_called(self):\n with mock.patch(\"requests.post\") as mock_requests_post, mock.patch(\n \"requests.get\"\n ) as mock_requests_get, mock.patch(\n \"sewer.tests.test_utils.ExmpleDnsProvider.delete_dns_record\"\n ) as mock_delete_dns_record:\n mock_requests_post.return_value = test_utils.MockResponse()\n mock_requests_get.return_value = test_utils.MockResponse()\n self.client.cert()\n self.assertTrue(mock_delete_dns_record.called)\n\n def test_right_args_to_client(self):\n def mock_instantiate_client():\n self.client = sewer.client.Client(\n domain_name=self.domain_name,\n dns_class=self.dns_class, # NOTE: dns_class used here\n domain_alt_names=\"domain_alt_names\",\n **usual_ACME(),\n )\n\n with self.assertRaises(ValueError) as raised_exception:\n mock_instantiate_client()\n self.assertIn(\"None or a list of strings\", str(raised_exception.exception))\n\n\nclass TestClientUnits(TestCase):\n\n # pylint: disable=E1125\n\n def __init__(self, *args, **kwargs):\n super().__init__(*args, **kwargs)\n self.mock_args = {\"domain_name\": \"example.com\", \"LOG_LEVEL\": LOG_LEVEL}\n self.mock_args.update(keys_for_ACME())\n self.mock_challenges = [{\"ident_value\": \"example.com\", \"key_auth\": \"abcdefgh12345678\"}]\n\n def mock_sewer(self, provider):\n return sewer.client.Client(provider=provider, **self.mock_args)\n\n # sleep_iter is a prerequisite for the prop_timeout machinery\n\n def test01_sleep_iter_sticky(self):\n p = test_utils.ExmpleDNS(1, prop_sleep_times=[1, 2, 3, 4])\n sleep = self.mock_sewer(provider=p).sleep_iter()\n self.assertEqual([1, 2, 3, 4, 4, 4, 4, 4], [next(sleep) for i in range(8)])\n\n # prop_timeout mechanism (Client.propagation_delay)\n\n def test02_prop_timeout_okay(self):\n p = test_utils.ExmpleDNS(prop_timeout=1, fail_prop_count=0)\n self.mock_sewer(provider=p).propagation_delay(self.mock_challenges)\n\n def test03_prop_timeout_timeout(self):\n # with default [1,2,4,8] sleep times and timeout of 2, needs 3 failures to timeout\n p = test_utils.ExmpleDNS(prop_timeout=2, fail_prop_count=3)\n with self.assertRaises(RuntimeError):\n self.mock_sewer(provider=p).propagation_delay(self.mock_challenges)\n\n def test04_prop_timeout_delayed_okay(self):\n p = test_utils.ExmpleDNS(prop_timeout=20, fail_prop_count=2)\n self.mock_sewer(provider=p).propagation_delay(self.mock_challenges)\n" }, { "path": "sewer/tests/test_auth.py", "content": "import unittest\n\nfrom .. import auth, lib\n\n\ndef pbj(**kwargs):\n return auth.ProviderBase(chal_types=[\"dns-01\"], **kwargs)\n\n\nclass TestAuth01(unittest.TestCase):\n \"tests of __init__\"\n\n ### probing the required parameter - chal_types\n\n def test01_requires_chal_types(self):\n with self.assertRaises(TypeError):\n auth.ProviderBase() # pylint: disable=E1125\n\n def test02_accepts_valid_chal_types(self):\n chal_types = ([\"dns-01\"], [\"dns-01\", \"http-01\"], (\"dns-01\",), (\"dns-01\", \"http-01\"))\n for ct in chal_types:\n with self.subTest(ct=ct):\n self.assertTrue(auth.ProviderBase(chal_types=ct))\n\n def _rejects_invalid_value_chal_types(self, chal_types):\n with self.assertRaises(ValueError):\n auth.ProviderBase(chal_types=chal_types)\n\n def test03_rejects_str_chal_types(self):\n self._rejects_invalid_value_chal_types(\"naked string is most likely error\")\n\n def test04_rejects_iter_chal_types(self):\n self._rejects_invalid_value_chal_types(iter([\"dns-01\", \"http-01\"]))\n\n ### quick check for handling of unrecognized or surplus parameters\n\n def test05_rejects_unknown_parameters(self):\n with self.assertRaises(TypeError):\n pbj(jelly=\"strawberry\")\n\n ### optional, one but not both - logger and LOG_LEVEL parameters?\n\n def test06_accepts_logger(self):\n self.assertTrue(pbj(logger=lib.create_logger(\"\", \"INFO\")))\n\n def test07_accepts_log_level(self):\n self.assertTrue(pbj(LOG_LEVEL=\"INFO\"))\n\n # current implementation allows both parameters :-(\n @unittest.expectedFailure\n def test08_rejects_logger_and_log_level(self):\n with self.assertRaises(ValueError):\n pbj(logger=lib.create_logger(\"\", \"INFO\"), LOG_LEVEL=\"INFO\")\n\n ### optional prop_timeout, prop_sleep_times\n\n def test09_prop_timeout_and_times_default(self):\n p = auth.ProviderBase(chal_types=[\"dns-01\"])\n self.assertEqual(p.prop_timeout, 0)\n self.assertEqual(p.prop_sleep_times, (1, 2, 4, 8))\n\n def test10_prop_timeout_accepted(self):\n self.assertEqual(pbj(prop_timeout=30).prop_timeout, 30)\n\n def test11_prop_sleep_times_int_accepted(self):\n self.assertEqual(pbj(prop_sleep_times=4).prop_sleep_times, (4,))\n\n def test12_prop_sleep_times_list_accepted(self):\n self.assertEqual(pbj(prop_sleep_times=[2, 4, 6, 8, 10]).prop_sleep_times, (2, 4, 6, 8, 10))\n\n def test13_prop_sleep_times_tuple_accepted(self):\n self.assertEqual(pbj(prop_sleep_times=(2, 4, 6, 8, 10)).prop_sleep_times, (2, 4, 6, 8, 10))\n\n def test14_prop_sleep_times_rejects(self):\n with self.assertRaises(ValueError):\n pbj(prop_sleep_times=[1, \"b\", 4, 8])\n\n\nclass TestAuth02(unittest.TestCase):\n \"tests of abstract methods\"\n\n def test01_notimplemented_setup(self):\n with self.assertRaises(NotImplementedError):\n pbj().setup([{}])\n\n def test02_notimplemented_unpropagated(self):\n with self.assertRaises(NotImplementedError):\n pbj().unpropagated([{}])\n\n def test03_notimplemented_clear(self):\n with self.assertRaises(NotImplementedError):\n pbj().clear([{}])\n\n\nclass TestAuthHTTP(unittest.TestCase):\n \"tests for the new-model HTTP sub-base class\"\n\n def test01_requires_nothing(self):\n self.assertTrue(auth.HTTPProviderBase())\n\n def test02_accepts_chal_types(self):\n self.assertTrue(auth.HTTPProviderBase(chal_types=[\"http-01\"]))\n\n\nclass TestAuthDNS(unittest.TestCase):\n \"tests for the new-model DNS sub-base class\"\n\n def test01_requires_nothing(self):\n self.assertTrue(auth.DNSProviderBase())\n\n def test02_accepts_chal_types(self):\n self.assertTrue(auth.DNSProviderBase(chal_types=[\"dns-01\"]))\n\n def test03_accepts_alias(self):\n self.assertTrue(auth.DNSProviderBase(alias=\"dns-01\"))\n\n def test04_without_alias(self):\n p = auth.DNSProviderBase()\n chal = {\"ident_value\": \"example.com\"}\n self.assertTrue(\n p.cname_domain(chal) is None and p.target_domain(chal) == \"_acme-challenge.example.com\"\n )\n\n def test05_with_alias(self):\n p = auth.DNSProviderBase(alias=\"valid.com\")\n chal = {\"ident_value\": \"example.com\"}\n self.assertTrue(\n p.target_domain(chal) == \"example.com.valid.com\"\n and p.cname_domain(chal) == \"_acme-challenge.example.com\"\n )\n" }, { "path": "sewer/tests/test_catalog.py", "content": "import unittest\n\nfrom .. import auth, catalog\n\n\nclass TestCatalog(unittest.TestCase):\n def test01_ProviderCatalog_create(self):\n cat = catalog.ProviderCatalog()\n self.assertIsInstance(cat, catalog.ProviderCatalog)\n\n def test02_catalog_get_item_list_okay(self):\n cat = catalog.ProviderCatalog()\n self.assertIsInstance(cat.get_item_list(), list)\n\n def test03_catalog_get_descriptor_okay(self):\n cat = catalog.ProviderCatalog()\n self.assertIsInstance(cat.get_descriptor(\"unbound_ssh\"), catalog.ProviderDescriptor)\n\n def test04_catalog_get_provider_okay(self):\n cat = catalog.ProviderCatalog()\n provider = cat.get_provider(\"unbound_ssh\")(ssh_des=\"noone@nowhere\")\n self.assertIsInstance(provider, auth.ProviderBase)\n\n def test_05_catalog__str__okay(self):\n self.assertTrue(str(catalog.ProviderCatalog()))\n" }, { "path": "sewer/tests/test_lib.py", "content": "import logging\nimport unittest\n\nfrom .. import lib\n\n\nclass response:\n def __init__(self, *, content_val=\"\", json_val=None):\n self.content = content_val\n self._json = json_val\n\n def json(self):\n if self._json is None:\n raise ValueError(\"No json here\")\n return self._json\n\n\nclass TestLib(unittest.TestCase):\n def test01_log_response_json_okay(self):\n self.assertEqual(lib.log_response(response(json_val=\"{}\")), \"{}\")\n\n def test02_log_response_content_okay(self):\n self.assertEqual(lib.log_response(response(content_val=\"{}\")), \"{}\")\n\n def test11_create_logger_okay(self):\n logger = lib.create_logger(\"silly_test_logger\", 42)\n self.assertIsInstance(logger, logging.Logger)\n self.assertEqual(logger.getEffectiveLevel(), 42)\n self.assertTrue(logger.hasHandlers())\n\n def test21_safe_base64_str_or_bytes_okay(self):\n self.assertIsInstance(lib.safe_base64(\"test string\"), str)\n self.assertIsInstance(lib.safe_base64(b\"test bytes\"), str)\n\n def test31_dns_challenge_okay(self):\n res = lib.dns_challenge(\"a most spurious and unlikely key auth string\")\n self.assertEqual(res, \"lNNwvD6ceN7n6Iugd3m3k6HQD8Wk6ytGvKkwhHAV_Hw\")\n\n def test41_sewer_meta_okay(self):\n res = lib.sewer_meta(\"license\")\n self.assertEqual(res, \"MIT\")\n" }, { "path": "sewer/tests/test_utils.py", "content": "import json\n\nfrom ..auth import ProviderBase\nfrom ..dns_providers.common import BaseDns\n\n\nclass ExmpleDnsProvider(BaseDns):\n def __init__(self, **kwargs):\n self.dns_provider_name = \"example_dns_provider\"\n super().__init__(**kwargs)\n\n def create_dns_record(self, domain_name, domain_dns_value):\n pass\n\n def delete_dns_record(self, domain_name, domain_dns_value):\n pass\n\n\nclass ExmpleDNS(ExmpleDnsProvider):\n \"fail unpropagated first n times DNS mock provider\"\n\n def __init__(self, fail_prop_count, **kwargs):\n super().__init__(**kwargs)\n self.fail_prop_count = fail_prop_count\n\n def unpropagated(self, challenges):\n if self.fail_prop_count <= 0:\n return []\n self.fail_prop_count -= 1\n return [(\"unready\", \"\", c) for c in challenges]\n\n\nclass ExmpleHttpProvider(ProviderBase):\n def __init__(self):\n super().__init__(chal_types=[\"http-01\"])\n\n def setup(self, challenges):\n return []\n\n def unpropagated(self, challenges):\n return []\n\n def clear(self, challenges):\n return []\n\n\nclass MockResponse(object):\n \"\"\"\n mock python-requests Response object\n \"\"\"\n\n def __init__(self, status_code=201, content=None):\n if not content:\n content = {}\n self.status_code = status_code\n content.update(\n {\n \"newNonce\": \"http://localhost/newNonce\",\n \"keyChange\": \"http://localhost/keyChange\",\n \"newAccount\": \"http://localhost/newAccount\",\n \"newOrder\": \"http://localhost/newOrder\",\n \"revokeCert\": \"http://localhost/revokeCert\",\n \"challenges\": [\n {\n \"type\": \"dns-01\",\n \"token\": \"example-token\",\n \"url\": \"http://localhost/challenge-url\",\n },\n {\n \"type\": \"http-01\",\n \"token\": \"example-token\",\n \"url\": \"http://localhost/challenge-url\",\n },\n ],\n \"authorizations\": [\"http://localhost/authorization-url\"],\n \"finalize\": \"http://localhost/finalize-url\",\n \"status\": \"valid\",\n \"certificate\": \"http://localhost/certificate-url\",\n \"meta\": {\"termsOfService\": \"http:localhost/termsOfService\"},\n \"dummy-certificate\": \"-----BEGIN CERTIFICATE----- some-mock-certificate -----END CERTIFICATE-----\",\n \"identifier\": {\"value\": \"example.com\"},\n \"wildcard\": None,\n }\n )\n\n self.content = json.dumps(content).encode()\n self.content_to_use_in_json_method = self.content\n self.headers = {\n \"Replay-Nonce\": \"example-replay-Nonce\",\n \"Location\": \"https://localhost/acme/acct/1\",\n }\n\n def json(self):\n json_d = json.loads(self.content_to_use_in_json_method.decode())\n return json_d\n" }, { "path": "tests/crypto_test.py", "content": "import unittest\n\nfrom typing import Sequence, Tuple\n\nfrom sewer.crypto import AcmeAccount, AcmeCsr, AcmeKey\n\n\nKeyType = Tuple[str, int]\n\nrsa_key_types = ((\"rsa2048\", 2048), (\"rsa3072\", 3072), (\"rsa4096\", 4096))\nsecp_key_types = ((\"secp256r1\", 256), (\"secp384r1\", 384))\n\n\ndef fromfile_privbytes_frombytes_sign_key(key_type: KeyType) -> None:\n \"\"\"\n this has grown into a test for almost everything we do with keys\n \"\"\"\n\n type_name, key_size = key_type\n filename = \"tests/data/%s.pem\" % type_name\n\n # read_pem\n loaded_key = AcmeKey.read_pem(filename)\n assert loaded_key.pk.key_size == key_size\n\n # to_pem (\"assert\" no exception)\n loaded_pb = loaded_key.to_pem()\n\n # to_pem matches original file\n with open(filename, \"rb\") as f:\n file_bytes = f.read()\n assert loaded_pb == file_bytes\n\n # from_pem\n reloaded_key = AcmeKey.from_pem(loaded_pb)\n assert loaded_key.pk.private_numbers() == reloaded_key.pk.private_numbers()\n\n # sign_message\n assert len(loaded_key.sign_message(b\"Taketh uth to thine leaderth\"))\n\n\ndef test11_rsa_kitchen_sink():\n for kt in rsa_key_types:\n fromfile_privbytes_frombytes_sign_key(kt)\n\n\ndef test12_secp_kitchen_sink():\n for kt in secp_key_types:\n fromfile_privbytes_frombytes_sign_key(kt)\n\n\ndef generate_test(key_types: Sequence[KeyType]) -> None:\n for type_name, key_size in key_types:\n key = AcmeKey.create(type_name)\n assert key.pk.key_size == key_size\n\n\ndef test21_generate_rsa_keys():\n generate_test(rsa_key_types)\n\n\ndef test22_generate_ec_keys():\n generate_test(secp_key_types)\n\n\ndef test31_read_key_write_acct_read_acct():\n acct = AcmeAccount.read_pem(\"tests/data/secp256r1.pem\")\n assert isinstance(acct, AcmeAccount)\n test_kid = \"https://acme-v02.api.letsencrypt.org/account/abc123def456ghi789\"\n acct.kid = test_kid\n tmpfile = \"tests/tmp/test31_acct_key.pem\"\n acct.write_key(tmpfile)\n acct2 = AcmeAccount.read_key(tmpfile)\n assert acct2.kid == test_kid and acct2._timestamp == acct._timestamp\n\n\n### TODO ### CSR tests\n" }, { "path": "tests/data/README", "content": "Directory for test data and artifacts.\n" } ]